sixpack handling of palettes for multiple images?

Talk about development tools here

Moderator: BigEvilCorporation

ammianus
Very interested
Posts: 124
Joined: Sun Jan 29, 2012 2:10 pm
Location: North America
Contact:

sixpack handling of palettes for multiple images?

Post by ammianus » Mon Feb 13, 2012 3:47 am

I sort of mentioned this question in this thread but maybe it could be called out in a separate discussion.

I am trying to use sixpack 1.3 for compressing some images and generating the palettes. The target I am using is 32X, the format is l8 (8bpp). Lets say you have multiple 256-color bitmap image files, A.bmp, B.bmp, C.bmp. A and B might be similar, but C may be different (another sprite or a background image). The way you process this with sixpack is through individual runs of the tool, so as a result you get 3 binary files in some format, e.g. LZSS, and 3 separate palette files. In the case of 32X images, 256 color mode, Each byte in the frame buffer corresponds to one entry in the CRAM. So how do you handle multiple images with different palettes?

In my case, A, and B images have about 20 colors, image C has 84 colors. I want to display all 3 at once. Should be possible right?

I guess a couple of problems I don't really know how to solve:

1) Repetitive colors in CRAM, I can see from reviewing colors that are stored if I load my palettes in end to end, that there are duplicates, which makes sense, sixpack doesn't know about the other images and their palettes.

2) Rhyme or reason to generated palette order? Is it based on some rule? Is there a sort order to the colors?

3) Lets say I could manually create the full palette I need (the generated .pal files are binary and not conducive to editing), how does sixpack account for this in the created image binary files? I mean each run of sixpack must index the image output with the palette it generates at runtime? Is there some trick like putting 1 pixel of all the colors in my palette in each image that I process?

4) How does the '-base n' argument affect the generated image and the .pal? If I want to start the index of the colors an image references in CRAM, can I put the offset of the image's palette in the CRAM in this argument? e.g. in my case, I put first 20 colors from Palette A/B in CRAM [0-19], then I put the 84 colors for Palette C in CRAM[20-103], do I call -base 20 when processing image C with sixpack?

I am wondering how others may have solved these kinds of problems before? Or is there another tool, besides sixpack, that can assist with compression and image formatting specifically for 32X? There is no undocumented feature that lets you tell sixpack to reference an external palette, right?

ammianus
Very interested
Posts: 124
Joined: Sun Jan 29, 2012 2:10 pm
Location: North America
Contact:

Post by ammianus » Mon Feb 20, 2012 1:20 am

Well in case anyone is interested, I've solved at least part of the problem by modifying sixpack source to export a file containing all of the colors it finds in the image. I also added a flag so that it can read the file in that format and then sixpack won't duplicate colors but refer to the same set.

From this I was able to pass each of my images through sixpack, and build a single common palette containing all the colors used by all the images that I wanted to display.

It actually worked the first time I tried it in an emulator.

I'd be happy to share the changed code, please let me know anyone is interested or if mic_ wants to incorporate it back into his stuff.

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Post by Chilly Willy » Mon Feb 20, 2012 1:40 am

Nice work! I'm sure mic would be interested in that. Go ahead and post it here, as well.

ammianus
Very interested
Posts: 124
Joined: Sun Jan 29, 2012 2:10 pm
Location: North America
Contact:

Post by ammianus » Mon Feb 27, 2012 2:23 am

It took me a while to get my hosting squared away.

Here is the new version of sixpack. I've included the source (note, it was built for Windows using Eclipse, so I had to change some things unrelated to my changes to get it to work).
https://github.com/ammianus/sixpack-ammianus

I've added two new arguments:

-palref <filename>
-savepal <filename>

palref reads the colors from the provided file, in that order(can be produced using -savepal argument).

savepal writes an output file with all the colors found in the image by sixpack, plus any from the palette reference file if provided.

The idea is that you could use the same palette file for multiple images and the end .pal file would be the one you actually use in your program. This would solve the duplication issue, as well as ensure the colors referred to in your images would always be in the same place in your palette.

I haven't done any backwards compatibility testing, I just used it for 32X 8bpp VDP mode.

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Sat Jan 07, 2023 7:02 pm

Ok, for the life of me, i cannot get this to work. I've got it compiled and if i use the basic functionality like the original sixpack then things work, but I cannot figure out what in the world ammianus is talking about (so i'm sure i'm just missing something on my end). Below is what i'm trying to do

I've got three images bg.png (which is a back ground), s1 which is my first sprite, and s2 which is my second sprite. So, in theory I should be able to generate a single .pal file that i can load for all three of these images (.apx files).

I must be a little confused on how to use the latest changes (extra flags he added) Here are the commands i've ran:

1) sixpack -image -target 32x -format l8 -q 256 -width 96 -height 56 -sizefilter 1 -o s2.apx -palref main.pal -savepal main.pal images/s2.png
2) sixpack -image -pack -target 32x -codec aplib -format l8 -q 256 -width 320 -height 224 -sizefilter 1 -o bg.apx -palref main.pal -savepal main.pal images/bg.png
3) sixpack -image -target 32x -format l8 -q 256 -width 96 -height 96 -sizefilter 1 -o s1.apx -palref main.pal -savepal main.pal images/s1.png

All three commands produce their .apx files along with their respective .pal files and it seems like each command keeps adding more to the main.pal file. So, you'd think i should be able to load up just the main.pal file; but when i do that all the colors are off...way off

I even tried to just load up the main.pal after running just the first command just to see if the image colors look right with only 1 image in main.pal .... and the colors are still off, but if i just load up the normal .pal file that is produce along with that command that corresponds to the s1.png then that one sprite looks normal

I've tried changing the -palref to the previous ran command, but the new sixpack doesn't like that (seems to only like the main.pal file that was generated ... if its working at all)

then i thought (well maybe the author meant to "just" run his flags and reference the images originally produced by sixpack). So I tried a command like: sixpack -palref images/s1.png -savepal main.pal But nothing happens

So i am at a loss and I hope that i'm truly just doing somthing dumb here. Any help?

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: sixpack handling of palettes for multiple images?

Post by Chilly Willy » Mon Jan 09, 2023 3:03 pm

palref wants a palette file, not an image. It has the same format as the generated savepal file. For the first image, you want to not use palref so that it only uses the colors in the image, and save that palette using savepal. For the other images, you then load that palette file with palref, and save a newer palette file with savepal. I'm not certain, but I think your main problem is you're quantizing all your images to 256 colors... but you're doing that for EVERY image while you only have 256 colors in total. You need to quantize each image to fewer colors. Now the processing will find matching colors in the loaded palette, if they exist, so you only need 256 / number images in the case where all images have nothing but unique colors. You'll have to play around with it to find the right balance... or do what I do - load ALL sprites and backgrounds into a single image (in RGB mode to preserve as many colors as possible), load that one image and quantize it down to 256 colors and save the palette and quantized image. Then chop that quantized image into individual images for each object, keeping the colors unchanged, making your final bg images and sprite sheets.

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Tue Jan 10, 2023 3:30 am

Chilly Willy wrote:
Mon Jan 09, 2023 3:03 pm
palref wants a palette file, not an image. It has the same format as the generated savepal file. For the first image, you want to not use palref so that it only uses the colors in the image, and save that palette using savepal. For the other images, you then load that palette file with palref, and save a newer palette file with savepal. I'm not certain, but I think your main problem is you're quantizing all your images to 256 colors... but you're doing that for EVERY image while you only have 256 colors in total. You need to quantize each image to fewer colors. Now the processing will find matching colors in the loaded palette, if they exist, so you only need 256 / number images in the case where all images have nothing but unique colors. You'll have to play around with it to find the right balance... or do what I do - load ALL sprites and backgrounds into a single image (in RGB mode to preserve as many colors as possible), load that one image and quantize it down to 256 colors and save the palette and quantized image. Then chop that quantized image into individual images for each object, keeping the colors unchanged, making your final bg images and sprite sheets.
Chilly! You are my hero:). Loved your interview with retrorgb btw! This is not me blowing smoke, but without your works and help the modern retro wave just wouldn’t exist

Also, all my previous games i had been working on are done. One of which got published via iphone and android devices. So i say that all to say that im 100% committed to building a new 32x game and I will do it ( i may beat my head against a wall figuring things out. But its going to happen)

So, back to your help:)! When you say “load them all up into one image and chop them that quantized image down…” are you meaning chop them into physical files again or chop them up during runtime?

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: sixpack handling of palettes for multiple images?

Post by Chilly Willy » Wed Jan 11, 2023 10:39 pm

Chop them up before compiling the rom. That way they use the least amount of space. When you combine all the images together into one, it's unlikely that it will all fit together into a nice rectangle. I tend to have gaps and parts sticking out this way and that... so I just chop them up into individual pieces that hold just the background/object in question. Or you could just leave it all as one big image if you have the rom space and don't care. Maybe just keeping offsets into one image is easier for your game. Either way will work.

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Wed Jan 11, 2023 10:58 pm

Chilly Willy wrote:
Wed Jan 11, 2023 10:39 pm
Chop them up before compiling the rom. That way they use the least amount of space. When you combine all the images together into one, it's unlikely that it will all fit together into a nice rectangle. I tend to have gaps and parts sticking out this way and that... so I just chop them up into individual pieces that hold just the background/object in question. Or you could just leave it all as one big image if you have the rom space and don't care. Maybe just keeping offsets into one image is easier for your game. Either way will work.
Thanks Chilly. I was going nuts writing a routine that only displays certain sprite from my sprite sheet. So knowing that you meant to actually have individual sprite files (sprite1.apx, 2.apx, etc) with one .pal file to rule them all is very helpful

Question though, im having a hard time getting the colors to still match even after chopping images up into separate files. Below are the steps i take

1). Open up my master_sprites.png sheet in asesprite
2). Select just one 32x32 sprite from the sheet
3). New document within asesprite (make sure canvas size is 32x32, rgb mode, and transparency)
4). Paste copied sprite into new document (interesting though looks like asesprite removed the unused colors from the palette)
5). Save as a new png file. (Sp1.png)
6). Using sixpack create .pal and .apx files from master_sprites with -q 256
7). Using sixpack create .pal and .apx files from newly created sp1.png with -q 32 (according to asesprite that one sprite only uses 32 colors i guess)
8 ). Only use the .pal from the master_sprite
9). Draw the newly created sp1.apx

But the colors are still all messed up:(. What may i be doing wrong? I thought I need to quantize the master_sprite.png before cutting out a new sprite with asesprite. But i cannot open up the .apx file generated by sixpack tool in asesprite or any image editor. Feels like im missing something

Thanks again for all your help

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: sixpack handling of palettes for multiple images?

Post by Chilly Willy » Thu Jan 12, 2023 9:57 am

Each time you quantize, it generates new colors. When you quantize for 256 colors, you're done if the images are for the 32X. There's no need to further quantize to fewer colors since everything is in the image when quantized to 256 colors. When you chop the big image up, you do nothing other than chop, so no new colors are generated, and it keeps the palette the big image had.

1 - combine all images into big RGB image
2 - quantize to 256 colors (keep as 256 color indexed BMP format image)
3 - chop into pieces - do nothing that would alter the colors or palette (each one a smaller 256 color indexed BMP file)

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Thu Jan 12, 2023 8:39 pm

Chilly Willy wrote:
Thu Jan 12, 2023 9:57 am
Each time you quantize, it generates new colors. When you quantize for 256 colors, you're done if the images are for the 32X. There's no need to further quantize to fewer colors since everything is in the image when quantized to 256 colors. When you chop the big image up, you do nothing other than chop, so no new colors are generated, and it keeps the palette the big image had.

1 - combine all images into big RGB image
2 - quantize to 256 colors (keep as 256 color indexed BMP format image)
3 - chop into pieces - do nothing that would alter the colors or palette (each one a smaller 256 color indexed BMP file)
Hey Chilly! thanks for the reply and the above makes sense. Unfortunately, you'd think simple steps would be ... simple. So, today I've decided to create this reply with all the steps and screenshots that i've been doing (with your new changes in mind). As something is still not quite right. Even after doing the above the sprite from the master sprites comes out with less colors (or completely messed up looking if i try to just load the sprite.bmp with the sprites.pal instead of the sprite.apx)

1. Here is the sprites.png file that I'm starting with in all its glory, opened up in asesprite.
1.png
1.png (126.99 KiB) Viewed 6427 times
2. Quantize down to 256 colors step. I tried two approaches yielding the same bad results:

a). Use the preview .bmp file that gets generated by sixpack tool running this command to get my newly created sprites.bmp "sixpack -image -target 32x -format l8 -width 320 -height 202 -sizefilter 1 -preview sprites.bmp -q 256 -o sprites.apx images/sprites.png" The sprites.bmp is now seen in the image below.
2.png
2.png (67.43 KiB) Viewed 6427 times
I then re-run the sixpack tool with that new sprites.bmp file with this command "sixpack -image -target 32x -format l8 -width 320 -height 202 -sizefilter 1 -o sprites.apx images/sprites.bmp" in order to get the newly created sprites.pal file that i'll use as the master .pal file for all sprites. I put that .pal file to the side for now and move on to creating a single sprite.bmp file

I go back to asesprite and edit the sprites.bmp. The only editing i'm doing is removing all sprites but the one sprite i want and resizing the canvas. Then i save as sprite.bmp. See the image below
3.png
3.png (27.59 KiB) Viewed 6427 times
b) Feel free to skip this section until later; but this goes through using gimp to generate those same sprites.bmp and sprite.bmp file. Interesting though, they yield different colors than the above method.

3. Now that I have my sprites.pal file and my single sprite.bmp to test with, I will try two different approaches.

a). This approach is to simple use the sprite.bmp file as it is without generating the sixpack .apx file. So i copy both the sprites.pal and the sprite.bmp file into my projects directory and then i make the following changes in the code so that the rom sees them. See below for the code and changes:

...to be continued on next post (think i hit a limit on how many images i can attach)

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Thu Jan 12, 2023 8:55 pm

Continuation of Step 3:

Changes are to the image.s file so that the rom see's where the images are at.
4.png
4.png (92.28 KiB) Viewed 6427 times
Here is where I load up the palette and draw the sprite within code. Just in case i'm doing something squirrely here. I'm using a lot of aminnaus' code from earlier examples (which includes his 32x_image library for drawing sprites and loading palettes). The red circle is the call to the loadpalette function and if you're curious both pal and pal_end are declared as "extern char pal[], pal_end[];" The blue circle is the drawsprite function and if you're curious ship is declared as "extern vu16 ship" and g_parallax_x is a global int that the slave manipulates for testing controller input (thinking of having my slave do that long with sound perhaps).
5.png
5.png (155.71 KiB) Viewed 6427 times
Here is what the loadpalette function and the drawsprite function look like
6.png
6.png (183.25 KiB) Viewed 6427 times
...to be continued on next post

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Thu Jan 12, 2023 9:10 pm

Ok, so now still in step 3, you have seen that i've edited the image.s file to include the sprite.bmp and the sprites.pal file. You also can see where i load up the palette and what that function looks like along with the drawsprite section as well. So below is what the screen looks like when i run the newly compiled rom
7.png
7.png (15.65 KiB) Viewed 6427 times
That fugly looking blob in the top left corner is suppose to be the spaceship sprite ... very depressing lol. So, now let's go back to sixpack and go ahead and generate the sprite.apx file from the sprite.bmp file and see if we have any better results.

b). This is the second method of testing the sprite.bmp. As described above i'm simply using sixpack tool again to create my sprite.apx from the sprite.bmp file using this command (note, i'm not quanitizing anymore) "sixpack -image -target 32x -format l8 -width 32 -height 32 -sizefilter 1 -o sprite.apx images/sprite.bmp"

I then copy over the newly created sprite.apx file only and change the image.s file from "sprite.bmp" to "sprite.apx" and recompile the rom. Below is the results i get this time. Still depressing :(
8.png
8.png (11.06 KiB) Viewed 6427 times
Now, for sanity purposes. I went ahead and copied over the sprite.pal (that's the single sprites palette) and made the change in my image.s to load that sprite.pal. This will work and beautifully show the sprite (and only that sprite) in all its true colors and glory. See below to what this should be doing :(
9.png
9.png (10.14 KiB) Viewed 6427 times
...now back to step 2.b for the gimp steps that i tried in my next posting

matthewnimmo
Very interested
Posts: 87
Joined: Thu Jan 07, 2021 8:04 pm

Re: sixpack handling of palettes for multiple images?

Post by matthewnimmo » Thu Jan 12, 2023 9:29 pm

Having to step back to step 2.b on a separate posting because i'm only allowed 3 screen shots attached per posting.

2) ...
...
b) This option I'm going to use GIMP to create my sprites.bmp file and my sprite.bmp file. First I open up the original sprites.png file that I showed in asesprite (so no need to show that picture again). Then I go to Image -> Mode -> indexed. And then i leave everything else alone and click convert.
10.png
10.png (18.67 KiB) Viewed 6427 times
Then, I go to File -> export as and type in "sprite.bmp" and click export. I get prompted letting me know that the transparency layer will be ignored and then i get another prompt and i selected "Do not write color space information" (i've tried both ways and nothing changes the outcome. If i leave it unchecked then sixpack doesn't like it and won't load it).
11.png
11.png (14.19 KiB) Viewed 6427 times
I then do the same thing i did with asesprite and crop down the newly created sprites.bmp file to the one sprite i want. I resize the canvas, and I export the same way i did earlier but its now just the single sprite as a sprite.bmp file.

Now that I have my sprites.bmp, time to get that juciy .pal file! So i head back to my sixpack tool and generate the respective sprites.apx and .pal using the command i mentioned earlier (just this time i dont quanitze down since it "should" have been done by gimp already). this step is just to secure that .pal file

So now, i have both my sprites.pal and sprite.bmp file and ready to try again with these files generated via gimp. Ultimately, they end with similar results with ugly representations of my single sprite.

...

So, at this point I do not know what to do? I've tried countless iterations/variations of the steps above. Seems like i'm missing a critical step (or is it the tools that im using?) Any help would be greatly appreciated.

thanks again!

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: sixpack handling of palettes for multiple images?

Post by Chilly Willy » Fri Jan 13, 2023 10:22 pm

Okay, probably the key thing to converting things to 256 colors and splitting them up correctly... use GIMP. :lol:

I'm not certain where you're having trouble with asesprite, but GIMP never gives me any trouble. Use a hex editor to double check the header your version of GIMP is saving. Even being off a few bytes will cause images to be whack. When I last updated the version of Xubuntu I use, GIMP also updated, and the header changed, meaning I had to change a number of offsets I use in dealing with GIMP bumps to make them look right.

Post Reply