SpritesMind.Net

Posted: **Fri Mar 05, 2010 1:36 pm**

Hi Everybody!!!

Shiru have suggested me to publish in this forum my MC68000 port of the aPLib decruncher, i hope that it can be useful for somebody.

The main features is that the decruncher are assembler source, small size (only 182 bytes), fast and good compress ratio.

To crunch files, you can use the appack.exe (and compiling for linux or other systems, inclusive make your custom compressor) that appears in the "examples/c" folder of this file.

The sources of the aPLib decruncher ready to use with gcc are:

Code: Select all

| -------------------------------------------------------------------------------------------------
| Aplib decruncher for MC68000 "gcc version"
| by MML 2010
| Size optimized (164 bytes) by Franck "hitchhikr" Charlet.
| -------------------------------------------------------------------------------------------------

| Make the function visible to the linker
.global aplib_decrunch

| -------------------------------------------------------------------------------------------------
| aplib_decrunch: A0 = Source / A1 = Destination
| void aplib_decrunch(unsigned char *source, unsigned char *destination); /* c prototype */
| -------------------------------------------------------------------------------------------------
aplib_decrunch:         movem.l 4(a7),a0-a1     | Recover params from the stack
                        movem.l a2-a5/d2-d6,-(a7)
                        lea     (24,a0),a0      | Skip 24 bytes from the header added for appack.exe
                        lea     32000.w,a3
                        lea     1280.w,a4
                        lea     128.w,a5
                        moveq   #1,d5           | Initialize bits counter
copy_byte:              move.b  (a0)+,(a1)+
next_sequence_init:     moveq   #0,d1           | Initialize LWM
next_sequence:          bsr.b   get_bit
                        bcc.b   copy_byte       | if bit sequence is %0..., then copy next byte
                        bsr.b   get_bit
                        bcc.b   code_pair       | if bit sequence is %10..., then is a code pair
                        moveq   #0,d0           | offset = 0 (eor.l d0,d0)
                        bsr.b   get_bit
                        bcc.b   short_match     | if bit sequence is %110..., then is a short match

                        | The sequence is %111..., the next 4 bits are the offset (0-15)
                        moveq   #4-1,d6
get_3_bits:             bsr.b   get_bit
                        roxl.l  #1,d0
                        dbf     d6,get_3_bits   | (dbcc doesn't modify flags)
                        beq.b   write_byte      | if offset == 0, then write 0x00

                        | If offset != 0, then write the byte on destination - offset
                        move.l  a1,a2
                        suba.l  d0,a2
                        move.b  (a2),d0
write_byte:             move.b  d0,(a1)+
                        bra.b   next_sequence_init
| Code pair %10...
code_pair:              bsr.b   decode_gamma
                        move.l  d2,d0           | get the new offset
                        subq.l  #2,d0           | offset == 2?
                        bne.b   normal_code_pair
                        tst.w   d1              | LMW == 0?
                        bne.b   normal_code_pair
                        move.l  d4,d0           | offset = old_offset
                        bsr.b   decode_gamma
                        bra.b   copy_code_pair
normal_code_pair:       add.l   d1,d0           | (d1 is either 1 or 0)
                        subq.l  #1,d0           | offset -= 1 (or 0)
                        lsl.l   #8,d0           | offset << 8
                        move.b  (a0)+,d0        | get the least significant byte of the offset (16 bits)
                        bsr.b   decode_gamma
                        cmp.l   a3,d0           | >=32000
                        blt.b   compare_1280
                        addq.l  #2,d2           | length += 2
                        bra.b   continue_short_match
compare_1280:           cmp.l   a4,d0           | >=1280 <32000
                        blt.b   compare_128
                        addq.l  #1,d2           | length++
                        bra.b   continue_short_match
compare_128:            cmp.l   a5,d0           | >=128 <1280
                        bge.b   continue_short_match
                        addq.l  #2,d2          | length += 2
                        bra.b   continue_short_match

| get_bit: Get bits from the crunched data (D3) and insert the most significant bit in the carry flag.
get_bit:                subq.b  #1,d5           | D5 = bit counter
                        bne.b   still_bits_left
                        moveq   #8,d5
                        move.b  (a0)+,d3        | Read next crunched byte
still_bits_left:        add.b   d3,d3           | D3.b << 1 (lsl.b #1,d3 ó roxl.b #1,d3)
                        rts

| decode_gamma: Decode values from the crunched data using gamma code
decode_gamma:           moveq   #1,d2
get_more_gamma:         bsr.b   get_bit
                        addx.l  d2,d2
                        bsr.b   get_bit
                        bcs.b   get_more_gamma
                        rts

| Short match %110...
short_match:            moveq   #3,d2           | length = 3
                        move.b  (a0)+,d0        | Get offset (offset is 7 bits + 1 bit to mark if copy 2 or 3 bytes)
                        lsr.b   #1,d0           
                        beq.b   end_decrunch    | if offset == 0, end of decrunching
                        bcs.b   continue_short_match
                        moveq   #2,d2           | length = 2
continue_short_match:   move.l  d0,d4           | old_offset = offset
copy_code_pair:         subq.l  #1,d2           | length--
                        move.l  a1,a2
                        suba.l  d0,a2
loop_do_copy:           move.b  (a2)+,(a1)+
                        dbf     d2,loop_do_copy
                        moveq   #1,d1           | LWM = 1
                        bra.w   next_sequence   | Process next sequence

end_decrunch:           movem.l (a7)+,a2-a5/d2-d6
                        rts

If you have any questions, don't hesitate to ask.

Bye!!!

NOTE: Post edit to put the last version of the code ready to use with the various genesis sdk based in gcc.

Posted: **Fri Mar 05, 2010 5:57 pm**

How much ram does it use?

Posted: **Fri Mar 05, 2010 10:27 pm**

this could be very interesting

compression ratio seems to float around 50% for md roms.

Posted: **Fri Mar 05, 2010 10:41 pm**

It would be interesting to see some figures on low/avg/high ratios for things like text vs 8 bit PCM vs 16 color BMPs.

Posted: **Fri Mar 05, 2010 10:57 pm**

I tried a couple of 32bit bmps, and got ratios between 20-80%.
for .vgm files it gives slightly better result than gzip (10-20% compression ratio).

tiles from Uwol, 16 colors

24,694 credits.bmp
3,481 credits.bmp.ap
24,694 finbad.bmp
1,279 finbad.bmp.ap
24,694 finend.bmp
472 finend.bmp.ap
24,694 fingoo.bmp
2,774 fingoo.bmp.ap
8,310 finscr.bmp
1,959 finscr.bmp.ap
24,694 gover.bmp
518 gover.bmp.ap
24,694 mojon.bmp
495 mojon.bmp.ap
3,190 sprites.bmp
1,423 sprites.bmp.ap
1,142 tilesa.bmp
373 tilesa.bmp.ap
1,142 tilesb.bmp
604 tilesb.bmp.ap
1,654 tilesc.bmp
788 tilesc.bmp.ap
24,694 title.bmp
4,209 title.bmp.ap
8,310 warp.bmp
1,998 warp.bmp.ap

Posted: **Sat Mar 06, 2010 1:40 am**

Seems to do pretty good on those 16 color BMPs.

Posted: **Sat Mar 06, 2010 4:58 am**

Just for information, results for title.bmp (most complex from all these bmp's) for different packers:

aPLib 4209
BitBuster 4258
Hrust 4376

To my knowledge, for now we have assembly depacker for first, and C for two others. Because the aPLib shows good results, it would be nice to have C calling interface for it.

Posted: **Sat Mar 06, 2010 1:12 pm**

Sorry for the delay in answer...

Chilly Willy wrote:How much ram does it use?

Well, only need a few bytes in the stack for the return address of the call to the rutine and for the BSRs to the subrutines getbit and decode_gamma (nothing recursive)... well and in the case that you need to save the registers that i use in the rutine (only have to discomment the first and the last line, the lines with the movem).

Shiru wrote: To my knowledge, for now we have assembly depacker for first, and C for two others. Because the aPLib shows good results, it would be nice to have C calling interface for it.

Depends of the calling conventions of your C compiler, i'm using vbcc and on this compiler i can mix assembly code files very easy, only have to define the prototype for the assembly function in the c code:

Code: Select all

Normal prototype:
void aplib_decrunch(unsigned char * source, unsigned char *destination);

Passing parameters in registers:
void aplib_decrunch(__reg("a0") unsigned char * source, __reg("a1") unsigned char *destination);

And modify the assembly code to get the parameters from the stack or pass the parameters in registers and mark the assembly function how public:

Code: Select all

    code
    idnt    "aplib.s"
    public  _AplibDecrunch

_AplibDecrunch:
.
.
.

I think, that it'll be very similar in other compilers.

Posted: **Sat Mar 06, 2010 1:22 pm**

These times the main C compiler for SMD development is GCC. Honestly I don't know it's calling conventions, haven't needed yet to pass arguments to the function.

Posted: **Sat Mar 06, 2010 4:04 pm**

Shiru wrote:These times the main C compiler for SMD development is GCC. Honestly I don't know it's calling conventions, haven't needed yet to pass arguments to the function.

GCC M68K calling convention always uses stack (even with maximum optimisation level). I also already tried to use "register" keyword or reg pragma but without success on parameters...

Posted: **Sat Mar 06, 2010 7:52 pm**

Yes, it's how Stef said, gcc use the stack.

I found in this forum an example to make the necessary changes, in the files vdp.h and vdp.s in the archive mddev.tar.gz that appears on this thread.

You will need to change the syntax of the source code for GAS, adding a % before the registers, recover the parameters from the stack with movem.l 4(%a7), %a0-%a1 and after that, you must save the rest of registers in the stack and recover them before exit from the rutine with the final rts (because gcc only let you use with freedom a0-a1/d0-d1, and you must preserve the rest of them).

Posted: **Sat Mar 06, 2010 8:37 pm**

SyX wrote:You will need to change the syntax of the source code for GAS, adding a % before the registers,

Ugh - no. Sorry, but it's unnecessary. Add this rule to the makefile and just use "normal" syntax.

Code: Select all

%.o: %.s
	$(AS) -m68000 --register-prefix-optional $< -o $@

The only things that need changing are the directives, which you can find here.

The ABI gcc uses for the 680x0 is as follows:

The return address is pointed to by the stack pointer, with all arguments following it as LONGs from left to right.

0(sp) = return address
4(sp) = first (leftmost) arg
8(sp) = second arg
etc.

You need to save and restore any register you use other than d0, d1, a0, and a1.

Posted: **Sun Mar 07, 2010 9:34 am**

Chilly Willy wrote:
SyX wrote:You will need to change the syntax of the source code for GAS, adding a % before the registers,
Ugh - no. Sorry, but it's unnecessary. Add this rule to the makefile and just use "normal" syntax.
Code: Select all
%.o: %.s
	$(AS) -m68000 --register-prefix-optional $< -o $@

Great Chilly Willy!!!

I made the decruncher for use in my Amiga projects, and how i told, i have been using vbcc, thanks for your comments. I have made all the modifications at the source code and the result would be:

Code: Select all

*** CODE DELETE, LAST VERSION IN THE FIRST POST ***

I hope that everything is correct, anybody can test it?

Posted: **Sun Mar 07, 2010 10:11 am**

Can't compile, the rule does not help, ton of error messages during compilation of the aplib.s (with the code above).

I've also tested appack with one of test pictures from SixPack, BitBuster wins for ~500 bytes in this particular case. This does not mean the appack is generally worse, of course.

Posted: **Sun Mar 07, 2010 6:01 pm**

GAS doesn't use ; for comments. You need to use |. This varies by the processor. The 680x0 version of GAS uses |, while the SH version uses ! instead. You can also use c-style comments in any version of GAS.

Also, a period before a label doesn't make it local. GAS local labels have a very specific form. The easiest form is a number followed by a colon. Those are local to a file. If you put a $ after the number (before the colon), it's local between two absolute labels (more like what most people think of local labels).

Check this for more on gcc labels: http://sourceware.org/binutils/docs/as/ ... mbol-Names

I just use the number form and insert comments when I wish to make known what a local label is for.

SpritesMind.Net

aPLib decruncher for 68000

aPLib decruncher for 68000