MK3 sound encode algorytm? (genesis\megadrive)

For anything related to sound (YM2612, PSG, Z80, PCM...)

Moderator: BigEvilCorporation

Post Reply
SeregaZ
Interested
Posts: 17
Joined: Mon Aug 03, 2015 12:45 pm

MK3 sound encode algorytm? (genesis\megadrive)

Post by SeregaZ » Fri Oct 30, 2020 5:10 pm

last time i am ask to help with move some data from original MK3 rom, but then i am do this my own... it was interesting experience. it was made not realy correct, but fine for me (i have not realy romhacking knowledge, just very basic :))

it was made for this case: i want to make some sound manager for MK3 GEMS. MK3 have non standart GEMS audio driver and my multi game GEMS GUI is not working with it. it like ZT sound manager - ZT have standart GEMS only for songs, but sfx is lay in special table. so i want to do same like ZT, but for MK3 (WWF Arcade have same - probably in a future i can make it too). so that MK3 have additional banks - it have not only standart 4 banks: samples, sequences, modulations and instruments - but additional samples and additional sequences. second nonstandart item is - 4bit DPCM samples.

so my plan was:
1. move that data at the end of rom a little higher. for write GEMS banks at the end of file. done (not realy 100% work probably, but it is fine)
2. unpack GEMS banks with r57shell's splitter-combaine done
3. make GUI for easy access to replace sound with online GEMS database almost done
4. decode 4bit DPCM into standart 8bit PCM (just for demo playing with GUI - because i have standart GEMS core, that cant play MK3's DPCM) done
5. additional check and reencode samples, if they have non 6.5mhz frequency (MK3 samples is playing with this frequency) - when i make some replacing sound. done
6. dismorale thing, that is killing all my plans about conquer the universe, without take attentions of doctors in psychiatry clinic. - how to make encode samples?????? not done

i have decode table, that makes 4bit DPCM into 8bit PCM - it is fine and easy to understand and i encode all original samples for my GUI for playing with standart samples playing. then i try to make backward operation - and it is work... i am dancing all this time, until... until i decide add some another song into MK3... as i found out my procedure work fine with original samples. i mean from 4 to 8 bit and back 8 to 4bit. because that samples was prepeared by creators of the game! but when i try to encode another samples, not MK3's - this my encode procedure makes wrong sound :) it have some scratch, hiss, noice and etc ugly things. i have some ideas, that fly in my head... but i cant capture it. system is - take two bytes, make second - first, then find value from table for this residue. the problem is table have values 0, 1, 3, 7... and etc. but real residue can have 2, or 4, or 5 and etc "between" value. so when i encode 8bit into 4bit, then try to decode this 4bit into 8bit - new 8bit is not a same as old one 8bit. and it cant be 100% same, but i need as much close to original.

so if some one is make somekind of this task - any advice will be nice :) or ready to eat algorytm... so now i start think about residue from... residue and keep it for next residue :))) but it is not sure... i am like Scarlett O'Hara - will think about it tomorrow. (now nothing going into my head)

SeregaZ
Interested
Posts: 17
Joined: Mon Aug 03, 2015 12:45 pm

Re: MK3 sound encode algorytm? (genesis\megadrive)

Post by SeregaZ » Sat Oct 31, 2020 10:24 am

Code: Select all

DataSection

  datajim8bit: ; original
  Data.a  $80,$7F,$80,$80,$80,$7F,$81,$7D
  Data.a  $82,$81,$7D,$83,$7E,$80,$81,$7E
  Data.a  $80,$80,$7D,$81,$7F,$7F,$80,$80
  Data.a  $7F,$80,$80,$7E,$81,$7E,$80,$7F
  Data.a  $7F,$80,$80,$7D,$82,$7E,$7F,$80
  Data.a  $7F,$81,$7E,$7F,$80,$7F,$80,$81
  Data.a  $7D,$83,$7E,$80,$7F,$81,$7C,$83
  Data.a  $7C,$83,$7C,$85,$7A,$87,$76,$8B
  Data.a  $6E,$A0,$6C,$60,$AB,$4D,$A1,$75
  Data.a  $73,$82,$85,$73,$85,$81,$75,$88
  Data.a  $77,$84,$7E,$7C,$81,$7D,$80,$82
  Data.a  $7B,$82,$7D,$81,$7E,$7F,$81,$7D
  Data.a  $80,$7F,$7F,$81,$7E,$80,$7F,$7F
  Data.a  $80,$7F,$7E,$81,$7C,$80,$80,$7F
  Data.a  $7D,$81,$7F,$7D,$80,$7F,$7E,$7F
  Data.a  $81,$7E,$7F,$83,$7B,$83,$7E,$7D
  Data.a  $82,$7C,$81,$7F,$7E,$83,$7E,$7E
  Data.a  $82,$7D,$81,$7E,$7F,$7F,$7E,$80
  Data.a  $7F,$7F,$80,$80,$7E,$80,$80,$7D
  Data.a  $80,$7E,$7F,$7E,$80,$80,$7E,$80
  Data.a  $80,$7D,$81,$7F,$7E,$80,$7E,$81
  Data.a  $7D,$80,$7F,$7F,$80,$80,$7F,$7F
  Data.a  $7F,$80,$7F,$7E,$81,$7E,$7E,$80
  enddatajim8bit:
  
EndDataSection

Enumeration
  #Window
  #Canvas
  #Button
  #TrackBar
EndEnumeration

; original decode table. not need to edit. as is.
Global Dim pikarray.b(15) ;{
pikarray(0)  = 0
pikarray(1)  = 1
pikarray(2)  = 3
pikarray(3)  = 7
pikarray(4)  = $D   ; 13
pikarray(5)  = $15  ; 21
pikarray(6)  = $1F  ; 31
pikarray(7)  = $2B  ; 43
pikarray(8)  = 0
pikarray(9)  = -1
pikarray(10) = -3
pikarray(11) = -7
pikarray(12) = -$D  ; -13
pikarray(13) = -$15 ; -21
pikarray(14) = -$1F ; -31
pikarray(15) = -$2B ; -43
;}

;{ bits operations
Macro NumToBit(Num)
  (1<<(Num))
EndMacro
Macro GetBits(Var, StartPos, EndPos)
  ((Var>>(StartPos))&(NumToBit((EndPos)-(StartPos)+1)-1))
EndMacro
;}

; paint image on a window
Procedure CanvPaint(forot.l, fordo.l, box.a, xshif.a)
  
  If StartDrawing(CanvasOutput(#Canvas))
    If box
      Box(0, 0, 880, 280, 0)
      Line(0, $80, 880, 1, RGB(0, 200, 0))
      color = RGB(240, 240, 240)
    Else
      color = RGB(80, 80, 250)
    EndIf
    x = 10
    oldx = 0
    oldy = $80
    For m = forot To fordo
      y = PeekA(m)
      ; count direction
      ; x always bigger oldx
      If y <> oldy
        height = oldy - y
      Else
        height = 1
      EndIf
      Line(x, y, oldx - x, height, color)
      oldx = x
      oldy = y
      x + xshif
    Next
    StopDrawing()
  EndIf
  
EndProcedure

Procedure.a GetEncodeValue(value.b) ; and this is one too.
  
  ret.a = 0
  
  Select value
    Case 0
      ret = 0
    Case 1 To 2
      ret = 1
    Case 3 To 6
      ret = 2
    Case 7 To 12
      ret = 3
    Case 13 To 20
      ret = 4
    Case 21 To 30
      ret = 5
    Case 31 To 42
      ret = 6
    Case 43 To 127
      ret = 7
    Case -2 To -1
      ret = 9
    Case -6 To -3
      ret = 10
    Case -12 To -7
      ret = 11
    Case -20 To -13
      ret = 12
    Case -30 To -21
      ret = 13
    Case -42 To -31
      ret = 14
    Case -127 To -43
      ret = 15
  EndSelect
  
  ProcedureReturn ret
  
EndProcedure

Procedure DPCMEncode(forstart.l, forend.l, memory.l) ; THIS MAIN PART
  
  Number.a
  OldNumber.a
  TestValue.b
  FlagOrder.a
  First.a
  Second.a
  MemShift.l
  
  OldNumber = $80 ; 0x80
  FlagOrder = 0
  MemShift  = 0
  For m = forstart To forend
    If FlagOrder = 0
      FlagOrder = 1
      
      Number = PeekA(m)                  ; read from mem
      TestValue = Number - OldNumber     ; count value
      First = GetEncodeValue(TestValue)  ; get value from table
      OldNumber = Number
    Else
      FlagOrder = 0
      
      Number = PeekA(m)                  ; read from mem
      TestValue = Number - OldNumber     ; count value
      Second = GetEncodeValue(TestValue) ; get value from table
      OldNumber = Number
      
      PokeA(memory + MemShift, second << 4 + first) ; write into memory encoded byte
      MemShift + 1                       ; move memory pointer to next byte
    EndIf
  Next
  
EndProcedure

Procedure DPCMDecode(forstart.l, size.l, memory.l)
  
  Number.a
  MemShift.l
  MemWriteValue.b
  
  MemShift = 0
  MemWriteValue = $80
  For m = forstart To forstart + size - 1
    Number = PeekA(m)
    
    ; split 8bit value into two 4bit
    DPCMfirst  = GetBits(Number, 0, 3) ; get %0000xxxx
    DPCMsecond = GetBits(Number, 4, 7) ; get %xxxx0000
    
    MemWriteValue + pikarray(DPCMfirst) 
    PokeB(memory + MemShift, MemWriteValue)
    MemShift + 1
    
    MemWriteValue + pikarray(DPCMsecond) 
    PokeB(memory + MemShift, MemWriteValue)
    MemShift + 1
  Next
  
EndProcedure

Procedure WavHeaderCreation(*memst, freq.l, size.l, bits.a)
  
             ;RIFF
             PokeB(*memst, $52):PokeB(*memst+1, $49):PokeB(*memst+2,$46):PokeB(*memst+3, $46)
             
             ;size
             PokeL(*memst+4, size+44-8)
             
             ;WAVE
             PokeB(*memst+8, $57):PokeB(*memst+9, $41):PokeB(*memst+10,$56):PokeB(*memst+11, $45)
    
             ;fmt
             PokeB(*memst+12, $66):PokeB(*memst+13, $6d):PokeB(*memst+14,$74):PokeB(*memst+15, $20) 
    
             ;size
             PokeB(*memst+16, $10)
    
             ;PCM 01
             PokeB(*memst+20, $01)
    
             ;mono stereo
             PokeB(*memst+22, $01)
    
             ;freq - 10400
             PokeL(*memst+24, freq)
    
             ;kbs
             PokeL(*memst+28, freq)
    
             ;bytes - 1
             PokeB(*memst+32, $01)
    
             ;bit
             PokeB(*memst+34, bits)
        
             ;data    
             PokeB(*memst+36, $64)
             PokeB(*memst+37, $61)
             PokeB(*memst+38, $74)
             PokeB(*memst+39, $61)
             
             ;sizedata
             PokeL(*memst+4, size)             
  
EndProcedure

OldTrackBarValue = 5
If OpenWindow(#Window, 100, 100, 900, 340, "")
  
  CanvasGadget(#Canvas, 10, 10, 880, 280)
  
  ButtonGadget(#Button, 10, 310, 50, 20, "play")
  
  TrackBarGadget(#TrackBar, 100, 310, 100, 20, 1, 10)
  SetGadgetState(#TrackBar, OldTrackBarValue)

  
  CanvPaint(?datajim8bit, ?enddatajim8bit - 1, 1, OldTrackBarValue)
  
  ; encode
  size = ?enddatajim8bit - ?datajim8bit ; count memory size, what need for encoded
  size = size / 2
  If size
    EncodedMem = AllocateMemory(size)
    If EncodedMem                           ; * 2 = for avoide odd-numbered
      DPCMEncode(?datajim8bit, ?datajim8bit + (size * 2), EncodedMem)
    Else
      Debug "mem problem"
    EndIf
  EndIf
  
  ; decode
  If EncodedMem 
    
    decodedsize = size * 2
    DecodedMem = AllocateMemory(decodedsize)
    If DecodedMem
      DPCMDecode(EncodedMem, size, DecodedMem)
      CanvPaint(DecodedMem, DecodedMem + decodedsize - 1, 0, OldTrackBarValue)
      WavMem = AllocateMemory(decodedsize + 44)
      If WavMem
        CopyMemory(DecodedMem, WavMem+44, decodedsize)
        WavHeaderCreation(WavMem, 6500, decodedsize, 8)
      EndIf
    Else
      Debug "mem problem"
    EndIf
    
  EndIf
  
  Repeat
     Select WaitWindowEvent()

       Case #PB_Event_Gadget

         Select EventGadget()
           
           Case #Button
             If EventType() = #PB_EventType_LeftClick
               If WavMem
                 ; not plays :((( too short piece?
                 sndPlaySound_(WavMem, #SND_MEMORY | #SND_ASYNC | #SND_NODEFAULT)
               EndIf
             EndIf
           Case #TrackBar
             If EventType() = #PB_EventType_LeftClick
               NewTrackBarValue = GetGadgetState(#TrackBar)
               If NewTrackBarValue <> OldTrackBarValue
                 OldTrackBarValue = NewTrackBarValue
                 CanvPaint(?datajim8bit, ?enddatajim8bit - 1, 1, OldTrackBarValue)
                 If DecodedMem
                   CanvPaint(DecodedMem, DecodedMem + decodedsize - 1, 0, OldTrackBarValue)
                 EndIf
               EndIf
             EndIf

         EndSelect

       Case #PB_Event_CloseWindow
         qiut = 1
   
     EndSelect
   Until qiut = 1

EndIf

End
and i have this result. white - original, blue that was encode and decode back. yes, it cant be 100% same, but need to be closer to original. it is without that residue count. not sure how to correct count that.
Image

SeregaZ
Interested
Posts: 17
Joined: Mon Aug 03, 2015 12:45 pm

Re: MK3 sound encode algorytm? (genesis\megadrive)

Post by SeregaZ » Sat Oct 31, 2020 12:23 pm

with this part becomes to more accurate, but anyway too many hiss :(((

Code: Select all

Procedure DPCMEncode(forstart.l, forend.l, memory.l)
  
  Number.a
  OldNumber.a
  TestValue.b
  FlagOrder.a
  First.a
  Second.a
  MemShift.l
  
  OldNumber = $80 ; 0x80
  FlagOrder = 0
  MemShift  = 0
  For m = forstart To forend
    If FlagOrder = 0
      FlagOrder = 1
      
      Number = PeekA(m)                  ; read from mem
      TestValue = Number - OldNumber     ; count value
      First = GetEncodeValue(TestValue)  ; get value from table
      OldNumber = OldNumber + pikarray(First) ;Number
    Else
      FlagOrder = 0
      
      Number = PeekA(m)                  ; read from mem
      TestValue = Number - OldNumber     ; count value
      Second = GetEncodeValue(TestValue) ; get value from table
      OldNumber = OldNumber + pikarray(Second) ;Number
      
      PokeA(memory + MemShift, second << 4 + first) ; write into memory encoded byte
      MemShift + 1                       ; move memory pointer to next byte
    EndIf
  Next
  
EndProcedure
Image

SeregaZ
Interested
Posts: 17
Joined: Mon Aug 03, 2015 12:45 pm

Re: MK3 sound encode algorytm? (genesis\megadrive)

Post by SeregaZ » Wed Mar 09, 2022 11:55 pm

i am found this news: if romspace in not a problem - i mean over 4mb - it can be used 6500 frequency 8bit PCM samples. no need to packed it into DPCM 4bit. but better is find how to correct prepare sample to decode into DPCM and get nice quality as original samples have. but no one know it :)

later need to check highest quality than 6500... who knows maybe it work fine too.

Post Reply