Interest in a custom FMV format?

Ask anything your want about Mega/SegaCD programming.

Moderator: Mask of Destiny

Post Reply
KillaMaaki
Very interested
Posts: 84
Joined: Sat Feb 28, 2015 9:22 pm

Interest in a custom FMV format?

Post by KillaMaaki » Fri Feb 01, 2019 7:35 pm

Since I just got my hands on a SegaCD this past Tuesday I got a little curious and started diving into the inner workings of FMV.
The product of this has been some tinkering with vector quantization / codebook compression techniques similar to what is used in the Sega Cinepak format.

This is what I have now. It compresses single frames of 256x160 into a single palette row of 16 colors (it assumes it can use color 0 since it can also set the VDP's background color to that), optionally bayer-dithered, and split into cells of 8x8 where each cell can either be compressed with 4x2 blocks, 2x2 blocks, or uncompressed. There can be up to 256 4x2 blocks and up to 256 2x2 blocks. Kinda want to experiment with splitting palettes on individual tiles, but I get the feeling this would very much complicate the codebook generation / quantization step, and despite some hefty optimizations (shy of attempting to SIMD optimize it) it already takes a pretty long time to encode a frame (about 10 ish seconds right now) :T

Results with a few different images:
Image

I guess at some level the format would work almost identically to Sega Cinepak, but I get the benefit of being able to document my format way better and also being able to write modern conversion tools for it (ideally I'd like to make a one-stop commandline utility that could, say, convert MP4s directly into the format) ;)

Don't suppose there'd be any interest in sharing this work as I go along? Originally just intended as a hobby project but wondered if somebody'd find it useful.

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: Interest in a custom FMV format?

Post by Chilly Willy » Sat Feb 02, 2019 1:33 am

I'm always interested in audio and video codecs, especially ones that work on old consoles. :D

It does sound quite a bit like cinepak. I don't suppose you've looked into motion estimation yet? You might take a look at RoQ format. That breaks frames into 16x16 macro blocks that are subdivided into 8x8 blocks for motion estimation, which can be broken into 4x4 blocks which can in turn be broken into 2x2 blocks. The codebook holds up to 256 2x2 blocks, and 256 4x4 blocks.

KillaMaaki
Very interested
Posts: 84
Joined: Sat Feb 28, 2015 9:22 pm

Re: Interest in a custom FMV format?

Post by KillaMaaki » Sat Feb 02, 2019 2:40 am

I haven't looked into motion estimation yet, no, though I did a quick glance through google after you mentioned it ;)

I guess it's something like estimation motion vectors from one frame to another, and using that to allow some blocks to just refer to a pixel offset to copy pixel data from in the previous frame? Guess that could help reduce number of necessary blocks in the codebooks and allow the remaining used ones to be re-adjusted afterwards to better fit their matches. :thinking:

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: Interest in a custom FMV format?

Post by Chilly Willy » Sat Feb 02, 2019 4:46 pm

Pretty much. RoQ allows a block to be copied from the previous frame at the same position -7 to +8 pixels horizontally and vertically (one byte that would have been a codebook index is now two 4 bit motion vectors). It only allows it on 8x8 blocks. 4x4 and 2x2 blocks aren't motion estimated. A 4x4 block can be either from the 4x4 codebook, or a scaled up 2x2 codebook entry. 2x2 blocks only come from the codebook.

KillaMaaki
Very interested
Posts: 84
Joined: Sat Feb 28, 2015 9:22 pm

Re: Interest in a custom FMV format?

Post by KillaMaaki » Sat Feb 02, 2019 7:40 pm

I could see that getting a little complex though... easy enough for a more classic framebuffer (like, say, playing back fmv on a CD+32X game) to calculate offset to copy from and just do some quick memory copies, but at the moment my codec is pretty custom-tailored for the base MD and so it splits the framebuffer into 8x8 chunks to make it easy to copy into VRAM as a tileset. Trying to copy a block of data that straddles tile boundaries could get a bit complicated I would imagine.
('course looks to me like RoQ is pretty much designed for more classic framebuffer anyway, so it be a good source of inspiration for a 32X variant?)

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: Interest in a custom FMV format?

Post by Chilly Willy » Sun Feb 03, 2019 1:00 am

Yeah. I've been working on a 32X RoQ player for the CD32 for a while when I get time. I made my own custom build of ffmpeg where I added bit rate management for RoQ format. If you think it's slow encoding normally, try needing to retry frames to adjust the bit rate. :lol:

I did a RoQ player for the N64, and it's nice. Plenty of power to decode. The 32X needs to be a bit more optimized. The Saturn would be easier as the processors are clocked faster, and have 32-bit ram for half the memory. I'll probably work on a Saturn version at some point.

bioloid
Very interested
Posts: 169
Joined: Fri May 18, 2012 8:22 pm

Re: Interest in a custom FMV format?

Post by bioloid » Tue Feb 05, 2019 5:35 pm

I'm interested of using it in my next demo.
But its more to play animated gif than video.

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: Interest in a custom FMV format?

Post by Chilly Willy » Wed Feb 06, 2019 1:19 am

Something like that might do better with an "old school" format, like the old Amiga ANIM formats. Those stored the first two frames, then the difference between the next frame and the second to last frame (for screen flipping). If not much is changing, formats like that look good and compress decently.

KillaMaaki
Very interested
Posts: 84
Joined: Sat Feb 28, 2015 9:22 pm

Re: Interest in a custom FMV format?

Post by KillaMaaki » Sat Feb 09, 2019 5:43 am

Oh, I just did some more digging... So it sounds like there's a way I can just treat frames like a plain old single bitmap but have it pretty much automatically translated to 8x8 tiles for easy VDP upload?
That would make that motion estimation you mentioned far easier... I may want to play with this now.

But first I wanna see about just encoding an image like I already have and just display that via, say, a simple Genesis rom (not involving the CD yet)

Post Reply