Bike shedding a new CD-ROM format

[Huge]

As I recall some PC protections used purposefully bad sectors, which got automatically "corrected" when dumping them. I don't remember the details, but this may be a problem if you decide to not store ECC.

I should note that if you use the disc, or mount an image with the intact subcode data of a CD+G disc, then Kega Fusion can play them back. I don't know if any other emulators can do that, I only tried with SSF and that one ignored the CD+G graphics.

[F1ReB4LL]

For most "normal" CD-ROM imaging needs, I kind of feel like MAME's CHD format is probably the right way to go.

It's not. The idea is good, the format itself is terrible. It needs a BCD image inside to be the proper MAME-grade format; right now it doesn't even have a proper TOC inside, let alone lead-in/lead-out/first pregap and even a multisession support. Nobody in MAME cares about optical formats (I only remember GDs having some love over the last few years).

The data is going to be in a consistent format 99% of the time anyway (I can easily replicate CCD SUB files for CDs that don't use R-W subchannels), so it's similar to stripping the data track parity information and regenerating it on the fly.

What are you going to do with non-zero bytes in R-W area on non-CD+G discs? Some of them are read errors, but others are mastering errors and exist on the disc, how are you going to sort them out to fix the read errors and leave all the mastering errors intact? If you're simply going to erase everything unusual, to fill R-W with 0x00, to fill F with 0x00 and 0xFF and to fix all the Qs to match the Q-CRC - yeah, the task is easy. To generate such subs you would only need to know the first EAN sector address (if exists), the first ISRC sector address (if exists), the 'distance of the step' between 2 EANs and/or 2 ISRCs, the gap type (Philips or Sony) and some kind of cue/ccd with the TOC and indices. Though, there are case like Detonator Orgun, where the audio pregap is marked as data in the Q-channel (unless my memory is tricking me, since such unusualities are mostly on the CD32 platform).

[Near]

The idea is good, the format itself is terrible.

I have to agree. Five major versions of it and counting (no attempt to cull older versions to reduce complexity), multiple compression algorithms (including gimmicks like RLE), SHA1 for some reason, a huge amount of code that's mostly tied into MAME rather than being fully independent, etc.

If a CUE is included, it would take 2-3 minutes for an emulator that can play IMG/SUB files (or even just BIN/CUE files) to support my proposed BCD format. If the CUE is omitted, make it 5-10 minutes to include a simple parser.

Nobody in MAME cares about optical formats

Last I read, their position seemed to be, "since we can't optically record every pit and landing, ISO/BIN is good enough."

What are you going to do with non-zero bytes in R-W area on non-CD+G discs?

CD+G discs must be ripped from drives that can read subchannel data. Multiple reads, preferably from multiple discs and multiple drives, should be done to reduce errors as much as possible. Unfortunately, there's no guaranteed way to read it because the CD-ROM standard designers stupidly didn't include the subchannel bytes inside the C1/C2 error codes. RS(25,29)+RS(29,33) instead of RS(24,28)+RS(28,32)+SUBCODE would have made this all a non-issue.

Auto-generating missing data is a *last resort* desperation move, and we would need to go out of our way to make it as clear as possible this was done. I'd even go so far as to add an intentionally bad W-channel string in the first TOC sector to identify the data as being faked.

Ideally, we want the officially ripped data. But, I mean, good luck getting redump.org to start over again to go back and rip the lead-in and lead-out sections on the few drives that can even do that.

The only reason to have a tool to fake the data is so that emudevs only have to support one format, and it will automatically run good, complete image rips when available.

[MetalliC]

It's not. The idea is good, the format itself is terrible. It needs a BCD image inside to be the proper MAME-grade format; right now it doesn't even have a proper TOC inside, let alone lead-in/lead-out/first pregap and even a multisession support. Nobody in MAME cares about optical formats (I only remember GDs having some love over the last few years).

can't agree, there was at least me and smf, who has been working on CHD improvements, but had no big success, so it is not in MAME mainline.
also, MAME is open project, during last years many / most of contributions made by 3rd-party people, so "nobody in MAME" sentence have no sense, anybody in this world may work on it, if he/she wish.

technically, even current CHDv5 can be used for most of applications, if add to images original TOC as binary metadata blob (similar to HDD's ATA identity dumps), and add some code which will read and parse TOC in this form.

[F1ReB4LL]

CD+G discs must be ripped from drives that can read subchannel data. Multiple reads, preferably from multiple discs and multiple drives, should be done to reduce errors as much as possible. Unfortunately, there's no guaranteed way to read it because the CD-ROM standard designers stupidly didn't include the subchannel bytes inside the C1/C2 error codes. RS(25,29)+RS(29,33) instead of RS(24,28)+RS(28,32)+SUBCODE would have made this all a non-issue.

CD+G has its own error correction, IIRC. Some drives (including Plextors) have a special (packet, 0x100b) mode with some kind of hardware error correction for the R-W channels, wiping the random errors on "normal" discs and providing the constant output for CD+Gs. Wondermega disc and Rock Paintings (both use CD+G) subs are absolutely constant across different drives/discs/dumpers if dumped in this mode. This likely wipes the possible mastering deviations, though.

Ideally, we want the officially ripped data. But, I mean, good luck getting redump.org to start over again to go back and rip the lead-in and lead-out sections on the few drives that can even do that.

We could try to resurrect the rawdump.net project or to try to build a new one using the MAME and its softlists as a base, the main problem was (and is) a lack of automatic tool (or a skilled coder to write it), because combining all the pieces manually is terrible. There were also issues with the last sectors of lead-out (different drives give up at different points), first sectors of lead-in (plextors read the lead-in data a little randomized, so you can't easily say when your lead-in dump is complete) and the discrepancies between 0xBE and 0xD8 subchannel outputs (like, 0xD8 wipes certain vital Securom-related bytes, while 0xBE fails on certain Neo-Geo CD and PC Engine discs).

can't agree, there was at least me and smf, who has been working on CHD improvements, but had no big success, so it is not in MAME mainline.
also, MAME is open project, during last years many / most of contributions made by 3rd-party people, so "nobody in MAME" sentence have no sense, anybody in this world may work on it, if he/she wish.

"I only remember GDs having some love over the last few years" was about your work

[MetalliC]

"I only remember GDs having some love over the last few years" was about your work

I was talking about "Bike shedding" new CHD revision, which will include lead-in/lead-out areas with subcodes or at least RAW TOC. as was said, there was at least smf and me who has been working on it.