XGM / VGM player for Megadrive

[Stef]

Thanks for your tests Moon (discussed that on twitter)
Ok so, so far test 2 doesn't make any improvements on VA0 and even produce new bugs on EUR model.
test 3 seems to fix the hang/freeze issues on VA0 but also bring some minors graphics glitchs.

That does mean Japanese VA0 model can crash if you try to access the 68K bus from the Z80 CPU when a DMA is occurring... i tested on many MD models i owned and never meet this problem so i guess that only JAP VA0 is impacted. If you have some weird / rare MD models please test the XGM Player on it and report any observed issues !

So that explains why almost games disable Z80 before doing DMA operations... a method to avoid that is to cleverly design the sound driver so it doesn't access the 68K BUS during DMA operations. If your game is correctly designed any DMA operation should happen in VBlank so basically you should prevent the Z80 CPU to access the 68K BUS during the vblank period. Definitely something possible and in fact the XGM driver already try to take care of that but in some situation it cannot guarantee it. I think my latest changes may have broken something on that part, need some investigations...

[Moon-Watcher]

Thanks for your tests Moon (discussed that on twitter)

It was a pleasure

[Manveru]

We had got an unknown glitch for a year that provoques the game having a random frozen. We try in various models and all freeze randomly, but in 50hz models it happened rarely, while at 60hz it froze very often. Finally one day we try to avoid Z80 access 68k during DMA ops, and the bug dissapeared forever

[Stef]

Honestly on all MD models in my possession i never observed freezing because of that, that's really weird... I do know you may need to disable Z80 when 68000 does read/write operation on controllers port but again on any of my MD that has never been a problem.
Definitely need more tests...

[Sik]

I recall the actual issue is much more nuanced than that so depending on the sound engine it may very well never happen (or it may reliably happen within a few seconds every time...).

That said:

So that explains why almost games disable Z80 before doing DMA operations...

Not just that. If you read all of the Sega addenums they basically add up to tell you to disable the Z80 any time you attempt to access anything in the $A00000-$DFFFFF region i.e. anything that isn't cartridge or 68000 RAM. It seems the bus arbitrer really doesn't like the Z80 accessing the cartridge while the 68000 is touching the hardware ports. Something must have been altered after the first revision since Z80 access timings in general seem to have changed after that, I wonder if it was an attempt to fix this.

[Stef]

If you read all of the Sega addenums they basically add up to tell you to disable the Z80 any time you attempt to access anything in the $A00000-$DFFFFF region i.e. anything that isn't cartridge or 68000 RAM

Well in fact the only bulletin notes i remember were about controllers access and maybe i saw one which mentioned it may be better to disable Z80 during DMA operation but just as an advice (not mandatory). I never saw information about all the $A00000-$DFFFFF range and i'm almost certain that almost game doesn't disable Z80 when they are doing VDP accesses, are you certain of that, do you have that bulletin note somewhere ?
Anyway after all the solution is the same :
- make all your IO accesses at same time (during VBlank presumably)
- prevent Z80 to access the 68k BUS during that period.

Still depending how your Z80 driver is working that is not necessary easy to do.
Hopefully we have the V-int on Z80 :p

[Sik]

I never saw information about all the $A00000-$DFFFFF range

I/O area is in $A00000-$BFFFFF
VDP area is in $C00000-$DFFFFF
This is accounting for mirroring

And yeah I've seen games halt on every VDP access. I need to look up but it seems all that range is unstable (and DMA just happens to be notoriously bad at it). Then again apparently Sega's code would go further and read back every byte it writes in Z80 RAM and retries if there's a mismatch, was the whole thing really that unstable early on?

[Stef]

I/O area is in $A00000-$BFFFFF
VDP area is in $C00000-$DFFFFF
This is accounting for mirroring

Yeah i understand the mirroring but in fact i was really surprised by the VDP region access... You access it a lot so disabling Z80 each time you need to access it is terrible for PCM playback :-/

And yeah I've seen games halt on every VDP access. I need to look up but it seems all that range is unstable (and DMA just happens to be notoriously bad at it). Then again apparently Sega's code would go further and read back every byte it writes in Z80 RAM and retries if there's a mismatch, was the whole thing really that unstable early on?

What would be nice is to have MD1 VA0 user's which can run different tests case on their machine. At least with XGM player, disabling Z80 on DMA was enough but that is not a satisfying solution for me.
I really need to grab one of those MD1 VA0 but unfortunately they seems quite difficult to find out (no luck on Yahoo auction nor ebay)...

[Sik]

You access it a lot so disabling Z80 each time you need to access it is terrible for PCM playback :-/

You're saying that as if Sega had cared (look how much of a trainwreck PCM is).

[Mask of Destiny]

I'm pretty sure whatever issues there are with the VDP region are separate from the issues with the I/O region. From the Rex Sabio scans, the problem seems to be that the bus arbiter does not properly assert !WAIT to delay the Z80 until the 68K asserts !BGACK if the bus arbiter is currently servicing a request to the IO area from the 68K. This results in the read potentially returning garbage. If it's just PCM data, this is no big deal as you'll just corrupt a sample, but if the Z80 is instead reading code or some complicated data structure it can obviously be a bigger problem.

I don't see anything in the bulletins about bus requesting the Z80 during general VDP access, but I do see a note about bus requesting it during 68K->VDP DMA. Unfortunately, that one doesn't really go into detail why that is required; however, if I had to guess it's probably some kind of race condition in the bus mastering logic. Both the bus arbiter and VDP use the 68K's bus grant feature to steal the bus from the 68K. I'm guessing that one or the other can get confused and think it's being granted the bus when in fact it's the other component receiving the bus. Writing to the data port when the FIFO is full doesn't have this problem as it just uses !DTACK to delay the write which will also prevent the 68K from acknowledging any bus mastering requests until the request finishes.

[Stef]

You access it a lot so disabling Z80 each time you need to access it is terrible for PCM playback :-/

You're saying that as if Sega had cared (look how much of a trainwreck PCM is).

Probably they didn't care at beginning but with time the bad reputation of the MD for voice reproduction was definitely hurting.

I'm pretty sure whatever issues there are with the VDP region are separate from the issues with the I/O region. From the Rex Sabio scans, the problem seems to be that the bus arbiter does not properly assert !WAIT to delay the Z80 until the 68K asserts !BGACK if the bus arbiter is currently servicing a request to the IO area from the 68K. This results in the read potentially returning garbage. If it's just PCM data, this is no big deal as you'll just corrupt a sample, but if the Z80 is instead reading code or some complicated data structure it can obviously be a bigger problem.

Yeah i exactly remember about this note, the only problem is that you could potentially obtained corrupted data.
In my PCM only drivers that is not an issue as corrupted data is PCM data only, with the XGM driver the story is different as i also fetch XGM/VGM music data from rom and having corrupted data can definitely crash the Z80 but to be honest i never experienced it (even on MD1 VA0). I heard this specific problem in fact happened only on some specifics MD2 revisions (can't remember which ones).

I don't see anything in the bulletins about bus requesting the Z80 during general VDP access, but I do see a note about bus requesting it during 68K->VDP DMA. Unfortunately, that one doesn't really go into detail why that is required; however

Again, i remember exactly the same... and as you said, they were not really specific about the potential issue with the DMA :-/

if I had to guess it's probably some kind of race condition in the bus mastering logic. Both the bus arbiter and VDP use the 68K's bus grant feature to steal the bus from the 68K. I'm guessing that one or the other can get confused and think it's being granted the bus when in fact it's the other component receiving the bus. Writing to the data port when the FIFO is full doesn't have this problem as it just uses !DTACK to delay the write which will also prevent the 68K from acknowledging any bus mastering requests until the request finishes.

Something i was thinking, about the first issue of getting corrupted data when 68K is doing IO accesses. I guess i can just do "safe read" by reading the value twice and "xoring" obtained values, if result is not 0 then i redo the read operation.
But the issue with the DMA is more obscure and i don't really know what happen maybe what you described... also it seems to be present only on first models (MD1 VA0).