Scaling hardware?

[Eke]

Just a few more progress:

Sangokushi III - Romance of the Three Kingdoms (hangs at Sega logo)

appeared to be a bad dump, a good dump works fine

Earnest Evans (hangs after Wolf Team logo)
Seirei Shinseiki - Fhey Area (hangs after Wolf Team logo)

needed SUB-CPU sync before triggering the level 2 interrupt from MAIN-CPU side

i'm problably a spammer International Soccer (hangs after Sega logo)
Jeopardy (hangs during intro)
NHL '94 (hangs after intro)
Seima Densetsu 3x3 Eyes (hangs after Sega logo)

were fixed by improving MAIN-CPU polling loop detection, i solved the issue by clearing CPU polling detection flag every time a "DBF" instruction is executed: it appeared that successive readings of the same register were stopping the CPU when they should only have been done a dozen time in a row. Reducing needed cycle gap between two successive read for polling detection would not help: it makes other sensitive games hang since CPU sync is highly tied to MAIN-CPU being stopped on polling loop. So it was tight but it now works for every sync-sensitive games.

ESPN Sunday Night NFL (hangs when starting new game + gfx errors during intro)
NFL Football Trivia Challenge (hangs when starting a game)

gfx errors in the former where fixed by having Word-RAM state being maintaned on mode switching
game not starting were because i didn't noticed you needed to use 2nd player controller to go further

Corpse Killer (random "drive error" hangs during game)
Joe Montana's NFL Football (hangs when starting a game)

were apparently fixed by some of the above changes, though I'm not sure which ones

Battle Frenzy / Blood Shot (graphic issues)
Chuck Rock (corrupted tiles)
Ecco the Dolphin (random corrupted tiles)
Ecco: The Tides of Time (random corrupted tiles)
World Cup USA 94 (corrupted tiles)

i do not know exactly why, on real hardware, when doing DMA from Word-RAM, setting the source address one word above the wanted address solves the VDP "slow" read-out problem but instead of simply decrementing the source address in VDP register, I handle it by dismissing the first read: VDP address is incremented and DMA length decremented but no word is written to VDP and source address is not incremented. It's a little bit different from what Gens/Picodrive are doing (i think).

Anyway, those games do not use an auto-increment of 2 like most other games and I was not taking care of this properly, they were fixed by properly using the auto-increment register when dismissing this first read

A-Rank Thunder Tanjouhen (hangs during intro cinematic )

expects the TOC to be correct and especially, play some audio track and expects the track index to increment itself up to some point before continuing so if track lengths are not correct, you can get stuck.

Cosmic Fantasy Stories

was fixed by improving SUB-CPU polling detection for some undetected cases where the same register was indeed read twice in a loop at two different PC locations


In the end, this leaves out only two remaining games with graphical bugs:

Adventures of Willy Beamish, The (gfx error in the first logo)
AH3 Thunderstrike / Thunderhawk (gfx errors in radar window, low-half is corrupt)

and about that again

Is Radical Rex working for you, Eke? Never got that one running (neither did Gens at that time), I did some research into it but don't really remember anything (it has been some years)..

I somehow broke it when experimenting with CDD latency: it appears that it requires at least one interrupt delay before effectively reading data after a PLAY command is sent. It seems like you should always apply at least one interrupt delay when a PLAY command is sent, no matter what is LBA offset or drive status since the drive was already in PLAY "status" when this PLAY command is sent.

[Eke]

Adventures of Willy Beamish, The (gfx error in the first logo)

I figured this one too: what happened is that this game does not always reset VDP DMA source registers $21 and $22 before triggering a DMA and I always assumed those registers got automatically incremented during DMA, while they should keep their initial value.

[Stef]

Adventures of Willy Beamish, The (gfx error in the first logo)

I figured this one too: what happened is that this game does not always reset VDP DMA source registers $21 and $22 before triggering a DMA and I always assumed those registers got automatically incremented during DMA, while they should keep their initial value.

That would fix also some genesis games
Look like you figured almost all segacd games, and very quickly !

[Eke]

Surprisingly, it didn't seemed to affect any mega drive games before as there weren't any graphical bug left on that side as far as i tested (although, even if i tested the whole library to make sure emulator was 100% compatible, i could not test every games from start to end).

Yes, i'm pretty satisfied with CD compatibilty so far, although again, i'm convinced bugs will arise when people start testing games more deeply
Most of the bugs or sync issues seem to be catched on startup however.

The only remaining bug I'm facing is in AH3 Thunderstrike, there are graphic errors in the bottom right radar panel and water, which i still have to figure out.
No clue so far, does not seem to be related to GFX operation timings...

[notaz]

That's some useful notes, thanks Eke.

So you managed to avoid lockstepping for all those games? If so that's pretty cool, as some of those supposedly locktep-needing games were not fast enough to be pleasantly playable on GP2X/PSP on PicoDrive, because lockstep is expensive.

[Eke]

Yes, it seems so, even if I could obviously not test all games throughfully and only checked they started fine but i got to through the hassle of testing every games listed for mega/sega cd (not couting region variations if any) to reduce the chances of leaving a bug for later.

Basically, both CPU got resynced everytime communication registers (including int2 and memory mode) are written to, plus also when a few ones are read by sub-cpu (was required for power rangers at least). I take care of one cpu being beyond the other (since they are still executed successively for one line) by being able to stop either cpu when register polling loop is detected and resync cycle count when cpu is restarted by the other cpu writing the polled register. It was not easy to achieve but testing all these sensitive games and finding why exactly they were hanging helped me adjusting my original concept idea and perfecting it, i think.

About Thunderstrike, i have analyzed the game engine on both cpu and i'm starting to think it needs fine tuned timings for gfx operation of variable width/height. Indeed, you basically have two concurrent process:

1) one being hint handler near the end of screen that handle full screen refresh (at 30 fps or 20 fps, still not sure) by doing DMA from Main-RAM to VRAM

2) another one where MAIN-CPU is waiting for graphics being rendered to Word-RAM to copy them to Main RAM (among other things ?) and SUB-CPU is triggering multiple GFX operations to render the main field, then the trees, then the map panel and finally a 2x1 pixel operation which i assume is the radar moving cursor (?)

The problem is that there is no sync between the two processes, only some kind of buffering which seems to be every 3 frames (plane A vscroll is changed every 3 frames to apparently point to one or another tile map) but there does not seem to be any control (still need to verify) when DMA is processed to check if data was in the middle of being copied from word-ram...[/img]

[Eke]

I figured the problem with Thunderstrike was that I was rendering the whole image buffer at once when the trace vector register was written (for optimization purpose) BUT this game actually updates Word-RAM DURING the graphics operation and expect the timing to be right and data to be ready for when specific part are being rendered.

So I needed to make graphic rendering timings more accurate and updated on each CPU executed "line" instead of all at once, and it indeed fixed the "map".

However, there is still that weird issue in the "Panama Canal" level with the riverside (cf previous pic) which seems corrupted or more precisely mirrored so I guess it has something to do with ROTATION bits. I wonder if the documentation on which my code is based is correct ? I am currently handling these bits like this, in the specified order:

Code: Select all

if (attr & 4) {x = x ^ sizemask;}
if (attr & 2) {x = x ^ sizemask; y = y ^ sizemask;}
if (attr & 1) {x = y ^ sizemask; y = x;}

EDIT: well, i kinda figured my stupid mistake while typing it here. To be correct, it should actually be coded like this:

Code: Select all

if (attr & 4) {x = x ^ sizemask;}
if (attr & 2) {x = x ^ sizemask; y = y ^ sizemask;}
if (attr & 1) {temp = x; x = y ^ sizemask; y = temp;}

...and this fixes the last issue i was having

[Eke]

Adventures of Willy Beamish, The (gfx error in the first logo)

I figured this one too: what happened is that this game does not always reset VDP DMA source registers $21 and $22 before triggering a DMA and I always assumed those registers got automatically incremented during DMA, while they should keep their initial value.

That would fix also some genesis games
Look like you figured almost all segacd games, and very quickly !

Well... about that. WHile testing all sensitive games once again with my final implementation, it appears that Mary Shelley's Frankenstein and Johnny Mnemonic won't work anymore if I don't update the DMA source registers during DMA.
Now, how odd is this ? Maybe it depends on DMA destination (VRAM, CRAM, etc ?). If I only update DMA source registers when DMA is done to CRAM, it's fine... I guess this should be tested on real hardware just to be sure.

For reference, the code does this:

Code: Select all

[4(4)][20282(3182)] HVC read -> 0x479 (ffff0a2c)
[4(4)][20394(3294)] VDP register 19 write -> 0x10 (ffff0a32)
[4(4)][20478(3378)] VDP register 20 write -> 0x0 (ffff0a36)
s68k run from 15963 to 19104 (PC=81F8)
m68k run from 20562 to 23940 (PC=FFFF0A36)
[5(5)][20562(42)] VDP register 1 write -> 0x34 (ffff0a38)
[5(6)][20674(154)] DMA type 0 (167 access/line)(3266 cycles left)-> 159 access (16 remaining) (ffff0a3c)
[5(5)][21001(481)] CRAM 0x40 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x42 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x44 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x46 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x48 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x4a write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x4c write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x4e write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x50 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x52 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x54 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x56 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x58 write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x5a write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x5c write -> 0x0 (ffff0a3c)
[5(5)][21001(481)] CRAM 0x5e write -> 0x0 (ffff0a3c)
[5(5)][21057(537)] VDP register 1 write -> 0x74 (ffff0a3e)
[5(5)][21225(705)] HVC read -> 0x5fd (ffff0a48)
[5(5)][21351(831)] HVC read -> 0x504 (ffff0a48)
[5(5)][21589(1069)] HVC read -> 0x50f (ffff0a2c)
[5(5)][21715(1195)] HVC read -> 0x516 (ffff0a2c)
[5(5)][21841(1321)] HVC read -> 0x51c (ffff0a2c)
[5(5)][21967(1447)] HVC read -> 0x522 (ffff0a2c)
[5(5)][22093(1573)] HVC read -> 0x529 (ffff0a2c)
[5(5)][22219(1699)] HVC read -> 0x52f (ffff0a2c)
[5(5)][22345(1825)] HVC read -> 0x535 (ffff0a2c)
[5(5)][22471(1951)] HVC read -> 0x53c (ffff0a2c)
[5(5)][22597(2077)] HVC read -> 0x542 (ffff0a2c)
[5(5)][22723(2203)] HVC read -> 0x548 (ffff0a2c)
[5(5)][22849(2329)] HVC read -> 0x54e (ffff0a2c)
[5(5)][22975(2455)] HVC read -> 0x555 (ffff0a2c)
[5(5)][23101(2581)] HVC read -> 0x55b (ffff0a2c)
[5(5)][23227(2707)] HVC read -> 0x561 (ffff0a2c)
[5(5)][23353(2833)] HVC read -> 0x568 (ffff0a2c)
[5(5)][23479(2959)] HVC read -> 0x56e (ffff0a2c)
[5(5)][23605(3085)] HVC read -> 0x574 (ffff0a2c)
[5(5)][23731(3211)] HVC read -> 0x57b (ffff0a2c)
[5(5)][23843(3323)] VDP register 19 write -> 0x10 (ffff0a32)
[5(5)][23927(3407)] VDP register 20 write -> 0x0 (ffff0a36)
s68k run from 19139 to 22288 (PC=81F4)
m68k run from 24011 to 27360 (PC=FFFF0A36)
[6(6)][24011(71)] VDP register 1 write -> 0x34 (ffff0a38)
[6(7)][24123(183)] DMA type 0 (167 access/line)(3237 cycles left)-> 158 access (16 remaining) (ffff0a3c)
[6(6)][24450(510)] CRAM 0x60 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x62 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x64 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x66 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x68 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x6a write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x6c write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x6e write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x70 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x72 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x74 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x76 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x78 write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x7a write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x7c write -> 0x0 (ffff0a3c)
[6(6)][24450(510)] CRAM 0x7e write -> 0x0 (ffff0a3c)
[6(6)][24506(566)] VDP register 1 write -> 0x74 (ffff0a3e)

and expects the DMA source address to be incremented

while Willy Beamish does this when displaying first logo:

Code: Select all

[227(227)][779761(1)] VRAM 0x2800 write -> 0x0 (ff0468)
[227(227)][780153(393)] VDP register 20 write -> 0x14 (ff046e)
[227(227)][780153(393)] VDP register 19 write -> 0x0 (ff046e)
[227(227)][780237(477)] VDP register 23 write -> 0x10 (ff0470)
[227(227)][780237(477)] VDP register 22 write -> 0x3c (ff0470)
[227(228)][780321(561)] DMA type 1 (102 access/line)(115719 cycles left)-> 3451 access (5119 remaining) (ff0472)
[227(261)][896040(0)] VRAM 0x7802 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7804 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7806 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7808 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x780a write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x780c write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x780e write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7810 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7812 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7814 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7816 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7818 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x781a write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x781c write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x781e write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7820 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7822 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7824 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7826 write -> 0x0 (ff0472)
[227(261)][896040(0)] VRAM 0x7828 write -> 0x0 (ff0472)

and expects DMA source address to be reloaded using reg 22-23 new values and reg 21 previous value.

The most logical to me would be that there is an internal source address register which increments itself during DMA and is reloaded when any of regs 21-22-23 is updated, NOT when DMA is started by writing to CTRL port. Also, the fact that DMA is limited to 128K region seems to indicate that this source address register is only 16-bit (enough for VRAM copy) while the remaining address bits are directly made of reg23 value during DMA from 68k bus, so it's very likely source address is not even reloaded when only reg23 is updated...

[Stef]

I've to admit that i'm not sure about what i'm doing in Gens, i think i'm updating DMA source register when you write register 21/22/23... i did it only for convenient reasons and never realized it had to be done that way :p

[Nemesis]

I have a LOT of unpublished information currently on the correct operation of the VDP in a massive number of corner-cases, so I can provide some answers here.

None of the DMA register settings are "cached", they are used live. In particular, the DMA source address and transfer count are actively modified during DMA operations. You can, for example, perform a DMA transfer for a count of 0x1000, then only rewrite the lower transfer count byte with 0x80, and trigger another DMA transfer, and it will only perform a transfer for 0x80 steps.

I can tell you that the DMA source address registers are never cleared under any circumstances, but a DMA operation will actively modify them as it runs, so you would expect the source address registers to be incremented by a DMA operation. I can tell you that the correct behaviour is for the combined DMA source address registers 21 and 22 to be incremented by 1 on each DMA update step, with every DMA operation, including a DMA fill for what it's worth, even though it doesn't use the source address. DMA source address register 23 is never modified under any circumstances by the DMA operation. This is why a DMA transfer wraps at 0x20000 byte boundaries: it's unable to modify the upper DMA source address register. If it was able to do this, it could inadvertantly modify the DMA mode during a DMA operation, which would be very bad.

Here are some other mitigating factors which may cause you problems:
-DMA operations to invalid write targets still run to completion, the result is simply not stored, so if a DMA operation is triggered to an invalid target, the DMA source address still needs to be updated.
-DMA operations always run to completion, they never abort, IE, when you reach the "end" of CRAM or VSRAM. Writes to CRAM and VSRAM wrap at an 0x80 byte boundary. Writes to the upper portion of VSRAM in this region (0x50-0x80) are discarded.

Any information you may read which is contrary to this info (IE, in genvdp.txt) is incorrect.

[Eke]

None of the DMA register settings are "cached", they are used live. In particular, the DMA source address and transfer count are actively modified during DMA operations. You can, for example, perform a DMA transfer for a count of 0x1000, then only rewrite the lower transfer count byte with 0x80, and trigger another DMA transfer, and it will only perform a transfer for 0x80 steps.

yes, I verified that already for some games, and I already do this (at least for DMA length now.. see below)

I can tell you that the DMA source address registers are never cleared under any circumstances, but a DMA operation will actively modify them as it runs, so you would expect the source address registers to be incremented by a DMA operation. I can tell you that the correct behaviour is for the combined DMA source address registers 21 and 22 to be incremented by 1 on each DMA update step, with every DMA operation, including a DMA fill for what it's worth, even though it doesn't use the source address

That's what I was doing before but it appeared it broke Willy Beamish Logo: see how it only rewrites reg 22 (and reg 23) but expects LSB to be reloaded with reg21 as well. If I leave reg21 to the final value it reached after previous DMA, I got those graphic errors. Hence why I suggested that source register was 16-bit and was FULL reloaded when either reg22 OR reg21 was modified. I'm planning to do complementary testing around that one.

. DMA source address register 23 is never modified under any circumstances by the DMA operation. This is why a DMA transfer wraps at 0x20000 byte boundaries: it's unable to modify the upper DMA source address register. If it was able to do this, it could inadvertantly modify the DMA mode during a DMA operation, which would be very bad.

yes, that's what I figured out too

-DMA operations to invalid write targets still run to completion, the result is simply not stored, so if a DMA operation is triggered to an invalid target, the DMA source address still needs to be updated.

yes, I already do that, I use a common "data_bus_w" routine for DMA and normal CPU access

-DMA operations always run to completion, they never abort, IE, when you reach the "end" of CRAM or VSRAM. Writes to CRAM and VSRAM wrap at an 0x80 byte boundary. Writes to the upper portion of VSRAM in this region (0x50-0x80) are discarded.
Any information you may read which is contrary to this info (IE, in genvdp.txt) is incorrect.

I already do that too, some games (Todd's Adenture World & ANdre Agassi tennis) are told to have graphics error if DMA is not stopped" after 0x80 bytes have been written but I never noticed any error with my current implementation, so I guess it's due to some incorrect handling of DMA registers in other emulators. This does not seem very logical either on an hardware point of view to have DMA be stopped like this.

[TascoDLX]

That's what I was doing before but it appeared it broke Willy Beamish Logo: see how it only rewrites reg 22 (and reg 23) but expects LSB to be reloaded with reg21 as well. If I leave reg21 to the final value it reached after previous DMA, I got those graphic errors.

I see what it's doing. But if the size of the previous transfer is 0x2800 bytes, shouldn't the low byte of the source address be the same in the end? You must be clearing the low bit or messing up the increment because of the wordram transfer issue.

[djcouchycouch]

Any information you may read which is contrary to this info (IE, in genvdp.txt) is incorrect.

Any chance there will be a more correct doc released sometime in the future?

[Charles MacDonald]

Any information you may read which is contrary to this info (IE, in genvdp.txt) is incorrect.

EDIT:

I'll try to withhold my grumpy rantings in the future.

[Nemesis]

Any information you may read which is contrary to this info (IE, in genvdp.txt) is incorrect.

That document is twelve years old, and I'm still waiting for somebody to do a better job. Just publish your damn notes already instead of pointing out how flawed mine are all the time.

Sorry, I really didn't mean it as a criticism, you've done an amazing job of documenting the VDP, and I've found your notes invaluble, as I'm sure a lot of other people have. Your notes are very accurate, and I still don't know how you managed to do such accurate analysis back in the day. I'm not meaning to replace or invalidate what you've done, just build on it.

I haven't released my notes yet because they're not yet complete. I'm still doing more tests which change my understanding of how the chip works as I go, in particular in areas such as DMA, where there's a lot of undefined behaviour, and some very unusual interaction is possible. I'm trying to go beyond describing behaviour under various conditions, and definitively prove points of the internal implementation, like I did with the YM2612, which takes a bit of time. As I come to understand a particular point of the implementation, that often leads to thinking of new tests too, which can give surprising results sometimes and reveal more points about the implementation, and so on.

The fact is, most emulators are accurate enough to play basically every game ever made without glitches, and your notes are accurate enough to understand how to write code that uses the VDP without issue. My testing is just about going to the extra-obsessive level of detail where you can understand what the VDP will do if you do things you really should never ever do.

Again, I'm sorry if I upset you, it wasn't my intention.