GEMS YouTube video

[MintyTheCat]

That's some good detective work there, R57Shell

I'd be happy to look inside the ROM if no one else volunteers.

[powerofrecall]

I tried to check some things about GEMS.EXE, and I found it works glitchy in DOSBox.

It may be DOSBox glitching but I know you will also get a result similar to this if certain files are missing (I think EGAVGA.BGI, maybe part of a GUI library).

The GEMS ROM is included with GEMS, at least the 2.8 version I am using has both an s-record GEMS.ABS and 2 ROMs interleaved, GEMS.C04 and GEMS.C05. I really should take a look at it myself also.

Here's what it should look like anyway:

[powerofrecall]

A quick peek that the ROM is already showing it's not quite a "stock" GEMS. It has all the regular GEMS functions that I can cross reference to the 2.8 source, but other functions for what looks like parallel communication on the EXT port (as we'd expect), and references to variables/addresses in the Z80 RAM space that the stock GEMS doesn't have. I wouldn't be surprised if the Z80 driver is different as well.

edit: I was wrong, the Z80 drivers are the same, and the GEMS stuff isn't as different as I had previously assumed--but there are still some differences

Fortunately, we've got the z80 source too, so this shouldn't be too hard to figure out. I have a feeling the communications between GEMS and the PC aren't very complex.

[r57shell]

It may be DOSBox glitching but I know you will also get a result similar to this if certain files are missing (I think EGAVGA.BGI, maybe part of a GUI library).

Yep, EGAVGA.BGI was missing. Now everthing looks same as on your screenshot.

Ok, I took a look into ROM, and what can I say? After throwing off stuff like VDP load palette, clear CRAM, VSRAM, VRAM and many common stuff, it was easy to find where packet receiving was located. More clear: ROM:000011EC byte_read. It's not easy for me now to understand how exactly it works though. You need to be at least familiar with reading joypads.

Ok, after that, it's easy to understand that it's reading standard GEMS FIFO commands, except some additional extension to be able to update patches, reload Z80 driver, and other stuff.

So, main question now: how to get it from DOSBox or other dos emulator?
I'll try redirection into sockets / pipe tomorow. By the way, I hope GEMS does not using writing into ROM region (such as SDRAM and so on).

[powerofrecall]

I don't know how to do it with DOSBox or if it can be done. Normal DOSBox doesn't have any kind of MIDI in or pipe/socket type structure. I do know there is a patch for DOSBox MIDI input, but I didn't try building it. I believe the newest version of the patch is on sourceforge. It may be part of an existing unofficial SVN, but I do not know. I'm not sure what version of DOSBox the patch applies to. If you want to try this, I can try to find it but googling it should turn something up.

Dosemu is the only one I have used that works and it uses a pipe that is not very well documented. I am also fairly sure it is not available on Windows or Mac. It is available for nearly any Linux. If anyone is interested in how to set this up, I can type something up about it. Basically it involves loading the ALSA 'virmidi' device to use as a port, then making a symlink from the /dev MIDI device to $HOME/.dosemu/run/dosemu_midi-in. From there, you can set your sequencer to output on that port when you want to record, or do as I do and use JACK/qjackctl to "wire" it together. You may have to use the GEMS commandline options -a (port address) and -i (IRQ) to make GEMS match what Dosemu presents.

As far as the GEMS dev cartridge I know it has some RAM (64k) to hold samples, so it almost certainly writes into an area mapped to the cart, but judging from the documentation, the GEMS binary itself was sent out on EPROMs. So it shouldn't do anything strange like write over itself. We don't really know much about the hardware, do we?

Some other stuff I've noticed: while I was disassembling it I noticed there's a jump table of stuff nothing seems to cross-reference in IDA. Have you figured this out yet? Starting at 0x220 forward, branches and jumps. I wonder if this is a table for something external in the hardware. Also, the ROM I made from the ABS file is very small. Something like 18kb. It looks complete. Does this match the ROM you have?

Were there any external connections from the Genesis other than a parallel cable connected to the EXT port?

Also, doesn't TmEE own one of the dev cartridges if I remember right? I remember seeing a decent pic or scan of it at some point.

[Mask of Destiny]

At one point, the local video game museum here had the GEMS hardware on loan. If they still have it, I could probably get you some information about it.

[r57shell]

Normal DOSBox doesn't have any kind of MIDI in or pipe/socket type structure.

I don't asking about MIDI now, because it's worked on your side. I don't care about MIDI now.

So it shouldn't do anything strange like write over itself.

I hope so .

We don't really know much about the hardware, do we?

I don't care about hardware now. Of course it is one of ways to find out how it works all together, but looking in hardware much harder for me .

Some other stuff I've noticed: while I was disassembling it I noticed there's a jump table of stuff nothing seems to cross-reference in IDA. Have you figured this out yet?

It's not jump table... but you may call it so. In any point of code execution, A5 always $200, so jsr $4A(a5) is actually jsr $24A, which is where located "new data check".

Also, the ROM I made from the ABS file is very small. Something like 18kb.

Yes, it's almost full filled with 0xFF. And actual rom size very small.

Were there any external connections from the Genesis other than a parallel cable connected to the EXT port?

Check GEMS.DOC in requirements there is only: LPT cable, hardware loader.


Ok, Genesis side completely revealed.

All works in following way:

EXT byte format: ?a?n dddd
a - acknowledge
n - new data not available
d - 4 bits of data

Code: Select all

byte ext_read()
{
    while (EXT&0x10); // wait for data
    low = EXT;
    EXT = 0; // may be for next part, but there is no wait after it.
    high = EXT;
    EXT = 0x40; // acknowledge
    return (low & 0xF) | ((high&0xF)<<4);
}

byte read_byte()
{
 do { low = ext_read(); } while (!(low & 0x80));
 do { high = ext_read(); } while (!(high & 0x80));
 return ((high << 1) & 0xF0) | (low & 0xF);
}

I don't know how it works, and why in this way, but only read_byte used for receiving commands from PC.

[powerofrecall]

I go to bed and when I wake up, great news!

I don't care about hardware now. Of course it is one of ways to find out how it works all together, but looking in hardware much harder for me .

You're right, knowing the hardware isn't as important as knowing how it works.

In any point of code execution, A5 always $200, so jsr $4A(a5) is actually jsr $24A, which is where located "new data check".

I should have noticed this, its a quirk of the Microtec C compiler. I think it always uses this form of short addressing when set to 'small code' model. It's also why IDA couldn't figure it out. Sometimes I let IDA do most of the work and forget to use my brain.

Not related but one interesting thing I noticed about that type of addressing when I first saw it is the compiler would generate code that could access 32k with positive displacements but could also point to the bottom half of RAM with negative displacement. I didn't understand it at first glance but then realized it that going negative would wrap around into RAM. I thought that was clever.

Check GEMS.DOC in requirements there is only: LPT cable, hardware loader.


Ok, Genesis side completely revealed.

All works in following way:

EXT byte format: ?a?n dddd
a - acknowledge
n - new data not available
d - 4 bits of data

Code: Select all

byte ext_read()
{
    while (EXT&0x10); // wait for data
    low = EXT;
    EXT = 0; // may be for next part, but there is no wait after it.
    high = EXT;
    EXT = 0x40; // acknowledge
    return (low & 0xF) | ((high&0xF)<<4);
}

byte read_byte()
{
 do { low = ext_read(); } while (!(low & 0x80));
 do { high = ext_read(); } while (!(high & 0x80));
 return ((high << 1) & 0xF0) | (low & 0xF);
}

I don't know how it works, and why in this way, but only read_byte used for receiving commands from PC.

But this is what I'm excited about. Nice work! I am going to keep going through the ROM and try to learn about that extra RAM that is on the cartridge to see if it might be possible to hack support into an emulator.

[r57shell]

EXT = 0; // may be for next part, but there is no wait after it.

Actually, after thinking a little bit about it... may be it's acknowledge, and $40 is - ack clear.

DOS side revealed too.

Code: Select all

void write_byte(byte b)
{
    DATA = b;
    int c = CONTROL;
    CONTROL = c|1;
    while (STATUS & 0x40);

    c = CONTROL;
    CONTROL = c & (~1);
    while (!(STATUS & 0x40));
}

It is used for feeding same bytes as genesis reading at high level.
So, all I need to do now, is writе code for interaction between genesis emulator and dos emulator. I think, I'll choose easiest way: patch GEMS ROM to replace polling code. And change DOS Box at parallel port emulation side.

Now I'm going to bed too.

[powerofrecall]

I'm trying another approach: using a pipe from Dosemu to redirect parallel output to a file and then using a modified emulator (Dgen-SDL) to read that pipe when the EXT port is accessed. Modifying the emulator should be the easy part; I'm having more trouble figuring out how to get Dosemu to properly redirect to a file. It is frustrating as it has been developed since the 90s but the documentation is really terrible.

I'd love to get this working and see it in action.

[powerofrecall]

I finally figured out how to get Dosemu to pipe out parallel data. From what I can tell GEMS.EXE doesn't really do any kind of checking on the status of the Genesis at all (makes sense, it's one-directional). It just dumps data out for the GEMS driver to load. Lots of different operations in GEMS.EXE result in data being put over the port--loading a .MOD, loading a sample, playing a sequence, the patch editor. It looks like it loads everything over to the Genesis first and then just sends commands. If you're curious about any of the raw parallel data I can post it here, but I don't know how helpful it'll be. Now I just have to figure out the best way to implement reading the pipe in the emulator. First time I've ever done anything like this.

[MintyTheCat]

Hi all,

I am not at all in any way experienced in GEMS or indeed making Music on the MD but a few points have sprung to mind:

1. Would it be possible to replicate GEMS and to use it on modern systems?
2. What are the shortcomings of GEMS for Musicians and could an improved version of GEMS be developed as a community?

It is very interesting for me to see you two chaps developing insight into how GEMS works and I hope that it will benefit us all in the long run.

Cheers,

Minty.

[r57shell]

1. Would it be possible to replicate GEMS and to use it on modern systems?
2. What are the shortcomings of GEMS for Musicians and could an improved version of GEMS be developed as a community?

It's obvious, if you read this topic. I don't like to type things to those who don't read it, so reread topic.

My current trouble, is: I can't build "DOSBox Daum" with VS2010 dosbox from sources now, I'm trying to solve this problem. It requires many of libraries, not all of them providing developer kits, so I have to build myself those libraries myself. I spent a lot of time to build SDL_Sound, but it looks like it's not compatible with "DOSBox Daum".

Still in progress.

[MintyTheCat]

1. Would it be possible to replicate GEMS and to use it on modern systems?
2. What are the shortcomings of GEMS for Musicians and could an improved version of GEMS be developed as a community?

It's obvious, if you read this topic. I don't like to type things to those who don't read it, so reread topic.

My current trouble, is: I can't build "DOSBox Daum" with VS2010 dosbox from sources now, I'm trying to solve this problem. It requires many of libraries, not all of them providing developer kits, so I have to build myself those libraries myself. I spent a lot of time to build SDL_Sound, but it looks like it's not compatible with "DOSBox Daum".

Still in progress.

You have missed my points entirely.

[r57shell]

You have missed my points entirely.

I think I don't. Make questions more clear.