GEMS YouTube video

Talk about anything else you want

Moderator: BigEvilCorporation

r57shell
Very interested
Posts: 478
Joined: Sun Dec 23, 2012 1:30 pm
Location: Russia
Contact:

Post by r57shell » Mon Oct 20, 2014 8:33 pm

powerofrecall wrote:So you're saying I'll need 0x7fff of RAM starting at 0x10000 for the cart RAM?
I was using 0x10000 - 0x1FFFF range, see how I'm changing ROM to define SRAM.

It was research of connection format, there is now troubles to send MIDI into DOSBox :D.

If you wanna try it in Windows then I can send builds to you. It's vast of time to build all things. I was doing many things for hours to be able to compile DOSBox from sources.
Last edited by r57shell on Mon Oct 20, 2014 8:43 pm, edited 1 time in total.
Image

powerofrecall
Very interested
Posts: 237
Joined: Fri Apr 17, 2009 7:35 pm
Location: USA

Post by powerofrecall » Mon Oct 20, 2014 8:35 pm

Chilly Willy wrote:A lot of us old-timers are more comfortable with the tried and true tracker style editors, but there's more out there. Perhaps you prefer a midi sequencer like MusE or RoseGarden. Maybe you prefer editing a score with standard musical notation, like Denemo. There's plenty of editors to check out, but I think most folks would prefer a converter, like the MIDI<>GEMS converters posted earlier.
This is why I'm so in love with Renoise. It's a tracker through and through, and can be used with plain samples and IT-style instruments, exactly like an old tracker, or used with softsynths, or external MIDI. It is also MIDIfied in some ways, like having polyphonic channels and MIDI CC. It's a shame it isn't free (though it is pretty cheap for what you get). To get the most out of GEMS though, the MIDI aspect is really a strength, not something that has to be worked around.

powerofrecall
Very interested
Posts: 237
Joined: Fri Apr 17, 2009 7:35 pm
Location: USA

Post by powerofrecall » Mon Oct 20, 2014 9:02 pm

r57shell wrote:It was research of connection format, there is now troubles to send MIDI into DOSBox :D.
This was the problem that got me using Dosemu in the first place... DOSBox was made for games, and games don't use MIDI input...
r57shell wrote:If you wanna try it in Windows then I can send builds to you. It's vast of time to build all things. I was doing many things for hours to be able to compile DOSBox from sources.
I believe you about DOSBox. It is kind of a mess of varying code quality and all the forked SVNs. The MIDI input patch isn't easy to apply either, I think it was created for an older version of the code. If you have binaries ready to go I could check it out, I don't have any kind of environment to build anything on Windows.

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Post by Chilly Willy » Mon Oct 20, 2014 9:07 pm

powerofrecall wrote:
Chilly Willy wrote:A lot of us old-timers are more comfortable with the tried and true tracker style editors, but there's more out there. Perhaps you prefer a midi sequencer like MusE or RoseGarden. Maybe you prefer editing a score with standard musical notation, like Denemo. There's plenty of editors to check out, but I think most folks would prefer a converter, like the MIDI<>GEMS converters posted earlier.
This is why I'm so in love with Renoise. It's a tracker through and through, and can be used with plain samples and IT-style instruments, exactly like an old tracker, or used with softsynths, or external MIDI. It is also MIDIfied in some ways, like having polyphonic channels and MIDI CC. It's a shame it isn't free (though it is pretty cheap for what you get). To get the most out of GEMS though, the MIDI aspect is really a strength, not something that has to be worked around.
Huh - that does look pretty good. I'm downloading their demo version to try it out. You never really know if you will like something until you actually try it. :D

Mask of Destiny
Very interested
Posts: 615
Joined: Thu Nov 30, 2006 6:30 am

Post by Mask of Destiny » Mon Oct 20, 2014 9:29 pm

I was at the MADE this weekend (the aforementioned video game museum) and so I took some closeup pictures of the GEMS dev hardware that should have sufficient resolution to figure out how the parallel port is conntected to the EXT port. Not sure if you still need that, but I figure it can't hurt to post them.

Top of transfer board
Bottom of transfer board
Top of RAM/ROM board
Bottom of RAM/ROM board
The transfer board is pretty simple so it shouldn't be hard to trace. I can do it myself, but I'm not sure when I'll have time to get to it. There was also a second EPROM cart board, but it didn't seem to be anything out of the ordinary so I did not take a picture of it.

Also, I was planning on adding some mechanism for piping data in and out of controller ports to BlastEm. I'd like to keep it somewhat generic so it can support things other than this particular board, but if you have any input on how you would like that to work I'd like to hear it. My general instinct would be to use a socket (both unix domain and TCP would be supported). Whenever, the emulated Genesis writes to the port, the byte written would be masked with the output bits and written as hex. The socket would be polled for new data both when the emulated Genesis reads from the port and at certain intervals. There should probably be a way to indicate serial data as well other than toggling the Tx/Rx bits.

Does that sound good to you?

powerofrecall
Very interested
Posts: 237
Joined: Fri Apr 17, 2009 7:35 pm
Location: USA

Post by powerofrecall » Mon Oct 20, 2014 11:05 pm

Those are really nice pictures, thanks

It's starting to become increasingly apparent to me that I wouldn't be the person to ask about what method of IPC should be used here. :)

For my own uses it's going to need to be something I can get out of Dosemu, at least for now. I'm already using it in a way I don't believe was really intended: in the configuration file (dosemu.conf) there is a section for three virtual LPT's. This section of the config file is filled in already for "lpt" and "lpt -l" for parallel port printing. The string in the config file is apparently appended to a stdout pipe and run in the shell; so whatever is entered in the config file is used as the target of the pipe. Normally piping to 'lpt'. So you can enter something like 'cat >whatever' and the raw parallel data will go into that file, or named pipe, or you can dump it straight to the terminal. The named pipe approach seems to work since Dosemu will start blocking if the FIFO isn't read. If I 'cat' the named pipe back out, I can see the data in the terminal and the FIFO starts filling up again. So to my untrained eyes, that appears to be 'working.'

I am (was?) trying to use this named pipe approach to read a byte out of the FIFO when data was requested from the EXT port. I figured this would work as it appears the MD-side GEMS software reads from 0xA10007 every so often, perpetually waiting on commands/data. I'm having a lot of trouble since I've never actually coded anything like this before and I'm not really sure of the best approach to take. Most information about this online is kind of over my head as I've never done any sort of IPC and my understanding of it is shaky.

r57shell's approach is clever and seems to work pretty well but unfortunately we're left without MIDI input, which makes the Genesis communication less useful.

If BlastEm had this kind of support, my needs would basically be to get a stdout pipe into it somehow. Could there be possibly a way to just direct stdin to it? That way I could put a blastem line directly into Dosemu's config file, something like a flag to enable stdin to a specific controller port, and select parallel or serial mode. Then again, I don't know if that is feasible with the structure of the emulator or if Dosemu internally keeps the "port" open (I think it does, but I'm not sure). Like I said before I understand the concept here but not the execution of it.

r57shell
Very interested
Posts: 478
Joined: Sun Dec 23, 2012 1:30 pm
Location: Russia
Contact:

Post by r57shell » Mon Oct 20, 2014 11:18 pm

I'm still wondering why it's so huge, if it's required only SRAM... Also, why there is additional LPT port?
Mask of Destiny wrote:Also, I was planning on adding some mechanism for piping data in and out of controller ports to BlastEm.
When I was thinking about my implementation of GEMS connection, I had few ideas. First, it would be best, if you could implement it as plugins (btw do you know how to make plugins in unix?), so anyone who wants his own protocol could implement some interface, and install ext-port as plugin. For example it can be abstract class with virtual methods: read, write.

Sockets / Pipes.

There is sync problem. You must understand that it is often situation when one side is working in loop of awaiting, and after condition is triggered, some modification happens right at that moment. I don't know how to describe... I just want to show why following setup is not good:

Code: Select all

PSEUDO-PSEUDO-CODE

// Client Side
Wire(byte b)
{
    send(sock, b); // using locking mode
    port = b; // update local port value
}

byte Read()
{
    return port; // always up to date
}

// running somewhere in loop
if (len = recv(sock,&v,1))  // no locking mode
    port = v; // updaring port

// Server Side
// COPY-PASTE ROFL
Wire(byte b)
{
    send(sock, b); // using locking mode
    port = b; // update local port value
}
................
Where is issue? This situation may and will happen:
1) Server write in port.
2) Client read from port.
3) Client recieve what Server wrote.
And client see some impossible situation so exception may happen.

Sockets / Pipes attempt two.

Sync must be strict. You can't achieve that by timestamps for example. You can't achieve that inside of connection, so you have to deal with it on some side. For example, let Server side do all sync work.

Code: Select all

PSEUDO-PSEUDO-CODE

// Client Side
Write(byte b)
{
    send(sock, "write", val); // using locking mode
}

Read(byte b)
{
    send(sock, "read"); // using locking mode
    recv(sock, &v, 1); // using locking mode
    return v;
}

// Server Side
Write(byte b)
{
    port = b;
}

Read(byte b)
{
   return port;
}

// running somewhere in loop
if (len = recv(sock,&type))  // no locking mode
{
    if (type == "read")
        send(sock, port); // locking mode
    else
    {
        recv(sock,&v); // locking mode
        port = v; // write port
    }
}
It will work, but if Client running some check in loop, he will spam packets.

Sockets / Pipes attempt three.

You can try to implement Bus hold and Bus release over sockets, it will make all sync. But when it comes to BUS, it requires Bus Arbiter, and again, it will be one of sides :) . So actually, it's same as previous one. For example, if arbiter is Server, then Client will send "server, stop operations now", and waits for his response "stoped", and after it, Client execute operation, after that, he is sending "server resume" without response. I don't know about performance, it may be slower.
Mask of Destiny wrote:Does that sound good to you?
I don't know. I don't get it what you said. :D
Image

TmEE co.(TM)
Very interested
Posts: 2440
Joined: Tue Dec 05, 2006 1:37 pm
Location: Estonia, Rapla City
Contact:

Post by TmEE co.(TM) » Tue Oct 21, 2014 4:40 am

I actually have a schematic of the transfer board somewhere on paper.
I had it in my posession for a while.
I recall it doing nothing more than taking a byte from LPT side and allowing MD to read it out in two steps, but that was some 4 or 5 years ago when I did the schematic and had that bit of hardware, I still lived at my parents place hahaha.
Mida sa loed ? Nagunii aru ei saa ;)
http://www.tmeeco.eu
Files of all broken links and images of mine are found here : http://www.tmeeco.eu/FileDen

Mask of Destiny
Very interested
Posts: 615
Joined: Thu Nov 30, 2006 6:30 am

Post by Mask of Destiny » Tue Oct 21, 2014 6:18 am

TmEE co.(TM) wrote:I actually have a schematic of the transfer board somewhere on paper.
I had it in my posession for a while.
I recall it doing nothing more than taking a byte from LPT side and allowing MD to read it out in two steps, but that was some 4 or 5 years ago when I did the schematic and had that bit of hardware, I still lived at my parents place hahaha.
I've been able to piece together a partial schematic from the pictures I took. It seems that the data is sent over the low nibble of the Genesis I/O port. TH is used to switch between the high and low nibble of the byte written from the parallel port. It looks like it's using the standard printer handshake signals, but it's hard to say for sure. If you have a complete schematic, it would be nice to see to confirm that.

This is mostly a good thing. Dosemu's parallel port emulation seems to assume that a printer (or something similar) is attached to the port and handles the handshaking internally. This means we can't send the output from TH back to Dosemu, but we don't really need to. We can just read the data in from the pipe and buffer it. It can then be fed into the emulator as the Genesis asks for it. The only downside is that GEMS may think that the transfer ended before it actually did.
r57shell wrote:I'm still wondering why it's so huge, if it's required only SRAM... Also, why there is additional LPT port?
I think that's a generic RAM based dev cartridge. It needs to have enough RAM to hold a reasonably sized game. The LPT port is probably for uploading binaries to the cartridge. GEMS can probably also run off of a mostly EPROM based cart which wouldn't have that connector.
r57shell wrote:When I was thinking about my implementation of GEMS connection, I had few ideas. First, it would be best, if you could implement it as plugins (btw do you know how to make plugins in unix?), so anyone who wants his own protocol could implement some interface, and install ext-port as plugin. For example it can be abstract class with virtual methods: read, write.
I'd prefer to avoid plugins for this. I/O port peripherals are pretty low speed and a socket based solution makes it easy to use whatever language you want.
r57shell wrote:There is sync problem. You must understand that it is often situation when one side is working in loop of awaiting, and after condition is triggered, some modification happens right at that moment.
Well for things using the I/O port in this fashion, the sync problem already exists when working with real hardware so the software running on the Genesis needs to deal with it anyway.

For things that interface with hardware that can respond nearly instantaneously to a pin change (like a game pad), the approach I outlined is potentially problematic. However, that kind of hardware is something I would probably just emulate in BlastEm directly.

Anyway, I'll try to add support for something like I outlined above in BlastEm soon.

r57shell
Very interested
Posts: 478
Joined: Sun Dec 23, 2012 1:30 pm
Location: Russia
Contact:

Post by r57shell » Tue Oct 21, 2014 8:43 am

Mask of Destiny wrote:I've been able to piece together a partial schematic from the pictures I took. It seems that the data is sent over the low nibble of the Genesis I/O port.
Why you still guessing? I already post code from both sides (MD, DOS) see above.
Mask of Destiny wrote:I'd prefer to avoid plugins for this.
Why? Because you are too lazy to implement plugins support?
Mask of Destiny wrote:I/O port peripherals are pretty low speed and a socket based solution makes it easy to use whatever language you want.
Orly? Slow?

Code: Select all

@loop:
move.b ($A10007),d0
andi.b #10,d0
beq.s  @loop
How many M68k cycles it? 16+12+10=38. 8MHz / 38 = ~210526 Hz. In other words, send(byte) 210526 times in second! Also, don't forget about Ethernet and TCP/IP headers, so it's like 210526*(packet_size) bytes in second. Let's say packet_size = 20 bytes, then it's 2MB in second, actually not so big but don't forget that it's many separate packets - not few packets of big size.
Mask of Destiny wrote:Well for things using the I/O port in this fashion, the sync problem already exists when working with real hardware so the software running on the Genesis needs to deal with it anyway.
Where is this sync issue? I don't see difference between EXT port and Joypad.

EDIT
powerofrecall wrote:I am (was?) trying to use this named pipe approach to read a byte out of the FIFO when data was requested from the EXT port. I figured this would work as it appears the MD-side GEMS software reads from 0xA10007 every so often, perpetually waiting on commands/data. I'm having a lot of trouble since I've never actually coded anything like this before and I'm not really sure of the best approach to take. Most information about this online is kind of over my head as I've never done any sort of IPC and my understanding of it is shaky.
Sorry, I missed your post. Ok.
You have lack of understanding. As I said, you must get same value as in pipe when ext_read reach rts. I think, you just not look at code. It's checking "is it byte available". If you able to replace value in register (d0) at some PC, you may just replace d0 at offset 2F26 for "is it data available" 0x00 = yes, 0x10 - no. and when rom check that it's availabe: he will read from EXT low and high 4 bits separately, mix them, and return at rts. So you may replace d0 to correct value there. Check my lua code for implementation of this method.

If you only able to set "what was read", and "what was writen", you have to go with another method:
1) initialize port value = 0x40
2) When something available: return it's 4 bits, and all high = 0. If 0x40 bit set, then return low bits of byte recieved from pipe. If 0x40 clear: return high 4 bits of received byte.
3) When 0x40 bit goes from 0 to 1 -> byte was read, so proceed to next byte.
4) when nothing available: return 0x10.
Image

Mask of Destiny
Very interested
Posts: 615
Joined: Thu Nov 30, 2006 6:30 am

Post by Mask of Destiny » Tue Oct 21, 2014 6:06 pm

r57shell wrote:
Mask of Destiny wrote:I've been able to piece together a partial schematic from the pictures I took. It seems that the data is sent over the low nibble of the Genesis I/O port.
Why you still guessing? I already post code from both sides (MD, DOS) see above.
I had forgotten about that; however, there was some uncertainty about certain details of the hardware from your pseudocode so I'm glad I went through the exercise anyway.
r57shell wrote:
Mask of Destiny wrote:I'd prefer to avoid plugins for this.
Why? Because you are too lazy to implement plugins support?
I don't like emulator plugins in general.
r57shell wrote:
Mask of Destiny wrote:I/O port peripherals are pretty low speed and a socket based solution makes it easy to use whatever language you want.
Orly? Slow?

Code: Select all

@loop:
move.b ($A10007),d0
andi.b #10,d0
beq.s  @loop
How many M68k cycles it? 16+12+10=38. 8MHz / 38 = ~210526 Hz. In other words, send(byte) 210526 times in second!
~210KB/s sounds pretty slow to me. It's not like I'm planning on sending this over the Internet or something. Also I think you misunderstand how I was proposing this would work. I'm not going to send an RPC request or something over the socket every time there's a read. Whenever the state of the port changes (i.e. the 68K writes to the port) I would do a write of the new value. Similarly the "device" on the other side would do the same.
r57shell wrote:Also, don't forget about Ethernet and TCP/IP headers, so it's like 210526*(packet_size) bytes in second.
Unix domain sockets don't have ethernet and TCP/IP headers. They're basically a bi-directional pipe you can use with the socket API. TCP sockets would have been an option mostly so that it would work in Windows when that port is done.
r57shell wrote:
Mask of Destiny wrote:Well for things using the I/O port in this fashion, the sync problem already exists when working with real hardware so the software running on the Genesis needs to deal with it anyway.
Where is this sync issue? I don't see difference between EXT port and Joypad.
If you're implementing a handshake in software the basic procedure is change the state of some pin and then poll the port until you see something that indicates that the device on the other hand has responded. The reason for this is that you can't guarantee how quickly the device on the other end will respond. You can see this in the pseudocode you posted. The 68K side software repeatedly reads the IO port until it sees bit 4 set. This means there's no tight deadline for the value to arrive.

In contrast, a gamepad just has a simple mux IC in it. You change the value of TH and the values sent on the other pins changes almost instantaneously. The 68K doesn't need to loop waiting for the value.

That said, I screwed up earlier when I suggested the scheme I proposed would work well for this. While there is some manual handshaking going on, there's also a hardware mux for selecting nibbles. That's why there's no delay in the code between when it writes 0 to the port and when it reads the high nibble. While my proposed scheme might work if the device program can respond quickly enough, it won't be reliable. I either need to emulate the mux in BlastEm or I need to wait for some acknowledgement from the device after I write the updated port value.

I'll probably do the first of those. I might even just add an option for emulating this particular device. Then you could just send the output of Dosemu directly to BlastEm without anything in between.

powerofrecall
Very interested
Posts: 237
Joined: Fri Apr 17, 2009 7:35 pm
Location: USA

Post by powerofrecall » Tue Oct 21, 2014 6:44 pm

I think I might have some success using netcat to send Dosemu's output to a TCP port, and then use a TCP socket on the emulator end (in this case, dgen-sdl). This or unix domain sockets is probably the best bet but I'm going to try TCP to see if I can also make it work over a local network.

HardWareMan
Very interested
Posts: 745
Joined: Sat Dec 15, 2007 7:49 am
Location: Kazakhstan, Pavlodar

Post by HardWareMan » Wed Oct 22, 2014 1:43 am

I've a WiFi Plus Click. And I have some WizNet based one. Maybe I should make a wireless RAM based MD devcart? :3

r57shell
Very interested
Posts: 478
Joined: Sun Dec 23, 2012 1:30 pm
Location: Russia
Contact:

Post by r57shell » Wed Oct 22, 2014 8:52 am

Mask of Destiny wrote:I don't like emulator plugins in general.
I don't like to rebuild stuff every now and then to change configuration. Also, if somethink does not support plugins, it has many different forks for additional features, like: dosbox, gens. And, you have to make another fork to addd some your own feature: it's pain in ass.
Mask of Destiny wrote:~210KB/s sounds pretty slow to me.
Yeah it's slow if socket works like pipe. But for packets count... I don't know, for me looks terrible.
Mask of Destiny wrote:Whenever the state of the port changes (i.e. the 68K writes to the port) I would do a write of the new value.
So you're talking about first approach in my list. Ok.
Mask of Destiny wrote:The 68K side software repeatedly reads the IO port until it sees bit 4 set. This means there's no tight deadline for the value to arrive.
Deadline is in other point. It's when EXT = 0; high = EXT; This is going without timeout. It's actually switching from low 4 bits to high 4 bits, so your approach will fail.
Mask of Destiny wrote:In contrast, a gamepad just has a simple mux IC in it. You change the value of TH and the values sent on the other pins changes almost instantaneously. The 68K doesn't need to loop waiting for the value.
Yes, you don't need to wait joypad. Here, you don't need to wait 0x10 bit, it's showing that byte is available on other side. So it works as joypad too. When you need to find out "is data available" it responsing instantaneously. When you need to swap from low bits to high bits: it respond instantaneously too.
Mask of Destiny wrote:I either need to emulate the mux in BlastEm or I need to wait for some acknowledgement from the device after I write the updated port value.
Why I was typing so much about sockets? :C What do you mean by "mux"?
Mask of Destiny wrote:I might even just add an option for emulating this particular device. Then you could just send the output of Dosemu directly to BlastEm without anything in between.
And then again, forks forks and recompile recompile. HATE.
HardWareMan wrote:I've a WiFi Plus Click. And I have some WizNet based one. Maybe I should make a wireless RAM based MD devcart? :3
Of course you can do such things. One suggestion from me: it would be nice to use other communication method instead of EXT port. It might be: Joypad, cartridge build-in port... (in ROM address space or may be somewhere else).

Also we can make GEMS ROM API as library, to inserting it right in the game, and test music on fly right in game.
Image

Mask of Destiny
Very interested
Posts: 615
Joined: Thu Nov 30, 2006 6:30 am

Post by Mask of Destiny » Fri Oct 24, 2014 4:09 am

r57shell wrote:
Mask of Destiny wrote:In contrast, a gamepad just has a simple mux IC in it. You change the value of TH and the values sent on the other pins changes almost instantaneously. The 68K doesn't need to loop waiting for the value.
Yes, you don't need to wait joypad. Here, you don't need to wait 0x10 bit, it's showing that byte is available on other side. So it works as joypad too. When you need to find out "is data available" it responsing instantaneously.
Not that this matters anymore, but in my original proposal there would be exactly 0 communication over the socket when the 68K does a read. When the "device process" had another byte ready, it would have written the new status of the pins to the socket. The emulator would save the last byte received as the current state and whenever the 68K read from the port it would get that value. For the 0x10 bit and the first nibble this works great without any explicit synchronization between the two processes. This would also work well for something like my transfer cable (also uses the parallel port, but doesn't have the extra hardware to make the Genesis look like a printer). It just won't work reliably for the nibble selection without some extra machinery around writes to ensure I've got the new value from the device before the next time the 68K does a read.
r57shell wrote:What do you mean by "mux"?
A multiplexer. In this case a 74HC157 quad 2-input multiplexer. It's the IC that lets the Genesis side switch between the two nibbles of the byte the PC side wrote by toggling the TH pin. Seems kind of pointless to be honest given that the byte ends up getting sent twice anyway (presumably so it can use the high bit for some sort of synchronization).
r57shell wrote:
Mask of Destiny wrote:I might even just add an option for emulating this particular device. Then you could just send the output of Dosemu directly to BlastEm without anything in between.
And then again, forks forks and recompile recompile. HATE.
You don't need to recompile BlastEm to change the config. That's what the config file is for. =P

Anyway, if you want to discuss the merits of plugin architectures in emulators we should probably take it to another thread.

Post Reply