UMDK manufacturing, part 2: Software

[mikejmoffitt]

I apologize that I won't be able to proceed further with getting my UMDK board to work until some time around Christmas, as my Sega system, UMDK, and lots of other things are currently being mailed home while I move.

I began working on a VM image that is all set up and good to go with Debian for SGDK / UMDK development. I already have the SGDK toolchain set up and ready to go, so once I get the UMDK working with everyone's help then I would like to add it to the VM as an option for people who may have trouble setting it up. That way a user can choose to install VirtualBox and attach my VM image, and proceed from there to get started with MD development.

[prophet36]

I apologize that I won't be able to proceed further with getting my UMDK board to work until some time around Christmas, as my Sega system, UMDK, and lots of other things are currently being mailed home while I move.

No need to apologise, you can proceed at your own pace.

I began working on a VM image that is all set up and good to go

VM images are explicitly not supported:

STEP 0: Preparation
Unless you know what you're doing, you should test your new UMDK cart using a Debian-based Linux distribution, on "real" (as opposed to virtual, e.g on VMware or VirtualBox) x86 or x64 hardware.

That said, you're very welcome to try it out, and iron out any bugs.

On a related note, I've been thinking about your binary-distribution idea. It turns out the main difficulty was DDD, and modern Linux distros have all my patches applied, so there's no need for me to supply a DDD binary. The rest of the binary tools may well work on non-Debian distros (Arch, Fedora, etc). So we can try that out. The other option that's worth someone investigating is the new GDB plugin to Visual Studio, or the Eclipse GDB plugin, for a native Windows UMDK environment. I'll leave that for Windows-lovers to try out though; I'm kinda allergic.

[mikejmoffitt]

VM images are explicitly not supported:

It's not something I'd want to use since I have an actual machine available, but if I can get it working it's an option I want to provide to others. It makes sense to say that it's not officially supported, though, as it can introduce extra less predictable behavior as far as IO is concerned.

On a related note, I've been thinking about your binary-distribution idea. It turns out the main difficulty was DDD, and modern Linux distros have all my patches applied, so there's no need for me to supply a DDD binary. The rest of the binary tools may well work on non-Debian distros (Arch, Fedora, etc). So we can try that out. The other option that's worth someone investigating is the new GDB plugin to Visual Studio, or the Eclipse GDB plugin, for a native Windows UMDK environment. I'll leave that for Windows-lovers to try out though; I'm kinda allergic.

Because I haven't set it up yet, it's less clear to me, but using regular gdb from the terminal is a supported operation as well, right?

[prophet36]

it's an option I want to provide to others

That's great, as long as you make it very clear to those others that if/when they run into weird problems, I will be absolutely unable to help them. I'm not trying to be awkward, but I do want to avoid getting blamed when people encounter problems whilst running with explicitly unsupported configurations. Also, I'm not making VMs unsupported on a whim - there are some pretty nontrivial problems with the interaction between libusb and the FPGALink FX2 firmware in some (but not all!) VM configurations, which I've been unable to fix.

using regular gdb from the terminal is a supported operation as well, right?

Of course. In fact, the "-tui" option to GDB provides a pretty good curses-based interface.

[Montserrat]

I began working on a VM image that is all set up and good to go with Debian for SGDK / UMDK development. I already have the SGDK toolchain set up and ready to go, so once I get the UMDK working with everyone's help then I would like to add it to the VM as an option for people who may have trouble setting it up. That way a user can choose to install VirtualBox and attach my VM image, and proceed from there to get started with MD development.

I configured a VirtualBox debian 8 to use UMDK, loading games and such is ok, but DDD give a ton of problems, better to stay out out vm.

Also, about the new testers gonna ask for 5 more to make sure its working before giving manufacturer details to public.

[mikejmoffitt]

Of course. In fact, the "-tui" option to GDB provides a pretty good curses-based interface.

I had no idea about the curses interface. This is great! Thanks for the tip.

[prophet36]

OK here's a new software build, incorporating all the necessary fixes we discovered so far, and a few more tweaks added for good measure. It's also a binary distribution, so no need to compile everything from source. I built it on a Debian 7.9 machine, and tested it on Debian 8.1 and Fedora 23, and it seems to work fine.

Code: Select all

export ARCH=x64  # set this to x86 if your OS is 32-bit
export RELEASE=20151220
cd ${HOME}
mkdir umdk-old
mv umdkv2-bin umdk-old/
mv x-tools umdk-old/
wget -q https://dl.dropboxusercontent.com/u/80983693/umdkv2/umdk-l${ARCH}-${RELEASE}.tar.gz
tar zxf umdk-l${ARCH}-${RELEASE}.tar.gz
wget -q https://dl.dropboxusercontent.com/u/80983693/umdkv2/xtools-l${ARCH}-${RELEASE}.tar.gz
tar zxf xtools-l${ARCH}-${RELEASE}.tar.gz

You can now install SGDK and build a debug version of the menu program to play with:

Code: Select all

mkdir ${RELEASE}
cd ${RELEASE}
wget -qO- http://tiny.cc/msbil | tar zxf -
cd makestuff
scripts/msget.sh makestuff/hdlmake/${RELEASE}
cd hdlmake/apps
mkdir makestuff
cd makestuff
../../../scripts/msget.sh makestuff/umdkv2
cd umdkv2/scripts
./build-sgdk.sh 
cd ../m68k/menu
./dbg.sh

Note that these "RELEASE" and "ARCH" variables are only used for this install operation; there's no need to permanently add them to your .bash_aliases or whatever.

I felt the need to change a couple of file-names, to make things more consistent. To upgrade your FPGA & MD firmware, do:

Code: Select all

flcli -v 1d50:602b -p J:A7A0A3A1:${HOME}/umdkv2-bin/spi-talk.xsvf
gordon -v 1d50:602b -t indirect:1 -w ${HOME}/umdkv2-bin/fpga-prod.bin:0
gordon -v 1d50:602b -t indirect:1 -w ${HOME}/umdkv2-bin/firmware.bin:0x60000

I also found a potential reason for why others were getting different results from the sig-test thing we tried (I doubled the depth of the trace FIFO in the FPGA, and increased the size of each chunk of trace data read over USB to 64KiB). It would be good if everyone could try that again. There's no need to hit ctrl-c this time; it'll stop after grabbing 2048 chunks of trace data, which is more than enough.

Code: Select all

cd ${HOME}
mkdir sigtest
cd sigtest
wget -q https://dl.dropboxusercontent.com/u/80983693/umdkv2/sigtest.bin.bz2
bunzip2 sigtest.bin.bz2
flcli -v 1d50:602b -p J:A7A0A3A1:${HOME}/umdkv2-bin/fpga-test.xsvf
loader -w sigtest.bin:0 -x 2 -t trace.bin:2048
logread trace.bin | grep -v HEARTBEAT | head -8388250 | awk '{print $2$3$4$5}' | md5sum

As before, the resulting MD5 should be d5a670db486de8597e214c6849fdfacb.

Probably the most noticeable difference is I no longer build DDD myself, but rely on you having DDD installed, which you can do with something like sudo apt-get install ddd or sudo yum install ddd.

Finally, as promised in the separate thread, I removed gdb-bridge's "-d" option, and replaced it with a couple of GDB monitor commands. So if you want a WRAM dump, enter this in the DDD command-window:

Code: Select all

mon rd dump.bin

And if you want to get a trace of the next continue-to-breakpoint or similar command, do this:

Code: Select all

mon tr trace.log

As always, any problems, let me know. I'll be moving the "getting started" posts at the beginning of this thread to the UMDK wiki in GitHub in the next few days, with suitable edits to reflect all we've learned recently.

[prophet36]

For the adventurous amongst you, I also built a set of armhf binaries (UMDK tools, and GDB & 68000 cross-compiler) suitable for use on a Raspberry Pi. With DDD it's not the most responsive of development environments, but it's usable. Dumping large trace-logs is a problem, probably because of the RPi's limited SD-card bandwidth. Perhaps a RPi2 with a fast SD-card would be able to keep up.

umdk-larmhf-20151220.tar.gz
xtools-larmhf-20151220.tar.gz

[Montserrat]

OK here's a new software build...

Ok i've loaded the new firms and such, the md5 checksum is OK

d5a670db486de8597e214c6849fdfacb

But now, loading roms from SD, or launching DDD justs resets the megadrive. Loading a ROM via terminal still works btw.

[prophet36]

the md5 checksum is OK

Great!

But now, loading roms from SD, or launching DDD justs resets the megadrive

Doh. I made a stupid mistake. Fixed now. Please try loading the updated firmware.bin from here. That should fix loading from SD-card. The problem with DDD I'm not so sure about, but I do remember something weird happening when I switched from my own from-source build of DDD to the Debian-supplied build; I had to do rm -rf ~/.ddd to fix it. If that doesn't work, try running which ddd and dpkg-query -l ddd, and pasting the output of both commands here.

[Montserrat]

New firmware solves the SD-card loader thing, about DDD:

If that doesn't work, try running which ddd and dpkg-query -l ddd, and pasting the output of both commands here.

Well this is what GDB-bridge returns

Code: Select all

Waiting for GDB connection on :8000...
Got GDB connection from 127.0.0.1:56288
umdkDirectReadBytes(): Illegal direct-read from 0xFFFFFFF0-0xFFFFFFF1 range!
umdkDirectWriteBytes(): Illegal direct-write to 0xFFFFFFF0-0xFFFFFFF1 range!

which ddd:

Code: Select all

/usr/bin/ddd

And dpkg-query -l ddd:

Code: Select all

Deseado=desconocido(U)/Instalar/eliminaR/Purgar/retener(H)
| Estado=No/Inst/ficheros-Conf/desempaqUetado/medio-conF/medio-inst(H)/espera-disparo(W)/pendienTe-disparo
|/ Err?=(ninguno)/requiere-Reinst (Estado,Err: mayúsc.=malo)
||/ Nombre                 Versión          Arquitectura     Descripción
+++-======================-================-================-==================================================
ii  ddd                    1:3.3.12-5       amd64            The Data Display Debugger, a graphical debugger fr

[prophet36]

Well this is what GDB-bridge returns

Code: Select all

Waiting for GDB connection on :8000...
Got GDB connection from 127.0.0.1:56288
umdkDirectReadBytes(): Illegal direct-read from 0xFFFFFFF0-0xFFFFFFF1 range!
umdkDirectWriteBytes(): Illegal direct-write to 0xFFFFFFF0-0xFFFFFFF1 range!

Oh, perhaps DDD was still running when you did rm -rf ~/.ddd? Try making sure DDD is definitely not running, and then try rm -rf ~/.ddd again.

[Montserrat]

Well this is what GDB-bridge returns

Code: Select all

Waiting for GDB connection on :8000...
Got GDB connection from 127.0.0.1:56288
umdkDirectReadBytes(): Illegal direct-read from 0xFFFFFFF0-0xFFFFFFF1 range!
umdkDirectWriteBytes(): Illegal direct-write to 0xFFFFFFF0-0xFFFFFFF1 range!

Oh, perhaps DDD was still running when you did rm -rf ~/.ddd? Try making sure DDD is definitely not running, and then try rm -rf ~/.ddd again.

i did always use pidof command, and kill -9 to make sure both GDB-bridge and ddd are terminated between test.

But ill try againg later just to be sure.

[Montserrat]

Ok, i've found the problem, it wasn't either of those.

I was running Sonic 3 & Knucles, that roms generates the GDB-bridge error upon launching DDD. May be its due to the fact that this rom is dumped from a lock-on cartrigde sonic 3 + sonic & knuckles, and the behaviour may be different...

I've tried with sonic and other games and it works normaly.

This made me think about some of the problems burbruee had. So i've tried Mega turrican and it doesnt even load (black screen), so there is some incompatible games.

And found some more problems on other games, ive found that soleil (crusader of centy in us) and shinning force had some problems with DDD. If you interrupt execution, it is halted but music continue playing, like the Z80 is ignoring the halt order and continue procesing code for the FM and the PSG.

Bonus: i was curious and tried the Bad apple demo (check here: viewtopic.php?t=1229), it also didnt worked.

[prophet36]

I was running Sonic 3 & Knucles, that roms generates the GDB-bridge error upon launching DDD

OK great, I can reproduce this error. I can't immediately see how to debug it so I logged it as a bug.

So i've tried Mega turrican and it doesnt even load (black screen)

Mega Turrican works fine on my MD2, and DDD interrupts it OK too. If you grab an execution trace of it failing to start I might be able to find out what's going wrong by comparing it with a log of mine starting correctly. Be sure to use a dump of an EU cart.

If you interrupt execution, it is halted but music continue playing, like the Z80 is ignoring the halt order and continue procesing code for the FM and the PSG

That's expected behaviour. There is actually no "halt" order; even the 68000 doesn't really halt when you run DDD. What happens is that gdb-bridge re-writes the illegal instruction vector to point at the UMDK monitor, then it inserts an illegal instruction opcode right at the beginning of the VBlank routine, so when the 68000 gets a VBlank interrupt, it ends up in the monitor code, where GDB can interact with it. But the 68000 carries on executing throughout. And everything else in the machine that is not coordinated by 68000 game code will also continue. Some games evidently use the Z80 and other sound hardware purely for sound generation, but rely on the 68000 to supply the timing and/or the sequence of actual musical notes to play, whilst others use the Z80 to play music completely autonomously. The music in these latter games will therefore continue playing when DDD starts.

It may be possible to implement Z80 breakpoints like I did with the 68000, but I have not tried (I never owned a Z80-based machine when I was a kid, so I'm not as familiar with it as I am the 68000).