SPC7110 Reverse Engineering Project

[kammedo]

A-1 is "address negative 1", because it's one bit lower than A0 which selects a WORD on a 16-bit ROM. When a 16-bit ROM can be put into 8-bit mode (/BYTE signal), D15 generally becomes an input which selects which byte of the word is output on D0-7. This is why in the document you mention that D0-7 and strangely D15 are connected, because D15 is really A0 and 16-bit A0 = 8-bit A1 etc.

Hm. Like the /BHE on the 8086. Only that there A0 does the job. Got it.
So that means that the MROM U2 is actually internally 16 Bytes. Thanks kyuusakku, one more mystery solved (for me at least ).

I have some USB 2.0 documentation, but didnt' had a chance to look at that closer now. Will do so the next days. Any suggestions from you?

Right now i don't have much free time, and the little i have is being devoted to another project that have higher priority in my list. In some weeks' place, however, i should have some time to try it but, as said, i'm not very optimist about it...

Well, I could shift the DD project a bit on lower priority..I got a blank new FEoEZ cart just yesterday and she eyes me all the time from the shelf ^^;

Finally something seems to start moving. Its about time!

[kyuusaku]

I have some USB 2.0 documentation, but didnt' had a chance to look at that closer now. Will do so the next days. Any suggestions from you?

What do you expect to do with that documentation? lol If you want to just interface USB to a one time project, use a FTDI module like everyone else.

BTW, USB 2.0 is way not necessary for SNES and USB modules are like US$30 after shipping... you could interface a ROM emulator to the EPP parallel port with less logic than the USB module's glue too.

Do you plan on building a ROM emulator for this project? I've designed an 8/16-bit ROM emulator (13x 74 series chips - $5.50), and I could turn it into a dual ROM emulator and add an EPP interface but it'd be a big chore so I'd only do it if you're serious. PS: you know I'm Calpis on Assembler right? :)

[kammedo]

Do you plan on building a ROM emulator for this project? I've designed an 8/16-bit ROM emulator (13x 74 series chips - $5.50), and I could turn it into a dual ROM emulator and add an EPP interface but it'd be a big chore so I'd only do it if you're serious.

Perhaps I explained myself wrong. I don't want to build an on-cart emulator, I want to directly supply the data in real time from the pc to the SPC7110.
I imagined it could work as follows :

SPC asserts address + read to USB interface
USB forwards address + read request to the PC
A listening client program on the PC supplies the required memory content
back
the USB interface forwards it to the SPC.

Tadà. Piece of cake. Only issue? Speed.

This way we would have a fast and reliable tool to use "on the fly".
And for that, im sure you'll agree, you need something damn fast - USB 2.0 seems just the best choice. You know what I mean : If I have the cannon, why can't i shoot the bird with it?
And since I don't care about spending something on that (knowing the results that may pop out), let's make something good and reliable rather than something that, well, may work but not as expected (like the CELL processor , j/k).

The device you listed is maybe too slow? Consider that the SPC7110 bus may not run at the same clock as the SNES bus.
Also, does the device support bidirectional tx or is it only one-way?

Perhaps parts like those would be more suitable

DLP-245PA, DLP-245PB-G, DLP-245SX and so on?

Only "problem" is that those are 40 pins instead of the 44 pins on the cart, but who cares? If you confirm they are suitable, I'm going to buy them.

Damn you tempted me to buy it straight away

PS: you know I'm Calpis on Assembler right?

No, really??
And yes, I'm damn serious about this whole thing, been like that for the last two years 8), but DD has always been my top priority so far cause I didnt know where to start. Of course I would be into building it and paying for it!

[neviksti]

No need to bug me. I have consistently expressed my interest on this for a long time, from the time when i was bugging The Dumper for him to construct a hardware interface to the last Kammedo-Charles MacDonald attempt to build such a thing.

Can someone give a historical rundown of what has been done so far?

I remember there was a whole discussion on CherryRoms, and at one point I understood what was in the files at least since some digging around showed I wrote some programs and began analyzing some things. I however do not remember any of those things.

All I have left is this apparrently:

focussing on just the first few input bytes -> output bytes
feoez_0.txt
feoez_all.txt

similar for the other game
mdh_all.txt

Some indication that the algorithm is adaptive, which considering the SDD1 had changing contexts to maximize compression is not unreasonable.
allign0.gif
allign2.gif

[kyuusaku]

Perhaps I explained myself wrong. I don't want to build an on-cart emulator, I want to directly supply the data in real time from the pc to the SPC7110.

When you said you wanted a USB-"U2" interface, I assumed that was the ROM.

Unless you manually clock the SPC, this way just doesn't make any sense, and clocking the SPC isn't practical since USB has huge latency! That's why all USB devices have FIFO, because it's not close to instantaneous.

I imagined it could work as follows :

SPC asserts address + read to USB interface
USB forwards address + read request to the PC
A listening client program on the PC supplies the required memory content
back
the USB interface forwards it to the SPC.

Tadà. Piece of cake. Only issue? Speed.

This way we would have a fast and reliable tool to use "on the fly".
And for that, im sure you'll agree, you need something damn fast - USB 2.0 seems just the best choice.

You don't understand, USB is fast only because it's packets have big payloads. If every payload was a couple of bytes representing control signals to the SPC interface, it'd be slow as hell to interface since the process relies on FEEDBACK. Not only that but unless the SPC has an acknowledge signal, you'll still have to manually give the FIFO fixed wait states since it's not real time. Honestly, because of this using the parallel port will be much faster than USB if you directly interface with the chip. It's not possible for the 8 MHz parallel port to beat the propagation time of the chip.

The device you listed is maybe too slow? Consider that the SPC7110 bus may not run at the same clock as the SNES bus.
Also, does the device support bidirectional tx or is it only one-way?

The ROM emulator works at over 1MiB/s, only limited by A) the operating system's LPT calls, B) the (pseudo) ISA bus (~4x 8 MHz cycles) C) the EPP device's wait state (adjusted for maximum RAM propagation).

It doesn't matter what speed the SPC operates at with a ROM emulator since SRAM is without a doubt faster than the SPC's mask ROM.

Of course the device is bidirectional, as I wrote initially I think the best plan of attack is to upload SPC data ROM, run a program on SNES which dumps decrypted data to the other RAM, which is then read back by the PC. Because the ROM emulator can reset the SNES, this can be entirely automated (though cycle speed would be severely limited by the SNES).

Perhaps parts like those would be more suitable

DLP-245PA, DLP-245PB-G, DLP-245SX and so on?

Only "problem" is that those are 40 pins instead of the 44 pins on the cart, but who cares? If you confirm they are suitable, I'm going to buy them.

Damn you tempted me to buy it straight away :P

??? Those are the same thing I linked, but with PICs! How will that help?! You will now have to do I/O through the MCU lol Talk about slow! The FT245's maximum throughput is 1 MiB/s (like half maximum EPP), but with a MCU now it's gotta be less than a quarter of that... You should have bought the plain module and connected it to a CPLD (or better yet, just get the CPLD and connect directly to LPT).

[Andreas Naive]

Can someone give a historical rundown of what has been done so far?

There is no much to say... There haven't any advance since your discussion in cherryroms' forum back in 2004; i think by that date i did an attempt to look at the data, but i failed to see anything interesting. However, i didn't employed any type of automatic processing that time, so i have always felt i could have done a more serious attempt; that's what is in my to-do list.

In the while, i have been eagerly awaiting for somebody to construct a hardware interface, but the only serious attempt in this time has been Kammedo/Charles MacDonald's one.

From what i remember, you did an analysis of the compressed data starting with variable amounts of zeros to conclude which ordering the input and output streams used (i mean, which bits were used first within every byte), and that the algorithm showed an adaptive behaviour, being stabilized the input bits/output bits rate in 1/128 (just like the S-DD1).

[blargg]

Of course the device is bidirectional, as I wrote initially I think the best plan of attack is to upload SPC data ROM, run a program on SNES which dumps decrypted data to the other RAM, which is then read back by the PC. Because the ROM emulator can reset the SNES, this can be entirely automated (though cycle speed would be severely limited by the SNES).

I agree that this is the only sane and reasonable approach. Also, correct me if I'm wrong, but isn't this a compression chip, not encryption, and hasn't all the data from the games already been decompressed and is known? I thought the main purpose was so that an emulator could have the original 2 MB or whatever compressed data, rather than something like 14 MB of decompressed data (compressed with zlib, which only halves it or so).

[kammedo]

Of course the device is bidirectional, as I wrote initially I think the best plan of attack is to upload SPC data ROM, run a program on SNES which dumps decrypted data to the other RAM, which is then read back by the PC. Because the ROM emulator can reset the SNES, this can be entirely automated (though cycle speed would be severely limited by the SNES).

I agree that this is the only sane and reasonable approach. Also, correct me if I'm wrong, but isn't this a compression chip, not encryption, and hasn't all the data from the games already been decompressed and is known? I thought the main purpose was so that an emulator could have the original 2 MB or whatever compressed data, rather than something like 14 MB of decompressed data (compressed with zlib, which only halves it or so).

Well actually its all about the adventure for me. Getting rid of those 14 MB of data is secondary . And it's actually a decompressor, not a compressor or decrypter.

I confirm what you said, : the SPC7110 can only _read_ (Charles MacDonald confirmed this, but im still hoping he was wrong) from the U2 bus, not _write_ on it.
This means that for dumping the decompressed data you would need to make it pass through the SPC7110 first, which means you would have to dump it either from the save chip (after a monitor program would had copied it there), or from the SNES's memory, which means the whole system would need :

a) the feoez cart with the rom emulator connected to a second pc par port
b) a GDSF7 (in my case) connected via a par port to the pc for dumping the results
c) a monitor program to control the SPC on the GDSF.

Messy. There is still hope that one of the grounded pins is a write latch signal for the U2 pins, but how the hell should we find out which it is?

OR, you bridge the ROM emulator to the SNES, through a latch barrier.
Then you can do everything from the GDSF and the monitor program.
Kyuusaku? Do you think that would be feasible?

Can someone give a historical rundown of what has been done so far?

I remember there was a whole discussion on CherryRoms, and at one point I understood what was in the files at least since some digging around showed I wrote some programs and began analyzing some things. I however do not remember any of those things.

Hey neviksti, great to see you are still alive ^^ ! Did you manage to work through the GDSF bios some more? I took the liberty to post the files on the ynt website btw (the Header file from TheDumper anbd the BIOS map from you). If you feel I should take them down, just tell me (cause i posted the map without your permission).

[kyuusaku]

isn't this a compression chip, not encryption

I misspoke.

This means that for dumping the decompressed data you would need to make it pass through the SPC7110 first, which means you would have to dump it either from the save chip (after a monitor program would had copied it there), or from the SNES's memory, which means the whole system would need :

a) the feoez cart with the rom emulator connected to a second pc par port
b) a GDSF7 (in my case) connected via a par port to the pc for dumping the results
c) a monitor program to control the SPC on the GDSF.

GDSF7?! This outline is ridiculous, the system only needs TWO small ROM emulators, one for each SPC bus. The ROM emulator which stores the SNES code also stores the results. BOTH emulators can be connected to the SAME parallel port.

Messy. There is still hope that one of the grounded pins is a write latch signal for the U2 pins, but how the hell should we find out which it is?

Yeah, YOUR way is messy lol. What's a write latch? Memory control signals are active low, so any grounded pins will always be active (and not the write signal). So U2 *is* the SPC's ROM?

OR, you bridge the ROM emulator to the SNES, through a latch barrier. Then you can do everything from the GDSF and the monitor program. Kyuusaku? Do you think that would be feasible?

What's a latch barrier? There's no reasonable way to use a GDSF7; The GDSF's I/O port must handshake in software, which means even more work and speed loss and that's assuming the GD and SPC can somehow cohabit.

[neviksti]

Hey neviksti, great to see you are still alive ^^ ! Did you manage to work through the GDSF bios some more? I took the liberty to post the files on the ynt website btw (the Header file from TheDumper anbd the BIOS map from you). If you feel I should take them down, just tell me (cause i posted the map without your permission).

Those were notes I compiled for myself while I was working on that. So, like most of my notes unfortunately, there's enough to be somewhat helpful but not enough to let me pick up were I left off (I'm not sure what some of the comments in those notes even mean now). If you or anyone else find them useful, by all means feel free to share it.

I haven't touched any SNES projects in quite awhile now, so forgive my rustiness. Due to the fact that you have a cartridge, and a SF7, I'd suggest we start nice and simple: make a SNES program that dumps decompressions from the cartridge to BRAM. Using the copier menu you can then save the BRAM and check if your program worked fine.

While we can't do completely arbitrary decompressions this way, it is likely that we can see the output from all possible first two bytes, and much other data to get us going.

Basically, to get this moving as soon as possible, let's start as simple as possible.

[kammedo]

I misspoke.

Oh god, kill him xD.

GDSF7?! This outline is ridiculous, the system only needs TWO small ROM emulators, one for each SPC bus. The ROM emulator which stores the SNES code also stores the results. BOTH emulators can be connected to the SAME parallel port.

Yep true. But then again, you have to mess around with two chips, not just one. Invasive, but surely effective.

Yeah, YOUR way is messy lol. What's a write latch? Memory control signals are active low, so any grounded pins will always be active (and not the write signal).

I misspoke. I meant attached to Vcc

So U2 *is* the SPC's ROM?

Yep, I think you can pretty much see it like that, because the U2 is only accessible by the SPC7110, and nothing else.

And another thing which you (kyuusaku) probably dont know : the SPC has 64K (Charles Mac Donald is convinced of that, I for myself think its only 32K) of dedicated RAM (mapped at bank $50) in which it decompresses all the data it reads from the U2 chip. As by now, we dont know if there is a way to tell the chip where to decompress the data (or I dont remember, have to take a look at the notes). So that means you wouldnt even need another ROM emulator, but only a transfer port, or a GDSF mapped in pass-through on bank $50 - god plese correct me if I am wrong.

What's a latch barrier? There's no reasonable way to use a GDSF7; The GDSF's I/O port must handshake in software, which means even more work and speed loss and that's assuming the GD and SPC can somehow cohabit.

No wait, now you are going out of context. I meant you just use the GDSF to actually sit between the SPC and SNES and thus dump the result from SNES memory (as a savestate for an example?).
Actually, neviksti said pretty much the same what I meant:

make a SNES program that dumps decompressions from the cartridge to BRAM. Using the copier menu you can then save the BRAM and check if your program worked fine.

A latch barrier is what we (in Italy, or at least where I studied) call a serie of 244/245, which are usually used to enable write data / address from a (for example) CPU like the 8086 on the data / address bus.

Because of that they are also called "drivers" here, dont know if tis the correct way to name them buut what wuold you expect from Italy

Those were notes I compiled for myself while I was working on that. So, like most of my notes unfortunately, there's enough to be somewhat helpful but not enough to let me pick up were I left off (I'm not sure what some of the comments in those notes even mean now). If you or anyone else find them useful, by all means feel free to share it.

I haven't touched any SNES projects in quite awhile now, so forgive my rustiness. Due to the fact that you have a cartridge, and a SF7, I'd suggest we start nice and simple: make a SNES program that dumps decompressions from the cartridge to BRAM. Using the copier menu you can then save the BRAM and check if your program worked fine.

While we can't do completely arbitrary decompressions this way, it is likely that we can see the output from all possible first two bytes, and much other data to get us going.

Basically, to get this moving as soon as possible, let's start as simple as possible.

I recall having tried to contact you for a while back then - welcome back ^^! Oh, and if Im not wrong, you said you had more notes about that somewere. Would you mind taking a look? I had planned to put the whole CR forums for read-only on the YnT (or the romhacking.net) site , but sadly I havent been able to contact CR's owner. And believe me, I tried.

[neviksti]

I misspoke.

Oh god, kill him xD.

There's a bit more bickering going on here than is comfortable for a topic that should be friendly collaboration. May I please request that everyone stop pouncing on mistakes and focus on the fun of reverse engineering this beast?

And another thing which you (kyuusaku) probably dont know : the SPC has 64K (Charles Mac Donald is convinced of that, I for myself think its only 32K) of dedicated RAM (mapped at bank $50) in which it decompresses all the data it reads from the U2 chip. As by now, we dont know if there is a way to tell the chip where to decompress the data (or I dont remember, have to take a look at the notes). So that means you wouldnt even need another ROM emulator, but only a transfer port, or a GDSF mapped in pass-through on bank $50 - god plese correct me if I am wrong.

This sounds like merely an additional method of reading data obtainable from the $00:48xx registers mentioned previously. For our purposes this doesn't seem to give an advantage, unless I am missing something. For example if there is some way to get the chip to use some of the RAM as input to the decompression. Are the tests the cartidge runs completely understood?

No wait, now you are going out of context. I meant you just use the GDSF to actually sit between the SPC and SNES and thus dump the result from SNES memory (as a savestate for an example?).
Actually, neviksti said pretty much the same what I meant:

make a SNES program that dumps decompressions from the cartridge to BRAM. Using the copier menu you can then save the BRAM and check if your program worked fine.

To be clear, what I was suggesting involved no modifications. Therefore it would only allow decompressions of byte patterns we can already find on this chip.

Assuming we are on the same page, the idea is then:
If you get this to work, it would give us some great data to get us started. This will hopefully give us enough data to figure out the class of compression used, and allow us to formulate precise test cases. At this point these test cases will probably not be able to be found in the ROM randomly and therefore we will need a way to run arbitrary data through. At this point we can add a ROM emulator to the board to replace U2. It can be very small, 8kb would probably be more than enough.


So first things first:
Can you play the game on passthrough on the SF7?
(basically checking if the delays are too long, and if the clock signal is making it to the cartridge)

EDIT: Yes, kyuusaku's idea (which sounds not to need the SNES, and everything controlled from a parallel port) is the most automated and has a faster throughput (insert->decompress->readout) rate. But the method I suggested above gets us data to play with almost immediately, which we can use to formulate some more specific test cases while you can work on extending your system with a rom emulator (of which you would only need one).

As always, this is merely a suggestion.

[kammedo]

I misspoke.

Oh god, kill him xD.

There's a bit more bickering going on here than is comfortable for a topic that should be friendly collaboration. May I please request that everyone stop pouncing on mistakes and focus on the fun of reverse engineering this beast?

Hehe no worries, that was actually just a friendly way to make fun, nothing more. But you are right, and I think we should focus more on the beast.

This sounds like merely an additional method of reading data obtainable from the $00:48xx registers mentioned previously. For our purposes this doesn't seem to give an advantage, unless I am missing something. For example if there is some way to get the chip to use some of the RAM as input to the decompression. Are the tests the cartidge runs completely understood?

the $50 bank is the place where the decompressed data gets placed by the decompression process of the SPC7110 by default.

That is, you supply the table offset, the pointer number, the number of bytes to be decompressed (or the numer of compressed bytes to process, I dont remember) to the relative regs, activate the chip, and you get the decompressed data from bank $50.

The point of seeing if it can decompress data from the same bank is intresting tho. We can investigate more about it.

Im not sure about it being the same if you use manual decompression (that is you read the data from the 00:48xx interface). I have to check that. Also, part of the tests I reversed, I will have a look at the rest too. I still have the old trace files i think.

To be clear, what I was suggesting involved no modifications. Therefore it would only allow decompressions of byte patterns we can already find on this chip.

Assuming we are on the same page, the idea is then:
If you get this to work, it would give us some great data to get us started. This will hopefully give us enough data to figure out the class of compression used, and allow us to formulate precise test cases. At this point these test cases will probably not be able to be found in the ROM randomly and therefore we will need a way to run arbitrary data through. At this point we can add a ROM emulator to the board to replace U2. It can be very small, 8kb would probably be more than enough.

Hm. Lets see if I got it right : you would suggest trying to find some specific patterns to look at, starting from the data present in the original U2 chip? Remember, we have both compressed and uncompressed data (aka graphics packs) already, and I even had started to write an analyzer tool which would allow you to handle comparisons on compressed / uncompressed data.
Hope to have it somewhere, If I do, @ Andreas: what functionality would be required for you to put the tool to good use?

So first things first:
Can you play the game on passthrough on the SF7?
(basically checking if the delays are too long, and if the clock signal is making it to the cartridge)

EDIT: Yes, kyuusaku's idea (which sounds not to need the SNES, and everything controlled from a parallel port) is the most automated and has a faster throughput (insert->decompress->readout) rate. But the method I suggested above gets us data to play with almost immediately, which we can use to formulate some more specific test cases while you can work on extending your system with a rom emulator (of which you would only need one).

As always, this is merely a suggestion.

I think that shouldn't be a big issue (playing the game in passthrough).

[Andreas Naive]

Hope to have it somewhere, If I do, @ Andreas: what functionality would be required for you to put the tool to good use?

That's impossible to say at this point, and i would suggest you to completely forget about it. All the people who could be interested in taking a look at the data (neviksti, byuu, Nach, me, ...) is perfectly able to make his own tools on the fly as the research progress. So raw compressed/uncompressed data is all we need.

[jolly_codger]

Could be coincidental. This 'pseudo-pattern' caught my eye.

http://www.mediafire.com/?l39ddrmhtye