SPC7110 Reverse Engineering Project

[kyuusaku]

And another thing which you (kyuusaku) probably dont know : the SPC has 64K (Charles Mac Donald is convinced of that, I for myself think its only 32K) of dedicated RAM (mapped at bank $50) in which it decompresses all the data it reads from the U2 chip. As by now, we dont know if there is a way to tell the chip where to decompress the data (or I dont remember, have to take a look at the notes). So that means you wouldnt even need another ROM emulator, but only a transfer port, or a GDSF mapped in pass-through on bank $50 - god plese correct me if I am wrong.

My way requires a second ROM programmer just to get the results back to the PC, any way you wish... You could choose to make a SNES control program as I was talking about earlier or if you don't want two RAM, just use the emulator's bus as a "transfer port" as you say, which is still better than using the GDSF7 since you don't need to waste any time working on I/O! In fact instead of writing these few responses I could have probably finished upgrading my emulator and written the control functions.

No wait, now you are going out of context. I meant you just use the GDSF to actually sit between the SPC and SNES and thus dump the result from SNES memory (as a savestate for an example?).

I understand what you mean, but it's not so simple, you can't get the GDSF to just arbitrarily access memory, you can only do I/O (slowly) through SNES software which means rewriting or hooking the BIOS which means wasting time. Or you could manually dump data by loading a SNES program and "Save BRAM", but that's by far the most time consuming and most error-prone way discussed so far...

Actually, neviksti said pretty much the same what I meant

It doesn't sound like it to me, you want PC control right? "Save BRAM" is obviously human control just meant to get you started.

A latch barrier is what we (in Italy, or at least where I studied) call a serie of 244/245, which are usually used to enable write data / address from a (for example) CPU like the 8086 on the data / address bus.

Because of that they are also called "drivers" here, dont know if tis the correct way to name them buut what wuold you expect from Italy :wink:

This is a bad translation, what you mean is a tri-state buffer (also called bus drivers), no latches involved. Some people say latch to express the latching action in a D flip flop aka edge triggered register, but most of the time latch means a "transparent latch" like a 74373, where the data is kept after enable is de-asserted but while it's asserted data passes through. Anyways,interfacing the SPC ROM emulator to SNES registers would be way more complicated, convoluted and again slower than just using a single PC interface, but would probably be smarter if you're using a GDSF7 since ideally you want only one PC interface.

While we can't do completely arbitrary decompressions this way, it is likely that we can see the output from all possible first two bytes, and much other data to get us going.

Basically, to get this moving as soon as possible, let's start as simple as possible.

If the ROM data has already been entirely decompressed, what's the point?

I recall having tried to contact you for a while back then - welcome back ^^! Oh, and if Im not wrong, you said you had more notes about that somewere. Would you mind taking a look? I had planned to put the whole CR forums for read-only on the YnT (or the romhacking.net) site , but sadly I havent been able to contact CR's owner. And believe me, I tried.

Even as the biggest CR poster, I think it's best to leave it in the past instead of retroactively worshiping it. Really how hard would it be to rewrite every spec of informative content on CR anyway? Not very.

EDIT: Yes, kyuusaku's idea (which sounds not to need the SNES, and everything controlled from a parallel port) is the most automated and has a faster throughput (insert->decompress->readout) rate. But the method I suggested above gets us data to play with almost immediately, which we can use to formulate some more specific test cases while you can work on extending your system with a rom emulator (of which you would only need one).

Well, my intention was to use the SNES to gather decompressed data, though it would be possible to bypass the SNES entirely by using the SNES ROM emulator's RAM bus to directly access the SPC registers and buffer. It would take significantly more logic however, but require one less RAM, and it'd be faster than using the SNES.

So how wide is the SPC ROM and how much data needs to be emulated? More thank 8KiB?

[Andreas Naive]

If the ROM data has already been entirely decompressed, what's the point?

Having the decompressed data *only* allow us to play the game, not to really *emulate* how the physical thing works. And that's the holy grial of emulation, after all...

Of course, you can also count with the intellectual challenge, and the fun of being capable of understanding how stuff works. Knowing how the algorithm works, and not only having the output data, is an important thing that have a real added value for many of us.

Knowledge. Knowledge is the point.

(And, well, romhackers could say you that they would like to decompress, modify and recompress some data, but that's secondary for me, so i won't speak of it. )

[neviksti]

Hm. Lets see if I got it right : you would suggest trying to find some specific patterns to look at, starting from the data present in the original U2 chip? Remember, we have both compressed and uncompressed data (aka graphics packs) already, and I even had started to write an analyzer tool which would allow you to handle comparisons on compressed / uncompressed data.

No, you misunderstand.
We have the decompression of only the data starting on the chunks that the game uses. We can however ask the chip to start decoding many MANY other places. So even without changing anything, we can get a much better dataset. For example the decompression results for most of all possible first two input byte combinations.

Does that make sense? Maybe I'm describing this badly.

So first things first:
Can you play the game on passthrough on the SF7?
(basically checking if the delays are too long, and if the clock signal is making it to the cartridge)

I think that shouldn't be a big issue (playing the game in passthrough).

Please, PLEASE test this as soon as possible and let us know. (For example pass-through access is often too slow for the DSP processor... hence the special DSP piece that can attach _between_ the SF7 and the SNES.) If the game can't be played on passthrough, then we'll have to drop this idea.

Also, pin 1 isn't passed straight through to the cartridge on the SF7, so I want to make sure that doesn't cause problems as well (I may be remembering that wrong, but I believe it is inverted before sending it to the cartridge).

[neviksti]

My way requires a second ROM programmer just to get the results back to the PC, any way you wish... You could choose to make a SNES control program as I was talking about earlier or if you don't want two RAM, just use the emulator's bus as a "transfer port" as you say, which is still better than using the GDSF7 since you don't need to waste any time working on I/O! In fact instead of writing these few responses I could have probably finished upgrading my emulator and written the control functions.

Do you have a cartridge you'd be willing to use for this?
Your idea is much more automated and quick once it is built, so if you could build this and run tests for us it would be much MUCH appreciated.

For kammedo I would still advocate getting a SF7 dumper working first as it is more appropriate to the learning curve (and let's us learn before he has to risk potentially destroying a cartridge).

So how wide is the SPC ROM and how much data needs to be emulated? More thank 8KiB?

The docs say the ROM has a 16bit data bus. If the setup is automated, 8kbits should be plenty for test data.

While we can't do completely arbitrary decompressions this way, it is likely that we can see the output from all possible first two bytes, and much other data to get us going.

Basically, to get this moving as soon as possible, let's start as simple as possible.

If the ROM data has already been entirely decompressed, what's the point?

I for one fall under the category Andreas stated as "you can also count the intellectual challenge, and the fun of being capable of understanding how stuff works."

When I hacked StarOcean to work without the SDD1 chip, it was almost purely academic. Similarly with reverse-engineering the "authentication algorithm" of another chip that Andreas worked on for me (thank you again so much Andreas! that was a lot of fun). How's it work? and Can we figure it out? are always fun questions for me.

I know it may sound like a waste of time, but I truely do find it fun. If you'd be willing to build your setup to help, it would be much appreciated (and hopefully you'll have fun too).

[blargg]

Having the decompressed data *only* allow us to play the game, not to really *emulate* how the physical thing works. And that's the holy grial of emulation, after all...

Why stop there? If you're not emulating the physical gates in the ROM containing the compressed data, you're merely providing the same data that would be read out of it when accessed. The line is always arbitrary, and there's always something unknown on the other side of it. You're always treating some parts as black boxes and simply emulating the high-level behavior, not how they work internally.

I'm not saying that there's no reason to RE this decompression chip, just that it seems much less significant than other unknown things related to SNES emulation, like whether the S-DSP has a processor or is all hardware.

[neviksti]

other unknown things related to SNES emulation, like whether the S-DSP has a processor or is all hardware.

If someone has a broken SNES (or a spare they don't care if I break), I can decap the chip and probably answer this. It is even possible that I could read out the ROM inside of this (assuming it is a processor which seems likely to me).

[kyuusaku]

Do you have a cartridge you'd be willing to use for this?
Your idea is much more automated and quick once it is built, so if you could build this and run tests for us it would be much MUCH appreciated.

I do have a Tengai Makyou cartridge (though I'd prefer to sacrifice one in worse condition) and nearly all the parts, but without a PCB which I'm not prepared to make, I don't think I can build it successfully with ~20 chips. That's why I'd like to help with the logic, but not the actual building.

I for one fall under the category Andreas stated as "you can also count the intellectual challenge, and the fun of being capable of understanding how stuff works."

+

I know it may sound like a waste of time, but I truely do find it fun. If you'd be willing to build your setup to help, it would be much appreciated (and hopefully you'll have fun too).

I didn't know it could decompress anything but the ROM so I thought it would be a waste of time to redump.

EDIT3: I came up with a suitable circuit on paper which trades off speed for simplicity. It's made with extremely common chips and only emulates a 2Kx16 bit ROM, as well as provide a SNES R/W interface (PC decoded /ROMSEL). The chips are currently:
3x74374, 3x74157, 1x7400, 2x74244, 2x6116/6264, maybe I'll try to build it.

[kammedo]

other unknown things related to SNES emulation, like whether the S-DSP has a processor or is all hardware.

If someone has a broken SNES (or a spare they don't care if I break), I can decap the chip and probably answer this. It is even possible that I could read out the ROM inside of this (assuming it is a processor which seems likely to me).

I have two loose SPC actually, desoldered with hot air pistols at my old university. I have to see if I have them here tho :/

[kammedo]

If the ROM data has already been entirely decompressed, what's the point?

Having the decompressed data *only* allow us to play the game, not to really *emulate* how the physical thing works. And that's the holy grial of emulation, after all...

Of course, you can also count with the intellectual challenge, and the fun of being capable of understanding how stuff works. Knowing how the algorithm works, and not only having the output data, is an important thing that have a real added value for many of us.

Knowledge. Knowledge is the point.

(And, well, romhackers could say you that they would like to decompress, modify and recompress some data, but that's secondary for me, so i won't speak of it. )

Ha. You wish it would be that easy... The Chip itself expects the decompression offsets to be in a table, which is in the U2 ROM. What you supply to the chip is in fact the offset of the table, the width of the pointers (usually 3 / 4 bytes), the number of the pointer to decompress.

You see, this is all in the U2 ROM, where we dont have direct access to...

[kammedo]

EDIT3: I came up with a suitable circuit on paper which trades off speed for simplicity. It's made with extremely common chips and only emulates a 2Kx16 bit ROM, as well as provide a SNES R/W interface (PC decoded /ROMSEL). The chips are currently:
3x74374, 3x74157, 1x7400, 2x74244, 2x6116/6264, maybe I'll try to build it.

Could you provide the schematics? I could give it a go also.

[caitsith2]

True that the pointer may be expected, but what if you could store some arbitrary value in bank $50:0000, say the table that is expected, and point it there, to decompress just entry 0 of that table. If the spc could do that, then you could use nearly every 2 byte combination of the existing U2 rom. If however you can't directly manipulate the $50:0000 bank, or the system won't allow a table to be pointed from there, then you will have to either look for table entry values that the. In any case, you could point the table over to 0x000000, (Type 1 compression, at data rom offset 0x020408), or at 0x000001, (Type 2 compression, at data rom offset 0x040810). Look for any bytes that are not part of actual tables, or if they are, part of the actual tables, off by 1, 2, or 3 bytes. A good example, is point the Table to 0x000009, (Offset for Tengai Makyo Zero would be type 0 decompression, offset 0x031802. (Would result in first 2 bytes being 0x4171).

[kyuusaku]

Could you provide the schematics? I could give it a go also.

IRQ can be connected to a status bit for testing.

[neviksti]

Kammedo,
have you tried to play the game on pass through on the SF7 yet? If that works I will write some code to get things moving if you'd like.

Ha. You wish it would be that easy... The Chip itself expects the decompression offsets to be in a table, which is in the U2 ROM. What you supply to the chip is in fact the offset of the table, the width of the pointers (usually 3 / 4 bytes), the number of the pointer to decompress.

You see, this is all in the U2 ROM, where we dont have direct access to...

You quoted Andreas, but seem to be replying to my comments. If so, I obviously failed to explain well again. Caitsith has the right idea. We can use any random data in the ROM as the table (I'm not sure about allignment issues, but we can figure that out or avoid it in the beginning).

True that the pointer may be expected, but what if you could store some arbitrary value in bank $50:0000, say the table that is expected, and point it there, to decompress just entry 0 of that table. If the spc could do that, then you could use nearly every 2 byte combination of the existing U2 rom.

That would be awesome if you could figure out a way to do that.

If however you can't directly manipulate the $50:0000 bank, or the system won't allow a table to be pointed from there, then you will have to either look for table entry values that the.

Yes, exactly. I think that would give us some good data to get going. At the moment we don't even have all the possible FIRST byte combinations.

Do you still have your system?
If so, could you get that data for us?
I unfortunately don't have any cartridges with the SPC7110 chip, so if your system still works it would be the quickest way to get new data here.

[caitsith2]

Unfortunately, I will have to rebuild my cart dumper. The only thing I have left of it, is the AT90S8535, and the ZIF socket it is in. Would have to pretty much rebuild it from scratch at this point. If I do that, it won't be with Wirewrap wire. I will more than likely have a PCB made for the purpose. (Most likely through batchpcb.com) (I have designed and built a few other things on my own.)

[neviksti]

kyuusaku,
I'm sorry, I was confused by the documents. The doc that was linked here said the U2 rom interface used a 16bit data bus. But later the pin listing shows that in fact it is only an 8bit data bus. And later the document says "The SPC7110 uses only the lower 8 bits of the U2 on type 1, and D15, but I still can't think about its use."

So the datarom is probably a 8/16 bus capable chip, and wired permanently for just an 8bit databus to the SPC7110. (EDIT: I see now that this was already discussed before I joined the discussion. Sorry about that.)

Unfortunately, I will have to rebuild my cart dumper.

Don't worry about it then. Do you have a SF7 or SF3? We can use that instead. If not, Kammendo has an SF7 and can probably do it for us.

Can you remind me what the format is for the files in the graphics packs?

Also, in your Atmel program, you have a routine called "init_spc7110_access" which enables decompression (write $02 to register $4834) and then writes a weird series of bytes toggling bit7 on register $4330. Is the weird series necessary for decompressing (which calls this routine), or just for dumping the DataROM (which also calls this routine)?

One final thing, is there ever a case where reading a location in bank $50 is NOT exactly like reading $4800? It's not clear to me what the bank $50 does, for reading DarkForce's comments (on the saved ZSNES thread) makes it sounds like bank $50 is not just accessing some internal 32kByte RAM as Kammendo suggests above, but is merely register $4800.