Page 4 of 21

Posted: Sat Jun 07, 2008 4:24 am
by caitsith2
Don't even have a copier that I could run the tests on, so Kammendo will have to do the runs for us.

As for whether the toggling is required for compression, I can't remember if it was or not. Needs to be tested for sure. It is quite possible that it may be required. Regardless, it only has to be done once per power up/reset IIRC.

As for graphics pack format,

Code: Select all

while (not EOF(index.bin)
{
    Read Table Index (1 byte)
    Read Table file name (3 bytes)
    Read Offset in Table file (4 bytes.)
    Read Size of entry in table file. (4 bytes)
    Seek to Offset in table file, and read entry size, pass back as decompressed data.
}

Posted: Sat Jun 07, 2008 6:00 am
by blargg
neviksti wrote:
blargg wrote:other unknown things related to SNES emulation, like whether the S-DSP has a processor or is all hardware.
If someone has a broken SNES (or a spare they don't care if I break), I can decap the chip and probably answer this. It is even possible that I could read out the ROM inside of this (assuming it is a processor which seems likely to me).
I believe that an S-DSP has already been arranged to be decapped. I think Lord Nightmare knows more.

Posted: Sat Jun 07, 2008 8:45 am
by kammedo
caitsith2 wrote:True that the pointer may be expected, but what if you could store some arbitrary value in bank $50:0000, say the table that is expected, and point it there, to decompress just entry 0 of that table. If the spc could do that, then you could use nearly every 2 byte combination of the existing U2 rom. If however you can't directly manipulate the $50:0000 bank, or the system won't allow a table to be pointed from there, then you will have to either look for table entry values that the. In any case, you could point the table over to 0x000000, (Type 1 compression, at data rom offset 0x020408), or at 0x000001, (Type 2 compression, at data rom offset 0x040810). Look for any bytes that are not part of actual tables, or if they are, part of the actual tables, off by 1, 2, or 3 bytes. A good example, is point the Table to 0x000009, (Offset for Tengai Makyo Zero would be type 0 decompression, offset 0x031802. (Would result in first 2 bytes being 0x4171).
Hm. Where's the point in using the U2 ROM if the SPC can access the table in the $50 bank - at that point it should be able to access compressed data in there also, or what do you think? I think that _is_ actually the only thing that we should check, along with the pass throug (which I didnt had a chance to check by now, am out of home until monday :/, but I have everything ready).

nevitksti : in case the clock doesn't get passed through, I presume a simple cable from the gdsf's snes slot pin 1 to the top slot pin should do the job, right?

Posted: Sat Jun 07, 2008 8:51 am
by kammedo
neviksti wrote: One final thing, is there ever a case where reading a location in bank $50 is NOT exactly like reading $4800? It's not clear to me what the bank $50 does, for reading DarkForce's comments (on the saved ZSNES thread) makes it sounds like bank $50 is not just accessing some internal 32kByte RAM as Kammendo suggests above, but is merely register $4800.
Well, register $4800 returns a "continuous value" from bank $50.
What I have seen in FEoEZ (and mds, if i remember it correctly), is that the games

a) setup the SPC
b) activate the decompression
c) dma the data from bank $50 to WRAM

I recall having been searching for some $4800 register use, but I couldnt find it..

Posted: Sat Jun 07, 2008 8:54 am
by Andreas Naive
blargg wrote:
Andreas Naive wrote:Having the decompressed data *only* allow us to play the game, not to really *emulate* how the physical thing works. And that's the holy grial of emulation, after all...
Why stop there? If you're not emulating the physical gates in the ROM containing the compressed data, you're merely providing the same data that would be read out of it when accessed. The line is always arbitrary, and there's always something unknown on the other side of it. You're always treating some parts as black boxes and simply emulating the high-level behavior, not how they work internally.

I'm not saying that there's no reason to RE this decompression chip, just that it seems much less significant than other unknown things related to SNES emulation, like whether the S-DSP has a processor or is all hardware.
I'm not suggesting we should stop here, indeed. ;) I would like seeing the hardware emulated to the atomic level but, as it wouldn't be doable, you have to stablish a level of abstraction that separate the things you are emulating from the things you are only simulating. I'm conscious of that

As to whether this is less or more important that other unemulated behaviors, you can be right, but i'm not saying this should be given higher priority. Simply i volunteer for this because, with my current skills, i can help with it, while possibly i lack the skills and the background to help with other issues. Besides, i like studying certain types of algorithms, so potentially this can be funnier for me. ;)

Posted: Sat Jun 07, 2008 9:44 am
by blargg
Andreas Naive wrote:[...] i'm not saying this should be given higher priority. Simply i volunteer for this because, with my current skills, i can help with it, while possibly i lack the skills and the background to help with other issues. Besides, i like studying certain types of algorithms, so potentially this can be funnier for me. ;)
I can't argue with that. It's a problem that's well-defined, and there's no loss if no progress is made, since the higher-level behavior (access to the decompressed data) is already emulated.

Posted: Sat Jun 07, 2008 11:12 am
by kyuusaku
OK, maybe now it would be best to just interface the SPC ROM emulator directly to the SNES. I see that you guys really want to reuse your devices.

Image

Is there a free space that can be decoded with a 74139 like SRAM? The "emulator" is now about as compact as possible ;)

Posted: Sat Jun 07, 2008 4:23 pm
by kammedo
Idea :

Suppose we replace the U2 ROM with an SRAM (with /WR coming from the SNES), bridge it over to the SNES bus over a set of drivers. That would give us access to the U2 rom, wouldnt it?

Posted: Sat Jun 07, 2008 9:38 pm
by kyuusaku
Look at the post before yours.. :?

Posted: Sun Jun 08, 2008 12:40 am
by kammedo
kyuusaku wrote:Look at the post before yours.. :?
Argh xD damn quick posting. I think that would be easier, cheaper and quicker, or?

Posted: Sun Jun 08, 2008 5:47 am
by neviksti
blargg wrote:I believe that an S-DSP has already been arranged to be decapped. I think Lord Nightmare knows more.
That's great! I didn't realize others had gotten into that hobby as well. If you hear of anyone figuring out a better way to stain the n versus p doping please _please_ have them contact me. I'd love to swap recipes.

----------
kammedo wrote:Hm. Where's the point in using the U2 ROM if the SPC can access the table in the $50 bank - at that point it should be able to access compressed data in there also, or what do you think? I think that _is_ actually the only thing that we should check, along with the pass throug (which I didnt had a chance to check by now, am out of home until monday :/, but I have everything ready).
No, you are still missing the point here. Caitsith suggested a very optimistic possibility that we can write to some internal RAM in the chip and use that for the table and/or the compressed data. Yes that is worth checking (although it is not clear what to check), but that is _not_ the "only thing" worth checking. In fact, the actual idea (which caitsith was just expanding upon) most likely will work given our current understanding (we don't need to search for new features, but merely check that our understanding is correct).

So the short explanation is this:
- we have some decompression data from the chip using particular 4 byte entries (24bit address plus some indicator byte)
- so now let's try every possible 4 byte entry to get more data to study (more likely we'll have a program search the compressed data ROM for some good entries to look at instead of dumping them all)

Hopefully the idea makes more sense now. Currently we don't even have data for all the possible first byte combinations... that would be a great start and quite feasible. We can also try to get as many first two byte combinations as possible.
kyuusaku wrote:OK, maybe now it would be best to just interface the SPC ROM emulator directly to the SNES. I see that you guys really want to reuse your devices.

[...image...]

Is there a free space that can be decoded with a 74139 like SRAM? The "emulator" is now about as compact as possible ;)
I thought about this some more and I think I have a one chip solution that requires much fewer connections.

an asynchronous unidirectional FIFO
any of the Cypress CY7C419/21/25/29/33 chips would work great.


steps for modification:
- cut trace to /CE on the compressed data ROM (U2), and tie pin to Vcc
- attach Vcc and GND to FIFO, and attach /XI, D8, /FL to Vcc
- attach FIFO output data lines 0-7 to U2-ROM data lines
- attach FIFO input data lines 0-7 to SRAM data lines
- attach FIFO /R to U2-ROM /OE from SPC7110
- attach FIFO /W to SRAM /CE
- attach FIFO /RESET to SNES /RESET (or RTC /CE if you want to have sofware reset capability)

So from the SNES you can load up the FIFO memory by writing to SRAM. You can also do a software reset of the FIFO if need be by doing any access to the RTC (software reset probably isn't needed, you could just attach that to the SNES /RESET instead).

The Datarom dumping feature of the SPC7110 can be used to easily test the setup before tackling decompressions. For decompressions, if the SPC7110 makes some kind of "spurious" reads to the ROM that we can't figure out, then this setup wouldn't work. Considering that it is crunched for time, I can't think of any reason it would have any "spurious" reads. It probably just reads:
... 4 bytes (24bit address + 1 byte indicator) from "table"
... and then sequentially reads from the compressed data

Sounds do-able to me.
Comments anyone?

Kammedo, does this sound okay to you?

EDIT:
In stock parts:
Digikey, or Mouser CY7C433-10JXC ... $10.58
Avnet CY7C421-20JXC .... $3.39

These are all 28-pin PLCC's.
If you prefer DIP packages for ease of hobby work, you can get all kinds of chips from: http://www.rocelec.com/ (They list stock, but not prices online. I sent for some quotes out of curiousity.)

Cypress also says on their site they will give people 10 free samples. I haven't looked into it though. (Back in the day, companies would give free samples because if people got hooked on their product they would likely use it at 'work' too. Now a days very few companies give free samples unless you are from a company and it is understood that if the product prototype works you will buy large quantities from them.) If you're lucky they'll give you some samples without hassle if you say you're a student... might as well give it a try.

Posted: Sun Jun 08, 2008 10:36 am
by kammedo
kammedo wrote:Hm. Where's the point in using the U2 ROM if the SPC can access the table in the $50 bank - at that point it should be able to access compressed data in there also, or what do you think?
neviksti wrote:No, you are still missing the point here.
Hehe by "in there" i actually meant the SPC - internal RAM (or bank $50) :). No worries, we are talking the same from two different views.

neviksti wrote: I thought about this some more and I think I have a one chip solution that requires much fewer connections.

an asynchronous unidirectional FIFO
any of the Cypress CY7C419/21/25/29/33 chips would work great.


steps for modification:
- cut trace to /CE on the compressed data ROM (U2), and tie pin to Vcc
- attach Vcc and GND to FIFO, and attach /XI, D8, /FL to Vcc
- attach FIFO output data lines 0-7 to U2-ROM data lines
- attach FIFO input data lines 0-7 to SRAM data lines
- attach FIFO /R to U2-ROM /OE from SPC7110
- attach FIFO /W to SRAM /CE
- attach FIFO /RESET to SNES /RESET (or RTC /CE if you want to have sofware reset capability)

So from the SNES you can load up the FIFO memory by writing to SRAM. You can also do a software reset of the FIFO if need be by doing any access to the RTC (software reset probably isn't needed, you could just attach that to the SNES /RESET instead).

The Datarom dumping feature of the SPC7110 can be used to easily test the setup before tackling decompressions. For decompressions, if the SPC7110 makes some kind of "spurious" reads to the ROM that we can't figure out, then this setup wouldn't work. Considering that it is crunched for time, I can't think of any reason it would have any "spurious" reads. It probably just reads:
... 4 bytes (24bit address + 1 byte indicator) from "table"
... and then sequentially reads from the compressed data

Sounds do-able to me.
Comments anyone?

Kammedo, does this sound okay to you?
Hm. beware, SRAM is decoded and mapped (! through a dedicated register) by the SPC itself.

But for the rest, there is one slight glitch. Remember, the U2 ROM has to hold both compressed data and the table of pointers used for decompression.

So this means you need random access, which obviously is not suitable for the FIFO. BUT. The other solution would be using the FIFO chip to store compressed data _only_, and the SPC internal RAM (bank $50 again) for storing the pointer table.
So far we don't know if the SPC RAM is accessible from the SPC decompression engine itself, in read operation. I presume its not, why provide a separate bus then? But untill im 100% sure of that I don't want to shot it down.

I dont think the SPC does some initial ROM checks by itself, all the checks are piloted at software level (at least in the ROMs i checked, all is possibleb obviously...). I have to retrieve the old traces I did. Mainly, most of the checks do write / read some data from / to the $50 bank, testing S-ram data and address bus, data port, compressed length, Data ports B,O,S (which I dont recall what they are).

Posted: Sun Jun 08, 2008 11:03 am
by kyuusaku
Game Doctors won't even start up with Tengai Makyou in their top slot so there must be a conflict.

Perhaps the game can be inserted into the slot after the GD is setup to passthrough conflicting areas?

A GDSF3 I tested is capable of running Mario Kart through the top, but I don't think it's memory mapping capabilities are as sophisticated as the GDSF7's, so maybe passthrough header flags aren't an option. I can test with a GDSF6/7 too but I think it's because of the passthrough feature that they're the ones that require the bottom DSP adapter.
kammedo wrote:Hm. beware, SRAM is decoded and mapped (! through a dedicated register) by the SPC itself.
What does this mean? There is a discrete SRAM chip on the board right? That's all that matters.
kammedo wrote:But for the rest, there is one slight glitch. Remember, the U2 ROM has to hold both compressed data and the table of pointers used for decompression.

So this means you need random access, which obviously is not suitable for the FIFO. BUT. The other solution would be using the FIFO chip to store compressed data _only_, and the SPC internal RAM (bank $50 again) for storing the pointer table.
As I understand it, the table/index/pointer data is really irrelevant because it doesn't affect the algorithm so you can give it fake setup data and then valid compressed data and achieve the same results. The only thing is this relies on predictable access from the SPC and the ability to find out the addressing sequence, which may be easy.

Posted: Sun Jun 08, 2008 1:12 pm
by kammedo
kyuusaku wrote: As I understand it, the table/index/pointer data is really irrelevant because it doesn't affect the algorithm so you can give it fake setup data and then valid compressed data and achieve the same results. The only thing is this relies on predictable access from the SPC and the ability to find out the addressing sequence, which may be easy.
Yes, figured it out 5 mins after I posted (again). No need to worry about that at all. So yes, it would reduce the number of components, but wouldnt a double-sram (with the U2 overbridged to the SNES bus) be more feasible?

As for the GDSF7, yes it doesnt run the FEoEZ cart from the top slot...I recall Darkforce telling that some changes to the GDSF were needed in order to make it run. I have the DSP exp, but at home. Ill see if I can get the GDSF to make it work. Any suggestions?

Posted: Sun Jun 08, 2008 2:27 pm
by caitsith2
The primary thing here, is that the game is expecting direct communication with the spc. If there is no clock going to the spc, then communication can not happen. You basically have to pass Pin 1 of the cartridge connector from the bottom side, directly to the top side somehow, as that is the clock for this chip.