Page 17 of 21

Posted: Fri Jul 11, 2008 3:38 pm
by caitsith2
Blast it, you just beat me to the punch in posting. Glad you figured out the fact that the compressed data was underdumped. I suspect that the mode 1 and mode 2 data both may be underdumped as well. Most likely, the developers have done some rom optimizations whenever possible. (Like if the last 20 bytes of the compressed data happened to also be the first 20 bytes of the next table entry compressed data, then they may have put them together, so that 20 less bytes of the datarom is used. In those days, every byte did matter.

Posted: Fri Jul 11, 2008 3:43 pm
by caitsith2
neviksti wrote:Wow, do I have some strange luck or what?
I only chose literally a couple files to focus on when I was trying to figure out what was wrong with my evolution table last night. It appears that probably the only actual bad decompression-dump is the confusing 2AE432-09.compressed file.

All other mismatches for 00 decompression (I checked the data supplied for all three games) can be explained by underdumps of the .uncompressed file.

I believe it is a bad dump since, as explained, running the prob calculator on the real hardware matched my code. And because only that one byte in the data doesn't match. If this was truely a decompressor problem, and the states were wrong, there should be no real recovery from a mistake.

Here's an output listing ALL mismatches

Code: Select all

Read 186 bytes from [feoez/00/2AE432-09.compressed]
Read 406 bytes from [feoez/00/2AE432-09.uncompressed]
byte 18: decomp[7E]->real[72]
****FAIL**** mismatch detected
It is really just one byte.
Seems that 18th byte of my feoez gfx pack dump came back to haunt me. It was always the 18th byte that was being inconsistent on my original cart dumper hardware, when doing decompression runs.

EDIT: Mode 0 decompression definitely seems to be bit perfect to real hardware. I found one 32KiB overdumped uncompressed entry in my FEOEZ-SJNS 00 set, and all 32KiBs of decompression match perfectly, when supplied with sufficient compressed data from the original rom. (One mismatch had been detected, but only because there was extraneious data written to the uncompressed file at the end, that should not have been there.)

Posted: Fri Jul 11, 2008 3:54 pm
by neviksti
caitsith2 wrote:Seems that 18th byte of my feoez gfx pack dump came back to haunt me. It was always the 18th byte that was being inconsistent on my original cart dumper hardware, when doing decompression runs.
What a bizarrely specific problem. Since you always read the data from the same address, it sounds like an actual SPC7110 hardware issue, and not a wire/connection issue. Maybe that was a common place for the SPC7110 to have trouble keeping up with the dumping (after all, it was being clocked at 8MHz instead of 21MHz... and the SNES access timing is fairly long, and even takes time to output and latch the bank every access).

Posted: Fri Jul 11, 2008 4:24 pm
by Near
Wow, so it's already perfected ... absolutely amazing.

It seems like very minor changes to mps/lps in the evolution table can get mode 2 data to start going farther, but that probably doesn't mean much. I wish I could help in getting the table updated ...

But I suppose I should work on getting this into an emulator for testing it live.

Can anyone please post all of the known technical data on the chip so far? Eg all the other registers, memory mapping, etc? I'd like to start emulating the other features of this chip as soon as possible.

I'll write the emulation as a generic class so we can put it in ZSNES and Snes9X.

Posted: Fri Jul 11, 2008 4:26 pm
by Lord Nightmare
neviksti: Any chance at posting source code to the decompressor?

LN

Posted: Fri Jul 11, 2008 4:48 pm
by caitsith2
I found my complete set of RAW gfx pack dumps. I will write the tester to check each and every entry, that uses mode 0 decompression.

Posted: Fri Jul 11, 2008 4:50 pm
by caitsith2
Lord Nightmare wrote:neviksti: Any chance at posting source code to the decompressor?

LN
http://neviksti.com/SPC7110/DecompTest.c

Posted: Fri Jul 11, 2008 5:55 pm
by neviksti
byuu wrote:Can anyone please post all of the known technical data on the chip so far? Eg all the other registers, memory mapping, etc? I'd like to start emulating the other features of this chip as soon as possible.
I strongly suggest figuring out how to use your copier to play with these cartridges. If need be you can always disable the CIC chip, load a program into ram with the copier, remove copier and insert cartridge. I can give you a memory explorer program which should let you then play with any cartridge.

DarkForce has a doc out on the net somewhere, but I couldn't find it. I have saved a post he put in a thread once

Code: Select all

When the cart powers up, banks $C0-$CF are mapped to the program rom and banks $D0-FF are mapped to the data rom. So like I said, if you can read properly from $D0, there is nothing stopping you from reading $E0 and $F0 as well.

See info below for how to map data rom banks.

DF

<SPC7110 Info, Reverse Engineered by Dark Force>
SPC7110 Rom Map
=====================

Far East of Eden Zero
40mb total: 8mb program rom (000000-0FFFFF) + 32mb data rom (100000-4FFFFF)

Super Momotarou Happy Train
24mb total: 8mb program rom (000000-0FFFFF) + 16mb data rom (100000-2FFFFF)

program rom mapped to $C0:0000-$CF:FFFF/$80:0000-$8F:FFFF always
data rom in 8mb chunks can be mapped to hirom areas using the bank mapping regs ($4831/2/3)
default mapping: first 24mb of data rom mapped to $D0:0000-$FF:FFFF

SPC7110 Decompression Ram Map
$50:0000 - $50:FFFF data decompressed from data rom by spc7110
when reading from bank $50, the offset address is ignored ($00:XXXX), instead bytes are
always read sequentially from the start of the bank ($50:0000) after each
new decompression. register $4800 is linked to the same read index as reading from bank $50

SPC7110 Sram Map
$00:6000 - $00:7FFF sram 8k

SPC7110 Mirroring
$xx:4800 - $xx:7FFF register and sram banks mirrored at $00 - $3F, $80 - $BF


defval: = default power-on/reset register value

SPC7110 Registers
$4800  DECOMPRESSED DATA CONTINUOUS READ PORT: returns a decompressed value from bank $50 and decrements 16 bit counter value at $4809/A by 1
$4801  COMPRESSED DATA TABLE POINTER: ($0000FF) low offset, defval:00
$4802  COMPRESSED DATA TABLE POINTER: ($00FF00) high offset, defval:00
$4803  COMPRESSED DATA TABLE POINTER: ($FF0000) bank, defval:00
$4804  COMPRESSED DATA TABLE INDEX: index of 32 bit compressed data pointer (big-endian), defval:00
$4805  DECOMPRESSED DATA OFFSET: ($00FF) low byte of offset in bank $50, defval:00
$4806  DECOMPRESSED DATA OFFSET: ($FF00) high byte of offset in bank $50, defval:00
$4807  DMA CHANNEL FOR DECOMPRESSION, set to match snes dma channel used for compressed data, defval:00
$4808  C r/w option, unknown, defval:00
$4809  COMPRESSION LENGTH COUNTER: ($00FF) low byte, defval:00
            write: set start counter value low byte
            read: get counter value low byte
            auto-increment: after each sequential read from bank $50
$480A  COMPRESSION LENGTH COUNTER: ($FF00) high byte, defval:00
            write: set start counter value high byte
            read: get counter value high byte
            auto-increment: after each sequential read from bank $50
$480B  DECOMPRESSION COMMAND MODE: see decompression command modes, defval:00
$480C  DECOMPRESSION FINISHED STATUS: high bit set = done, high bit clear = processing, cleared after successful read,
            high bit is cleared after writing to $4806, $4809/A is set to compressed data length, defval:00
$4810  DATA ROM CONTINUOUS READ PORT: returns a byte from data rom at data rom pointer location, defval:00
$4811  DATA ROM POINTER: ($0000FF) r/w low offset, defval:00
$4812  DATA ROM POINTER: ($00FF00) r/w high offset, defval:00
$4813  DATA ROM POINTER: ($FF0000) r/w bank offset, defval:00
            bank offset is zero based from start of data rom: banks $00-$3f data rom -> $10-$4f full rom
$4814  DATA ROM POINTER ADJUST: ($00FF) low byte, defval:00
$4815  DATA ROM POINTER ADJUST: ($FF00) high byte, defval:00
$4816  DATA ROM POINTER INCREMENT: ($00FF) low byte, defval:00
$4817  DATA ROM POINTER INCREMENT: ($FF00) high byte, defval:00
$4818  DATA ROM COMMAND MODE: bit field control of data rom pointer (see data rom command mode byte), defval:00
            write: set command mode,
            read: performs action instead of returning value, unknown purpose,
            command mode is loaded to $4818 but only set after writing to both $4814 and $4815 in any order
$481A  DATA ROM READ AFTER ADJUST PORT: returns a byte from data rom at data rom pointer location + adjust value ($4814/5), defval:00
$4820  16 BIT MULTIPLICAND: ($00FF) low byte, defval:00
        32 BIT DIVIDEND: ($000000FF) low byte of low word, defval:00
$4821  16 BIT MULTIPLICAND: ($FF00) high byte, defval:00
        32 BIT DIVIDEND: ($0000FF00) high byte of low word, defval:00
$4822  32 BIT DIVIDEND: ($00FF0000) low byte of high word, defval:00
$4823  32 BIT DIVIDEND: ($FF000000) high byte of high word, defval:00
$4824  16 BIT MULTIPLIER: ($00FF) low byte, defval:00
$4825  16 BIT MULTIPLIER: ($FF00) high byte, defval:00
$4826  16 BIT DIVISOR: ($00FF), defval:00
$4827  16 BIT DIVISOR: ($FF00), defval:00
$4828  32 BIT PRODUCT: ($000000FF) low byte of low word, defval:00
        32 BIT QUOTIENT:($000000FF) low byte of low word, defval:00
$4829  32 BIT PRODUCT: ($0000FF00) high byte of low word, defval:00
        32 BIT QUOTIENT:($0000FF00) high byte of low word, defval:00
$482A  32 BIT PRODUCT: ($00FF0000) low byte of high word, defval:00
        32 BIT QUOTIENT:($00FF0000) low byte of high word, defval:00
$482B  32 BIT PRODUCT: ($FF000000) high byte of high word, defval:00
        32 BIT QUOTIENT:($FF000000) high byte of high word, defval:00
$482C  16 BIT REMAINDER: ($00FF) low byte, defval:00
$482D  16 BIT REMAINDER: ($FF00) high byte, defval:00
$482E  MUL/DIV SIGN: bit 0: on = signed calculation, off = unsigned calculation, defval:00
$482F  MUL/DIV FINISHED STATUS: bit 7: on = processing, off = finished,
            high bit is set after a write to multiplier or divisor regs $4825/$4827, defval:00
$4830  SRAM CHIP ENABLE/DISABLE: bit 7: on = enable, off = disable, defval:00
$4831  ROM BANK MAPPING A $D0:0000-$DF:FFFF, see bank mapping modes, defval:00
$4832  ROM BANK MAPPING B $E0:0000-$EF:FFFF, see bank mapping modes, defval:01
$4833  ROM BANK MAPPING C $F0:0000-$FF:FFFF, see bank mapping modes, defval:02
$4834  HARDWARE DECOMPRESSION ENABLE: bit 0: unknown, bit 1: on = enable, off = disable, defval:00
$4840  RTC CHIP ENABLE/DISABLE: bit 0: on = enable, off = disable, defval:00
$4841  RTC INDEX/DATA PORT:
            first write after rtc enable: rtc command mode byte (see rtc command modes)
            subsequent writes: index of rtc register to read/write (00-0f)
            read: returns value of indexed rtc register
            auto-increment of register index occurs after each subsequent read/write
$4842  RTC READY STATUS: bit 7: on = ready, off = still processing, tested before reading rtc data
            high bit cleared after successful read


Decompression Command Modes:
00 - manual decompression, $4800 is used to read directly and sequentially from the data rom
02 - hardware decompression, data gets decompressed mapped to $50:0000-$50:FFFF, $4800 can be used to read sequentially from the compressed data pointer address

Note: decompression mode is activated after writing to $4806 and finishes after reading the high bit of $480C


Data Rom Command Mode Byte:
X6543210
||||||||
|||||| \__ : 00 - use 1 as the offset increment value, add immediately after reading $4810
||||||    : 01 - use $4816 as offset increment, add immediately after reading $4810
||||||    : 10 - use $4814 as offset increment, see below for when to add
||||||    : 11 - unused
||||||____ : 0 - unsigned calculation for $4816
|||||        1 - signed calculation for $4816
|||||_____ : 0 - unsigned calculation for $4814
||||        1 - signed calculation for $4814
||||______ : 0 - offset increment gets added to $4811/2/3            
|||          1 - offset increment gets added to $4814/5
| \_______ : 00 - disable offset addition
|          : 01 - 8 bit offset addition using $4814, immediately after writing to $4814/5
|          : 10 - 16 bit offset addition using $4814/5, immediately after writing to $4814/5
|          : 11 - 16 bit offset addition using $4814/5, only after reading $481A
|_________ : unused

Note: the data rom command mode is activated only after registers $4814 and $4815 have been written to, regardless of the order they were written to


Bank Mapping Modes Values for $4831/2/3:
00 - use data rom 000000-0FFFFF for mapping
01 - use data rom 100000-1FFFFF for mapping
02 - use data rom 200000-2FFFFF for mapping
03 - use data rom 300000-3FFFFF for mapping

Note: the data rom areas are mirrored according to rom size, i.e. the value 02 for a 24mb rom means use data rom 000000-0FFFFF


RTC Command Modes:
03 - normal sequential read/write mode
0c - sequential read/write mode 2
other commands unknown as of yet


RTC Registers (8 bit values):  [default values]
00 - seconds 1's digit                00
01 - seconds 10's digit                00
02 - minutes 1's digit                00
03 - minutes 10's digit                00
04 - hours 1's digit                  00
05 - hours 10's digit                  00
06 - day of month 1's digit            01
07 - day of month 10's digit          00
08 - month 1's digit                  01
09 - month 10's digit                  00
0a - year 1's digit                    00
0b - year 10's digit                  00
0c - day of week                      00
0d - control register                  01
0e - control register                  0F
0f - control register                  06


RTC Control Register Bits:
Register 0d
XXXXXX10
      ||_ : 0 - normal timer operation
      |    1 - pause rtc registers updating, time still increments normally
      |__ : 0 - disable rtc interrupts
            1 - enable rtc interrupts

Register 0e
XXXX3210
    \\\|_ : unknown

Register 0f
XXXXX210
    |||_ : 0 - normal
    ||    1 - stops timer and resets seconds to 00 (date gets set to 01/01/00 00:00:00 ??)
    ||__ : 0 - normal timer operation
    |      1 - stop timer
    |___ : 0 - 12 hour time, bit 2 of "hours 10's digit" register contains am/pm bit (am=0,pm=1)
            1 - 24 hour time
The comments on 480A/9 should say auto-decrement not increment. (as even he hints in discussion of reg 4800)

Here's some more data (posted earlier in this thread) copied here for reference

Code: Select all

http://nesdev.com/bbs/viewtopic.php?p=34184#34184
Charles MacDonald:

About the FEoEZ memory map, I took my SNES memory map viewing program and adapted it to run on the Game Genie, so I could plug in the FEoEZ cart and examine it after my program relocated itself to work RAM. (and yeah, I added a wire to connect the missing clock signal that isn't passed through the Genie to the cart edge connector). 

What I got was this: 

Banks 
00-1F : SRAM @ 6000-7FFF, U1 ROM @ 8000-FFFF (1MB) 
20-3F : SRAM @ 6000-7FFF, U2 ROM @ 8000-FFFF (banked?) 
40-4F : Unused (reads return last value on data bus; e.g. bank address) 
50 : 64K SPC7110 internal RAM  
51-57 : Unused (as above) 
58 : SPC7110 data port 
59-7D : Unused (as above) 
80-9F : 8K SRAM @ 6000-7FFF, U1 ROM @ 8000-FFFF (1MB) 
A0-BF : 8K SRAM @ 6000-7FFF, U2 ROM @ 8000-FFFF (banked?) 
C0-CF : U1 ROM (1MB) 
D0-DF : U2 ROM (banked) 
E0-EF : U2 ROM (banked) 
F0-FF : U2 ROM (banked) 

The SRAM is the 8K RAM chip on the PCB. The internal RAM is, I assume, internal the SPC7110 because there is no other RAM on the board to account for it, and it makes sense the chip would have some place to store its results. 
As discussed at length earlier in this thread. bank $50 is not ram. It is just reg $4800 (although 480b settings may affect whether it is active or not, I forget at the moment). I don't know what he means about bank $58.

If anyone has more info please post.

Posted: Fri Jul 11, 2008 6:17 pm
by caitsith2
Im uploading the FULL raw packs, (no index.bins included, but the original data rom, is included, specifically for use with the decompressor. (The program rom is NOT included.)

So far, All of SPL4's mode 0 decompressions, and ALL of MDH mode 0 decompressions are BIT PERFECT. (Other than one isolated case in MDH, where the real hardware decompressor crashed, causing a repeated byte to be output for the rest of that 32KiB run. I have seen that crash before. I bet if I redumped that exact entry, it might be BIT PERFECT as well.)

I believe 100% of the mismatches in the feoez-sjns raw packs, are way past the normal requested size for each of the entries, as are 100% of the feoez mismatches that are past 25 bytes.

ETA to upload completion is about 1 hour 20 minutes from the time of this post.

Posted: Fri Jul 11, 2008 6:42 pm
by Near
Thank you, neviksti. I'll get started, then.
I strongly suggest figuring out how to use your copier to play with these cartridges. If need be you can always disable the CIC chip, load a program into ram with the copier, remove copier and insert cartridge.
Hmm, I have an FEoEZ cart, and two SNES units. I'd be willing to try and perform this mod, though I'm 98% certain I'll just ruin hardware. What exactly would I need to do to allow swapping the copier and cart while the system is running? The SNES CIC, the cart CIC, both, or possibly neither? I can Google if need be, but do you have a preferred document with pictures on how to make the mod by chance?

I should be able to write up a test program to let me strobe the cart's registers. I'll make it upload to RAM, then wait for a keypress. I can get it perfected through emulation, then just leave the SNES on for days until my testing is done, so as not to wear out the system via hardware swapping ...

Thanks again for everything. Getting the SPC7110 emulated was one of my main goals for getting involved in SNES emulation, so you've really made my day/week/month here :)

EDIT: hmm, looks like I want to disconnect pin 4 on the SNES console's CIC to turn it into a "key" chip. Wonder if the SPC7110 is like the SA-1 and S-DD1, where it can detect that and stop working if so ... hope not. But, I have two consoles anyway, worth a try I guess.

I don't have desoldering braid, so I guess I'll try using a needle to break the pin off myself, if the surface mount isn't too tiny ...

Posted: Fri Jul 11, 2008 7:15 pm
by caitsith2
The SPC7110 is not like the SDD1 or SA1. It can not detect the lack of the CIC chip, in fact, the cartridge has a CIC chip like most others.

The one other thing you have to mod besides CIC, is the plastic power switch setup, if you are going to have the cover on the system. (Remove the power switch cartridge lock.)

Posted: Fri Jul 11, 2008 7:33 pm
by Near
Alright, I successfully broke off pin 4 on the F411A chip. I started counting at 1, as per the PCB's numbering.

It was a clean break, and most of the metallic trace on the board came off with it. The other pins do not appear to be affected.

I'm still able to boot up both FEoEZ and Star Ocean. I don't have a PAL game to test, though. So ... hopefully this worked. Only way to be sure is to write up a test to run from RAM.

As for the power switch thing ... not sure how to do that exactly, I'll just leave the case off and be sure to prop up the copier.

Posted: Fri Jul 11, 2008 7:41 pm
by caitsith2
For the top piece/power switch cart lock, disassemble to top half completely, and you will see what needs to be removed. Reassemble without the pieces that need removing. The lock specifically prevents removing the cartridge while power is on. Of course, this means it is also preventing insertion.

Oh yeah. Raw packs are now uploaded. Grab from http://caitsith2.net/spc7110/.

Posted: Fri Jul 11, 2008 9:29 pm
by caitsith2
Now uploaded a modification of Neviksti's test code, to use the datarom (conveniently included with the raw packs.), to check all 0x8000 bytes of each and every raw pack entry, that used compression type 0.

Now is the time to write an snes side version of the decompressor, to test these on real hardware, and at the same time, do types 1 and 2 compression next. I checked up on the stats, and of official compressed/decompressed data, Type 2 is more common, with Feoez getting 3411, sjns getting 3409, spl4 getting 2835 and mdh getting 764. Data type 1 is 314, 318, 601, and 4 respectively.

EDIT: Just did a decompression comparison run, NOT skipping types 1 & 2. By the looks of things, it seems that they are also QM-Coder compression as well, with different tables. Most type 1 or 2 decompressions fail very early, like on bytes 0-20. Here are some highlights of decompressions that go quite a bit farther.

Code: Select all

Type 1
Table 0D33D1-AA-1 read --- ****FAIL**** mismatch on byte 1266 - Error in either decompressor or Raw Data
Table 0D4DE4-49-1 read --- ****FAIL**** mismatch on byte 102 - Error in either decompressor or Raw Data
Table 0D4DE4-4D-1 read --- ****FAIL**** mismatch on byte 357 - Error in either decompressor or Raw Data
Table 0D4DE4-4F-1 read --- ****FAIL**** mismatch on byte 357 - Error in either decompressor or Raw Data
Table 0D5B98-6A-1 read --- ****FAIL**** mismatch on byte 449 - Error in either decompressor or Raw Data
Table 0D5B98-6B-1 read --- ****FAIL**** mismatch on byte 92 - Error in either decompressor or Raw Data


Type 2
Table 0D5B98-28-2 read --- ****FAIL**** mismatch on byte 22330 - Error in either decompressor or Raw Data
Table 0D5B98-3D-2 read --- ****FAIL**** mismatch on byte 16057 - Error in either decompressor or Raw Data
Table 0D5B98-9C-2 read --- ****FAIL**** mismatch on byte 700 - Error in either decompressor or Raw Data
Table 0D5B98-A3-2 read --- ****FAIL**** mismatch on byte 2244 - Error in either decompressor or Raw Data
Table 28B68A-5C-2 read --- ****FAIL**** mismatch on byte 1515 - Error in either decompressor or Raw Data
Table 28B68A-5D-2 read --- ****FAIL**** mismatch on byte 1007 - Error in either decompressor or Raw Data
Table 28B68A-5E-2 read --- ****FAIL**** mismatch on byte 499 - Error in either decompressor or Raw Data
Table 28B68A-5F-2 read --- ****FAIL**** mismatch on byte 52 - Error in either decompressor or Raw Data

Posted: Sat Jul 12, 2008 1:57 am
by Near
This is what I have so far ...

Image

$4820+ was easy enough, but registers $4810 - $481a are confusing the ever living fuck out of me. And here I thought PPU $2115 was overly complex ...

The notes above are hard enough to understand, but then SNES9x' source for them just goes straight to hell with ~5x the complexity. I'm not just going to blindly implement that bit blending mess with absolutely no source comments. Will have to try out my own tests on hardware to figure these regs out, I suppose ...

Anyway, I can run the second test before the first and get in-game with SPL4, so I should at least be able to test decompression out in-game once we have the other two evolution tables ready. No hurry, of course.