FDS Internal Checksum Question

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

User avatar
ShaneM
Posts: 353
Joined: Wed Apr 04, 2012 4:15 pm
Location: United States of America (USA)
Contact:

FDS Internal Checksum Question

Post by ShaneM »

I have a question about FDS checksums. It seems that 0x28-2A within FDS ROMs (non-headered) represent some sort of checksum, I assume possibly for integrity purposes. Though a game would run fine if these three bytes are wrong. NESDev wiki labels the 10 bytes in that area as unknown but they really do seem to be some sort of checksum at those three bytes I mentioned above. I have compared Zelda I's 2 revisions on the FDS with those bytes and they all differ at those 3 bytes. Also, build date at 0x1f and 0x2c differ, too. But I know what they represent. Revision # is at 0x14 and 0x2b. So if you have any info on 0x28-2A and what kind of checksum it is, please fill me in. Thanks.

PS. I have discovered that offset 0x2b represents a second build number of some sort. Not listed on the Wiki (the lone byte preceding the second date of the build when you add 1925 + the byte...the last of the 10 unknown bytes. Or the last byte under "Disk info block (block 1)" 10 Unknown).
User avatar
MottZilla
Posts: 2837
Joined: Wed Dec 06, 2006 8:18 pm

Re: FDS Internal Checksum Question

Post by MottZilla »

The checksums/CRCs for Disk files are absent from .FDS files. I'm not sure what, if any, useful information exists in the FDS header. Remember that the .FDS "roms" are an unofficial creation. The actual data on the Quick Disk has more information on it than is represented in our FDS "roms".
User avatar
ShaneM
Posts: 353
Joined: Wed Apr 04, 2012 4:15 pm
Location: United States of America (USA)
Contact:

Re: FDS Internal Checksum Question

Post by ShaneM »

MottZilla wrote:The checksums/CRCs for Disk files are absent from .FDS files. I'm not sure what, if any, useful information exists in the FDS header. Remember that the .FDS "roms" are an unofficial creation. The actual data on the Quick Disk has more information on it than is represented in our FDS "roms".
I agree with you, for the most part. Though it is still interesting, since the "HVC" string onward to KYODAKU (meaning approved in Japanese) is official within Nintendo's original format. I was not referring to the first 16 bytes within headered ROMs. I was referring to the official header. I think up to the boot read file code (byte $0F at 0x19 [unheadered]) within that block is necessary. But the bytes afterward, before the "Created date" is what I'm referring to, here. The 3 bytes before appear to be some kind of manufacturer checksum, though it is unnecessary. I just thought it would be cool if there was a way to find the algorithm to it to see the logic behind it. Right now, it seems to be completely random, which is why I think the last three bytes of that unused string is a checksum of some sort. Here is what I'm referring to, the 10 "Unknown" area on this: http://wiki.nesdev.com/w/index.php/Fami ... block_1.29

(Still I agree with you that emulation of FDS is missing GAP and CRC 'dummy' bytes.)

EDIT: I think I made a sure discovery, here. On the last set of unknown 9 bytes. The 7th one (after the 5 $FFs on any disk) is a second disk side numery system (not as in a Side B one, but I mean since the first one is "1 Disk number (first disk is actually $00)" on the nesdev wiki). Though this one does nothing, not even affect emulation, etc. I compared about 7 2-sided games and 3 1-sided (Zelda I and II, Mahjong Goku, Metroid I and II, Nazoraa Land Dai 2, Knight Move, Karate Champ, SMB1 and SMB2J) to be sure. Side A is always $00 at that 7th unused byte within those last 9 unknown bytes and Side B is always $01.

I hope I wasn't too confusing in what I was trying to get across in saying.
Sik
Posts: 1589
Joined: Thu Aug 12, 2010 3:43 am

Re: FDS Internal Checksum Question

Post by Sik »

It's possible the firmware is using that byte (this wouldn't matter in emulation since as long as disk reads are emulated it will work, it's all done in software). What happens usually if you try to boot from side B? Did you actually change the value of that byte to make sure it isn't ignored? (e.g. swap sides A and B)
User avatar
ShaneM
Posts: 353
Joined: Wed Apr 04, 2012 4:15 pm
Location: United States of America (USA)
Contact:

Re: FDS Internal Checksum Question

Post by ShaneM »

Sik wrote:It's possible the firmware is using that byte (this wouldn't matter in emulation since as long as disk reads are emulated it will work, it's all done in software). What happens usually if you try to boot from side B? Did you actually change the value of that byte to make sure it isn't ignored? (e.g. swap sides A and B)
Yes. I actually replaced everything under the "Boot read file code" with $FFs. Those bytes under that boot code (represented by $0F in most games) are really nothing more than header info, up until the KYODAKU. When I switched disk sides using a double-sided game like The Legend of Zelda, it had no effect whatsoever on the game or within emulation. Byte 0x15 (Side number) in the header is responsible for disk sides within the game, and for which disk side is loaded. (When I say header, I always refer to the official Nintendo header. NOT the 16 byte header unofficially attached at the beginning of the ROM image.) It would seem that the 7th byte in that last unused string was really for the purpose of the assemblers. It seems that back then, assembly files were stored on different diskettes and then produced and assembled together at the end. Maybe that was a personal check, since both Sides A and B contain an internal header.

Now, I think I've sort of figured out what those 10 unknown bytes are (from the link I posted above, right after manufacturer permit date and right before created date) within the header. It seems that ALL FDS games have one thing in common with those 10 bytes. The first 5 of those bytes are always going to be $49, $61, $00, $00, $02 within all licensed games. In hex they spell "Ia". Since what's right before and what's following are dates, I believe that these 10 bytes are some sort of ID that any licensed FDS game must have (for integrity purposes). The first five bytes of that sting must be some sort of architecture while the last 5 bytes are the actual ID. Here, take a look at The Legend of Zelda 2 on the FDS. I'll be comparing v1.0 with v1.1:

===========================
Game: Legend of Zelda 2, The - Link no Bouken (Japan) (v1.0) - The internal name is LNK $20
Permit date within the header: 1987, January 14th ($62, $01, $14 in hex this date is in BCD so add 1925 + $62 for the year, according to the nesdev wiki)
10 Unknown bytes: $49 $61 $00 $00 $02 $00 $25 $02 $18 $00 (first five bytes are Ia string, which ALL licensed FDS games have in common)
Created date: 1987, January 14th ($62, $01, $14 in hex this date is in BCD so add 1925 + $62 for the year, according to the nesdev wiki)

Now let's take a look at v1.1, which was released less than a month later:

Game: Legend of Zelda 2, The - Link no Bouken (Japan) (v1.1) - The internal name is LNK $20
Permit date within the header: 1987, February 13th ($62, $02, $13 in hex this date is in BCD so add 1925 + $62 for the year, according to the nesdev wiki)
10 Unknown bytes: $49 $61 $00 $00 $02 $00 $26 $02 $19 $00 (first five bytes are Ia string, which ALL licensed FDS games have in common)
Created date: 1987, February 13th ($62, $02, $13 in hex this date is in BCD so add 1925 + $62 for the year, according to the nesdev wiki)

So let's compare the two set of unknown bytes within both revisions, released just 1 month from each other:

v1.0 = $49 $61 $00 $00 $02 $00 $25 $02 $18 $00
v1.1 = $49 $61 $00 $00 $02 $00 $26 $02 $19 $00

See how similar they are? The $25 and $18 went up by 1. Though this is not the case with all revisions. But I think the five bytes following that "Ia" string is some sort of internal ID.

Here are some more games to compare:

Donkey Kong Jr.: .........$49 $61 $00 $00 $02 $06 $75 $04 $84 $01
Nazoraa Land Dai 2 Gou: $49 $61 $00 $00 $02 $06 $71 $04 $80 $01
Super Mario Bros.:........$49 $61 $00 $00 $02 $06 $59 $04 $64 $01

Legend of Zelda I v1.0:...$49 $61 $00 $00 $02 $00 $26 $00 $18 $00
Donkey Kong:..............$49 $61 $00 $00 $02 $00 $45 $00 $43 $00
SMB2J:......................$49 $61 $00 $00 $02 $00 $1b $00 $97 $00

There are more patterns like this, but I just listed some of them. Notice that the 6th, 8th and 10th bytes are all also similar in both pattern sets. There are more 6th 8th and 10th byte differences but they always have a pattern. So, what do you think? Do you see this as an internal manufacturer ID like I do?
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: FDS Internal Checksum Question

Post by koitsu »

Shanem contacted me in PM about this. I've found some Japanese site that describes the block format in verbose detail (all bytes from what I can tell), so I'm having it translated and will post it here.
User avatar
ShaneM
Posts: 353
Joined: Wed Apr 04, 2012 4:15 pm
Location: United States of America (USA)
Contact:

Re: FDS Internal Checksum Question

Post by ShaneM »

koitsu wrote:Shanem contacted me in PM about this. I've found some Japanese site that describes the block format in verbose detail (all bytes from what I can tell), so I'm having it translated and will post it here.
You are awesome. Thanks, Koitsu!

Maybe then we can also add it to the Nesdev wiki, too.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: FDS Internal Checksum Question

Post by koitsu »

Sources:
This is kind of a combination of the 3 docs, and done very quickly. The 2nd doc has some kind of encoding or line break problem as well. If put into a Wiki doc or something formal, I would format this way better, particularly using offsets and sizes on the left, and the description on the right. But anyway...

Code: Select all

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−
GAP
DATA: $80
BLOCK CODE: $01, 1 BYTE
CHECK CODE: "*NINTENDO-HVC*", 14 BYTES
MAKER/MANUFACTURER CODE: 1 BYTE
GAME NAME: 4 BYTES
GAME VERSION: 1 BYTE
DISK SIDE: A=$00, B=$01, 1 BYTE
DISK ORDER (DISK NUMBER): USED BY GAMES WITH MORE THAN ONE DISK, 1 BYTE
DISK TYPE: YELLOW=$00, BLUE OR GOLD=$01, 1 BYTE -- OR -- ERR.9: EXTRA DISK NUMBER, 1 BYTE
NUMBER OF FIRST FILE READ: 1 BYTE -- OR -- ERR.10: EXTRA DISK NUMBER, 1 BYTE
BOOT READ FILE CODE: 1 BYTE
  -- The description of this byte is lost due to file formatting or encoding mistakes.
  -- The source material has line breaks where there should be proper characters, and it
  -- just so happens that the mistakes happen at the most important part of the sentence.
  -- All I was able to get:
  -- Defines the file number (possibly file index or file code?) to be read at boot time.
  -- There's a 2nd paragraph after this that is difficult to translate due to formatting mistakes
  -- or file format/encoding problems.
UNKNOWN: 5 BYTES
MANUFACTURING DATE: 3 BYTES
  -- Recorded using BCD encoding, year is from the Showa era/period (starts at 1925)
  -- Format is YEAR, MONTH, DAY.  See nesdev wiki for an example (although year 2010 isn't part of the Showa period! Ha ha ha!)
  -- For info on the Showa era, see Wikipedia
UNKNOWN (ALWAYS THE SAME VALUE): $49, $61, $00, $00, $02, 5 BYTES
REWRITTEN DISK DATE: 3 BYTES
  -- Speculate that this may be an "updated" date for MANUFACTURING DATE above, used when the
  -- game manufacturer releases an updated version of their game.  Ex: first release would have the
  -- same values for MANUFACTURING DATE and REWRITTEN DISK DATE, but a subsequent release
  -- (bugfixes, etc.) would have a newer REWRITTEN DISK DATE.
  -- Same format as above.
UNKNOWN: 1 BYTE
UNKNOWN: #$80, 1 BYTE
DISK WRITER SERIAL NUMBER: 2 BYTES
UNKNOWN: 1 BYTE
TIMES REWRITTEN: WRITTEN IN BCD, 1 BYTE
ACTUAL DISK SIDE (0=SIDE A, 1=SIDE B), 1 BYTE
UNKNOWN: 2 BYTES
CRC: 2 BYTES

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−
GAP
DATA: $80
BLOCK CODE: #$02, 1 BYTE
NUMBER OF FILES: 1 BYTE
CRC: 2 BYTES
−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−
GAP
DATA: $80
BLOCK CODE: #$03, 1 BYTE
SERIAL NUMBER: 1 BYTE
LOAD NUMBER: 1 BYTE
FILE NAME: 8 BYTES
LOAD ADDRESS: 2 BYTES (ORDERED FROM BOTTOM TO TOP)
FILE LENGTH: 2 BYTES (ORDERED FROM BOTTOM TO TOP)
FILE TYPE (0-2): $00=PROGRAM, $01=CHARACTER, $02=LICENSE FILE, 1 BYTE
CRC: 2 BYTES
−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−
GAP
DATA: $80
BLOCK CODE: $04, 1 BYTE
PROGRAM (DATA) MAIN BODY
CRC: 2 BYTES
−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−
User avatar
ShaneM
Posts: 353
Joined: Wed Apr 04, 2012 4:15 pm
Location: United States of America (USA)
Contact:

Re: FDS Internal Checksum Question

Post by ShaneM »

I have a question. Right after the "UNKNOWN (ALWAYS THE SAME VALUE): $49, $61, $00, $00, $02, 5 BYTES" and before the disk rewritten date, where are the other 5 unknown bytes I assumed are some sort of ID? But thank you for the documents. I am grateful for your time. I'm just a little confused. Maybe I read it wrong.
User avatar
freem
Posts: 176
Joined: Mon Oct 01, 2012 3:47 pm
Location: freemland (NTSC-U)
Contact:

Re: FDS Internal Checksum Question

Post by freem »

I found a version 0.4 of the first linked document (updated in what looks to be 2011) here. Doing a quick diff between the two files, it seems there's a lot more info in 0.4.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: FDS Internal Checksum Question

Post by koitsu »

That's a significantly updated doc, but any time you see 不明 it means "unknown".
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: FDS Internal Checksum Question

Post by koitsu »

Shanem wrote:I have a question. Right after the "UNKNOWN (ALWAYS THE SAME VALUE): $49, $61, $00, $00, $02, 5 BYTES" and before the disk rewritten date, where are the other 5 unknown bytes I assumed are some sort of ID? But thank you for the documents. I am grateful for your time. I'm just a little confused. Maybe I read it wrong.
No, you didn't -- the chat window with my translation colleague did not automatically scroll so I missed a bunch of lines. I've updated my previous post to reflect the missing data. Good catch -- thank you (and further points that organising this into a proper Wiki table would be worthwhile. I can do that some time if/when I have the time).
User avatar
ShaneM
Posts: 353
Joined: Wed Apr 04, 2012 4:15 pm
Location: United States of America (USA)
Contact:

Re: FDS Internal Checksum Question

Post by ShaneM »

I figured out what "UNKNOWN (ALWAYS THE SAME VALUE): $49, $61, $00, $00, $02, 5 BYTES" means from reading freem's link. Those bytes are always the same because those 5 bytes represent country code. (All FDS licensed games were released in Japan.) This is why they always started with "Ia". (I translated using Google Translate to figure this out.)

Here is the Japanese:

 国コード            : 1バイト $49=日本
 不明              : 1バイト $61 地域?


Though the second one claims to be unknown, the first one is definitely defined: "Country Code: 1 byte = $ 49 Japan" (The second unknown byte, $61 is thought to be area code. I'm not able to translate the next byte $00, though it adds a little more than 'unknown')

On to the last 5 bytes, the thing I've been wrestling with. This is what I got from Google Translate, using the newer 2011 doc:

不明(各ゲームの情報?)    : 5バイト = Unknown (Information of each game?) ; 5 bytes

I still feel Google Translate was a little less accurate with that last one. So if anyone knows Japanese, please fill me in on what the last one says instead of "Information of each game". I still think it is an ID of some sort.

EDIT: At least, I think that site agrees that it is not a checksum, unless the translator on Google was inaccurate.

EDIT2: I got to thinking that since $49 $61 spell "Ia" maybe that is the whole country code instead of it being limited to byte $49. I read on Google that Japan can be represented by either JP or JA. (This is just a theory.) But at least we know that $49 is country code. My theory is that Japan in Japanese makes a 'Y'/'I' sound when transliterated??? (Nippon is the word in Japanese for Japan. So if a Japan native tried to say 'Japan', maybe it'd sound like Y/Iapan?) I also guess they think the next byte means area because it starts with an 'a' ($61). Though I still stand by that string standing for Japan.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: FDS Internal Checksum Question

Post by lidnariq »

User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: FDS Internal Checksum Question

Post by koitsu »

How does this work for folks?

http://wiki.nesdev.com/w/index.php/User ... rmat_stuff

If folks like it, and it looks correct, I can move it into the main FDS wiki page. I have a list of the manufacturer codes which I can put up there as well (just haven't done it yet).

Tomorrow I'll bother the couple people I know who do the translation work to see if the "Price" field can be explained more clearly. I'm pretty sure (given what I know of Japanese) I understand what's being conveyed, especially in the case of rewritten/reformatted games, but I'd rather an accurate translation be provided.

That said: the translation of "不明(各ゲームの情報?)" is correct, by the way. The author simply does not know what the bytes represent and is speculating (based on analysis of other FDS images) that they values may represent some kind of game-specific data but he does not know what. The same doc has a lot of other speculative stuff in it, some of it justified/backed up in a way, but other parts are purely speculative.

The bottom line is that at this time nobody seems to know what those bytes represent. I'm sure if you could corner someone at Nintendo Co. who helped design the FDS they could probably tell you what they're for, but good luck with that. :P

Respectfully, I'd recommend simply unplugging the OCD chip in your brain for a while. Meaning don't worry/fret over 5 bytes in a file when (as far as we know so far) has no real impact on the behaviour of the game itself. These are proprietary formats that had no real documentation provided or disclosed anywhere (that we know of) publicly. Possibly someone at one of the companies who made FDS games would have an old Nintendo game submission sheet that might contain some of the fields on it (similar to the SNES/SFC game submission sheets), but who knows.

And finally, about the Ia (that's capital-EYE lowercase-A) thing and all this weird speculation about that it's a representation of Japanese transliterated speech: absolutely not. Not even remotely. It's much more plausible that the value just represents a particular country code. Don't ask me why $49 is Japan. There may be some reason behind it but I simply don't know. For sake of comparison, the SNES/SFC country code designator in the ROM header has $00=Japan, $01=North America, $0D=Korea, $11=Australia, etc. and the reason $00 is Japan is because that's how it was on the Gameboy ($00=Japan, $01=Everywhere else). It's just a number. Hell, for all we know $49 could be the lower 2-digit birth year of some particular Nintendo employee who worked on the thing (no it's not Masayuki UEMURA, he was born in 1943, nor Hiroshi YAMAUCHI (born in 1927)). I'm not so sure I even believe that $49=Japan, since the FDS was never released outside of Japan to begin with, so why would they just start the number at $49? I even checked things like EUC tables to see if maybe $4961 would turn out to be 日本 but it doesn't (it's $C6FC $CBDC).
Post Reply