NES 2.0 XML Database

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

NES 2.0 XML Database

Post by NewRisingSun »

Here is a complete (see note 1) database of NES 2.0 headers in XML format. The fields should be self-explanatory, for the most part:
  • Just "rom" is the entirety of the ROM file without header, including PRG-, CHR- and Misc ROM. It's what you look up when being confronted with a headerless ROM file (see note 2).
  • "prgram","prgnvram","chrram","chrnvram","chrrom","miscrom" tags are only present if the respective memory type is present.
  • "sum16" is a simple sum of all bytes in a particular ROM type, truncated to 16 bits; this value can be found in Nintendo's leaked internal spreadsheet.
  • "console type", "console region", "expansion type", "vs hardware" and "vs ppu" are just the direct values defined for the NES 2.0 header; I did not want to use text strings that have to be looked up.
  • "mirroring" can be:
    • For normal mappers:
      • "H", meaning that byte 6's LSB nibble is set to 0..0
      • "V", meaning that byte 6's LSB nibble is set to 0..1
      • "4", meaning that byte 6's LSB nibble is set to 1..0
    • For mapper 030's idiosyncratic use of the NES header's mirroring bits:
      • "H", meaning that byte 6's LSB nibble is set to 0..0
        "V", meaning that byte 6's LSB nibble is set to 0..1
        "1", meaning that byte 6's LSB nibble is set to 1..0
        "4", meaning that byte 6's LSB nibble is set to 1..1
    • For mapper 218's idiosyncratic use of the NES header's mirroring bits:
      • "H", meaning that byte 6's LSB nibble is set to 0..0
        "V", meaning that byte 6's LSB nibble is set to 0..1
        "0", meaning that byte 6's LSB nibble is set to 1..0
        "1", meaning that byte 6's LSB nibble is set to 1..1
Notes:
  1. The database should be complete for all dumped licensed and original unlicensed games, and will be continuously updated to include more homebrew games/hacks/translations, pirate dumps and plug-and-play consoles, as well as corrections. As it is created automatically from a master set of headered ROM files, I cannot process pull requests on the XML text data; instead, simply point out errors or omissions by posting replies to this thread.
  2. When comparing file hashes against the database's full "rom" hash, be sure to compute two hashes:
    • one that trusts and uses an existing header's ROM size fields to know how many bytes to hash,
    • one that ignores any existing header completely and just hashes the entire file sans header.
    Searching for the first hash value will fail if an otherwise complete ROM has a header with incorrect size fields. Searching for the second hash value will fail if the file has common trailing garbage at the end, such as "This file downloaded from website x". Searching for both hashes will maximize the chances of identifying ROM files with bad headers or trailing garbage bytes. ROM files that have simultaneously trailing garbage and incorrect sizes specified in the header, or no header at all and trailing garbage, cannot be identified without human intervention. ;)
Attachments
nes20db-2021-12-25.7z
(844.18 KiB) Downloaded 6617 times
Last edited by NewRisingSun on Sat Dec 25, 2021 1:00 pm, edited 10 times in total.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 XML Database

Post by NewRisingSun »

Update 2020-04-19 based on feedback. :)
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: NES 2.0 XML Database

Post by Dwedit »

What does Expansion type= mean?
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
User avatar
kitrinx
Posts: 3
Joined: Mon Feb 04, 2019 10:28 pm

Re: NES 2.0 XML Database

Post by kitrinx »

Thank you for this NRS. This will make it so much easier to batch repair collections of ROMs with janky headers.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 XML Database

Post by NewRisingSun »

Dwedit wrote: Sun Apr 19, 2020 1:47 pm What does Expansion type= mean?
The value of this field.
Great Hierophant
Posts: 780
Joined: Tue Nov 23, 2004 9:35 pm

Re: NES 2.0 XML Database

Post by Great Hierophant »

I now defer to this database, FWIW.
talbin
Posts: 4
Joined: Sat Apr 12, 2014 9:24 am

Re: NES 2.0 XML Database

Post by talbin »

What is the best way to apply this to a set? Is there also something like this for standard ines headers?
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 XML Database

Post by NewRisingSun »

A Python script has been written (not by me) for that purpose. As NES 2.0 is designed to be backwards-compatible to standard iNES headers, there is no separate database for that purpose.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 XML Database

Post by NewRisingSun »

The "name" tag has been removed, and my original filenames only included as XML comments.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 XML Database

Post by rainwarrior »

Thank you for doing this good work.
Ice Man
Posts: 547
Joined: Fri Jul 04, 2014 2:34 pm

Re: NES 2.0 XML Database

Post by Ice Man »

Awesome. Thanks alot!
KickMeElmo
Posts: 6
Joined: Fri Apr 24, 2020 2:24 am

Re: NES 2.0 XML Database

Post by KickMeElmo »

For your own information, I applied the 2020-04-22 xml to a 2020-04-22 no-intro set using the python script, and I've recorded the names of the unmatched entries in a text file. In addition to those, "Bible Adventures (USA) (v1.5) (Unl)" wasn't in the xml. Hope this helps.

https://telegra.ph/nes20db-misses-2020-04-22-04-24
Great Hierophant
Posts: 780
Joined: Tue Nov 23, 2004 9:35 pm

Re: NES 2.0 XML Database

Post by Great Hierophant »

KickMeElmo wrote: Fri Apr 24, 2020 2:33 am For your own information, I applied the 2020-04-22 xml to a 2020-04-22 no-intro set using the python script, and I've recorded the names of the unmatched entries in a text file. In addition to those, "Bible Adventures (USA) (v1.5) (Unl)" wasn't in the xml. Hope this helps.

https://telegra.ph/nes20db-misses-2020-04-22-04-24
For the (Asia) ROMs not in the database, the attached list should cover their header information for the meantime. For the Virtual Console and the Collection ROMs, I'm sure they can be figured out on their own.

Edit : Eliminated unused values for Poke Block.
Attachments
list.txt
(2.55 KiB) Downloaded 520 times
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 XML Database

Post by NewRisingSun »

Added many homebrew ROMs as well as some ROM hacks, and corrected the mirroring information on the Oeka Kids games. (NesCartDB incorrectly claims that mirroring is mapper-controlled, but it's not.)

Amended the description of the "mirroring" field to include the meanings in the two mappers 30 and 218 that use the NES header's mirroring bits idiosyncratically. Added a note on the pitfalls of hashing ROM files found "in the wild" with trailing garbage.
talbin
Posts: 4
Joined: Sat Apr 12, 2014 9:24 am

Re: NES 2.0 XML Database

Post by talbin »

Thanks for this great work! These Vs. came up as missing for me. Does it also make sense to add FDS?

ROM not found in database. SHA1: 611322F482CA236C42C7325D540DF7C906C1C330 File: Vs. Excitebike (set EB4-3 X).nes
ROM not found in database. SHA1: EC1BEA24B1F7E5A86006760EDF12B00A9922B3F5 File: Vs. Soccer (set SC4-3 X).nes
Post Reply