proper archival format discussion on bannister.org

[hap]

This post is a notification that a new discussion was started about a proper archival format for NES games, so if you're registered on the bannister.org (Nestopia) forums, join in: http://www.bannister.org/forums/ubbthre ... 717&page=1

[gannon]

Just did a quick read through, and some of the ideas talked over there seem a bit ridiculous to me.
If they really wanted a *proper* format, they could put a header, prg, and chr in a zip with the extension znes or something. This could be used for the double method of using a database based on the prg/chr hash and then falling back on the header if wanted I suppose, but the only reasons for database vs. header would be for people with bad roms that had improper/modified headers as far as I see it.

[Bregalad]

To me, they even seems totally ridiculous.
Headerless files cannot be used with the NES at all, because of all the different carts existing. And the idea to put separate PRG/CHR in a ZIP file is S T U P I D because :
- ZIP is an universal format, not a NES secific format.
- Even trough PRG and CHR are separate, they always goes together on a cartridge.
- Impossible to compile a file in a zipped archive directly for homebrewer
- Impossible to romhack the game without unzipping, modifying, rezipping. This include graphics hack as well, because you wouldn't be able to see the graphics of games with a tile editor unless it has built-in zipper and unzipper.
- Impossible to directly compare both revisions of a same ROM with a hex comparer.

I don't see any problem with iNES, exept for it's lack of information about WRAM sizes and the big mess with higher mapper numbers. Both aren't the fault of the iNES format : The SRAM size because the people who invented iNES didn't know that they were game with other size than 8kb and 0kb back at the time, and the big mess with mappers because proper mapper documenting and number assigning has been a mess for a while, wich isn't the fault of iNES.
iNES 2.0 fixes all problems by iNES, exept the mapper mess, wich I guess is unfixable at some point. There will even be supmapper mess in addiction with a bit of luck.

[tepples]

To me, they even seems totally ridiculous.
Headerless files cannot be used with the NES at all, because of all the different carts existing. And the idea to put separate PRG/CHR in a ZIP file is S T U P I D because :

Why? Is it stupid because MAME does it?

- ZIP is an universal format, not a NES secific format.
- Even trough PRG and CHR are separate, they always goes together on a cartridge.

They aren't concatenated in the cartridge. As I understand it, the rule in MAME is one file for each ROM chip.

- Impossible to compile a file in a zipped archive directly for homebrewer

That's a one line change to the makefile: replace cat with zip.

- Impossible to romhack the game without unzipping, modifying, rezipping.

Archival != hacking. It's already impossible to romhack Nintendo DS games without breaking apart the .nds, patching files, and rebuilding the .nds.

This include graphics hack as well, because you wouldn't be able to see the graphics of games with a tile editor unless it has built-in zipper and unzipper.

Zip files appear as a folder in Windows. User drags files in and out of the zip file as he would a folder.

And does NES 2.0 allow for samples in Famicom games that use a third ROM for samples (such as Moero Pro Yakyuu)?

[Bregalad]

MAME emulates not just one system, but many systems.

That's a one line change to the makefile: replace cat with zip.

No, because you'll get a unzipped file named ".zip". You'd have to place your PRG (and possibly CHR) file in a new archive each time.

Zip files appear as a folder in Windows. User drags files in and out of the zip file as he would a folder.

Yes, but it'd be annoying to have to drag 'n' drop your files each times you want to see them in a HEX heditor, see their graphics, or just modify them.

And does NES 2.0 allow for samples in Famicom games that use a third ROM for samples (such as Moero Pro Yakyuu)?

This is mapper specific. The mapper allows it to play samples, not just the ROM chips by themselves.

Anyway, regardless of what anyone say, the gaming comuunauty will never switch to another format by themselves, so I guess iNES 2.0 should be used for game having trouble running with the old iNES.

[commodorejohn]

Bregalad is right. MAME gets away with this because it emulates a huge number of systems with largely incompatible ROM arrangements. As much of a pain as it is to get a game and find one file is missing out of the ZIP (*cough*Kiki Kaikai*cough*) it saves a lot of pain for the emulator coders. NES, on the other hand, has at most two types of ROM, which are always (to my knowledge) separate, and can thus be smushed into one file without much trouble. We really don't have anything to gain by going to zipped-ROM format - each ZIP would contain about two files. Three if we add header information, which brings me to my next point. MAME emulates a fixed number of games, each of which is known, and therefore can have a set board assigned to it simply by the filename. NES, on the other hand, has new games that are constantly being discovered, a number of which even use their own mappers. It's bad enough having to update one's emulator every time a new mapper is discovered - imagine updating every time a new game is discovered. Not only would this slow down emulator development terribly, it would make homebrew a huge pain in the ass.

In conclusion, this might work for MAME but it's a horribly stupid and pointless idea for NES.

[tepples]

MAME emulates not just one system, but many systems.

MESS? Mednafen? Besides, didn't Pasofami use split ROMs?

That's a one line change to the makefile: replace cat with zip.

No, because you'll get a unzipped file named ".zip".

The one-line change I meant was the following:

Code: Select all

#Replace this
cat inesheader.bin mygame.prg mygame.chr > stuff.zip
#With this
zip stuff.zip mygame.json mygame.prg mygame.chr

The Info-ZIP Zip software is included with many operating system distributions; if not, it is free software under the zlib license, ported to many operating systems including Windows.

You'd have to place your PRG (and possibly CHR) file in a new archive each time.

The point is that this step can be done by Make every time you rebuild, and it's no harder than the step to build an iNES file.

And does NES 2.0 allow for samples in Famicom games that use a third ROM for samples (such as Moero Pro Yakyuu)?

This is mapper specific. The mapper allows it to play samples, not just the ROM chips by themselves.

But you still need to store a dump of the audio in the ROM chip connected to the mapper when the header indicates that such a dump is needed.

NES, on the other hand, has at most two types of ROM, which are always (to my knowledge) separate, and can thus be smushed into one file without much trouble.

Famicom has three: PRG, CHR, and the audio ROM that forms part of the mapper. Distributing this audio ROM with every copy of every emulator is both infringing and inefficient; therefore, it must be distributed with the ROM file. Besides, even a loose Game Pak is more than what is soldered onto its PCB; there's also a case around the PCB, and serious archiving needs to have a way to represent the label on this case. Given that many emulators already have support for iNES format ROMs in zip files, would it be that hard to introduce support for split ROMs with a new header in zip files?

[commodorejohn]

MAME emulates not just one system, but many systems.

MESS? Mednafen? Besides, didn't Pasofami use split ROMs?

And look at how many people use the PasoFami format...

Besides, adoption of an obnoxious format for "serious archiving" is only going to lead to the creation of conversion utilities to make a NES ZIP into an iNES file. Nobody's going to use it. As I said, there's really nothing to gain by doing this - iNES needs some retooling, sure, but the basic concept (one file containing all the ROM dumps) is fine - sample ROMs, on the few games that use them, can be supported by appending the ROMs to the file after the CHR-ROM, if the format were just slightly extended.

Also, who does "serious archiving" of NES games? MAME does that because of cabinet art and such and because they're obsessive-compulsive completionists. Nobody getting NES ROMs really cares about anything besides the game and a text transcription of the manual where necessary.

[gannon]

Wouldn't be too hard to make a converter to the new proposed format, but the hard part would be getting people to agree on one (I still think that the current nes 2.0 is good enough).
Here's a quick untested example script. It doesn't do anything to the header data though, and I have no clue if the prg & chr would even come out right

Code: Select all

<?php
if (!file_exists($argv[1]) || $argc != 2) {
echo 'Usage: ./make_znes.php {ROM FILE}'."\n";
die(1);
}
function write_file($file, $string) {
 if ($string == '') return;
 $fp = fopen($file, 'wb');
 $status = fwrite($fp, $string);
 fclose($fp);
 return $status;
}
$dir = substr($argv[1], 0, strrpos($argv[1], '.'-1));
$fp = fopen($argv[1], 'rb');
$header = fread($fp, 16);
$prg_banks = substr($header, 4, 1);
$chr_banks = substr($header, 5, 1);
$prg = fread($fp, $prg_banks * 16 * 1024);
$chr = ($chr_banks != 0) ? fread($fp, $chr_banks * 8 * 1024) : '';
fclose($fp);
mkdir($dir);
write_file($dir.'/header.bin', $header);
write_file($dir.'/prg.bin', $prg);
write_file($dir.'/chr.bin', $chr);
exec('zip -1 -r '.escapeshellarg($dir).'.znes '.escapeshellarg($dir));
unlink($dir.'/header.bin');
unlink($dir.'/prg.bin');
unlink($dir.'/chr.bin');
rmdir($dir);
?>

[Bregalad]

Besides, even a loose Game Pak is more than what is soldered onto its PCB; there's also a case around the PCB, and serious archiving needs to have a way to represent the label on this case.

Yes, and I think that's not too bad.
You get "what is soldered on the PCB" trough a NES ROM for free, but if you want the real game, then you have to find one and pay. I think this is good that real hardware has something better than plain emulation.
Also, ROMs are usefull for playing and/or collecting the games themselves, with nothing arround them such as artwork or whathever. There is plenty of websites arround that collects pictures of box and carts, such as Moby Games.
There is also on the internet people who published scans of whole manuals, or that typed it down in a text file.

[kyuusaku]

Hmm I don't have a problem with split roms in a container, it's certainly better than a database... If this had to be done, it would be nice to have a zip file containing a "PRG" file or few, a "CHR" file if applicable, a "BOARD"/"PCB"/"MMC"/something (which is only plaintext board name) and other files for whatever else is appropriate for that game. (Just not TOO much detail)

[Bregalad]

I wouldn't see how to do this file while avoiding details. Perfect emulation may describe all non-roms chip present on the PCB and how they are wired up (and it's up to the emulator to emulate the chips), and how ROMs are wired up and wich file they are associed too. That'd be compltetely crazy, but damn accurate.

[kyuusaku]

Why don't all emulator authors just implement a HDL interpreter so that mapper logic could be embedded into the ROM?

[tepples]

Why don't all emulator authors just implement a HDL interpreter so that mapper logic could be embedded into the ROM?

Because most of us don't have quad core PCs yet, or many of us want to make emulators run on handheld devices.

[Jagasian]

Why don't all emulator authors just implement a HDL interpreter so that mapper logic could be embedded into the ROM?

Even if you aren't serious, that would the logical ideal for "ripping" the contents of a cart to a pure software format. Anything short of that is really just a form of compression. For example, if every NES game's PRG ROM started with one of one hundred different 128 byte patterns, would it make sense to assign each pattern a number from 0 to 99 and instead of doing a straight dump of the ROM, replace the first 128 bytes of the PRG ROM with its pattern ID?