nesdev.org

There's a few guys out there that has disassembled (and commented) entire NES-games. Metroid and Super Mario Bros comes to mind. Probably more exists.
This sounds like a huge task for a mortal like myself. Does anyone know, is there a special technique involved in this process or is it just people with too much time to spare that's doing this?
I'm just curious how they're doing it..

First thing to do is figure out where the variables are, and what they do. You can use a cheat finder for that, or just poke around, or freeze bytes in memory.
Once you know what the variables do, your disassembles now have symbol names instead of numbers, so you have a much easier time guessing what the surrounding code does.

What kind of tools would allow this?

EDIT: Just noticed FCEUX can create symbolic names. But is there a disassembler that can make use of this data?

Just write simple text replacement tools to substitute your variables for the addresses.

You should also play through the game thoroughly and create a code/data log before you use your disassembler.

I've uploaded latest disassemble of Battletoads.
https://code.google.com/p/feos-tas/sour ... isAssemble
Format is IDB (IDAPRO). For people who principal, can't get IDA leaked version, I've made different outputs: *asm , hmtl and something else.

And about techniques, for initial disasm you can use scripts, that video of using on my channel:
https://www.youtube.com/watch?v=KZC_s0fGzoQ
This scripts exists into folder 'scripts' on feos-tas svn, but it's for anrom, if you need mmc3 or some else you should edit them.
that was used for romhack "battletoads-dark queen rematch", and some others.

Then continue disasm by finding variables. (like lifes,coordinates). by cheat search.
And at last name functions.

Good luck!.

Bisqwit wrote an amazing tool he called clever-disasm as part of his nescom tools. It's a tracing disassembler that takes a simple specification language for hints when it gets confused.

For simple programs it does the job almost entirely by itself; for more complex ones I usually find I only need to add 30-50 hints to get a usable output.

Mapper support is somewhat limited, however, and some issues require source diving/changing.

Start with graphical output and work backwards. You can knock out a lot of code just looking at the system i/o accesses and following the trail until the data format becomes apparent (Disassemblies need to come with a data format bible - that helps understanding the code a lot more than scattershot commenting). Well-known compression methods like Huffman and lz77 are likely to show up in the code.

Would the one I made for Bananmos's music engine and the one I made for an NES FMV player be a good example of a data format bible?

Certainly. From the P-frame explanation, it sounds like the frame is stored in 4 bytes + (n changed rows * 2) + n changed 4x2 pixel segments.