Finding NES code that is never executed

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

Post Reply
mattjustice
Posts: 9
Joined: Mon Aug 26, 2019 3:59 pm

Finding NES code that is never executed

Post by mattjustice »

Does anyone have recommendations for a tool or approach for detecting NES PRG code that is never executed? Ideally, I'd like to run my game, play through it for a while, and have some kind of report generated that tells me the addresses of code that wasn't executed at all.

I know the MESEN debugger has a color option for "Unexecuted code", but even with that configured I don't see where to view the results.

MESEN also has a profiler, which is kind of the opposite of what I want (since it shows the code with the greatest execution time).

I realize I could just set breakpoints all over the place and see what doesn't get hit, but I'm hoping for something simpler.

Thanks in advance!
- Matthew Justice
User avatar
nesrocks
Posts: 563
Joined: Thu Aug 13, 2015 4:40 pm
Location: Rio de Janeiro - Brazil
Contact:

Re: Finding NES code that is never executed

Post by nesrocks »

I use fceux and the code/data logger. You need to absolutely know that you've played through the entire game, seeing everything the game has to offer to the smallest detail, secrets, etc. I don't think there's another way.
Be mindful that some games still have unknown cheats, so you may want to take a quick look at the code to see if there's any nearby branches, or worse: jump routines elsewhere.
https://twitter.com/bitinkstudios <- Follow me on twitter! Thanks!
https://www.patreon.com/bitinkstudios <- Support me on Patreon!
mattjustice
Posts: 9
Joined: Mon Aug 26, 2019 3:59 pm

Re: Finding NES code that is never executed

Post by mattjustice »

nesrocks - Thanks! I'll check out fceux's code/data logger. And yes, understood about being sure that every code path is executed. Fortunately, in this case, the game in question is my own - I wrote all the code in assembly. I'm just looking to find any code that was used earlier in the development process but has since become obsolete/unused.
User avatar
Individualised
Posts: 310
Joined: Mon Sep 05, 2022 6:46 am

Re: Finding NES code that is never executed

Post by Individualised »

If I remember correctly BizHawk has this feature, though I haven't personally used it (the feature, that is, I have used the BizHawk emulator before) myself.
Bavi_H
Posts: 193
Joined: Sun Mar 03, 2013 1:52 am
Location: Texas, USA
Contact:

Re: Finding NES code that is never executed

Post by Bavi_H »

mattjustice wrote: Mon Sep 05, 2022 5:27 amI know the MESEN debugger has a color option for "Unexecuted code", but even with that configured I don't see where to view the results.
The "unexecuted code" color seems to only be used for small sections of code that are skipped over by short branches:

unexecuted-code.png
unexecuted-code.png (3.64 KiB) Viewed 2656 times

For larger sections of code (like entire subroutines) that never get called, the "unidentified code/data" color will be used, but you also have to turn on the options to show those:
  • Options
    • Disassembly Options
      • Disassemble...
        • Unidentified Code/Data
      • Show...
        • Unidentified Code/Data
As those names suggest, if the code is never executed, Mesen can't tell if it's code or data. Turning these options on will show "code" for every address that hasn't been accessed, which can show a lot of stuff that isn't really code.
mattjustice
Posts: 9
Joined: Mon Aug 26, 2019 3:59 pm

Re: Finding NES code that is never executed

Post by mattjustice »

Individualised - Thanks! I'll take a look at BizHawk; I've never used it.

Bavi_H - Thanks for your help! Your explanation of "unexecuted code" makes sense. I looked at the "unidentified code/data" option earlier. It seems that anything with a label (I'm importing labels from my build) is considered "identified" (verified?), even if it hasn't been executed yet. I guess I could try not importing labels, but then I still would see a lot of non-code that is interpreted as code, as you mentioned. I suppose what I want is something that shows all identified/verified code that is never executed, and I'm not sure if that exists in MESEN.
User avatar
oRBIT2002
Posts: 687
Joined: Sun Mar 19, 2006 3:06 am
Location: Gothenburg/Sweden

Re: Finding NES code that is never executed

Post by oRBIT2002 »

Interesting topic! I've stumbled upon blocks of NOP's sometimes in some games and wondered what actually what was removed back then.. But it's slightly harder to find out.. :)
Oziphantom
Posts: 1565
Joined: Tue Feb 07, 2017 2:03 am

Re: Finding NES code that is never executed

Post by Oziphantom »

there was a demo that did this and it ended up with not being able to go up in the menu, or left in some cases in the game.. its a nice idea but it doesn't really work.

64tass has two methods to help you.
a.) you can ask it to tell you all unused labels. So if it finds a label that is never JSR/JMP or referenced in a .byte or .word directive it will tell you.
b.) you can make .proc out of each function and if nothing explictly calls it, it won't assemble it into the file. Basically used for making large libs, and this way it will only link in used functions.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Finding NES code that is never executed

Post by rainwarrior »

Mesen produces a code-data-log file in is Debugger folder. I'm not sure if the format is documented, but I think it's a simple format where each byte corresponds to a byte in the ROM, and is a bitfield indicating whether that byte was ever used as code, data, etc. It runs whenever the debugger is running. I don't think the contents of the CDL are influenced by imported labels, they're just a log of accesses, more or less. The debugger uses the CDL and other clues like labels to display whatever it's going to display...

FCEUX has a CDL option that you have to enable. Its format is documented.

Either way, you can parse the CDL file yourself and generate a map of what's used and unused. I think in both cases, a byte that was never accessed will be 0, so even without knowing much else about the format, that might be enough for your purpose.
mattjustice
Posts: 9
Joined: Mon Aug 26, 2019 3:59 pm

Re: Finding NES code that is never executed

Post by mattjustice »

Oziphantom - Thanks! Interesting tips for using 64tass. I'll check it out.

rainwarrior - The CDL idea may be just what I need. I'll take a look. Thank you!

Everyone - Thanks so much for the responses. I'm really impressed by the community's friendly, helpful engagement!
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Finding NES code that is never executed

Post by tepples »

Mesen's CDL format is documented in CodeDataLogger.h. It's very similar to FCEUX's.

Code: Select all

7654 3210  FCEUX and Mesen PRG ROM CDL entry
|||| |||+- Code
|||| ||+-- Data
|||| ++--- A14-A13 of most recent access
|||+------ FCEUX: Target of JMP (aaaa); Mesen: Target of any jump
||+------- Read indirectly using (dd,X) or (dd),Y
|+-------- Read by DMC
+--------- Mesen only: Target of JSR

7654 3210  FCEUX and Mesen CHR ROM CDL entry
       |+- Read by PPU during rendering
       +-- Read by PPU through $2007
Mesen-S's CDL format is documented in DebugTypes.h.

Code: Select all

7654 3210
|||| |||+- Code
|||| ||+-- Data
|||| |+--- Target of jump
|||| +---- Target of JSR/JSL
|||+------ Accessed with flag X=1 (8-bit index)
||+------- Accessed with flag M=1 (8-bit accumulator)
|+-------- Accessed by GSU coprocessor
+--------- Accessed by CX4 coprocessor
mattjustice
Posts: 9
Joined: Mon Aug 26, 2019 3:59 pm

Re: Finding NES code that is never executed

Post by mattjustice »

Thanks tepples for the additional info, and thanks for everyone who gave some input.

I found a working approach, using the CDL file from FCEUX coupled with the labels text file generated by ld65 at build time. I wrote a Python script to read the CDL file, looking for bytes that are never accessed (value of 0). I then matched those byte addresses to the nearest label for easier readability. Using this approach happens to also include data bytes too, but that's OK.

Here's my script if anyone is interested: https://github.com/matthewjustice/nes-unexecuted

Edit: I just did a quick test of MESEN's CDL file, and it works too, once I realized that I need to reset the CDL log in MESEN. In fact, after doing that, I'm now seeing more useful highlighting in MESEN. Sections that were marked as executed are now the "unexecuted code" color.
User avatar
aquasnake
Posts: 515
Joined: Fri Sep 13, 2019 11:22 pm

Re: Finding NES code that is never executed

Post by aquasnake »

This should be the optimization function of compilers and linkers, but this optimization must be abandoned on the NES platform because it requires a bank. After introducing the concept of bank, all the data stored in the non first bank space cannot be detected and unknown. The program executes by assuming a forced jump address, which cannot be optimized by the high-level language. Because many code spaces are overlapped, it is impossible to de duplicate or delete unused code (even if subroutines have never been called)

Emulators traverse the code by running the TAS script, and then mark the executed and unexecuted parts. TAS will always approach the whole, but never complete
Oziphantom
Posts: 1565
Joined: Tue Feb 07, 2017 2:03 am

Re: Finding NES code that is never executed

Post by Oziphantom »

if you are compiling then you are compiling all banks, and as such the compiler will be able to optimise across the entire program, and remove any unused functions or data regardless of which bank it eventually places it in.
mattjustice
Posts: 9
Joined: Mon Aug 26, 2019 3:59 pm

Re: Finding NES code that is never executed

Post by mattjustice »

Interesting discussion on the role of optimizing compilers! For what it's worth, my original question was in the context of a game written in assembly, so there's no compilation step - just assembling/linking.
Post Reply