Is there an assembler that supports unit testing?

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

User avatar
SusiKette
Posts: 147
Joined: Fri Mar 16, 2018 1:52 pm
Location: Finland

Is there an assembler that supports unit testing?

Post by SusiKette »

I was writing unit tests at work today, which made me wonder if unit testing is currently possible to do in 6502. It would be definitely useful for keeping your code working right, especially when you need to edit something.
Avatar is pixel art of Noah Prime from Astral Chain
User avatar
Jarhmander
Formerly ~J-@D!~
Posts: 568
Joined: Sun Mar 12, 2006 12:36 am
Location: Rive nord de Montréal

Re: Is there an assembler that supports unit testing?

Post by Jarhmander »

Not that I'm aware, and frankly I'd be surprised to find one that does. However, with a suitable simulator, you can unit test your code.

I believe tepples has made such a simulator to achieve the same goal. Also, you can probably achieve the desired results with the cc65 suite: cc65+ca65+sim65. Create a main in C, call your subroutine in assembly (you may need to make some adapter stubs, to make your code correctly valable from C), link with your assembly code, then run through the simulator. You can even use stdio.h functions and print results, failures and exit with some code.
((λ (x) (x x)) (λ (x) (x x)))
User avatar
Movax12
Posts: 541
Joined: Sun Jan 02, 2011 11:50 am

Re: Is there an assembler that supports unit testing?

Post by Movax12 »

If you can run your code on an emulator that has Lua (or other?) scripting support that can access the machine state and allows calling script routines from 6502 code at key points you could probably do what you want. It might be nice if the script could set values and call your 6502 routine as well.
I am not sure if you can do that with Mesen, but you can with NintendulatorDX.
User avatar
Memblers
Site Admin
Posts: 4044
Joined: Mon Sep 20, 2004 6:04 am
Location: Indianapolis
Contact:

Re: Is there an assembler that supports unit testing?

Post by Memblers »

cc65 comes with sim65, if that helps. I've never tried to use it, myself.
https://cc65.github.io/doc/sim65.html
User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Is there an assembler that supports unit testing?

Post by rainwarrior »

I do find sim65 very useful for unit testing.

Example: https://github.com/bbbradsmith/huffmunc ... anger/test

The cc65 project had used it internally for this purpose for years without it being very well documented. I contributed a bit to the current documentation to help try and improve that situation, because I think it's a really great tool. Basically you can write normal C stdio command line apps in cc65 and run them through sim65. Assembly-only programs are possible too. I think it has other applicable uses besides just unit testing.

It can also be used to count cycles, in case you need an automated way to measure performance. I used that to test a variable cycle delay project.
User avatar
dougeff
Posts: 3078
Joined: Fri May 08, 2015 7:17 pm

Re: Is there an assembler that supports unit testing?

Post by dougeff »

If timing / efficiency is important...

MESEN has a "performance profiler" tool that tells you the average cycles of each function that is called.

Put the test code in a subroutine.

Have the main loop call it every frame.
nesdoug.com -- blog/tutorial on programming for the NES
Oziphantom
Posts: 1565
Joined: Tue Feb 07, 2017 2:03 am

Re: Is there an assembler that supports unit testing?

Post by Oziphantom »

unit testing is not a feature of the assembler, as a unit test suite is not a feature of a compiler.

Martin Piper and I use a full BDD suite with Gherkin support here https://github.com/martinpiper/BDD6502 you will want to stick to the Overclocked 6502 and ignore the WANG 2000 stuff. you can enable tracing between sections to help debug where you code goes wrong etc
Drag
Posts: 1615
Joined: Mon Sep 27, 2004 2:57 pm
Contact:

Re: Is there an assembler that supports unit testing?

Post by Drag »

I can think of a few ways to do it; the first way would be to assemble your rom image like normal, but instead of the usual entry point into your game, the entry point is an extra "trainer" of sorts which contains the individual tests to run, along with some basic library stuff to print the tests and results to the screen, or some other kind of port you can read from. This could be nice because your tests would run on the actual NES hardware (or the emulator at large).

Another way would be to write your tests in some other language (like a scripting language), and then you could either assemble your subroutines individually and run your tests that way, or some kind of just-in-time assembler can scan your source code for markers you insert yourself (though either way, you'd need something which can run assembled 6502 code). This would be more for checking your 6502 code for correct outputs given inputs, but seems like it'd be much easier to run tests with large swaths of data, probably more conveniently than option 1 would, but option 1 could give you a better idea for performance since page boundaries (and other 'invisible' things like that) are preserved.


Anyway, I'm not aware of any kind of test runner or testing framework for 6502 or NES homebrew code, but these are the first ideas that come to mind if someone asked me to create one from scratch. :P
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Is there an assembler that supports unit testing?

Post by Dwedit »

The "Trainer" flag of a NES header (512 bytes after the header to initialize a part of WRAM) is still honored by some emulators.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
Drag
Posts: 1615
Joined: Mon Sep 27, 2004 2:57 pm
Contact:

Re: Is there an assembler that supports unit testing?

Post by Drag »

512 bytes might be enough for a few unit tests, but it seems unreliable if its original purpose was to hack around incomplete mapper support and not all emulators support it, though maybe it's perfectly fine and I'm just being too stingy. :P

I had a good long think about it, and I think the ideal unit test runner for the NES would be something like this:

Forking an emulator and wrapping some kind of scripting language around it, where the scripting language contains instructions you write on how to preload the NES's memory with test data, how to initialize mappers, etc, then points the emulated CPU at the code you want to test, then emulates until return or breakpoint or something, then returns to the scripting language which contains instructions on what to check as the result (memory, registers, PPU memory, etc), and how to pass/fail it.

Something which would help is, if the assembler you use can output the symbols generated by the assembly process, so you could use the labels you used in your source code to tell your tests which routines to point to.

Mapper support would start out broad if you just use register writes in the scripting language to initialize the mapper you're using.

Another bonus is, your unit tests and test data would be separate from your source code.
User avatar
SusiKette
Posts: 147
Joined: Fri Mar 16, 2018 1:52 pm
Location: Finland

Re: Is there an assembler that supports unit testing?

Post by SusiKette »

Drag wrote: Thu Dec 30, 2021 8:35 pm Forking an emulator and wrapping some kind of scripting language around it, where the scripting language contains instructions you write on how to preload the NES's memory with test data, how to initialize mappers, etc, then points the emulated CPU at the code you want to test, then emulates until return or breakpoint or something, then returns to the scripting language which contains instructions on what to check as the result (memory, registers, PPU memory, etc), and how to pass/fail it.

Something which would help is, if the assembler you use can output the symbols generated by the assembly process, so you could use the labels you used in your source code to tell your tests which routines to point to.

Mapper support would start out broad if you just use register writes in the scripting language to initialize the mapper you're using.

Another bonus is, your unit tests and test data would be separate from your source code.
I've been thinking of something similar.

Another feature that may be useful in some cases is to be able to generate interrupt signals at specific cycle counts and the ability to use PPU warmup (ignored by default, since most of the code you test probably is not reset code).

For performance testing the emulator would give cycle count and percentage of used time from one frame's or VBlank's time depending on test type.
Avatar is pixel art of Noah Prime from Astral Chain
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Is there an assembler that supports unit testing?

Post by Dwedit »

When you say "Unit Testing", do you mean an emulator that can run from the console and can return Pass/Fail based on executing some 6502 code, then peeking at a RAM value?

If you have that, you can build in "Unit Test" mode with different macros, run it, then build in normal mode.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
User avatar
Movax12
Posts: 541
Joined: Sun Jan 02, 2011 11:50 am

Re: Is there an assembler that supports unit testing?

Post by Movax12 »

Drag wrote: Thu Dec 30, 2021 8:35 pm Forking an emulator and wrapping some kind of scripting language around it, where the scripting language contains instructions you write on how to preload the NES's memory with test data, how to initialize mappers, etc, then points the emulated CPU at the code you want to test, then emulates until return or breakpoint or something, then returns to the scripting language which contains instructions on what to check as the result (memory, registers, PPU memory, etc), and how to pass/fail it.

Something which would help is, if the assembler you use can output the symbols generated by the assembly process, so you could use the labels you used in your source code to tell your tests which routines to point to.
Maybe check out NintendulatorDX.
Drag
Posts: 1615
Joined: Mon Sep 27, 2004 2:57 pm
Contact:

Re: Is there an assembler that supports unit testing?

Post by Drag »

I've come up with a potential API we can use for unit testing: https://github.com/DragWx/nes-unit-testing-api

It's really basic for right now, but at least this is something which can be used to feel out how unit testing NES code would work at all. :P

Note that this is just an API definition and not a usable library. The idea is, someone could fork an emulator, strip it to just the emulation part, then implement this API with it, and finally package it as a library for others to import into their unit testing frameworks.

So now, if given a library which gives you these functions and data structures, would it be sufficient for writing some basic unit tests?
Oziphantom
Posts: 1565
Joined: Tue Feb 07, 2017 2:03 am

Re: Is there an assembler that supports unit testing?

Post by Oziphantom »

the emulators have LUA, just make your test system in LUA and then you can test it against the emulator. However you don't really want to do unit tests in the emulator as you want to test the code and at full speed not 1Mhz logically. And not have to deal with issues of the hardware. Like you forgot to enable rendering or forgot to turn it off or your set up code choose the wrong bank because you moved the function etc.
Post Reply