nesdev.org

During all these years I've been working on my emulator I've tried quite a few test-ROMs that some skilled people have written.
What they all seem to have in common is lack of human understandable-documentation. Tests may fail but I usually haven't got a clue what the problem is all about.
Yesterday I tried a bit with "NESStress". It actually comes with a textfile describing the tests (in a line or two), but it really doesn't tell me what's wrong with my emulator. So, I noticed I failed a few tests, but I simply don't understand what's wrong so I don't know what to fix.

What I really want is a detailed (but still readable) description of the tests, what they do, and the expected end result.
For example, documentation concerning a frame/vbl/timing-test should clearly state what it expects to pass the test. Which scanline is expected? Which state should all flags have etc.

I am not sure about the purpose of this post, but perhaps an idea for possible future test-ROM coders.

What do you think? Am I the only one that thinks the tests are usually somewhat hard to understand?

...

I always found the PPU testing part of NEStress to be quite cryptic. I do recommend using blarggs' ppu tests though, which comes with a neat little .txt-file which is a bit more verbose than NEStress' Error or OK!

When I was debugging my CPU and I got in to a trench, I'd always trace operations instruction-for-instructions and compare it with an emulator like Nintendulator, to see if they differed. And as Zepper suggested, just look at the disassembly, and see if the expected operation differ from that of your emulator.

The tests' purpose is to test some behavior, and make it clear what behavior they're testing. It's not to describe the correct behavior; that's what the Wiki is for. That said, I'm sure my tests often fail at making clear what behavior they're testing. BTW, I've found Nestress to give wrong results, one reason I started writing my own tests.

Any tests in particular that are unclear?

Many times blargg's test ROMs are obtained from ROM sets or ROM sites and don't come with source or any text documents. This was a problem I had early on.

blargg wrote:The tests' purpose is to test some behavior, and make it clear what behavior they're testing. It's not to describe the correct behavior; that's what the Wiki is for. That said, I'm sure my tests often fail at making clear what behavior they're testing. BTW, I've found Nestress to give wrong results, one reason I started writing my own tests.

Any tests in particular that are unclear?

Your tests are awesome Blargg, however I keep failing them all the time. I've obviously got my basic timings wrong. Your tests seems to be extremly cycle-accurate, if the emulator is 1 or 2 cycles off, the test fails(?).

So, I noticed I failed a few tests, but I simply don't understand what's wrong so I don't know what to fix.

The people writing the tests have a deep understanding of the behavior in question, whereas they have no idea what your knowledge set is. Does the term "DRAM refresh" confuse you? Do we expect you to look that up on Wikipedia, or should we make each of thousands of tests dozens of pages long in their explanations?

Documentation is also error-prone. You can't run your explanation through an emulator to make sure there are no mistakes in it. The more we have to explain, the more likely we say something wrong and it throws you off.

We could certainly do better, just explaining things from our side a little.

byuu wrote:Does the term "DRAM refresh" confuse you? Do we expect you to look that up on Wikipedia

That isn't a bad idea. That's why I made a list of URLs of Wikipedia articles to read for background information. Such a list might be useful to include with a test suite so that emulator authors can know what they don't know.

From the nesstress.txt file:
"*PPU $3FFF-$0000 Wrap Around.
To test if the address is correctly reset to $0000 after
access to $3FFF. One emulator actually crashed during this test."

What's the deal here? What happens if you read/write at $3FFF and the PPU increment bits are set to 32? How would the wrap around work?

Wraparound works the same way it always works. The value wraps around =P

The PPU address is actually 15 bits wide, so it would wrap at $7FFF->$0000, though, not $3FFF->$0000 (that extra bit controls the fine Y scroll). Wrapping can be accomplished by ANDing with $7FFF after you increment the address (though it's not entirely necessary to do that, but whatever)

When you read/write $2007, only the low 14 bits of the address are used, so you effectively AND the address with $3FFF (or you could say that $4000-7FFF mirrors $0000-3FFF -- it's the same thing)

So if the PPU is incrementing by 1:
$3FFF

I did not want to start a new thread to ask this question.

Can someone point me to some good test roms? I looked a bit through the forms and on the main page, but I'd like to get a consensus of what is good so I can test against a 'gold standard' and get things right.

I have the most of the cpu code implemented and a good portion of gpu code as well. I am starting to draw stuff to the screen, but there are obvious errors in my code somewhere. There is a resemblance of the correct image on the screen, but with a lot of graphical artifacts.

I would like to start out with some very basic tests to make sure everything is operating how it should be (basic cpu functions) and them move on to the more complex ones.

Anyway, I'm not really looking for help with a specific problem, just some good test roms. If anyone has any suggestions, I would really appreciate it.

Zepper wrote:http://wiki.nesdev.com/w/index.php/Emulator_tests

Awesome. Thank you. I have only glanced at the wiki a couple of times and didn't notice them there.

tineras wrote:

Zepper wrote:http://wiki.nesdev.com/w/index.php/Emulator_tests

Awesome. Thank you. I have only glanced at the wiki a couple of times and didn't notice them there.

I keep all the test ROMs [and some of my crap ROMs] I test NESICIDE against in my git repo on www.gitorious.org:

http://www.gitorious.org/nesicide/nesic ... /test_roms

[Click on the "Download master as tar.gz" button on the right side of the screen to get it all.]

And I also keep a status of test results on my wiki:

http://wiki.nesicide.com/doku.php?id=test_roms_status

I probably should have asked blargg and the other test ROM authors before putting all their hard work there, but I think it is all publicly available on other sites so I hope they're okay with that.

I'm not saying you have to keep a publicly visible test ROM status file, either...but in my opinion it gives potential users and/or development helpers the chance to see where a)your emulator might do a better job at emulating some esoteric system behavior that they'd like to take advantage of or b)they can help by using their knowledge of how to get tests to pass to improve your emulator.[/url]

Posting emulator test results is also an objective way of evaluating an emulator. Simply seeing it stated that "sprite hit timing is accurate" isn't very meaningful. I suppose I should be more diligent about versioning my tests.