Count Errors Test ROM and some new DMC glitch info

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

I had been working a few months ago on DMC emulation, and finally got around to testing count_errors.nes on a dev board from power on.

Here is a video with the results (one OEM NES controller plugged in, front loader NES):

https://youtu.be/m_-EzTS8pMY

Getting the emulator to match these results was actually pretty tricky, and I needed to implement a new behaviour that I don't believe had been documented before.

Basically, if the DMA address happens to have the same bits set as the controller register (I think it is (DMA ADDR & 0x401F) == 0x4016) then there will NOT be a controller read glitch.

I'm not yet sure how this impacts other potential DMC glitch accesses like those of the ppu.

Even when I implement this behaviour though I still can't get TASes on games that use the DMC channel to sync on console. I think something is still some unknown behaviour. Time Lord for example encounters a situation where the DMC channel is disabled as a DMA is about to be triggered. What happens?

Probably some new test ROMs are needed.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by lidnariq »

... we were just talking about that glitch on the Discord.

If DMA tries to happen, it cares about the CPU's address when it's fetching. If the CPU is accessing the internal register range when it's interrupted, OAMDMA will instead happen from the internal registers (which are mostly open bus).

Fiskbit might be able to provide the test ROMs
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

So new stuff happens with OAMDMA as well? Interesting, it seems there are still a variety of unexplored edge cases.

I never wrote anything down about it, so I don't quite remember the details, but I also did similar testing with count_errors_fast.nes:

https://www.youtube.com/watch?v=FJsX3M5ABJA

Despite there being nothing too obviously different, I never got this case to match emulator (at least not in a way that was consistent with everything else.) I haven't had a chance to focus on it in a while, I'll be quite interested to learn what the resolution is.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by lidnariq »

Yeah, if you look at Visual2A03, the address bus multiplexers select what goes out using the ab_use_pcm, ab_use_cpu, ab_use_spr_r, and ab_use_spr_w signals. However, the signal that controls whether the APU registers are enabled or not depends on only the CPU's address bus (apureg_rd), so the the DMA unit can be trying to override the address bus at the same time that the CPU is isolating the data bus from the outside world.

This can cause the DMCDMA to fetch open-bus or the status byte (or, on a modified NES with the test registers enabled, the test registers) instead of the byte it intended. It can also cause invisible bus conflicts, because the APU support circuitry can enable /RD4016 and /RD4017 at the same time that the address bus indicates something that points to not-open-bus.

Similarly, OAMDMA from page $40 will (usually) end up just fetching all open bus, and not any of the APU registers, because the CPU's address bus doesn't point to the apureg_rd region.
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

Cool stuff, that would make an interesting test rom as well. Still a lot of cases that can differentiate emulator from hardware I guess.

Oh yeah, another DMC edge case that seems to generate new effects is reloading DMC manually around the time an internal DMC reload occurs. At least, when I run Fiskbit's previous test ROM (dmc_dma_start_test_v2) with a nonzero length on hardware I get unexpected results. Tiny Toon Adventures and Teenage Mutant Ninja Turtles encounter such cases.

I tried looking into it but couldn't get the exact right answer.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Count Errors Test ROM and some new DMC glitch info

Post by Fiskbit »

Thanks so much for looking into this more; there are definitely some mysteries left. I've attached the test ROM lidnariq was referencing for the APU register access behavior. I had intended to do more work on this to make it release-ready, but haven't gotten to that yet and it'd clearly be useful to you now, so here's the current, WIP version. The goal of the test is to do DMA while the CPU is reading from $4000-401F and then read the values back out of OAM (requiring a 2C02G or H) and print them to the screen. The one major challenge here is that OAM DMA starts immediately after the CPU write that initiates it, so we need to be executing out of $4000-401F rather than just reading it. OAM will also decay before we can read out all the values, so we have to refresh it as we go. To make this work, I jump to $3FFE and use PPU open bus at $3FFE, CHR-ROM data at $3FFF, and open bus with a perfectly-timed DMC DMA at $4000 to initiate OAM DMA from the $4000 page while the CPU is executing in (and thus reading from) the $4000-401F range. This causes the registers to activate even on reads outside the register range. This test does not pass on any emulator or flash cartridge that I am aware of (flash cartridges tend to mess with open bus). Lidnariq got the following results on console: "0x15 bytes of 0x40, followed by 0x10, followed by 0x1F bytes of 0, followed by 0x10, followed by 0x1F bytes of 0, repeat until done."

In addition to what lidnariq's explained here, I did a writeup on this APU behavior here on the wiki (sorry for not having yet integrated this material elsewhere on the wiki. Anyone interested in doing so is welcome to!). One implication I've noticed and mentioned on the wiki is that reading from a joypad when DMC DMA strikes can cause a bus conflict, resulting in corruption to the value read by the DMA. I had previously thought that was the only real-world impact of this behavior, but if it affects the joypad bit deletion, that's interesting, too. I haven't thought through that behavior thoroughly yet.

Two other interesting points. First is that joypad bit deletion behavior differs between RF Famicom and NES frontloader (and I can't remember if we know what behavior each other model should have). The frontloader will have just 1 deletion event, while the Famicom will have multiple. I had a test in mind to try to figure out exactly which cycles the Famicom was having additional deletion on and how those extra reads might be suppressed, but I don't remember the details anymore and it wasn't simple to implement. If interested, I can track down my idea from Discord and post it here.

Second is that the CPU and APU randomly boot into either of 2 alignments relative to each other, similar to how the CPU and PPU have various alignments. That is, the CPU can start on an APU GET cycle or on an APU PUT cycle. If you're trying to play back TASes and the DMA timing matters, then this can be significant. There is a quirk where the CPU's OUT pins are only updated every APU cycle, so you can use single-CPU-cycle toggles of OUT0 to control a standard joypad and see if the strobe took, allowing you to identify the alignment. I've written tests for this, as well, but they're pretty messy, so I'll try to clean them up and put something together to post soon.
Attachments
apu_register_activation_test_wip1.zip
(52.19 KiB) Downloaded 119 times
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

Thanks for the test ROM, I'll look into it first chance I get.

I had already stumbled upon the fact that the APU could align differently with the CPU at startup based on the results from count_errors, but similarly to the ppu, I just kind of had to pick one for emulation anyway so it never really occurred to me to make note of it.

I don't have a famicom to test with, but when I was running your previous test rom I did notice on my NES that I never got any of the 4-cycle blips (and I ran it many times.)

I'm hoping to start looking into this again in detail in a few weeks, these DMC edge cases are really the last thing needed for TASes to work on console for games that use DMC channel. (Currently, I have several runs that purposely press start on frames I know there will be a bit deletion to check if the start press (i.e. pause) is skipped on console the same way as on emulator. The runs sync through about a dozen such skipped-pauses, but still ultimately desync due to bad RNG for unknown reasons, though I presume to be the reasons I mentioned above.)
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

I have a question about the OAM DMA. If the cpu is not accessing 0x4000-0x401F at the time of OAM DMA, and the DMA address is 0x4000, does that mean the DMA will access external (cartridge) memory at 0x4000-0x401F that the CPU itself can't otherwise access? (That would be pretty neat if so.)
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by lidnariq »

Yes, that's exactly what it means.

In practice, that only matters for $4015, unless the NES has its test pin enabled.

It also means that the controller ports won't be clocked by DMA
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Count Errors Test ROM and some new DMC glitch info

Post by Fiskbit »

Alyosha_TAS wrote: Sun Dec 05, 2021 1:01 pm I don't have a famicom to test with, but when I was running your previous test rom I did notice on my NES that I never got any of the 4-cycle blips (and I ran it many times.)
I found your result here very surprising, so I just did some testing on a rev E Twin Famicom and rev G and H AV Famicoms. I got the 4-cycle blips on the G and H systems, but not the E one. So, it looks like this behavior was introduced in G. Can you confirm that your NES has a 2A03E?
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

Here is a pic of the internals of my nes.
nes_brd.png
So I guess I have a G variant (but I'm not really sure what the -O at the end means.)

I will run up the test ROM again and record at 60 fps tomorrow to verify.
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

Following up, I did a console run of dmc_dma_start_test_v2 on my NES and once again did not see any 4 cycle blips in any of the places indicated in the read me (there were 1 cycle blips though, exactly as described.) I did 5 trials with at least 15 seconds in between each power cycle. I had 2 OEM controllers plugged in. The dev board I am using uses SST 39SF040 chips for both prg and chr, and I am using an original CIC. Aside from the lack of 4 cycle blips, the tests behave as described. For tests where only 4 cycle blips are noted, there are no blips at all (i.e. no 1 cycle blips instead.)

I have some video if interested.

Also to reiterate, I also previously have modified the ROM to write 1 instead of 0 to the length (0x4013), and in this case I did see 4 cycle blips.

Hopefully this is helpful. Currently in emulator I do not get any blips, so any new info would really help fill in the last missing pieces.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Count Errors Test ROM and some new DMC glitch info

Post by Fiskbit »

Thanks for following up. I'm pretty baffled about what could be causing the difference in our test results. I've been doing all of my testing on Everdrive, but I initialize all of the DMC-related registers, so in theory, the only difference between our setups should be where in the current DMC period the test starts. Since we're both seeing 1-cycle blips, I wouldn't expect there to be some kind of CPU/DMC alignment difference where you're always starting in an alignment that prevents the 4-cycle glitch while I'm able to run in one that encounters it. This also shouldn't be something that can differ by motherboard type. I'll think more about what additional testing can be done and pull out some more consoles to test to see if I can identify any sort of pattern.

I think it's expected that setting the length to something greater than 1 byte lets you see the blips on unaffected systems. Presumably there is a case where the first loaded byte immediately gets sent from the sample buffer to the output unit, allowing the memory reader to immediately fetch another byte (or, at least, within a few cycles before we turn off greyscale in this test). Test 5 should tell us if the 4-cycle blips you see are immediate or not because any delay would force the 2nd DMA to post-greyscale. Test C would also show this because you'd only get 2 extra joypad reads if both DMAs land on the joypad read cycle. In my testing, the 4-cycle blip is definitely immediate and it'd be interesting to know if this is possible from normal behavior.


I saw you added support to NesHawk for the APU register stuff, which is great to see. I want to make sure we're on the same page for a couple things, though. First, this stuff applies the same to both OAM and DMC DMA; the test is using OAM DMA because I can measure the result, but with DMC DMA, this behavior can presumably result in bus conflicts between the sample byte and a joypad byte, which will affect audio and perhaps be noticeable. A system modified to use test mode could detect this in software, though I guess you could also collide an open bus read from $4000-401F with a DMC DMA from an address that matches a joypad register.

Second, the only places where DMA can read something the CPU can't are $4015-4017. CPU reads to any location other than $4015 will see the external CPU bus like normal. That means you can have a cartridge map something in this region and the CPU will be able to read it as long as it's not $4015, though $4016-4017 may have bus conflicts with the value read from the joypads. DMA, on the other hand, will get the cart-mapped values at $4015-4017 and without risk of bus conflicts, assuming of course that the CPU isn't reading from $4000-401F.
Alyosha_TAS
Posts: 173
Joined: Wed Jun 15, 2016 11:49 am

Re: Count Errors Test ROM and some new DMC glitch info

Post by Alyosha_TAS »

The only other NES I have to test with is an E variant cpu, and I tested with that one and just got the 1 cycle blips as expected.

I ran up the tests on my G variant where I write 1 instead of zero, and it turns out in this case I ONLY get 4 cycle blips. Test 5 does NOT have any blips in this case. test 2 gets 4 cycle blips in both frame toggle 1 and 2.

Thanks for clarifying the apu register stuff. Just to make sure I'm understanding this right, if the cpu is reading from 0x4017, and a DMC DMA happens simultaneously wiih and address of XX16, will controller one be clocked?
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Count Errors Test ROM and some new DMC glitch info

Post by Fiskbit »

I've been chatting with lidnariq and writing and rewriting this post as I puzzle through how this must work cycle by cycle and think I have it worked out now. First off, though, let's talk 4-cycle blips. I've done some testing on a 2A03G Famicom and while I typically do get 4-cycle blips, sometimes I don't. Whether I do or not varies across both power-on and reset. I thought I also saw it change once while cycling through tests, but I can't reproduce that and might just be mistaken. Whatever the case, it's possible that you'll get the blips if you try across several resets rather than from power-on. I'm testing with Everdrive, which also means I get different timing every time I power on and enter the test. Changing up how long it takes the test to start might make it always happen for you.

Also, the fact that the 1-cycle blips disappear for you when you use a longer sample suggests that the issue is related to doing DMC DMA near the end of a DMC period where no additional DMA is required. This means it should only be able to happen with 1-byte samples, and maybe also only with loop disabled. Under normal circumstances, DMC DMA occurs near the very beginning of a period and so this issue can't happen. It's only the $4015-initiated ones that can trigger it.


Now, let's talk about joypad clocks. First off, results will differ between RF Famicom and frontloader. We'll want to figure out at some point exactly which models fall into which category. The frontloader category appears to only clock joypads once per contiguous series of reads. The Famicom category clocks on every cycle that reads.

With dmc_dma_start_test_v2, tests C-E tell us a lot about how things are clocking. Tests D and E collide a read with the 1-cycle blips, which looks like this:
  • Cycle 0 (put): CPU reads from $4017 <-- 1-cycle halt occurs
  • Cycle 1 (get): CPU reads from $4017
On a frontloader, we only clock 1 time here, while we clock 2 times on Famicom.

Test C collides a 3-cycle DMC DMA with joypad reads. Those look like this:
  • Cycle 0 (get): CPU reads from $4017 <-- DMA starts
  • Cycle 1 (put): CPU reads from $4017
  • Cycle 2 (get): DMA reads from normal address <-- DMA completes
  • Cycle 3 (put): CPU reads from $4017
There are 2 sets of joypad reads here, so the frontloader sees 2 clocks. The Famicom sees every read, so it sees 3 clocks.

When a 4-cycle blip occurs, we get this behavior:
  • Cycle 0 (get): CPU reads from $4017 <-- DMA starts
  • Cycle 1 (put): CPU reads from $4017
  • Cycle 2 (get): DMA reads from normal address <-- DMA completes
  • Cycle 3 (put): CPU reads from $4017 <-- DMA starts
  • Cycle 4 (get): CPU reads from $4017
  • Cycle 5 (put): CPU reads from $4017
  • Cycle 6 (get): DMA reads from normal address <-- DMA completes
  • Cycle 7 (put): CPU reads from $4017
Frontloaders see 3 clocks here and Famicoms see 6.

So, that's as far as I can go with just the results from that test. Your finding at the start of this thread was that sometimes the extra clock doesn't happen. In that case, the DMA is activating the joypad that's being read, bridging the gap between the sets of reads. The Famicom, though, will see an extra clock from that DMA. Depending on whether the DMA starts on a put or get cycle, it will look like either of these cases:
  • Cycle 0 (get): CPU reads from $4017 <-- DMA starts
  • Cycle 1 (put): CPU reads from $4017
  • Cycle 2 (get): DMA reads from $4017 and normal address <-- DMA completes
  • Cycle 3 (put): CPU reads from $4017
  • Cycle 0 (put): CPU reads from $4017 <-- DMA starts
  • Cycle 1 (get): CPU reads from $4017
  • Cycle 2 (put): CPU reads from $4017
  • Cycle 3 (get): DMA reads from $4017 and normal address <-- DMA completes
  • Cycle 4 (put): CPU reads from $4017
In both cases, it's all just 1 long series of reads from $4017, so the frontloader should see just 1 clock. The Famicom should see all of them, though, clocking either 4 or 5 times. The value seen by the DMA is a bus conflict between the joypad value and the value at the address it was trying to read from.

Now, if the DMA is activating the other joypad, then it'll look like either of these:
  • Cycle 0 (get): CPU reads from $4017 <-- DMA starts
  • Cycle 1 (put): CPU reads from $4017
  • Cycle 2 (get): DMA reads from $4016 and normal address <-- DMA completes
  • Cycle 3 (put): CPU reads from $4017
  • Cycle 0 (put): CPU reads from $4017 <-- DMA starts
  • Cycle 1 (get): CPU reads from $4017
  • Cycle 2 (put): CPU reads from $4017
  • Cycle 3 (get): DMA reads from $4016 and normal address <-- DMA completes
  • Cycle 4 (put): CPU reads from $4017
In these cases, I would expect the frontloader to clock $4016 once and $4017 twice, and the Famicom to clock $4016 once and $4017 3 or 4 times.

(As an aside, one irrelevant detail that came up while discussing this is that the open bus bit when reading $4015 will reflect open bus from before the internal bus is isolated from the external one, but can't be affected by whatever is driven to the external CPU bus on that cycle. Might be fun to think of an emulator test to write for this.)
Post Reply