Glitch-free controller reads with DMC?

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Glitch-free controller reads with DMC?

Post by Sour »

tepples wrote:Mail from the wiki is broken right now. Try not providing an email address when signing up. Because mail is broken, you'll need to autoconfirm (2 talk edits and 4 days) or wait for me or another administrator to wake back up and confirm your account.
Yup, that worked. If someone could confirm the account (same username as here) that'd be nice - no hurry though, I'm not planning on editing this right away :p
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

I tried to write a test that involved watching for corruption while doing repeated synced joypad reads, with the goal of identifying whether the emulator supported this method of reading. Surprisingly, despite being synced, I was encountering corruption when testing on Mesen.

What I learned was that if your synced region is long enough that two or more DMAs can occur during it (about 430 CPU cycles at minimum, I believe), then you need to ensure that all consecutive write cycles in the routine end on an even cycle; otherwise, DMA landing on that write will take 3 cycles instead of 4. For example, when using ROL on memory, the second write cycle must be even so that DMA that tries to occur on either one is forced onto the following odd cycle, where it will take 4 cycles. Without this, the cycle parity changes, allowing future reads of controller registers to occur on odd cycles where they're subject to bit deletion.

In addition to repeatedly reading the joypad, this seems likely relevant for handling Four Score because of how much time is required to read everything even once.


While I'm reviving this old thread, I'd like to note here on the topic of OAM DMA on PAL that testing done in this thread indicates that if your OAM DMA fits within NTSC vblank, it should also be fine on PAL, so it does not appear that different behavior needs to be taken depending on the region; ending vblank with OAM DMA and joypad read should work either way.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

By testing this on real hardware via Everdrive, I believe I've been able to confirm this write behavior, and also discovered that the only emulator that seems to be getting this totally correct right now is Nintendulator 0.980. Testing this is a bit tricky because it seems that you can have timing alignments where the DMA avoids landing on certain cycles of the loop, which can cause it to miss an odd write cycle (manifesting as tests showing black when they should show white), but I found that by adding variable delay into the loop, it seems to land on the target cycle reliably. I'd appreciate another set of eyes on this to make sure it all makes sense, but I'm fairly confident in the results because my test ROMs show different behavior despite the only meaningful difference being the timing of the ROL write cycles (all other timing, code positioning, and register values remain the same).

I've attached two modified versions of dma_sync_test_v2. Both of these change the loop to no longer include the OAM DMA, so code has to stay synced indefinitely for the controller port reads to work reliably. They also add a variably delay to try to mix up where DMC DMA lands within the function. The difference between these versions is in the second ROL instruction, which ends on an odd cycle in badrol and an even cycle in goodrol. badrol is expected to exhibit bit deletion (screen turns white) because DMA landing on the odd ROL write cycle (cycle 5/5) will be delayed by 1 to an even cycle and thus take only 3 cycles, changing the parity so that future joypad bits can be lost. goodrol is expected to never encounter bit deletion (screen stays black unless right is manually pressed) because DMA landing on the odd ROL write cycle (cycle 5/6) will be delayed by 2 to an odd cycle, where it takes 4 cycles as usual.

Both tests are verified to give the expected result on real hardware and Nintendulator 0.980, and but not on Mesen 0.9.7.37 (goodrol turns white), puNES 0.100 (goodrol turns white), Nestopia UE 1.49 (goodrol turns white), and Nintaco 2018.12.31 (goodrol turns white on some resets). Other emulators I'm aware of don't emulate the bit deletion at all and will show black on both ROMs. There hasn't been much progress in a couple years on emulator support for this, so I'm really hoping to get the ball rolling so hacks and homebrews can use synced reads and still work properly in emulators.
Attachments
dma_sync_test_loop.zip
(3.64 KiB) Downloaded 651 times
User avatar
Zepper
Formerly Fx3
Posts: 3262
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Re: Glitch-free controller reads with DMC?

Post by Zepper »

I didn't get it. Please, could someone explain this stuff to me? What "bit deletion" are you talking about?
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

Another wrinkle: DMC DMA landing at the end of OAM DMA only takes 1 or 3 cycles, immediately breaking sync and causing problems for synced read functions longer than one DMC period. Aligned write cycles alone do not solve this case. This is also relevant for the EPSM, where CPU/APU sync is required due to a quirk where the CPU's OUT pins only update every APU cycle, so the EPSM could see the write a cycle late and miss the data.

Synced reads are still fine for function lengths of less than the DMC period (so a second DMA can't land in the function after it has been desynced by the first). If you go longer than that, then not only do you need to properly align write cycles, but you need to ensure that the DMAs that come at multiples of the DMC period after the 2 possible desyncing DMAs either never cause a problem or land on a single write cycle before they can cause a problem, pushing them back into sync. Edit: While there are 2 possible desyncing DMAs, they both cause the synced code to begin executing after the DMA completes, producing the same behavior in either case, so the synced code only has to worry about 1 bad alignment for each rate rather than 2.
User avatar
Ben Boldt
Posts: 1149
Joined: Tue Mar 22, 2016 8:27 pm
Location: Minnesota, USA

Re: Glitch-free controller reads with DMC?

Post by Ben Boldt »

Is there an example existing that can do a synced read of the SNES mouse? I am thinking that this especially has the potential of multiple DMC DMA fetches during reading.

Also, quoting the wiki:
Because OAM DMA synchronizes the CPU and APU such that reads on an "even" CPU cycle never overlap a glitch, a program on an NTSC NES can miss all the glitches by triggering an OAM DMA as the last thing in vblank just before reading the controller, so long as all the reads are spaced an even number of cycles apart.
I am a little confused here. Does it have to be in v-blank, and does it have to be at the end of v-blank? Or does it actually mean that reading the controller must be the FIRST thing you do after starting OAM DMA? Regardless of when/where you start OAM DMA?


Edit:
The reason I ask is because it seems like adding SNES mouse support to Sunsoft Shanghai II would be really cool and relatively simple. This game uses lots of DMC. I did a little poking and I find that $0F is being written sometimes to $4010, which sets the DMC to run as fast as possible. My understanding, this makes it 1 delta per 54 CPU cycles (NTSC), which x8 deltas per byte means that a DMC DMA happens each 54*8=432 cycles. Does that sound reasonable to read the entire mouse in this worst case without getting more than 1 DMA?? This game’s normal controller reading code already must handle DMC glitches, so nothing to worry about with that. I am just trying to scope this out before I get my heart too set on it.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

I might work on a mouse example, unless someone already has example code they can post.

Regarding OAM DMA:
Q: "Does it have to be in vblank?"
A: No, but you probably want to do it in vblank. The vblank suggestion is because OAM DMA is only effective outside of rendering. You could do it entirely during rendering and have the OAM DMA ignored by the PPU, or do it whenever and not display sprites, or whatever.

Q: "Does it have to be at the end of v-blank?"
A. No, but it's best to do it last. Reading controllers doesn't need to be done during vblank, but pretty much any PPU interaction does. Since reading controllers has to be done very shortly after OAM DMA, doing OAM DMA last means controllers aren't eating up vblank time you might need.

Q: "Does it actually mean that reading the controller must be the FIRST thing you do after starting OAM DMA?"
A: For practical reasons, yes, though it isn't strictly required. There are multiple reasons why you would want to do it right away:
1. Any work between OAM DMA and reading the controllers needs to be of predictable timing. That is, it needs to be always even or always odd, or you need to have a way to know which parity it has (such as number of iterations through an odd-length loop). If you know how long it takes, then you can still maintain sync.
2. Any write cycles between OAM DMA and reading the controllers may need to occur on get cycles (just like the controller read cycles). DMC DMA reloads only try to initiate on put cycles, and if they land on the first of an odd number of write cycles, they'll steal only 3 cycles instead of 4 (because they are forced to halt on a get cycle instead of put). Stealing an odd number of cycles breaks the synchronization because controller reads will now occur on risky put cycles instead of safe get cycles. I say 'may need to' because if your last controller read is still within 1 DMC period of the end of OAM DMA, then it doesn't matter if sync has broken from a DMA because the bit deletion would require a 2nd DMA, which is 1 period after the 1st one. Note that interrupts are a problem, too; interrupts do 3 consecutive write cycles, and if the series starts on a put, there are 2 opportunities for sync to be lost. IRQs during vblank are probably not very common, though, maybe unless you're using the DMC IRQ, which will require additional consideration.
3. As described in my last post here, if you get unlucky and DMC DMA lands in the last 3 cycles of OAM DMA, then it will only take 1 or 3 cycles, immediately breaking sync. This means you'll want to fit your controller read within one DMC period of OAM DMA unless you want to write the intricate code necessary to realign those 2 sync-breaking events or avoid their DMAs. (It also makes the mitigations described in points 1 and 2 above harder to write.) Edit: While there are 2 sync-breaking events, they both cause the synced code to begin executing after the DMA completes, so the synced code only has to worry about 1 bad alignment for each rate rather than 2.

Q: "Regardless of when/where you start OAM DMA?"
A: The timing of OAM DMA doesn't matter for this trick. It can be anywhere or anytime and the rules are the same. The restrictions on when you do it are purely related to the PPU.


(Note that while this trick means that doing OAM DMA last on NTSC is ideal, I suggest doing OAM DMA first on PAL because this trick isn't necessary and the window to do OAM DMA is much shorter on PAL than the window to do other PPU work. Based on results I've seen, it may also be safer for OAM DMA to extend too late on NTSC than on PAL.)
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

Here's a first attempt at mouse reading code. It is untested. The synced portion takes 410 cycles, so it will fit in the fastest DMC rate. There is loop unrolling, and the first byte is read off in a way that is fast (looping would take 8 cycles overhead per loop) and still tells us if the bits were all 0 (if not, we jump to an error handler because this can't be a mouse).

Code: Select all

; These must be zero page and contiguous.
mouse_x: .res 1
mouse_y: .res 1
mouse_buttons: .res 1


  LDX #$01
  STX mouse_x
  STX mouse_y
  STX mouse_buttons
  STX JOYPAD1
  LDA #$00
  STA JOYPAD1

  INX  ; LDX #$02

  LDY #>oam_buffer
  STY OAM_DMA

  ; A == 0, X == 2.

  BIT $00      ; get put get

  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  ORA JOYPAD2  ; put get put GET
  LSR          ; put get
  BCS MouseReadFailed  ; put get (put)

  ; 39 cycles so far.

 :
  LDA JOYAPD2     ; put get put GET
  LSR             ; put get
  ROL mouse_x,X   ; put get put get put get
  LDA JOYAPD2     ; put get put GET
  LSR             ; put get
  ROL <>mouse_x,X ; put get put get put get put
  BCC :-          ; get put (get)
  DEX             ; get put
  BPL :-          ; get put (get)

  ; (28*4*3) - 12 + 8 = 371, + 39 = 410
User avatar
Ben Boldt
Posts: 1149
Joined: Tue Mar 22, 2016 8:27 pm
Location: Minnesota, USA

Re: Glitch-free controller reads with DMC?

Post by Ben Boldt »

Wow, thanks for all the info and the sample code. Amazing it can fit in the worst case/fastest DMC rate. I will see what I can do to shoehorn this into Shanghai II. The normal D-pad is already moving the “mouse cursor” so it couldn’t possibly be too complicated to take that apart and inject movements from the mouse.

I do not have an actual SNES mouse to test with but getting things working in an emulator should be a good milestone. From there, we can verify what it does on real hardware.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

Assuming BizHawk supports the mouse, the NesHawk core in the newest build (not sure if it's in an official release yet) should have the best DMC DMA timing of any emulator. Mesen is still very accurate for our purposes here, though.

You could actually modify this code without too much trouble to also do the normal joypad. In that case, all of the write cycles in the mouse reading code have to be safe, which means the ROL instructions' last 2 cycles need to be put get, not get put. If you can guarantee that the two branches in the read loop cross a page boundary, then they'll take an extra cycle and so you can make the 2nd ROL be zero page instead of absolute, which would still align correctly. Then, you'd need to make sure that the bad DMA that could occur exactly 428 cycles after the first cycle post-OAM DMA (so, 428 cycles after the first cycle of the BIT) lands on a single write cycle (to resync) or anywhere but a joypad read cycle (because we don't care about sync after this). It's a bit complicated, but it's faster than the repeated read solution, works reliably when implemented correctly, and won't cause corruption of the DMC sample data (which can happen when DMC DMA lands on a joypad read).

There is another caveat that you need it to work with all used DMC rates. I've put code below that works around rate F and ends before the bad cycle on other rates, but for rate E, if your code can't finish reads in 144 cycles after the indicated rate F cycle, you'll need to make sure that later cycle is safe, too. The code below does it in just 102 cycles. Note, again, that this is untested.

The code would look like this:

Code: Select all

; These must be zero page, contiguous, and in this order.
mouse_x: .res 1
mouse_y: .res 1
mouse_buttons: .res 1

; This must be zero page.
joypad1_down: .res 1


  LDX #$01
  STX mouse_x
  STX mouse_y
  STX mouse_buttons
  STX joypad1_down
  STX JOYPAD1
  LDA #$00
  STA JOYPAD1

  LDY #>oam_buffer
  STY OAM_DMA

  ; A == 0, X == 1.

  ; (desync case: PUT GET PUT GET)
  BIT $00       ; get put get

  INX           ; put get (LDX #$02)

  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  ORA JOYPAD2   ; put get put GET
  LSR           ; put get
  BCS MouseReadFailed  ; put get (put)

  ; 41 cycles so far.

 :
  LDA JOYAPD2     ; put get put GET
  LSR             ; put get
  ROL mouse_x,X   ; put get put get put get
  LDA JOYAPD2     ; put get put GET
  LSR             ; put get
  ROL mouse_x,X   ; put get put get put get
  BCC :-          ; put get (put get)  MUST CROSS PAGE BOUNDARY
  DEX             ; put get
  BPL :-          ; put get (put get)  MUST CROSS PAGE BOUNDARY

  ; 426 cycles so far.

  NOP             ; put GET (432 cycles after when DMC DMA could have desynced us during OAM DMA.
                  ;          This would swap all gets and puts, and so the last cycle here would be
                  ;          a put cycle and DMC DMA halt.)

  ; Safely read joypad 1. Takes 102 cycles to finish the joypad reads.
 :
  LDA JOYPAD1       ; put get put GET
  LSR               ; put get
  ROL joypad1_down  ; put get put get put
  BCC :-            ; get put (get)

  ; Sync is fully done. It's now safe to process inputs or whatever else.
CORRECTION: I had earlier said that there are 2 bad cycles to watch out for when DMC DMA lands near the end of OAM DMA, but in both of these cases where DMC DMA steals an odd number of cycles (1 or 3) instead of an even number, the synced code starts running immediately after the DMA completes. That means in both cases, the next DMA (at rate F) is 428 cycles after the first BIT cycle, so you only have 1 bad cycle to worry about.
User avatar
Ben Boldt
Posts: 1149
Joined: Tue Mar 22, 2016 8:27 pm
Location: Minnesota, USA

Re: Glitch-free controller reads with DMC?

Post by Ben Boldt »

I was able to put the mouse-reading code into the game pretty easily. Sunsoft wrote a dedicated function to start OAM DMA:

Code: Select all

>07:C0C8:A9 00     LDA #$00
 07:C0CA:8D 03 20  STA PPU_OAM_ADDR = #$00
 07:C0CD:A9 02     LDA #$02
 07:C0CF:8D 14 40  STA OAM_DMA = #$02
 07:C0D2:60        RTS -----------------------------------------
This calls it:

Code: Select all

>07:C3C0:20 C8 C0  JSR $C0C8

I found some free space in page $7, which is the fixed $4000 byte page at the end of PRG-ROM. I put your code there and tweaked to include the exact same orignal Sunsoft code that starts OAM DMA (i.e. writes the low byte of OAM address every time).


Call changed to:

Code: Select all

>07:C3C0:20 61 FF  JSR $FF61
New code at $FF61:

Code: Select all

>07:FF61:A2 01     LDX #$01
 07:FF63:86 7D     STX $007D = #$FF
 07:FF65:86 7E     STX $007E = #$FF
 07:FF67:86 7F     STX $007F = #$FF
 07:FF69:8E 16 40  STX JOY1 = #$00
 07:FF6C:A9 00     LDA #$00
 07:FF6E:8D 16 40  STA JOY1 = #$00
 07:FF71:E8        INX
 07:FF72:A9 00     LDA #$00
 07:FF74:8D 03 20  STA PPU_OAM_ADDR = #$00
 07:FF77:A9 02     LDA #$02
 07:FF79:8D 14 40  STA OAM_DMA = #$02
 07:FF7C:24 00     BIT $0000 = #$00
 07:FF7E:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF81:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF84:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF87:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF8A:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF8D:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF90:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF93:0D 17 40  ORA JOY2_FRAME = #$01
 07:FF96:4A        LSR
 07:FF97:B0 12     BCS $FFAB
 07:FF99:AD 17 40  LDA JOY2_FRAME = #$01
 07:FF9C:4A        LSR
 07:FF9D:36 7D     ROL $7D,X @ $009A = #$00
 07:FF9F:AD 17 40  LDA JOY2_FRAME = #$01
 07:FFA2:4A        LSR
 07:FFA3:3E 7D 00  ROL $007D,X @ $009A = #$00
 07:FFA6:90 F1     BCC $FF99
 07:FFA8:CA        DEX
 07:FFA9:10 EE     BPL $FF99
 07:FFAB:60        RTS -----------------------------------------

I converted all those opcodes manually with an instruction set and hex editor, then FCEUx disassembled back to the instructions seen here, so hopefully I didn't goof anything up. The game still does run completely normal with these changes. The mouse value should be getting written to zero-page addresses $7D/7E/7F (which aren't ever used before AFAIK), and next would be to use those bytes to move the existing cursor and handle mouse button clicks.

Controller 1 is still going to be required in order to press start and operate the menus to get the game started. This game also makes use of controller 2 in multiplayer modes. I am actually not too familiar with running controllers, I am kind of diving into something new here. What controller is intended to be replaced by a mouse with your code? Is it possible to connect the SNES mouse to the Famicom expansion port as a third controller somehow?


Edit: Something else I just realized that I did not do yet is initialize the mouse tracking speed, which may apparently be required on some revisions of the real hardware.


Edit2: bytes 7d/7e/7f are updating correctly when I set controller 2 to be an SNES mouse in FCEUx! :D
Last edited by Ben Boldt on Sat Feb 12, 2022 9:14 pm, edited 1 time in total.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

The mouse is on joypad 2.

That looks like it should work. You've got a bit of a mess with the A register around the time of OAM DMA, but it's OK because the ORA stuff only cares about bit 0, which is clear (A is 2). You can use the other version I posted to read the normal controller on joypad 1, as well.

Edit: Putting the mouse on the Famicom expansion port would require putting it on a different bit. That would make it take longer to extract the bit from the joypad read, and you'd have to save the other controller at the same time. Definitely a harder problem.

That said, you could use a normal controller on the expansion port on $4016.1 and use that for a 2nd normal controller.

Edit2: FCEUX doesn't emulate the DMC DMA joypad bit deletion issue. Good to see it's working in the normal case, though!
User avatar
Ben Boldt
Posts: 1149
Joined: Tue Mar 22, 2016 8:27 pm
Location: Minnesota, USA

Re: Glitch-free controller reads with DMC?

Post by Ben Boldt »

Okay thanks. I see what you mean with A being messy. I am loading it with #00 when it is already 0… I will see if I can get this working with the mouse as controller 2 as you have written it. I will also see if I can make the mouse apply for both players to work around that.

Edit: actually it does break it when I get to the play field. I was just checking the title screen that the sprites were still working before. I am finding the background all bouncing around and bad pixels in it when it is assembling the tiles on the playfield… but sprites are all okay and at the correct position. This game is unique in that it draws in CHR-RAM like Qix. And it is also getting some of its drawing wrong now. Anyway, I will keep working on it and let you know what I find out. I did try restoring A to 02 at the end before the RTS and that didn’t help. Hopefully it is something simple and not like I ran out of VBlank time.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Glitch-free controller reads with DMC?

Post by Fiskbit »

This game does not do OAM DMA last in vblank, so the extra vblank time wasted by reading the mouse is pushing later work that relies on vblank into rendering. You need to move OAM DMA to the end.
Post Reply