High color bitmap on the SNES

[tepples]

unlike DMA which is limited to about 6K/frame without letterboxing.

What's so bad about letterboxing though?

The fact that you'd need a heck of a lot of it to display at 60 fps. I imagine most people are seeking at least 256x160 pixels to fill an HDTV when zoomed.

There are 262 lines in an NTSC frame. Let's assume that one of those lines is needed as overhead to set up blanking, transfer, unblanking, and scroll, leaving 261 lines. These are divided into "display" lines and "load" lines. Each "load" line provides 165 bytes; each "display" line consumes 256 bytes.

So we have D + L = 261, and 256*D = 165*L.

Substituting:
256*D = 165*(261 - D)
Distributive law:
256*D = 43065 - 165*D
Add 165*D to both sides:
421*D = 43065

That leaves you with roughly a 256x102 pixel window that can be updated at 60 Hz.

I can try the calculation again with a 192-pixel-wide window:

D + L = 261, and 192*D = 165*L
Substituting:
192*D = 165*(261 - D)
Distributive law:
192*D = 43065 - 165*D
Add 165*D to both sides:
357*D = 43065

This leaves 192x120 pixels.

[93143]

dmac_align.sfc now seems to work flawlessly on the 2/1/1. Nice work!

Excellent.

Tested this on my 1/1/1 Super Famicom (PowerPak/MUFASA build #11331): No errors at all with dmacolor.sfc (downloaded Jan 09, 2016), but immediate glitches with dmac_align.sfc (no glitches after tapping Reset though).

1) I really hope you got that backwards.

2) If you didn't, what did the glitching look like? Was it an unholy mess like tepples and Revenant posted earlier, or was it localized around the DRAM refresh, or something else? A screenshot would be ideal, if you can get one.

Then again, with that poor resolution, I don't see the point of all this.

There are two: first, to answer Sik's question at the beginning of this thread (also because I wanted to see if it could be done).

Second, as tepples has suggested, it may be possible to get a useful video mode out of this. Depending on the application, it might be fine as it stands, but it would probably look much better if a higher-resolution luminance mask could be added. The way the trick works now, it's impossible, but if some predictable flickering and shifting of the direct-colour pixels is allowable it should be possible to just write directly to an active layer, freeing up either the main screen or the subscreen for colour math. The DMA transfers would have to be triggered by raster-aligned timed code rather than IRQs to get the necessary precision, but I'm pretty sure my buffering trick to hide the DRAM refresh would still work at the cost of a less straightforward data format.

...actually, it occurs to me that since subtraction is the most obvious way to implement a luminance mask, the active layer should probably be on the main screen. So if I can't rely on DMA consistently targeting the main screen after every reset, and I can't change the DMA/PPU alignment in software, the program might have to prompt the user to reset until the correct alignment shows up, either directly or by pretending to have failed to boot... but both of those are spectacularly ugly hacks, on top of the assumption that no SNES exists in which DMA reliably targets the subscreen after every reset... Maybe it'd be better to just use addition or averaging and accept the loss of saturation...

Is there an existing special chip that can allow software access to /RESET?

[Ramsis]

Tested this on my 1/1/1 Super Famicom (PowerPak/MUFASA build #11331): No errors at all with dmacolor.sfc (downloaded Jan 09, 2016), but immediate glitches with dmac_align.sfc (no glitches after tapping Reset though).

1) I really hope you got that backwards.

I did not.

2) If you didn't, what did the glitching look like? Was it an unholy mess like tepples and Revenant posted earlier, or was it localized around the DRAM refresh, or something else?

It looked almost exactly like in tepples' video.

[Revenant]

That kind of makes me want to try it again on my console just to make sure the good results weren't a fluke. I was able to get it to display correctly all 10 or so times that I tried with the newer ROM, but...

[Sik]

I guess you're going to need to test on a lot of revisions and flashcarts to be 100% sure.

[93143]

Yeah, it's a somewhat brittle method; even with the timing test it depends on assumptions that are hard to fully verify. Using timed code to line up the writes and just putting up with the shifting DMA alignment pattern would probably have been more reliable (since it wouldn't matter which screen it targeted, so I could just put the same thing on both), but I wanted straight columns because the MD version has them, and that necessitated the main/sub double shuffle that's the source of the weird timing requirements.

The newer code (dmac_align) tests DMA/PPU alignment by putting a constant nonzero index on the main screen and a different constant nonzero index on the subscreen, zeroing both corresponding entries in CGRAM, turning the screen on, triggering a two-byte DMA to CGRAM during active display, turning the screen back off, and checking where the write went. Unless the timing shifts during runtime, this should guarantee a working method, right? Except...

...based on testing in Snes9XW combined with sober second thought, there's a possibility that my timing test as originally written didn't have sufficient accuracy to guarantee the correct result. I have rewritten it to use an IRQ instead of polling H/V values; the write should now happen during the early portion of an active scanline, well before DRAM refresh and nowhere near the end of the colour fetch pattern. Once again, I've tested it in bsnes v072 and on my real SNES via Super Everdrive, and it works as expected in both.

If this doesn't work I'm not sure what to say, other than "how did the PowerPak manage to screw up (H)DMA with the old firmware, and could it still be doing something like it?". If the good boots stay good indefinitely, it means the half-dot alignment is consistent within a run. So as I see it, there should be no way for this to fail unless (a) there's sub-half-dot variance going on and quarter-dot alignment is too electrically finicky for reliable writes, or (b) something is kicking the alignment somehow in between the test and the display loop. Can anyone think of any other reason?

DMA is described as always aligning itself to an even multiple of 8 master clocks since reset before starting a transfer. Is this true of all models?

[Sik]

More like "how can flashcart firmware screw up with DMA at all?". Unless it's leaving some register in a non-default state...

Regarding timing, on the MD the HV counter is used for loose timing, then NOPs for some fine timing, then it causes a FIFO overflow to sync with the VDP (the 68000 on its own simply can't do it). Is there a way on the SNES to make the SPPU mess with the timing of the 65816 in a similar way?

Also I wonder what are the differences between slow ROM and fast ROM here.

[Ramsis]

"how did the PowerPak manage to screw up (H)DMA with the old firmware, and could it still be doing something like it?".

I used to leave $420B/$420C in whatever state they were in before starting a game. As of build #11331, I do stz $420B : stz $420C (details in the commit).

I can't test your new ROM right now but will do so once I get home.

EDIT: Okay, tried it just now on both the PowerPak and sd2snes. I didn't encounter any glitches (game loading: ~3 times, tapping Reset: ~20 times for each cart). Also, I tested the older dmac_align.sfc (without the _i in the file name) on sd2snes as well as I was curious if it would differ from the PowerPak. I got those moving glitches after about the fifth time of pressing Reset. (IRQ hooks on sd2snes were disabled, BTW.)

[93143]

That's great news; thanks!

I don't mean to torment you with goalpost-shifting, but did you power-cycle a few times? It might be different from reset.

Anyone else who wants to test the ROM and report the results is welcome to do so as well. Has anybody got a 1CHIP or SNES Jr.? I know it's not that important in the grand scheme of things, but I kinda want this to work across the board.

[Sik]

I imagine that any effect power cycling could have would be greatly nullified by a firmware running first. Is there any flashcart that allows running games directly without any firmware running first?

[tokumaru]

I imagine that any effect power cycling could have would be greatly nullified by a firmware running first.

But if the firmware code can change whatever power up state affects the visual effect (which you're assuming it does), couldn't the ROM itself do the same and completely avoid glitches, making power cycles pointless?

[AWJ]

More like "how can flashcart firmware screw up with DMA at all?". Unless it's leaving some register in a non-default state...

Regarding timing, on the MD the HV counter is used for loose timing, then NOPs for some fine timing, then it causes a FIFO overflow to sync with the VDP (the 68000 on its own simply can't do it). Is there a way on the SNES to make the SPPU mess with the timing of the 65816 in a similar way?

Also I wonder what are the differences between slow ROM and fast ROM here.

As far as I know, nothing external can halt the S-CPU or affect its timing. That's why the SA-1, for example, has to defer to the S-CPU when both access ROM at the same time.

Since the NES, Nintendo has liked to cram as much logic as possible on die on their CPUs; on the SNES, WRAM refresh cycles and memory wait states are actually generated by the S-CPU itself, and the S-CPU also decodes the WRAM and cartridge ROM address ranges and outputs the corresponding chip select signals directly.

[Near]

But if the firmware code can change whatever power up state affects the visual effect (which you're assuming it does), couldn't the ROM itself do the same and completely avoid glitches, making power cycles pointless?

Pretty much all hardware will interfere.

The Super UFO resets all WRAM to 0x00, leaves its logo in the PPU WRAM (enable the display register and you'll see the UFO loading screen), and somehow manages to shatter open bus. When you read from eg $20ff, you're supposed to get back $20 (last byte the CPU fetched successfully), but instead you get 0x00. This causes bugs in DKC2 with barrel rolling, and completely breaks Speedy Gonzales in level 6-2.

The sd2snes seems to be the best so far, but it also destroys state upon loading games, and the reset trick invokes very weird issues reminiscent to the NES' different PPU phases at startup (asserting reset from the cartridge seems to only reset the CPU/APU, and not the PPUs.)

[Sik]

Well to be fair you want the ROM to work even if somebody presses Reset so that means having it to work from a completely unpredictable state.

[93143]

Well, it looks like we're about done here for the time being. Considering I seem to have correctly guessed why the code wasn't working on my first try, both times, I'm going to just assume it works properly until such time as I can justify buying a SNES Jr. (for unrelated reasons: I'll want to do compatibility tests for my shmup port).

Thanks to everyone who helped out! Maybe something interesting will come of this knowledge someday...

...

Only one thing still bugs me. It looks like the dead line problem doesn't happen on a real SNES. So why does it happen on all versions of bsnes/higan?

it also allows me to start the DMA before the beginning of the active scanline and buffer enough colours to get through the DRAM refresh

I still don't entirely get this because DRAM refresh is smack in the middle of the screen for 40 clocks.

The main/sub trick allows me to preload a number of colours in advance by increasing the offset between the loading and display patterns. This allows me to start with a buffer that stays filled until I need it, without requiring me to break up the linear bitmap format of the image.

I start each line well back in HBlank. I manually set the CGRAM index to the index used at the beginning of the display pattern and start the DMA early enough that four colours get written during HBlank, with the CGRAM address incrementing normally without interference from the PPU. The first column of the loading pattern is actually the same index as the fifth column of the display pattern, and it remains four columns (16 dots) ahead until DRAM refresh.

DRAM refresh is handled by making one of the columns in the loading pattern 14 pixels wide instead of 4. This leaves the loading pattern only 6 dots ahead of the display pattern rather than 16. (If what fullsnes says about half-dot CPU 'wakeups' during refresh is true, it should be possible in principle to hit one of those and have the write go to the wrong pattern, but I... guess I dodged a bullet?)

loading.png

display.png

You may notice that the actual data in the ROM has some extra colour and palette indices offscreen. That's a relic of my (successful) attempt to fix the dead line problem in bsnes/higan. But if the problem doesn't show up on a real SNES, why then it doesn't need fixing, does it?

As for why there's a colour #1 stripe at the right-hand edge of the loading pattern... I was trying to get the PPU to reset the CGRAM index for me, so I wouldn't have to waste time doing it manually. But it didn't work.

...

Source is attached. There are a few inconsequential details that are different from last time, such as the IRQ clear during the NMI routine - I was paranoid about breaking it this time, so I didn't change anything but the comments and formatting. The checksum should be the same as dmac_align_i.sfc.

@tepples: Should I try to find a different picture? I've just kinda been using your photo as a reference standard...