Nine sprites overflow doesn't work in the beginning

[DRW]

I found some strange behavior:

When you put nine sprites on a scanline and check for the sprite overflow flag (bit 5 in register $2002) during rendering, then it tends not to work if you do it too early in the game.

I.e. the game freezes at startup, but it works after a reset.

The freezing only happens on a real cartridge. Not in emulators and not on a PowerPak.

It only happens at startup.

If the game is already playing for some seconds and you only check for the sprite overflow then, it never freezes.
I.e. it doesn't spontaneously happen in the middle of the game. Seems to have more to do with the PPU not being ready yet.

(That's probably also the reason why you never see this on a PowerPak: Once you navigated through the menu to get to your game, the time when this freeze can happen has long gone.)

However, the same issue does not happen if you check for the sprite 0 flag instead of the sprite overflow flag, even if you keep everything else the same.
(Don't forget to put a non-transparent background tile at the location of sprite 0 in this case, so that it works at all.)

I put a sample source code and the ROM that demonstrates the behavior into the attachments.


The issue was originally discussed here: viewtopic.php?f=2&t=15737

But this is not a problem regarding my game anymore.
(I simply skipped the oveflow check when I'm in text screens and only checked for it during gameplay.)

Instead, this is about general analysis of the NES' behavior.
If I haven't made an obvious mistake, this is maybe actually undocumented behavior that belongs to the NESDev wiki.

Also, the last thread was more about shots in the dark (trying this, trying that) while the current one is about the definite source of the freeze, with a test ROM and code.

So, we don't need to speculate whether the sprites in the PPU are properly set etc.
If you have an idea what might be wrong, you can just have a look at the source code of the minimalistic test ROM and see for yourself whether I actually made the mistake you're suspecting.

That's why I decided this warrants a new thread, so that people don't have to scroll through all the other stuff from the old thread that turned out not to be the issue.

[DRW]

Source code for quick view:

Code: Select all

.segment "HEADER"

	.byte "NES", $1A
	.byte 1
	.byte 1
	.byte 1


.segment "ZEROPAGE"

	Pointer: .res 2
	WaitForNmi: .res 1
	Counter: .res 1

.segment "SPRITES"

	Sprites: .res $100

.segment "CODE"

Reset:

	; Initialization
	SEI
	CLD
	LDX #$40
	STX $4017
	LDX #$FF
	TXS
	INX
	STX $2000
	STX $2001
	STX $4010

	; VBlank 1
	BIT $2002
@waitForVBlank1:
	BIT $2002
	BPL @waitForVBlank1

	; RAM to 0
	TXA
	TAY
	STA Pointer
	STA Pointer + 1
@initializeRamOuterLoop:
@initializeRamInnerLoop:
	LDA #0
	STA (Pointer), Y
	INY
	BNE @initializeRamInnerLoop
	INC Pointer + 1
	INX
	CPX #$08
	BNE @initializeRamOuterLoop

	; VBlank 2
	BIT $2002
@waitForVBlank2:
	BIT $2002
	BPL @waitForVBlank2

	; Sprites out of screen
	LDX #0
	LDA #$F4
@spritesLoop:
	STA Sprites, X
	INX
	BNE @spritesLoop

	; Nine sprites (not sprite 0) in one row
	LDX #4
@nineSpritesLoop:
	LDA #88
	STA Sprites, X
	INX
	LDA #$0F
	STA Sprites, X
	INX
	LDA #0
	STA Sprites, X
	INX
	STA Sprites, X
	INX
	CPX #40
	BNE @nineSpritesLoop


	; Sprites for counter
	LDA #120
	STA Sprites + 40
	STA Sprites + 43
	STA Sprites + 44
	LDA #0
	STA Sprites + 42
	STA Sprites + 46
	LDA #120 + 8
	STA Sprites + 47

	; First thing to run: NMI
	LDA #1
	STA WaitForNmi

	; NMI enabled
	LDA #%10010000
	STA $2000

	; Game logic
@gameLogic:

	LDA WaitForNmi
	BNE @gameLogicEnd

	; Wait for sprite overflow bit
@waitForNotOverflow:
	LDA $2002
	AND #%00100000
	BNE @waitForNotOverflow
@waitForOverflow:
	LDA $2002
	AND #%00100000
	BEQ @waitForOverflow

	; Update counter on screen
	LDA Counter
	LSR
	LSR
	LSR
	LSR
	STA Sprites + 41
	LDA Counter
	AND #%00001111
	STA Sprites + 45

	INC Counter

	; Wait for NMI
	LDA #1
	STA WaitForNmi

@gameLogicEnd:

	JMP @gameLogic


Nmi:

	PHA
	TXA
	PHA
	TYA
	PHA

	LDA WaitForNmi
	BEQ @end

	LDA #0
	STA WaitForNmi

	; PPUMASK enabled
	LDA #%00011110
	STA $2001

	; Sprite OAMDMA
	LDA #$00
	STA $2003
	LDA #>Sprites
	STA $4014

	; Palette
	LDA $2002
	LDA #$3F
	STA $2006
	LDA #$11
	STA $2006
	LDA #$0D
	STA $2007
	LDA #$05
	STA $2007
	LDA #$02
	STA $2007

	; Scrolling
	LDA #0
	STA $2005
	STA $2005

	; Name table
	LDA #%10010000
	STA $2000

@end:

	PLA
	TAY
	PLA
	TAX
	PLA

	RTI


.segment "VECTORS"

	.word Nmi
	.word Reset
	.word 0


.segment "CHARS"

	.incbin "Graphics.chr"

Config file:

Code: Select all

MEMORY
{
	HEADER:  type = ro, start = $0000, size = $0010, file = %O, fill = yes;
	PRG_ROM: type = ro, start = $C000, size = $4000, file = %O, fill = yes;
	CHR_ROM: type = ro, start = $0000, size = $2000, file = %O, fill = yes;
	ZP:      type = rw, start = $0001, size = $00FF, file = "";
	RAM:     type = rw, start = $0200, size = $0400, file = "";
}

SEGMENTS
{
	HEADER:   load = HEADER,  type = ro;
	CODE:     load = PRG_ROM, type = ro;
	RODATA:   load = PRG_ROM, type = ro;
	VECTORS:  load = PRG_ROM, type = ro,  start = $FFFA;
	CHARS:    load = CHR_ROM, type = ro;
	ZEROPAGE: load = ZP,      type = zp;
	SPRITES:  load = RAM,     type = bss, align = $0100;
	BSS:      load = RAM,     type = bss;
}

[tokumaru]

Could this mean that it takes longer than the typical PPU warm up time for sprite behavior to stabilize? Hopefully someone will be able to test this properly.

BTW, if you're using the sprite overflow to time raster effects, you may want to make the wait loop tighter in order to minimize jitter:

Code: Select all

   LDA #%00100000
@waitForOverflow:
   BIT $2002
   BEQ @waitForOverflow

[tepples]

Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame:

Code: Select all

   LDA #%10100000
@waitForOverflow:
   BIT $2002
   BEQ @waitForOverflow

[DRW]

Could this mean that it takes longer than the typical PPU warm up time for sprite behavior to stabilize?

That's probably what it leads up to.

BTW, if you're using the sprite overflow to time raster effects, you may want to make the wait loop tighter in order to minimize jitter

Yeah, I've heard this somewhere.

That one only works for this specific bit, right? It wouldn't work for sprite 0?

In my actual game code, I used the "wait for certain $2002 bit" functionality as a macro, so that I only had to pass the actual value since I called this both, for sprite overflow and sprite 0 on two different locations on the screen.

Also, in my specific case, this didn't make a practical difference. The place where the scrolling change happened was a scanline where the whole line consists of one continuous color. (Top: Status bar. Middle: Fog.) So, graphical artifacts like in "Journey to Silius" were a non-issue.

But yeah, in general, this might be a nice tip.

Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame

I would advise against this. If the flag doesn't turn on, there's some mistake and the code needs correction. I wouldn't want to hide this.

[tokumaru]

It wouldn't work for sprite 0?

For sprite 0 hits you can use BVC, just like you use BPL for the vblank flag. There's no need to bother setting up a mask, since BIT copied bits 7 and 6 to the N and V flags respectively, so you can just check the V flag. But if you want to keep using a generic macro, there's nothing stopping you from using %01000000 as a mask.

One way to wait for the sprite 0 hit:

Code: Select all

WaitForSpriteHit:
  bit $2002
  bvc WaitForSpriteHit

Another way to do it, and the loop is just as fast:

Code: Select all

  lda #%01000000
WaitForSpriteHit:
  bit $2002
  beq WaitForSpriteHit

[tepples]

Could this mean that it takes longer than the typical PPU warm up time for sprite behavior to stabilize?

That's probably what it leads up to.

In particular, I'm under the impression that some of the OAM DRAM refresh circuitry doesn't stabilize until one frame has been completely rendered, or at least one scanline. A test ROM similar to OAM reset might help characterize this.

Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame

I would advise against this. If the flag doesn't turn on, there's some mistake and the code needs correction. I wouldn't want to hide this.

Some mistakes cannot be fixed; they must be worked around. One example of such a mistake is a mistake in the design of OAM DRAM refresh. Time is money, and it might take less time for you to put a workaround like this in place and get your game out the door than to wait until this misbehavior is nailed down.

[DRW]

Some mistakes cannot be fixed; they must be worked around.

Well, I did:

In my game, text screens don't actually need any scanline splits, only the level does. In my old version, the sprites for the split were still there, though.

So, I simply changed it to the following:
When we initialize a text screen, we put the 10 "system sprites" out of the screen. Only in levels are they set to their correct location.

At the code where the overflow and sprite 0 bit is waited for, I simply check: If address of Sprites + 0 = $F4 (i.e. if the y position of sprite 0 is out of the screen), then we skip the scanline split waits entirely.
I had to do a similar thing anyway, even in the old version: If PPUMASK = 0, then skip them as well.
So, the new version simply has one more check.

[rainwarrior]

Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame

I would advise against this. If the flag doesn't turn on, there's some mistake and the code needs correction. I wouldn't want to hide this.

There isn't really a danger that it will "hide" the problem because it still produces a visual corruption of the screen from missing the split point.

It does, however, make the difference between a total hang/crash and just some temporary visual problems and/or slowdown until the issue resolves. As a player, if I've been playing something for an hour I'd much prefer a few seconds of glitchy graphics that I can recover from than a hard crash I have to completely restart from.

The PPU isn't totally reliable even in the best of conditions. Vibrations, power fluctiuations, dirt in the pins, etc. can and do cause occasional CHR corruptions in otherwise "bug free" games. I test for vblank in my sprite-0 waits myself for this reason- I don't trust the hardware completely.

[DRW]

O.k., yeah, this is a good argument.

However, in this specific thread, the optimal way to wait for sprite 0 or failsafe ways to wait for scanline splits is really not my focus.
As I said, in my actual game, the issue is already circumvented by simply not checking for sprite overflow when you are in text screens.
The publisher already has the new ROM, so I won't work on the game anymore.


Instead, this thread was not created out of an immediate problem, but because I'm genuinely interested in the actual technical cause of this behavior.

Because this might be a discovery about the NES' PPU that isn't documented yet, so maybe people who have more knowledge than me and who own the required equipemt can analyze the issue.


The sprite overflow fails to be set at an early point in the game, unless you use a soft reset.

If you check for sprite 0 in exactly the same way (plus putting an opaque tile at the required location, of course), this does not fail.
It's only the overflow bit, not the sprite 0 bit.

Why is this the case?

Is there an oversight in my code (see above) or is this some wiki-worthy piece of information about the PPU that hasn't been discovered yet because nobody ever relied on the sprite overflow bit being set right at the start?

[tokumaru]

If you check for sprite 0 in exactly the same way (plus putting an opaque tile at the required location, of course), this does not fail.
It's only the overflow bit, not the sprite 0 bit.

Are you sure that the sprite 0 hit *never* fails? Or could it simply be that OAM corruption is more likely to affect 1 out of 9 sprites than 1 out of 1? What about the overflow, does it *always* fail on cold boot?

[tepples]

If the behavior is that the display list entries for sprites 0 and 1 overwrite the display list entries for another randomly chosen even-odd pair of sprites, then allow me to describe a situation in which sprite overflow fails.

Say sprites 1-9 are for the overflow trigger, and sprite 0 is at least 16 pixels below sprites 1-9. If sprites 0 and 1 overwrite sprites 2 and 3, 4 and 5, 6 and 7, or 8 and 9, then you'll have only eight sprites on that line. But because sprite 0 is not overwritten, it will still be displayed.

[tokumaru]

How can we reliably test this to confirm it as a new discovery?

[rainwarrior]

For the purposes of the test, maybe give the 9 sprites numbered tiles and spread them out horizontally so all of them can be seen individually. You might be able to see which one has failed this way when it hangs.

Edit: have made this, is attached.

[DRW]

Are you sure that the sprite 0 hit *never* fails?

When I had the problem, I asked my publisher to check some ROM versions that I created out of my game. In one of them, I disabled the overflow check, in one I disabled the sprite 0 check.

Disabling sprite 0 check still produced the error. (Because the overflow is the problem.)
Disabling overflow check, but keeping sprite 0 check, worked correcly.

Of course, I can't be sure. I assume he didn't test it 50 times. This would have to be done by someone who has the equipment, time and motivation.

But since sprite 0 check is pretty common in actual games (I assume "Super Mario Bros." also has it right at the beginning, doesn't it?) while no game really checks for overflow right at the start, this result is actually believable to me.

What about the overflow, does it *always* fail on cold boot?

Mostly, but not always. Just like I also had rare instances of my game still not working after a reset.

One interesting thing about this: The publisher sent me two cartridges of my game. One of them is more likely to fail at cold boot than the other one. One of them actually had a pretty high success rate (maybe 30-50 %) compared to the other one that almost never worked at startup.
So, the frequency of the error occuring also seems to be related to the individual boards.

Also:
No issue whatsoever with the same cartridges on a PAL system. (I tested this myself. I own an Amercian and a European NES.)
And a clone system that the publisher owns doesn't produce the behavior either.
Likewise, the problem occurs regardless of whether you use a front loader or top loader NES. (I have the front loader. And some person from NintendoAge confirmed me the bug on a top loader.)

Say sprites 1-9 are for the overflow trigger, and sprite 0 is at least 16 pixels below sprites 1-9. If sprites 0 and 1 overwrite sprites 2 and 3, 4 and 5, 6 and 7, or 8 and 9, then you'll have only eight sprites on that line. But because sprite 0 is not overwritten, it will still be displayed.

Huh? I'm not sure whether I get this. What do you mean with "overwriting"?
From a pure code logic, I can guarantee you that nothing overwrites anything. I reserved the first 10 sprites for this specific purpose. Neither in my game, nor in the above test code are they ever used for anything else but for the scanline splits, and sprite 0 is never part of the overflow sprites.
If you are referring to some internal NES-specific behavior: Maybe, maybe not. I don't know

By the way, is there a way to set the PowerPak into a mode that a game is already loaded at startup instead of the PowerPak going into its own menu first?