kulor wrote: ↑Sat Apr 30, 2022 11:43 pm
I also don't think VRAM is a concern, because, while it's true that you need 4x more VRAM for the same coverage for your graphics, you also lose out on a whole 4bpp layer's worth of VRAM consumption, since mode 5 trades a 4bpp layer for that extra horizontal crispness.
Wanted to elaborate on this, because after doing some more investigation, I'm definitely convinced VRAM is not a concern for mode 5. Something to consider is that character addressing still works in 8x8 tile chunks with 16x16 tiles, so the VRAM consumption of all the unique tiles you can address per-layer is actually the same regardless of if you use 8x8 tiles or 16x16 tiles, high-res or low-res.
For example, in mode 1, let's say you want as many unique tiles as possible for each layer. For each of the 4bpp layers, you need 1024x 8x8 tiles, which are 32 bytes each, for a total of 32KB VRAM, multiplied by two for the two layers. For the 2bpp layer, the tiles are 16 bytes each, so you need 16KB for that. Grand total is 80KB of VRAM use, which is more than you actually have, so you can't actually have the maximum number of unique tiles for all layers in mode 1.
In mode 5, because it's 16x16 tiles, you only get 256 unique tiles, as well as the probably-useless half-steps between unique tiles. For the 4bpp layer, you need 256x 16x16 tiles, which are 128 bytes each, for 32KB. For the 2bpp layer, they're 64 bytes each, so it's 16KB. Grand total is just 48KB, which is actually less than mode 1!
In fact, if you had 64x32 tilemaps for each of the layers, you could theoretically fill up your VRAM with 256 unique 16x16 tiles for the 4bpp layer, 128 unique 16x16 tiles for the 2bpp layer, and have your 8KB tilemaps and 16KB sprite space too. I think it's fair to say that "mode 5 uses too much VRAM" is a myth.