Success compressing tiles?

Bregalad · Post by **Bregalad** » Sun May 06, 2007 1:08 am

Wow, if you keep the same compression rate, you'll be able to put in 1.55 times the tiles you could put before, that means you can have more than a half more the data. I think this is great, I don't undestand why you're not looking satisfacted with that.

tokumaru · Post by **tokumaru** » Sun May 06, 2007 1:51 am

Bregalad wrote:I think this is great, I don't undestand why you're not looking satisfacted with that.

Hum... I guess it's because regular file compressors can compress the same data to about 60% of the size. I think I'd like to reach that mark too.

Anyway, I still have some tweaking to do, I'll experiment with other ways of transforming the tiles based on previous ones, maybe even implement multiple transforms so that the encoder picks the one that results in better compression, PNG style. The bad part is the overhead of indicating the type of transform used, but the numbers will tell. The delta-coding itself came with a lot of overhead, as each bit plane must indicate if it was XOR'ed with a previous one, and if so, which one. But it still improved compression.

Heh, it's fun to try this thing with other types of data... most things actually expand, but it did compress some text documents I had...!

Post by **Memblers** » Sun May 06, 2007 2:03 am

That does seem really good. For comparison I tried FilePack on SMB1's CHR, and it reduced to 71% of the original size. Your newer method did better than that.

I hope you'll release a utility for people like me, who always use CHR-RAM.

Compression makes a huge difference when memory is limited. Hot Seat Harry used RLE compression, even for a data table. On Garage Cart #1, a lot of the intro (characters for the menu and Nerdtracker 2 + 3 songs) fit in the space reclaimed by compressing CHR. It's a 64kB ROM, and Solar Wars on there was originally 64kB by itself.

If I would've had better compression than RLE, I could've thrown more music on there.

tokumaru · Post by **tokumaru** » Sun May 06, 2007 10:07 am

Memblers wrote:That does seem really good. For comparison I tried FilePack on SMB1's CHR, and it reduced to 71% of the original size. Your newer method did better than that.

I still have to test it with other data, SMB has only been my primary test material, but later today when I get the time I'll try other tile sets, larger ones, smaller ones, and so on.

I hope you'll release a utility for people like me, who always use CHR-RAM.

Heh, You know, I have been programming for the web for so long that I kinda forgot how to code anything other than that (and the NES)! I've been trying this out with FreeBASIC, but the code is really messy and I don't know if I could turn it into a good standalone utility. I'll try, though. I had some trouble switching to object-oriented programming, and I guess I still have trouble with that sometimes.

I'm also still experimenting a bit to see if I can improve compression even more, and when I think I can't improve it anymore I'll decide the final format and code the final compressors and decompressors.

I also have to code the decompression routines in 6502 asm, and see how fast it goes. Manipulating bits is never a fast thing! =)

Compression makes a huge difference when memory is limited. Hot Seat Harry used RLE compression, even for a data table. On Garage Cart #1, a lot of the intro (characters for the menu and Nerdtracker 2 + 3 songs) fit in the space reclaimed by compressing CHR. It's a 64kB ROM, and Solar Wars on there was originally 64kB by itself.

That is impressive! =)

dvdmth · Post by **dvdmth** » Sun May 06, 2007 10:11 am

You probably want to test your algorithm(s) on multiple tile sets. You don't want to optimize for one tile set only to make the algorithm less effective on the average case.

tokumaru · Post by **tokumaru** » Sun May 06, 2007 10:29 am

Yes, yes, of course. I just didn't have the time to test with other tile sets after modifying the compression scheme. I'm not at home, and if I wanted other tile sets, I'd have to get an hex editor, get ROMs, and so on.

I'll go back home tonight, where I already have some tile sets ready for testing.

Bregalad · Post by **Bregalad** » Sun May 06, 2007 11:14 am

dvdmth wrote:You probably want to test your algorithm(s) on multiple tile sets. You don't want to optimize for one tile set only to make the algorithm less effective on the average case.

If it would be possible to make a compression sheme effective for a particular tileset, then if you'd be coding a game with compressed tiles, you'd definitely want a compression sheme particulary efficient for your game's tileset. I don't think it works that way tough.

NB : A fun thing is that I remember Tokumaru stating earlier that CHR-ROM was better than CHR-RAM, and I was stating the opposite. And now, Tokumaru has a project where he's using CHR-RAM, and me I've a project where I use CHR-ROM. I think that's kinda ironical.

tokumaru · Post by **tokumaru** » Sun May 06, 2007 7:09 pm

Bregalad wrote:A fun thing is that I remember Tokumaru stating earlier that CHR-ROM was better than CHR-RAM, and I was stating the opposite. And now, Tokumaru has a project where he's using CHR-RAM, and me I've a project where I use CHR-ROM. I think that's kinda ironical.

You are absolutely right! =D I remember that! Know what? I got really spoiled by CHR-RAM now! =D I guess I do take back some of the things I said back then!

Turns out that, by taking some of the time of the rendered frame, I can simulate CHR bankswitching, in small chunks of 16 tiles per frame. I'm using this to update the main character, doing some background animations and so on.

Bregalad · Post by **Bregalad** » Mon May 07, 2007 8:51 am

Yes, but this is almost cheating (i.e Battletoads

)
I really think the best is to take the best from CHR ROM and CHR RAM, because both have their ups and downs, depending on what you want to show on the screen.

Post by **tepples** » Mon May 07, 2007 6:45 pm

Bregalad wrote:
tokumaru wrote:Turns out that, by taking some of the time of the rendered frame, I can simulate CHR bankswitching, in small chunks of 16 tiles per frame. I'm using this to update the main character, doing some background animations and so on.
Yes, but this is almost cheating (i.e Battletoads )
I really think the best is to take the best from CHR ROM and CHR RAM, because both have their ups and downs, depending on what you want to show on the screen.

It's not cheating. It's what just about every game for Genesis, Super NES, and Game Boy Advance does. Seriously, back up your Sonic Advance or Kirby: Nightmare in Dream Land Game Pak, load it up in VisualBoyAdvance, open the pattern table viewer (Tools > Tile Viewer), and turn on automatic refresh. Scroll down to 0x06010000 (sprite cel VRAM), make Sonic run and jump, and see what CHR changes. This white paper was written near the beginning of the GBA homebrew scene.

Bregalad · Post by **Bregalad** » Tue May 08, 2007 9:08 am

I know a lot of no-NES games does this (at the end they don't even bother mazing tiles, they just maze hard-coded tiles with hard cored positions on the screen, and put whathever graphics they need for the game's protagonist(s) here). I noticed at least Chrono Trigger, Golden Sun and Mega Man Battle Network does that (some of my all time favourite games), so I wouldn't be surprised if that would be the case of others as well.
However, Battletoads is the only one that was able to do this for the NES ever. I of course exaggerate and this is by no means cheating, but you have to admit doing this on the NES is insane, due to the slow CPU and lack of $2007 DMA.
I noticed Final Fantay IV advance goes a step further : In the menu, the map is just filled with continus tiles, and the whole tileset is modified to modify the content of the screen (like a bitmap image). This reduce the respond rate to scrolling menu commands, so they did it slightly differently in FF5 and FF6 advance.

Post by **tepples** » Tue May 08, 2007 9:27 am

Bregalad wrote:I noticed Final Fantay IV advance goes a step further : In the menu, the map is just filled with continus tiles, and the whole tileset is modified to modify the content of the screen (like a bitmap image).

Like how Qix (NES), Qix (Game Boy), or Videomation (NES) works, right?

Bregalad · Post by **Bregalad** » Tue May 08, 2007 10:05 am

Well, I think so, and that because they don't use a regular 8x8 font, but a smaller font (because the menu originally had to fit a larger screen on the SNES).

CartCollector · Post by **CartCollector** » Thu May 24, 2007 3:40 pm

I think Moon Crystal updates CHR-RAM to show animation, considering how smoothly the characters move. I'll have to check.

EDIT: Strangely enough, it doesn't use CHR-RAM, but it does dynamically update the characters (sort of). I've noticed that there is a 32-tile area in the pattern table that changes depending on how the player character is moved, but does not update if the character continually moves in the same direction. So I'm guessing that the ROM is bankswitched depending on how the character is moved, and that that bank holds all of the animations needed for that movement (since the character is 8 tiles, there is enough for 4 still frames per movement).

tokumaru · Post by **tokumaru** » Thu May 24, 2007 3:52 pm

Many games do this. I think most machines at the time had their patterns in RAM, and the NES was a rare exception that had many games with patterns stored in ROM.

It was only natural that programmers would find some free time to update the patterns during gameplay for more dynamic graphics.