nesdev.org

Blargg has an instantiation of streaming lossless 32kHz stereo 16-bit audio to the SPC.

More practically, psychopathicteen implied he had an instantiation of streaming 22kHz mono BRR audio using just HDMA, freeing up the main CPU to do useful things.

KungFuFurby wrote:There are two ways I can think of this. One of them is via the echo buffer, and the other is by streaming BRR samples.

The problem with streaming to the echo buffer is that you're forced to choose between low quality and high CPU time twice.

First, storage - if you compress it, the S-CPU has to waste time decompressing it, and if you don't, well, you're storing uncompressed audio in the ROM.

Second, bandwidth - you can't DMA to the APU, and it's not fast enough to get data from more than one HDMA channel at a time, so if you want to feed it high-quality uncompressed audio you have to spend a lot of time in a handshake loop.

Now, I've never actually used the audio module, so maybe I'm missing something...

Perhaps you speak of a sound driver that simply loads two BRR samples (one for the left side and one for the right side), plays them at the desired sample rate and has a good sized buffer (make sure at the end of your buffer you have a loop block in there... you can just embed them in the sample)?

I'm assuming it's pretty trivial to stream BRR once you know what you're doing (I don't). What I'm wondering is if there'd be enough processing time left to trigger additional highpassed 32 kHz samples for the crash, hat, snare, vocal sibilants and a few synth sounds that might need extra sparkle. That way we could do a decent imitation of 32 kHz 16-bit stereo without having to actually encode the whole song at that bitrate.

The third option would be to stream compressed 8-bit audio into the APU and have the SPC700 decompress it, saving to first order ~50% of the storage space at the cost of bit depth, but as I said I'm not sure how feasible that is or how good it would sound if it were...

lidnariq wrote:More practically, psychopathicteen implied he had an instantiation of streaming 22kHz mono BRR audio using just HDMA, freeing up the main CPU to do useful things.

Why 22 kHz? It seems like that just fits into a single byte per scanline (with a couple kHz to spare); can't you use more than one I/O port?

You know, why is storage space such a big issue here? I'd much rather have superior audio quality, even if it means using a 32Mb cartridge... (Which there are enough of to use, if this even sees cartridge form.)

Difference between Mb and MB. The video data, as matters now stand, takes up more than 64 Mb (8 MB), and 32 kHz stereo BRR would take almost as much. So the lazy ideal case doesn't even fit in the memory map.

I got the decoder to work.

Now how is this going to fit into a ROM? It needs to extend between both slowROM and fastROM regions, but get around the 128kB of RAM right in the middle of it. I think we need to split the video into 2 parts, and have some jump in the middle when it gets to a specific frame. It needs a slightly better compression to fit in 8MB. Maybe if the tile map is processed vertically instead of horizontally, we can squeeze it in there.

In the mean time, can you make a shortened 1 minute version of the video, so I can test the frame rate?

93143 wrote:Difference between Mb and MB. The video data, as matters now stand, takes up more than 64 Mb (8 MB), and 32 kHz stereo BRR would take almost as much. So the lazy ideal case doesn't even fit in the memory map.

Oh... Well, I guess you could compress the video, but leave the audio in tact and it would maybe fit in a 96 megabit cartridge? (I'm not basing this on anything.)

psycopathicteen wrote:It needs a slightly better compression to fit in 8MB.

Does it need to fit in 8 MB?

I mean, even if you get it down below 8 MB, you've still got no audio. And if the whole thing has to fit in 8 MB, you'd need to shrink the video a lot more than slightly before the audio would sound decent...

Espozo wrote:Well, I guess you could compress the video, but leave the audio in tact and it would maybe fit in a 96 megabit cartridge? (I'm not basing this on anything.)

They're already compressing the video. If it turns out they've got spare CPU time, the question then is: how much more can they compress it and still have the CPU be able to unpack it at speed?

Once the video is as small as it's going to get, we'll know how much space there is for audio.

psycopathicteen wrote:I think we need to split the video into 2 parts, and have some jump in the middle when it gets to a specific frame.

Bad Apple for NES put a special marker at the end of each 8192-byte bank's data that caused it to fetch the next frame from the start of the next bank. Once we get a full-size ROM, I could have it put such a marker after each 32K bank, which should allow use in both the low and high sections. I've made a copy of the first minute with each 32K bank padded out with $FF bytes.

I might investigate which 8x1-pixel slivers are most common. Allowing only 256 visually distinct slivers will slow decoding slightly and cause slight picture degradation but hopefully not too much.

I have the first half of Bad Apple working, but the frame rate jumps back and forth between 60fps and 30fps, depending on how much detail the frames are.

Is it supposed to jump between frame rates? it looks like the original video runs at 30fps.

My favorite part!

Not half bad. The fact that some frames are running at 60 fps shows that you can spend some unused time on uploading audio to the SPC. But when I ran it in NO$SNS, I noticed that some frames appear to be exceeding vblank time, causing a flickering black band at the top of more complex white frames.
biggestframe.png has 394 tiles, which means 6304 bytes. This exceeds a single vblank, which means you'll need to spread the copy over two frames. But it also occupies only (394+2)*16*2+2048*2=16768 bytes of VRAM, which means you'll be able to double- or even triple-buffer the tiles easily.

I've got a few lossy approaches to try over the next week, including 4-bit slivers, 8-bit slivers, and a constant tile bank. If you want, can send you simulations of these approaches in the existing compressed format so that you don't have to modify the decoder to play them before we decide on a final codec to use. Did you use the $FF-padded version?

Are the multitrack masters of "Bad Apple" available? If so, then perhaps the instrumental parts can be converted to sequenced music. And it appears a lot of the track is repeated twice.

Looks nice.

But not in everything - I get a black screen in Snes9X v1.53, and garbage in higan v094. ZSNES v1.51 won't even load it. Seems to work fine in bsnes v072 and no$sns, but it's too big for my Super Everdrive so I can't test it on real hardware.
Weird... EDIT: maybe it's because of the outsize memory map?

tepples wrote:I've got a few lossy approaches to try over the next week, including 4-bit slivers, 8-bit slivers, and a constant tile bank.

Could sliver compression be done losslessly? How often would you have to refresh a sliver dictionary to catch all of them?

Is there an easy-to-decode form of compression, like RLE or LZ4 or something, that could be productively applied to the whole compressed data stream without blowing the CPU budget? It seems like LZ77-type compression with a big enough window would do interframe compression automatically, and there should be plenty of RAM for that, but I'm not clear on how much CPU time it would use...

I know you have no obligations to me here, and I am by no means a compression expert, but the MD version that originally sparked this idea was lossless, and I don't like the idea of giving up on matching it so soon. As you once said, "The trolls will still claim superiority"...

...

Maybe in a week or so I'll have enough time to fiddle with the audio track a bit, if KungFuFurby or someone like that hasn't already finished it...

The fact that some frames are running at 60 fps...

Is it just me, or was there some actual slowdown in the part with Youmu and the tree? As in, below 30 fps?

EDIT: It seems to me that WRAM and VRAM combined should have enough room for a fairly substantial frame pipeline, if all else fails...

tepples wrote:Are the multitrack masters of "Bad Apple" available? If so, then perhaps the instrumental parts can be converted to sequenced music. And it appears a lot of the track is repeated twice.

I don't know about multitrack (you'd have to ask Alstroemeria Records for that), but the CD release included a karaoke track. I have yet to attempt subtracting them to separate the vocals, so I don't know if the mastering was done in a way that makes it possible.

It's pretty easy to get a "multitrack" from the original sequenced music, if that would help.

I think we can use a library of the 256 most used slivers. We can use this format for tiles:

IRIRIRIRIRIRIRIR
I: 0=indexed sliver, 1=literal sliver
R: 0=repeat sliver, 1=new sliver

So the "top 256" format has little-endian, LSB-on-top control words where 00 is a repeat, 01 is indexed (one of the top 256 slivers), and 11 is literal (any other sliver), with a separate 512-byte table of the top slivers. I'll assume 10 is undefined. I'll post a 1-frame test image for this format tonight, and while you're working on getting a decoder working, I'll be working on converting the existing bitstream to the new format to see how much space it saves.