APU External Channel Mixing?

[beannaich]

I recently got the Sunsoft 5B board working in my emulator, and was playing Gimmick! to see what the sound is like, when I ran across a horrible noise. I fell into a spike pit, and my speakers crackled loudly. I realize this is because I am not mixing the new sound channel properly.

Code: Select all

var output = (NesApuMixer.MixSamples(sqrSample, tndSample) * 128);

if (output > 0x80)
    output = 0x80;
if (output < 0x00)
    output = 0x00;

if (this.External != null)
{
    output += this.External.RenderSample(sampleRate);
}

this.soundBuffer[wPos++ % this.soundBuffer.Length] = (int)output;

My questions is, what is the proper way of doing it? I came up with a couple ideas, which I'm sure are wrong.

One of them included finding the relationship between the 2A07's maximum output: (45 + 30 + 127) + (15 + 15) = 232, and the external component's maximum output: (15 + 15 + 15) = 45. Is this the proper way of doing it? Forcing more room into the post-mixed 2A07 sample and adding the SS5B sample?

Seems like there is a better way to go about this, and I couldn't find anything using the search (big surprise there). So I feel it would be nice to have this post for others to learn from!

EDIT: For now, this is my solution:

Code: Select all

            var output = (NesApuMixer.MixSamples(sqrSample, tndSample) * OutputMul);

            if (this.External != null)
            {
                output += this.External.RenderSample(sampleRate);
                output *= (OutputMul / (OutputMul + External.MaxOutput));
                // (128 / (128 + 45)) for Sunsoft 5B
            }

            if (output > 0x80)
                output = 0x80;
            if (output < 0x00)
                output = 0x00;

            this.soundBuffer[wPos++ % this.soundBuffer.Length] = (int)output;

[ReaperSMS]

The NES APU isn't linear, see http://wiki.nesdev.com/w/index.php/APU_Mixer

To make sense of your code, could you say what types MixSamples produces, as well as what type the soundBuffer is, and what output format you're trying to mix to?

The formulas on that page spit out a unipolar floating point value (0-1) for the APU. If you have your external unit rendering to the same range, you can mix them by scaling each by 0.5, and adding them, resulting in a value that still remains in the range of 0-1. The downside is everything gets quieter, but you don't clip.

To convert to say, signed 16-bit output, you scale it by 65536, subtract 32768, and clamp to -32768..32767, then convert to int and store.

If you don't want to chop the volume down, you can try more complicated schemes to keep the audio within range. One simplistic auto-gain control would be to track the maximum amplitude (amplitude = 2.0f * fabs(sum - 0.5)) of the summed audio, and divide by the max before the clamp operation above. You'd probably hear the volume shift abruptly occasionally, but it should settle to the highest value that avoids clipping overall.

[Zepper]

Could someone "translate" the following:

Code: Select all

pulse_out = \frac{95.88}{\frac{8128}{pulse1 + pulse2} + 100}

...into a C-language equivalent expression, please?

[tepples]

\frac{a}{b} in TeX is something like (a)/(b) in C. Thus the assignment would translate as

Code: Select all

pulse_out = (pulse1 + pulse2) ? 95.88/(8128/(pulse1 + pulse2) + 100) : 0;

And you'd probably implement this by creating a table for pulse_out where (pulse1 + pulse2) varies from 0 to 30.

[ReaperSMS]

http://wiki.nesdev.com/w/index.php/APU_Mixer_Emulation has it in C.

[beannaich]

I am implementing the non-linear mixing scheme already in my code, this is what the call to MixSamples does.

Code: Select all

private static float[] sqrTable;
private static float[] tndTable;

static NesApuMixer()
{
    sqrTable = new float[0x0F * 1 + 0x0F * 1 + 1];

    for (int i = 0; i < sqrTable.Length; i++)
        sqrTable[i] = 95.52f / (8128.00f / i + 100);

    tndTable = new float[0x0F * 3 + 0x0F * 2 + 0x7F * 1 + 1];

    for (int i = 0; i < tndTable.Length; i++)
        tndTable[i] = 163.67f / (24329.00f / i + 100);
}

public static float MixSamples(int sqrSample, int tndSample)
{
    return (sqrTable[sqrSample] + tndTable[tndSample]);
}

My question was, Is there some sort of formula for mixing external channels? Or is it just "keep the normalized output range when adding other channels, using relational formulas"? If that's the case, the solution I have works damn fine!

Should I be converting my samples (currently ranging from 0-128) to 16-bit samples ranging from -32768 to 32767?

By the way, nothing is wrong with the 2A07 channels in my emulator, they all sound fine. I'm only worried about the external channels (MMC5, SS5B, VRC6, VRC7) here.

[ReaperSMS]

Docs on the actual mix levels for the external chips are rather thin on the ground. I believe most things assume the external chips are linear, and mix 50/50 with the NES output.

The format of your samples depends on what's actually playing them. The two most common output formats are unsigned 8-bit (0..255) and signed 16-bit (-32768..32767). You should probably be adjusting your output to fit the range of whichever your playback hardware/API wants.

If your hardware actually wants 8-bit signed, then that horrible noise you got before was the value passing 128 (and thus going negative) when you added the sunsoft in. If that is the case, you should be mixing them to 0..255, and subtracting 128 before storing in soundBuffer. Your current code might still be hitting 128 occasionally, which would do the same.

[beannaich]

Alright, so assumed 50/50 mix.. Here is my current (working) solution:

Code: Select all

public static float MixSamples(int sqrSample, int tndSample, int extSample = 0, float extMax = 0)
{
    var outputMax = (232f + extMax);
    var outputApu = (sqrTable[sqrSample] + tndTable[tndSample]);
    var outputExt = (extSample / extMax) * (extMax / outputMax);

    if (extMax == 0f)
        outputExt = 0f;

    var output = BlockDC(
        (outputApu * (232f / outputMax) + outputExt) * 255 - 128);

    if (output > +127)
        output = +127;
    if (output < -128)
        output = -128;

    return output;
}

And here is doing it your way:

Code: Select all

public static float MixSamples(int sqrSample, int tndSample, int extSample = 0, float extMax = 0)
{
    var outputApu = (sqrTable[sqrSample] + tndTable[tndSample]);
    var outputExt = (extSample / extMax);

    if (extMax == 0f)
        outputExt = 0f;

    var output = BlockDC(
        (outputApu * 0.5f + outputExt * 0.5f) * 255 - 128);

    if (output > +127)
        output = +127;
    if (output < -128)
        output = -128;

    return output;
}

Does that look correct? Also, I don't know if my DC Blocker is even proper, lol. Here it is:

Code: Select all

public static float BlockDC(float sample)
{
    var output = sample - lastAmp + 0.999f * lastOut;
    lastAmp = sample;
    lastOut = output;

    return output;
}

It should be correct, but I have no clue

[ReaperSMS]

looks correct for the 0.5f bit.

I don't know what you're trying to do with that 232 thing, and the outputExt calc looks a little fishy, since it works out to extSample/outputMax.

BlockDC looks like it might do the trick, maybe. A lot of the time, it is unnecessary, unless you're feeding this output to some other thing to mix. The audio card will have a DC blocking capacitor in there.

[tepples]

If you don't block DC in software, you'll have annoying pops as you start and stop the emulator, and the wave files you record (if you include that feature) will have DC in them.

[beannaich]

@tepples, is that DC blocking function correct? I still get those pop sounds when starting/stopping emulation. And that function I found doing a google search on DC Blocking filters, it was one of the first things that came up.

I noticed that Nestopia uses integer values throughout the entire audio rendering process, then does signed bit shifting to filter out DC.

@ReaperSMS, I got rid of all of that, and tried to mix 50/50 with the APU/Sunsoft 5B channels respectively, but the APU was too quiet. A mix of 66/33 sounds much better, as 75/25 is too loud.

[tepples]

Can you graph the waveform that the DC blocker produces?

[beannaich]

I'm not quite sure how to generate a waveform from the DC blocker, but I did plot it's output versus it's input (if that's what you meant, then hooray).

Code: Select all

var samples = new float[44100];

for (int i = 0; i < 44100; i++)
{
    samples[i] = BlockDC(
        (float)Math.Sin((i / 44100) * (Math.PI * 2)));
}

The resulting line was off. I believe the line should be as flat as possible, right?

Blue Line = 0
Green Wave = Input
Red Wave = Output

[Near]

@ReaperSMS, I got rid of all of that, and tried to mix 50/50 with the APU/Sunsoft 5B channels respectively, but the APU was too quiet. A mix of 66/33 sounds much better, as 75/25 is too loud.

Both volumes (NES APU+SS5B) can have audio scaled however they like, so it's possible to readjust your scaling bases to add at 50/50.

By the way, you are using a logarithmic scale on your 5B audio, yes?

Using something like:

Code: Select all

  for(signed n = 0; n < 16; n++) {
    double volume = 1.0 / pow(2, 1.0 / 2 * (15 - n));
    dac[n] = volume * 8192.0;
  }

You can add dac[pulse1.output]+dac[pulse2.output]+dac[pulse3.output] for a signed 16-bit sample that can be added 50/50 with NES APU output, assuming NES only uses 50% of the spectrum as well (so that you won't get clamping.)

8192 can be adjusted to some degree to modify output volume.

DC bias can be removed using integer math fairly well. For a 16-bit signed sample:

Code: Select all

signed run_hipass_strong(signed sample) {
  hipass_strong += ((((int64)sample << 16) - (hipass_strong >> 16)) * 225574) >> 16;
  return sample - (hipass_strong >> 32);
}

signed run_hipass_weak(signed sample) {
  hipass_weak += ((((int64)sample << 16) - (hipass_weak >> 16)) * 57593) >> 16;
  return sample - (hipass_weak >> 32);
}

And then mix it with:

Code: Select all

    output  = run_hipass_strong(output);
    output += cartridge_sample;
    output  = run_hipass_weak(output);
    output  = sclamp<16>(output);  //lock from -32768 to +32767

Code is from Ryphecha/Mednafen.

[tepples]

I'm not quite sure how to generate a waveform from the DC blocker, but I did plot it's output versus it's input (if that's what you meant, then hooray).

That's what I meant.

Code: Select all

var samples = new float[44100];

for (int i = 0; i < 44100; i++)
{
    samples[i] = BlockDC(
        (float)Math.Sin((i / 44100) * (Math.PI * 2)));
}

Now try graphing it with sine waves at various frequencies with 0.5 added to all samples.