Please test nemulator's audio resampling

[lidnariq]

re: 50Hz refresh rates. I don't know how common it is/isn't, but all of the LCDs that I have or have access to only run at 60Hz.

Most of the internal laptop LCDs I've seen are set to 50Hz.

[Grapeshot]

It's a power saving thing in the latest Intel IGPs and it's very difficult to turn off without registry hacking.

[koitsu]

What Grapeshot says is correct, to my knowledge. Some nVidia mobile/laptop chips are known to do this as well -- doesn't require registry hacking in that case though; in the Advanced area of the Power Management panel in Windows 7 there's a toggle for it. I know because on my work laptop, after upgrading my nVidia drivers one day, I noticed the display to be "flickery" intermittently ("the fuck? That looks like 50Hz. I'm NTSC, jackass!") and found the feature they had added + disabled it.

This is just another reason I said what I did (2nd paragraph) and why I disagree with this (last line). Expecting things like EIST, CPU P-states, and GPU power-saving features to be disabled = wrong mentality for software design.

[James]

Expecting things like EIST, CPU P-states, and GPU power-saving features to be disabled = wrong mentality for software design.

nemulator, if configured to do so (the default setting), will automatically disable CPU throttling upon startup, and restore the previous setting when exiting. So, I don't expect any particular configuration for it to work properly.

Re: 50/75Hz settings - 50Hz with vsync is just never going to work. 75Hz should work, albeit not as smoothly as 60Hz. I'm not sure why the timer sync stuff isn't working for you. With it enabled, and vsync disabled, you should be running at a rock solid 59.97Hz...

I found an LCD that will run at 75Hz. Will test on it when I get a chance.

[koitsu]

When you say "CPU throttling" what exactly do you mean? Are you talking about EIST? Are you talking about processor P-states? And how are you disabling it (explain the code)?

If you're doing something like actually disabling a setting (registry, power profile, etc.), this is completely unacceptable. If your emulator crashes, or gets stuck and someone end-tasks the process, that setting is going to remain disabled. This is akin (in my mind) to emulators which used to do things like change screen resolution + bit depth (I can think of quite a few Genesis emulators that did this). Please don't change peoples' system settings. I've also seen programs which to inhibit a screen saver coming on or monitor power-save (standby) while using their software, actually changing the OS/UI settings.

Windows offers a way to inhibit an OS feature by using a WM_* message (probably WM_SYSCOMMAND); that's the way to go. That method is completely fine. In fact, that's the proper way to inhibit the screen saver/monitor standby. Below is a small (4KByte) document I wrote back in 2006-2008 for the folks who maintained the Windower program (which allowed FFXI to run in a window), explaining to them how to inhibit such. Maybe others can benefit from what I've written.

FFXI, with Windower running (I haven't tested FFXI w/out Windower), does not inhibit Windows from powering off a monitor based on the Power Scheme chosen (e.g. under Display Properties -> Screen Saver tab -> Power button -> Turn off monitor: after X minutes or X hours). I can't remember if it inhibits Windows from inducing the screen saver either, but if not, the code below will stop it from happening.

I've discussed this in the past with StarHawk on IRC, but it was maybe 1.5 years ago (yes really!), and I doubt he remembers the conversation. He said he was personally interested in addressing the problem too, hence my write-up.

The solution is to catch WM_SYSCOMMAND, returning zero when wParam SC_SCREENSAVE or SC_MONITORPOWER bits are set. This will block the screen saver or monitor suspend/power down from happening while Windower is running.

Code: Select all

LRESULT CALLBACK WndProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
  switch (uMsg)
  {
        case WM_SYSCOMMAND:
        {
                switch (wParam & 0xfff0)
                {
                        case SC_MONITORPOWER:
                        case SC_SCREENSAVE:
                                return 0;
                }
                break;
        }
  }
  return DefWindowProc(hWnd, uMsg, wParam, lParam);
}

Documentation for WM_SYSCOMMAND: http://msdn.microsoft.com/en-us/library/windows/desktop/ms646360%28v=vs.85%29.aspx

There may be other ways to do this to inhibit P-states but I don't know how at this time. I'd have to dig deep in MSDN.

Edit: added & 0xfff0 per Microsoft requirement.

[James]

When you say "CPU throttling" what exactly do you mean?...Are you talking about processor P-states?

Yes: http://en.wikipedia.org/wiki/Dynamic_frequency_scaling

And how are you disabling it (explain the code)?

I make a copy of the current power scheme, adjust the appropriate settings in the copy, and activate it. Upon exit, I switch back to the scheme in use at startup and delete the copy.

If you're doing something like actually disabling a setting (registry, power profile, etc.), this is completely unacceptable.

If your emulator crashes, or gets stuck and someone end-tasks the process, that setting is going to remain disabled.

Yep. I'm not 100% happy with it either, but I know of no other way to handle it.

[koitsu]

What you're doing (changing the power scheme -- yes I know how to do this, i.e. POWER_POLICY, GetActivePwrScheme(), ReadPwrScheme(), SetActivePwrScheme) is bad for the reasons I listed. I can assure you that in the future you will get hounded and scolded for this. Please do not do this. And yeah, I know about this as well.

The effect is going to be awful if your emulator crashes or someone end-tasks it then wonders why their laptop battery life suddenly drops to crap -- permanently. If someone clever figures it out, they're going to go "what the hell is changing my power profile setting?!?!?!" Nobody is EVER going to consider that an emulator is the cause. Please think about the repercussions of this "solution".

Can you explain, verbosely, why it is you need to disable these CPU features?

Even on *IX systems I've never heard of anyone ever needing to disable these states to get smooth audio playback or to sync to a specific frequency/clock rate. I also don't see why it's necessary either -- if existing emulators don't need to do this (I can list off quite a few), why does yours? This sounds more like a software/application design problem that needs to be solved in a different manner.

[tepples]

I was under the impression that some operating systems determined whether a machine was idle based only on the keyboard and mouse, not any connected joysticks, and used the idle status to determine whether to enter deeper power-saving modes. In such a case, the user would see a problem only if playing the game with a joystick, not a keyboard.

[Dwedit]

I remember that one of Tepples's own PC programs had a problem where it would alter Windows settings (colors of controls) if it crashed or was killed.

[tepples]

Which program might that have been? I don't remember having inserted any calls to Windows services to change system color settings in any of my own programs. The only system setting that I remember changing is the screen resolution and bit depth when a program is run in full screen. Might Windows have altered the colors itself if it crashes in a 256 color mode and for some reason can't restore the 16-, 24-, or 32-bit mode that your desktop normally uses? Was it running under Windows 9x or under Windows XP?

Status: incomplete

[James]

I can assure you that in the future you will get hounded and scolded for this.

Perhaps. But you're the first since this functionality was introduced in 4/11

If someone clever figures it out, they're going to go "what the hell is changing my power profile setting?!?!?!" Nobody is EVER going to consider that an emulator is the cause.

The temporary power scheme name is "nemulator" and the description is "Copy of %s scheme. Used by nemulator to disable processor throttling." I do have one change in the works though: if nemulator starts up and detects that the nemulator scheme is already in use, indicating an unclean shutdown, it will revert to the scheme that was in use before the previous scheme change.

if existing emulators don't need to do this (I can list off quite a few), why does yours?

I'm unaware of any other emulator that simultaneously provides smooth video output (no tearing, no stuttering) and glitch-free audio. It's a differentiating feature of nemulator and one that I wish that you could try firsthand. I think it's a worthwhile trade-off.

Can you explain, verbosely, why it is you need to disable these CPU features?

I'll try. To recap, emulators generate video and audio output at slightly different frequencies. To keep everything in sync, most emulators block on audio output, effectively locking the frame rate to the audio output rate. This results in glitch-free audio output, but tearing or stuttering video output.

Alternately, an emulator could lock to 60Hz vsync to provide artifact-free video output, but there will be audio glitches when the audio output buffer over-/under-runs.

To address this issue, nemulator uses vsync for timing (except when it doesn't, but that's a different discussion), and dynamically adjusts the audio frequency to match the current frame rate. Video is output at a consistent 60Hz, audio output is never blocked waiting on the output buffer to drain, and the audio buffer is never under-run.

In order to determine the difference in audio generation and playback rates, I check the playback and write cursor positions of the sound card's primary output buffer. If the distance between the two pointers is consistent -- the goal -- that means that the buffer is being drained at the same rate that it's being filled. If it's not, audio frequency is adjusted to compensate.

So, why does processor throttling interfere with this? When processor throttling is enabled, processor frequency -- and therefore the time between vsync occurring and the audio buffer being filled -- will vary depending on system load. Usually, this 'jitter' isn't too bad, and can be accounted for. But on occasion, there will be large differences that can result in under-runs. Here's an example. The red line represents the number of bytes between the playback and write cursors:



With processor throttling disabled, latency is much more consistent, and large differences generally don't occur:



There's a couple of ways to account for the increased jitter with throttling enabled. I can increase the buffer size, but that can result in noticeable latency between video and audio (e.g., you hear Mario jump later then it happens on-screen). I can make larger/faster adjustments to audio frequency, but that can result in noticeable pitch changes.

So, at the end of the day, temporarily disabling processor throttling provides the best results. I agree that there are downsides, but I think the benefits outweigh those.

[koitsu]

I think I understand the concern -- thank you for taking the time to explain it -- but the screenshots in question are quite a bit confusing/difficult to interpret. At this point they mean more to you than they do to me.

All I can discern is that the top screenshot shows two red lines (am I supposed to be looking at the red gradient line, or the slightly-brighter-but-slightly-not line?), and that there is more variance there (on the vertical axis) than in the lower screenshot. This doesn't tell me how much (on a numeric level) delta there is between the audio playback position vs. the audio write position -- what's rendered in the top screenshot could indicate a 10ms delta, or it could indicate a 1000ms delta.

I also assume that the horizontal axis indicates time, i.e. this red-green-mess ( :-) ) is effectively panning/scrolling leftward at all times, with new information drawn on the right-hand side.

At some level I still do not understand why the audio and video pieces have to be "in sync" with one another -- the hardware doesn't operate this way, i.e. there's a separate oscillator/crystal for audio than there is for video/PPU and they operate at two different frequencies, correct? So why can't two separate threads (one for audio, one for video/PPU) be made to run at those rates?

[blargg]

The problem is of two master timebases which are inevitably slightly out of sync, video and audio. The monitor might claim to be 60.00 Hz, and audio 441000 Hz. If an emulator generated exactly 60.00 frames per second and 441000 samples per second, one would inevitably fall behind since their timebases aren't exactly in sync.

Some emulators use sound as the master timebase, periodically doubling/skipping video frames as necessary. Others use video as the master timebase, letting audio drop out periodically or skip forward. As I understand it, this emulator takes a different approach of catering to both video and audio timebases, fine-tuning its audio rate on-the-fly in response to variations. To do this, it needs to accurately measure how much audio is currently buffered, so that it can adjust its sample rate by small increments. It uses some functions that find how much audio is buffered and waiting to go to the DAC in the sound card, and apparently these give unreliable results when processor speed is changed. I believe that the graph shows the results of this audio buffering query, and how it jumps around a lot when processor speed changes. This causes nemulator's audio code to think it needs to make a large adjustment to the audio rate, which I take it noticeable audibly.

By the look of it, the graph has time on the X axis. The green is output sampling rate, higher towards the top (more samples generated per second), and the red is the number of samples buffered waiting for the sound card (higher near the top). I think that the red gradient background might be the desired level of buffering. So where the average number of samples buffered jumps up suddenly and stays there, the emulator lowers the number of samples generated per second so that the sample buffer will become less full on average. At some point, it will increase the number of output samples per second to keep the buffer at this less-full level.

Since it looks like the queries are mostly clean even when processor speed changes, I wonder whether nemulator can ignore these erratic values.

[James]

Thanks for the breakdown, blargg.

The audio buffer is circular. The red line represents the number of bytes that the write cursor is behind the playback cursor (the red area depicted below).



If the red area is less than one frame's worth of audio (red line low on the screenshot), then the emulator will block waiting until there is enough room to write pending data. That will lead to a delayed video frame and stuttering. If the green area gets too small (red line high on the screenshot), the playback cursor will pass the write cursor and there will be garbled audio.

On the screenshot, the red gradient represents the target area, and the green line represents the playback frequency. The graph does move right to left and represents about 4.25 seconds worth of data (256 frames).

The jitter in the first screenshot is not a big deal, but what is problematic is the large jump above Mario's head. That represents about a 16 millisecond gap... or one frame. And that got me thinking...

What seems to be happening is that the emulator is missing vblanks, and that problem is exacerbated when processor throttling is enabled. However, that's not the direct cause. I discovered that setting the a/v thread's affinity to one processor leads to performance in line with having processor throttling disabled. So, good news -- I can scrap the power management stuff.

I need to do some more work to discover why vblanks are being missed (it still, infrequently, happens).

koitsu -- thanks for shaming me into finding a better solution

natt -- I discovered why timer_sync wasn't working for you. That fix will be in the next release.

[miker00lz]

Hmm, I can't even run nemulator on the machine i'm using at the moment. It spits out the error "DX10CreateDeviceAndSwapChain". I'm using Windows 8 and DirectX11. I think the problem is that I am using the motherboard's integrated graphics, and it is only compatible with up to DirectX 9.0c functionality.

This raises a big question, do you REALLY need DirectX10 for a NES emulator? That seems completely unnecessary to me. You're really limiting the number of possible users by doing that. It won't be useable on almost any laptop ever made, on any netbook, and there are still a HUGE number of desktop systems out there without a decent video card like mine here. That's something to think about.

Anyway, this computer is not my main one. It's a spare with a 3.06 GHz Pentium 4 that I've been using to test/screw around with Windows 8. (Which sucks mad goat balls, btw) I'll try nemulator a bit later on my 4.6 GHz eight-core with Geforce 9800GT and Windows 7.