Various questions about the color palette and emulators

[JimDaBim]

Hello,
I've got some questions about the NES and since they're rather technical, I guess this forum here is the right one for me.

OK, here are my questions:


Why does every NES emulator have a different color palette?
Isn't it possible to design the definite palette? I understand that different TVs have different color adjustments and therefore look differently. But that's the same with computer screens and still, every picture file, be it BMP, JPG, GIF or PNG etc. has definite, "objetive" color values. Yet, all NES emulators define their colors individually.
Why is that the case? How were the colors transferred from the NES to the TV?

Wouldn't the following be possible:
Take an NES ROM that displays all the various colors.
Copy it on a cardridge.
Put it into a real NES.
Show that demo NES game on a TV.
On the same TV, show a PC bitmap file that includes various colors.
Look which of the bitmap colors look identical to the NES colors.
And so, you have the RGB value for the NES colors.


Is there any color palette that comes closest to the display of an actual NTSC TV screen as it "usually" looks like?


Where did they get the colors from when they published some of the NES classics for the Game Boy Advance for Game Cube or the Wii Virtual Console? And do these games even have the same color palette among each other? So, does "The Legend of Zelda" for the Game Boy Advance has the same colors as "The Legend of Zelda" for Game Cube?


I've heard that Nestopia is cycle-accurate? What's the practical difference to emulators that are not cycle-accurate? For example, I never hear about FCEUX being cycle-accurate, so what is it instead?


What would you say is the best/most popular emulator for the NES?


In FCEUX, there is an option called "NTSC Color Emulation". If you check it, the colors change from the default palette. So, does that mean that the default palette in FCEUX is based on a PAL TV? That wouldn't really make sense to me. Isn't FCEUX an American product? The default screen resolution is the NTSC one, the default speed is the one from NTSC as well. If you want PAL resolution or speed, you have to explicitly check it. So, why aren't the NTSC colors chosen by default?

[tepples]

Why does every NES emulator have a different color palette?

Because everybody's TV is calibrated differently. Do you know what the brightness, contrast, color, and tint knobs do on a TV?

But that's the same with computer screens and still, every picture file, be it BMP, JPG, GIF or PNG etc. has definite, "objetive" color values.

At what color temperature? With yellow boost on or off? Sure, there's an "objective" standard to which studio monitors are calibrated, but various "image enhancement" methods implemented in TVs sold to the public make live-action footage look "better" at the cost of deviating from this standard. Should an emulator be emulating an NES connected to a studio monitor or an NES connected to a home TV?

Wouldn't the following be possible:
Take an NES ROM that displays all the various colors.
Copy it on a cardridge.
Put it into a real NES.
Show that demo NES game on a TV.

So far, you've described running Loopy's palette demo on a PowerPak.

On the same TV, show a PC bitmap file

For one thing, most PCs can't output video to a standard-definition television. A scan converter adds its own coloration to the signal. But what I have done is run my PowerPak and Nestopia side-by-side using picture-in-picture on a Vizio VX32L.

Where did they get the colors from when they published some of the NES classics for the Game Boy Advance for Game Cube or the Wii Virtual Console? And do these games even have the same color palette among each other? So, does "The Legend of Zelda" for the Game Boy Advance has the same colors as "The Legend of Zelda" for Game Cube?

The gamma of the display of the original Game Boy Advance system is very, very dark, much darker than that of a TV or a PC monitor.

I've heard that Nestopia is cycle-accurate? What's the practical difference to emulators that are not cycle-accurate? For example, I never hear about FCEUX being cycle-accurate, so what is it instead?

An emulator is cycle-accurate if all operations on the CPU, PPU, and APU take exactly the same number of cycles that they take on an NES. Pretty much all popular emulators have a cycle-accurate CPU, but the PPU and APU aren't quite as accurate in every emulator. There are corner cases of timing that are still being figured out, such as what happens if the CPU and PPU start a fraction of a cycle out of sync, or what happens when sprites are being moved at the same time sampled sound is being played.

In FCEUX, there is an option called "NTSC Color Emulation". If you check it, the colors change from the default palette. So, does that mean that the default palette in FCEUX is based on a PAL TV?

A lot of the default color palettes shipped with emulators are based on old guesses that happened to look good at the time, not based on any theory of how the NES generates its signal. The NTSC NES picture contains artifacts on fine details, which may color the player's perception of particular colors.

[JimDaBim]

Thanks for the answers.

Should an emulator be emulating an NES connected to a studio monitor or an NES connected to a home TV?

I would say the emulator should emulate the NES in a way that the image on the screen looks exactly like if you connect an actual NES to that same screen.
Sure, a TV screen has brightness and all that stuff. But those things are there anyway. If you connect a PC to a screen, you still could adjust the screen. But the thing that I don't understand about emulators: Why do two emulators look differently, even on the same screen? Why is the hardware-related issue of screen adjustment even a topic in the emulator software? Doesn't the NES send its color information as RGB values, just like a computer or a DVD player also sends them as RGB values?
Sorry if I sound naive. I don't really know how anything of that works. I just write those things to understand it better.

So far, you've described running Loopy's palette demo on a PowerPak.

So, do you have an actual palette file with the results that can be loaded into an emulator?

An emulator is cycle-accurate if all operations on the CPU, PPU, and APU take exactly the same number of cycles that they take on an NES. Pretty much all popular emulators have a cycle-accurate CPU, but the PPU and APU aren't quite as accurate in every emulator.

So, Nestopia is an emulator that actually does the PPU and APU accurately?

By the way, this is a bit off-topic, but do you know if MAME is cycle-accurate in CPU, PPU and APU?

The NTSC NES picture contains artifacts on fine details, which may color the player's perception of particular colors.

I guess you just described the NTSC filter, that option where you can make the image look like on an old TV. The option that is choosable next to special effects like HQ2x and Scale2X. But I was just talking about the option that can be found in the menu under "Config", "Palette", not the one in "Config", "Video", "Special Scaler".
If you check it, the colors slightly change. But I would have assumed that the default colors are for NTSC anyway.

By the way, do you know why waiting for VBlank and double buffering doesn't work in FCEUX when you disable the hardware acceleration (which is necessary so that the image looks pixelated and not blurry), but VSync works fine in Nestopia, even without bilinear filtering? With the default settings, I just don't get FCEUX to scroll smoothly. (It always looks as if a horizontal line is moving from bottom to top.) I would need one of those sync methods, but they are completely ignored as soon as I disable hardware acceleration.

[Disch]

Doesn't the NES send its color information as RGB values, just like a computer or a DVD player also sends them as RGB values?

No. Not at all. Nothing on the NES is RGB. This is probably what is leading to your confusion.

This is what it sends to the TV:
http://wiki.nesdev.com/w/index.php/NTSC_video

Understand it? Me neither. Neither do most other emu authors. Hence the inconsistency. Emulating output with an RGB palette (while inaccurate), is easy and works well enough. Which is why so many emus do it.

So, do you have an actual palette file with the results that can be loaded into an emulator?

Many people have done the whole "take a screenshot of the video from a TV and make a palette off of it" to get a "100% accurate" palette.

The problem is:

1) output on everyone's TV is different, so any RGB value you pull from a screen capture is going to be wrong.
2) The signal the NES sends to the TV "blends" nearby colors. So a gray pixel isn't always gray if it's drawn next to a different colored pixel.

So, Nestopia is an emulator that actually does the PPU and APU accurately?

There are several accurate emus, but NEStopia and Nintendulator are generally considered to be the most accurate. Although no emu will ever be 100% accurate.

I guess you just described the NTSC filter,

That's the thing.. NTSC isn't really a filter, it's an output signal. The NTSC "filter" is actually emulating how the video signal gets sent from the NES to the TV. It's not just blurring the image to make it look like a TV.

[tepples]

If you connect a PC to a screen, you still could adjust the screen.

The "color" and "tint" are actually controlling the conversion from composite to RGB and thus don't apply to a PC. As Disch said, the NES generates the signal directly in the composite domain.

Doesn't the NES send its color information as RGB values

The PlayChoice does; the NES doesn't.

An emulator is cycle-accurate if all operations on the CPU, PPU, and APU take exactly the same number of cycles that they take on an NES. Pretty much all popular emulators have a cycle-accurate CPU, but the PPU and APU aren't quite as accurate in every emulator.

So, Nestopia is an emulator that actually does the PPU and APU accurately?

Nestopia and Nintendulator have more accurate timing than a lot of other emulators. But there's a reason I still use FCEUX on my 10" laptop: it's accurate enough for making game logic, and it runs at full speed even on an Atom CPU.

do you know if MAME is cycle-accurate in CPU, PPU and APU?

MAME isn't one emulator; it's several hundred emulators, one for each supported arcade system board. Are you asking whether the PlayChoice driver in MAME is accurate?

I guess you just described the NTSC filter, that option where you can make the image look like on an old TV.

The NTSC filter simulates an NES; everything else simulates a PlayChoice. A PlayChoice has a 52-entry lookup table from NES color numbers to RGB values, which it sends through three DACs to a standard-definition RGB monitor. An NES has only one DAC, and it generates a composite signal directly by rapidly alternating between the signal level for $00, $10, $20, or $30 and the signal level for $0D, $1D, $2D, or $3D. And as Disch pointed out, different TVs decode them differently.

But I was just talking about the option that can be found in the menu under "Config", "Palette"

Those are theoretically the color in the middle of a flat area. Different people eyeball the colors differently.

[JimDaBim]

Well, the whole color topic seems very complicated, but thanks for your help.
Is this only an issue on the NES? Or are the color palettes like for the Super Nintendo or other consoles that non-definite too?

The PlayChoice does; the NES doesn't.

So, shouldn't the colors of the PlayChoice be the basis for any emulator since those are basically the same games?

Since you use FCEUX, how do you make it scroll smootly? If I don't use any sync method, it will update the screen in a way that I always see a horizontal line going from bottom to top. But for some reason, the sync methods only work with enabled hardware acceleration (which I don't want since in this case, modern graphic cards use bilinear filter for stretching).

Are you asking whether the PlayChoice driver in MAME is accurate?

No, that was a completely unrelated question from the NES. I thought that cycle-related stuff works equally in every game in MAME. In my case, the original "Street Fighter II" would be of interest here.

Those are theoretically the color in the middle of a flat area. Different people eyeball the colors differently.

Sorry, I still don't get it. Especially the relation between the two palettes:

This the default palette in FCEUX:


And this is the "NTSC Color Emulation" palette:


If the second image is NTSC color emulation, what is the first image then? PAL color emulation? Unlikely, since it's an American emulator and all other options are by default set on NTSC, not on PAL. So, how can the first palette be called? And why isn't the NTSC color emulation enabled by default?

Last time, you said:

But what I have done is run my PowerPak and Nestopia side-by-side using picture-in-picture on a Vizio VX32L.

Did you save the color palette you devised with that method?

[tepples]

Is this only an issue on the NES? Or are the color palettes like for the Super Nintendo or other consoles that non-definite too?

It's mostly a problem with consoles from the NES era and earlier. Every console video chip since the Genesis has generated an RGB signal and encoded that to composite, though the encoding has its own artifacts, especially on the TG16 and Genesis.

So, shouldn't the colors of the PlayChoice be the basis for any emulator since those are basically the same games?

There are a couple browns that are noticeably wrong on the PlayChoice compared to how TVs interpret them, but yes, emulators should have a PlayChoice palette as one choice.

Since you use FCEUX, how do you make it scroll smootly?

By putting the ROM on a CF card and inserting it into a PowerPak. I use FCEUX for developing and an NES for play-testing.

But for some reason, the sync methods only work with enabled hardware acceleration (which I don't want since in this case, modern graphic cards use bilinear filter for stretching).

Perhaps you can tell your video card to use a different kind of filtering. Or perhaps you can enable software pre-filtering. I know VisualBoyAdvance has the "Simple 2x" filter that can be applied before the texture is sent to the video card.

I thought that cycle-related stuff works equally in every game in MAME.

Different boards are understood to different degrees.

Sorry, I still don't get it. Especially the relation between the two palettes

I'd bet a single TV could display both the "NTSC Color Emulation" palette and something close to the default palette simply by turning the color, brightness, and contrast knobs. The default palette is darker and more saturated, which would mean the "Color" knob is cranked up and "Brightness" and "Contrast" down somewhat.

Did you save the color palette you devised with that method?

No. It was years ago, and the computer with which I did that has long since been totaled. My TV isn't even on the same floor as a computer capable of running Nestopia.

[Drag]

This is what I use, and it's the result of me having Loopy's palette demo on my PowerPak, plugged into my Panasonic CRT television, while twiddling around in Nestopia's NTSC palette generator until I came up with something that looked as close as I could get.

Believe me, I've tried, I've looked at literature, written scripts, messed with matrices, played with clipping, but the simple fact is that YIQ isn't the same as RGB, and CRTs that use YIQ are capable of displaying colors that are actually outside of the RGB range. For example, I can't get color 22 to look correct, no matter what I do, because it uses a blue value that is brighter than what's possible with RGB, so no matter what, it's going to look washed out and dull.

However, designing NES graphics, I've been able to use this palette, and the results on the NES, using my specific television, are extremely close.

And yes, color 08 is that dark on my TV, it's darker than the darkest gray, but lighter than black. Most LCD tvs won't display it correctly unless I play with the settings.

Color xC is really strange, it's cyan, but as you get darker, it gets more blue, and again, I tried to match it as close as I could with the generator.

[kyuusaku]

In general MAME drivers can be considered some of the least accurate emulators. Almost all video hardware is "high-level" emulated, and the titles with line interrupts instead of a foreground layer are emulated with pseudo-line accuracy at best. I understand that some drivers are receiving more modern emulation treatment, but as a general rule MAME developers are working "blind", they have ROM dumps that they have to make run, but don't have access to the hardware to perform tests so they reverse engineer the hardware via the game code. MAME is a very helpful tool when you need to see a rough implementation of a system, but I do not trust even the tiniest details to be completely correct.

Fortunately with most arcade hardware there is little consequence of inaccurate video or audio emulation because they are often separate subsystems which under no circumstances influence the state of the CPU/game.

[JimDaBim]

Since you use FCEUX, how do you make it scroll smootly?

By putting the ROM on a CF card and inserting it into a PowerPak. I use FCEUX for developing and an NES for play-testing.

I was actually talking about the scrolling when you play a game, not about programming the scrolling in a self-made game. The way the image refreshes in an emulator looks faulty unless you use back buffering or vsync.

Perhaps you can tell your video card to use a different kind of filtering. Or perhaps you can enable software pre-filtering.

Neither the graphic card, nor FCEUX seems to offer something like that. And I can't believe that this issue never came up during the development process of the emulator. I mean, what does double buffering and vsync have to do with hardware acceleration anyway? Maybe I should ask one of the developers for that.

I've got two more specific questions:

I've often read about the fact that the NES doesn't have a proper yellow in its palette. When I saw the FCEUX palette, I thought that the value at (9,3) looks quite alright:


Sure, it's not perfect, but it's good enough to display for example blond hair. Then I saw this, the Nestopia palette:


And here, this color is actually green. Which would also explain why they used the color at (8,3) for Pac-Man and not the one at (9,3).

So, I know that this all has to do with color adjustment, but how would the color at (9,3) "usually" look on an NTSC screen? Is it yellowish or really a lemon-like green?
(Interestingly, both "Ms. Pac-Man" games use the color at (9,3) which means that Ms. Pac-Man looks greenish on Nestopia.)

The title screen of "Metroid": Again, on a "normal" NTSC TV screen: Is the ground supposed to look more like brown or more like green?

[lidnariq]

So, I know that this all has to do with color adjustment, but how would the color at (9,3) "usually" look on an NTSC screen? Is it yellowish or really a lemon-like green?
(Interestingly, both "Ms. Pac-Man" games use the color at (9,3) which means that Ms. Pac-Man looks greenish on Nestopia.)

The title screen of "Metroid": Again, on a "normal" NTSC TV screen: Is the ground supposed to look more like brown or more like green?

PAL aficionados, back in the day, would joke that NTSC stood for "never the same color". Please take that to heart — there isn't a single normal or correct color for any given output from a NES. Different TVs will reproduce those colors in all different ways.

Theoretically, the 8s column of the screen is the color of the NTSC colorburst, and should be pure color -U in the YUV colorspace. I made a graphic a few years ago of the NTSC colorwheel superimposed on NTSC's YIQ color axes on a vectorscope with the NES colors called out:

[tokumaru]

I was actually talking about the scrolling when you play a game, not about programming the scrolling in a self-made game. The way the image refreshes in an emulator looks faulty unless you use back buffering or vsync.

I think tepples meant he doesn't care about the tearing because he doesn't use FCEUX for playing, just for debugging. By playing on on an actual console he doesn't get any tearing.

And here, this color is actually green.

I guess this is just a side effect of the YIQ (or similar) color representation. The hue can easily shift depending on the monitor/TV.

(Interestingly, both "Ms. Pac-Man" games use the color at (9,3) which means that Ms. Pac-Man looks greenish on Nestopia.)

This just comes to show that developers back then also suffered with the way the colors change from TV to TV. They were as clueless as we are concerning how each color should actually look.

[JimDaBim]

Thanks a lot for all your help. I guess, after reading all this, that Nestopia, and not FCEUX, will be the correct choice for me when it comes to playing NES games.

[3gengames]

FCEUX has the best looking colors compared to my flat screen to me for sure. Nestopia is WAYYYYY too bright, at least the one I use. 1.39 is my nestopia version.

ETA:Just tried 1.40, the colors are darker, but they just look a little off. I don't think NESTOPIA is that good of a emulator for color accuracy. Like how vNES went with the "100% right" colors and made it look like complete shit.

[ccovell]

So, shouldn't the colors of the PlayChoice be the basis for any emulator since those are basically the same games?

You would think, but the RGB PPU (Playchoice one) likely was produced after the original composite one. They also are a bit different in some areas, so a few Famicom and NES games actually will display worse on an RGB PPU: