nesdev.org

I think I know what is going on: use squares to make the values non-linear, then use a square root to counter the bias towards black caused by that (to make it a bias towards white instead). Not sure how well that works though.

So basically the full calculation would be:
Y = sqrt(R * R * 0.299 + G * G * 0.587 + B * B * 0.114)

EDIT: did a quick check and it does seem that grays are preserved at least.

EDIT 2: OK, here we go again with the comparisons
Original image
Standard grayscale algorithm
Square-based grayscale algorithm

EDIT 3: and now I'm seeing shortcomings to both approaches =/ Lame (but of course it's possible that with this other algorithm different values are needed as well, remember the original algorithm was just a way to get reasonable results with the technology available at the time)

Sik wrote:So basically the full calculation would be:
Y = sqrt(R * R * 0.299 + G * G * 0.587 + B * B * 0.114)

EDIT: did a quick check and it does seem that grays are preserved at least.

It is easy to prove: x=r=g=b implies y=sqrt(.299x^2+.587x^2+.114x^2)=sqrt((.299+.587+.114)x^2)=sqrt(x^2)=x. This is not a proof of how good it is; only that it has one good quality which is that it is idempotent so that grayscale pictures are preserved.

EDIT 2: OK, here we go again with the comparisons
Original image
Standard grayscale algorithm
Square-based grayscale algorithm

One slight problem is that the pictures do not all represent the same frame. It would be easy to fix by capturing one frame and then applying the algorithm and making outputs based on it (this should be easy to do using ImageMagick, for example).

In fact, it's really easy to do with ImageMagick:
convert YZG20dL.png -fx 'sqrt(0.299*R*R+0.587*G*G+0.114*B*B)' sqrt.png
convert YZG20dL.png -type Grayscale gray.png

I actually think the sqrt version seems to do a respectable job at compensating for the H-K effect... at least for pure reds, greens, and blues.

zzo38 wrote:One slight problem is that the pictures do not all represent the same frame. It would be easy to fix by capturing one frame and then applying the algorithm and making outputs based on it (this should be easy to do using ImageMagick, for example).

Yeah, but the only true difference is the clouds having moved a little and maybe Sol's position being slightly different so it's not hard to do (by the way, in case you wonder those images were made by applying a color filter in-game).

As I said the factors with the square-based algorithm could be wrong, I was messing with these ones to see how well it works, does anybody else want to give them a try? I didn't check yet but at least I got the impression that it wasn't anywhere as off (this was mostly noticeable with red):

R = 0.25
G = 0.60
B = 0.15

Wikipedia suggests removing the gamma, and then Y = 0.2126R + 0.7152G + 0.0722B, and then readding the gamma.

A simple approximation would be something like:
R = pow(R, 2.2);
G = pow(G, 2.2);
B = pow(B, 2.2);

Y = 0.2126R + 0.7152G + 0.0722B;

Y = pow(Y, 1/2.2);

Huh, I thought we were already dealing with linear RGB in the first place. Also that grass became way too bright.

Also this is what I got with the modified values I mentioned before. It still can be tweaked, but it certainly seems to look a lot closer than all the attempts I've seen so far (although my perception for the blue component may be off...).

The formulas with sqrt in them implicitly assume 2.0 gamma. I know that 2.0 of the 2.2 gamma comes from the fact that signal levels represent potential (in volts), and power is proportional to the square of potential if impedance is constant. But where's the other 10%?

Instead of a square root, take the 2.2-root? x^(1/2.2)

Doesn't the gamma depend entirely on the display? (which is why images for Macs always had to deal with a different gamma value)