Bringing a new HI-DEF NES kit to market; where to start

Are you new to 6502, NES, or even programming in general? Post any of your questions here. Remember - the only dumb question is the question that remains unasked.

Moderator: Moderators

forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

I have jumped into the world of gaming retro analog consoles on today's HDTVs (specifically the NES) and with all of the options I have found (Analogue NT, RetroUSB AVS, Mister, Framemeister, Retrotink5, Hi-Def NES kit, OSCC, etc.) the one that I am most interested in is Kevtris's Hi-Def NES kit as it uses the original hardware versus FPGA solutions to mimic original hardware including the CPU and PPU.

Unfortunately it appears Kevtris is no longer making the kits for game-tech to sell.

I would like to create a new version of an HDMI upgrade kit for the NES. I have resources available to have PCB's designed and manufactured, but my larger hurdle with this goal is the programming aspect of the FPGA based upscaling portion. If my goal becomes a reality I would like to make this open source once I recoup startup costs.

While I could spend the time to learn how to do this myself, I'm hoping that there are some open source resources available given all the upscaling work that has already been done by so many specifically for retro gaming.

I've tried searching github and while I found good resources for NES projects and upscaling independently, I didn't find anything that explains upscaling directly from the NES's CPU and PPU.

I also tried emailing Kevin to ask if he could help or even sell his design, but I have not received a response.

With all that said, here are my questions:

1. Is there open source information available for the programing needed to upscale NES to HD digital video out?

2. Where might you direct me if there are no open source projects available that you know of?

3. Is there a contract type website tailored to this sort of need where I could possibly hire individual(s) to help me with that portion?
Last edited by forkram on Tue Mar 28, 2023 5:59 am, edited 2 times in total.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by lidnariq »

forkram wrote: Mon Mar 13, 2023 8:01 pm Unfortunately it appears Kevtris is no longer making the kits for game-tech to sell.
Unfortunately, selling these things is a tech support nightmare. The most recent revision (4) of the analog equivalent (NESRGB) has been full of problems.

HDMI is extra bonus mean, too. Analog standards have to put up with things being a bit out of spec, but HDMI TVs are allowed to entirely reject what you try to send them unless they're perfectly correct.
programming aspect of the FPGA based upscaling portion.
Upscaling is really not hard. You have two processes: one figures out what the color displayed was, and puts it in a queue; the other reads out of that queue at some faster rate but multiple times.

This is particularly straightforward in the case of "exactly 2x", where you just need two linebuffers:

[Fill left half of linebuffer A] [Play out entirety of linebuffer B]
[Fill right half of linebuffer A] [Play out entirety of linebuffer B]
[Fill left half of linebuffer B] [Play out entirety of linebuffer A]
[Fill right half of linebuffer B] [Play out entirety of linebuffer A]
I've tried searching github and while I found good resources for NES projects and upscaling independently, I didn't find anything that explains upscaling directly from the NES's CPU and PPU.
You don't actually care about the NES native output here. Upscaling is upscaling; that on some level the NES has a HSV stage instead of RGB doesn't change the upscaling.
1. Is there open source information available for the programing needed to upscale NES to HD digital video out?
Specifically as far as "how do you capture the equivalent digital contents of the NES", look at this thread.

Specifically for upscaling, look at OSSC or hoglet67's RGBtoHDMI.
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

Unfortunately, selling these things is a tech support nightmare. The most recent revision (4) of the analog equivalent (NESRGB) has been full of problems.

HDMI is extra bonus mean, too. Analog standards have to put up with things being a bit out of spec, but HDMI TVs are allowed to entirely reject what you try to send them unless they're perfectly correct.
Do full FPGA machines like the NT mini and AVS reduce these problems by not using original hardware?

I appreciate the quick response; I'll be sure to check out your suggestions for further learning.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by tepples »

Some of the complications of using original hardware are
  1. Supplies of working CPUs and PPUs are dwindling. This caused Analogue to replace the Nt, which used original parts on a motherboard with the Hi-Def NES mod preinstalled, with the Nt mini, which included an FPGA clone of the CPU and PPU.
  2. The NTSC NES PPU generates 60.0988 fields per second, not the standard 60000/1001 (about 59.94) or exactly 60 of a perfect signal. You will need to run the CPU and PPU slightly slower.
  3. The NTSC NES PPU generates 240 scanlines of picture out of 262 total scanlines per field, or picture for 91.6% of the time. This doesn't perfectly line up with 480p, which has 480 scanlines of picture out of 525 total scanlines per field (91.4%); 720p pursuant to SMPTE296M, which has 720 scanlines of picture out of 750 total scanlines per field (96.0%); nor 1080p pursuant to SMPTE274M, which has 1080 scanlines of picture out of 1125 total scanlines per field (96.0%). Accounting for this difference without causing the TV to reject a signal as imperfect requires a circular buffer a couple dozen lines tall, not just a single line buffer.
  4. The NTSC PPU has a "missing dot" at the end of every second pre-render line, but only if rendering is enabled, and this dot might cause TVs to reject a signal as imperfect. You'll need to account for missing dot 0 by slowing down the CPU and PPU.
  5. You'll need to digitize both of the CPU's analog audio outputs. If you plan to support cartridge audio on a Famicom or expansion audio on a front-loading NES (such as EPSM), you'll need to digitize that as well. This becomes more difficult with the FM synthesizers, such as VRC7 in Lagrange Point and YMF288 (OPNA) EPSM. Unlike other expansion audio chips, which are clocked by M2 (the CPU clock), FM synthesizers have their own oscillators, and these oscillators do not slow down when you slow down the CPU and PPU to account for the difference between 60 and 60.1 frames per second.
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

tepples wrote: Tue Mar 14, 2023 8:39 am Some of the complications of using original hardware are
Thanks, tepples. I appreciate your detailed response and I heard Kevtris talking about most of your points on his youtube videos and on this message board. Doesn't mean I know how to tackle those problems in the least bit, but that is why I am here in the newbie section :)
  • With your points mentioned regarding the troubles using original PPU and CPU's, perhaps you can shed some light on my assumption that using original chips will mitigate compatibility issues where not all games play on clone/FPGA chips solutions?
  • If my goal is to get HMDI output from an original cartridge type system is it your opinion that going the full FPGA route would be significantly less difficult than using original chips?
I'm beginning to think that the time investment needed to get me up to speed doing this solo and from ground zero will be more than I am willing to spend and perhaps going right to an open source project with paid contributors would be best.

I don't really care about trying to make a profit as much as I care about getting people more alternatives for HD NES systems that aren't emulators and can play cartridges. Not to mention how much I value having more time over more money.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by tepples »

The program loaded onto an FPGA is still an emulator (despite Analogue's marketing campaign to the contrary). It just isn't a conventional emulator for a serial computer. It emulates all the circuitry at the same time, which gives it some theoretical and practical advantages. I've compared and contrasted the two in the topic "Latency challenge of conventional emulation compared to FPGA".
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

tepples wrote: Wed Mar 15, 2023 9:33 pm The program loaded onto an FPGA is still an emulator (despite Analogue's marketing campaign to the contrary). It just isn't a conventional emulator for a serial computer. It emulates all the circuitry at the same time, which gives it some theoretical and practical advantages. I've compared and contrasted the two in the topic "Latency challenge of conventional emulation compared to FPGA".
I'm not sure what my take away is from your reply or from the linked thread. If you were trying to answer my two questions then I'd have to admit I would still need them answered in a way I might understand better.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by Fiskbit »

Compatibility of a clone or FPGA solution is only as good as its implementation. FPGA cores are hardware-based emulators, and just like software-based emulators, they can be very accurate or inaccurate. Our knowledge of how the hardware works is extremely good at this point and so emulators can be extremely accurate, if implemented correctly.

Using the official chips doesn't allow you to completely avoid problems. Both the NESRGB and Hi-Def NES rely on an unused PPU mode generally referred to as slave mode (though I am becoming fond of 'EXT output mode' for improved clarity) where the PPU outputs 4 of the 5 bits of the palette index used by each pixel via the PPU's EXT pins. Because the 5th bit isn't present, it has to be extracted from the analog signal, which can be challenging to reliably synchronize with the EXT output, causing the bit to sometimes be misaligned with the picture. Slow transition speed in the analog signal can also result in flickering pixels at transitions in the 5th bit. Slave mode itself is also normally disabled momentarily when writing to the $2000 register, which can cause a stray dot in the image.

Even with official chips, the mod still has to implement some of the PPU's functionality, so emulation is still involved, just of a much smaller part of the PPU than a whole new implementation or even what's required for krikzz's RGB Blaster.
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

Fiskbit wrote: Fri Mar 17, 2023 6:56 am Compatibility of a clone or FPGA solution is only as good as its implementation.....

Using the official chips doesn't allow you to completely avoid problems. Both the NESRGB and Hi-Def NES rely on an unused PPU mode generally referred to as slave mode.....

Even with official chips, the mod still has to implement some of the PPU's functionality, so emulation is still involved, just of a much smaller part of the PPU than a whole new implementation or even what's required for krikzz's RGB Blaster.
Thank you Fiskbit! This cleared up my assumptions in a way I understood. I realize you repeated some of the items tepples mentioned so I appreciate you taking the time to try and respond in a different way.
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

Fiskbit wrote: Fri Mar 17, 2023 6:56 am Using the official chips doesn't allow you to completely avoid problems. Both the NESRGB and Hi-Def NES rely on an unused PPU mode generally referred to as slave mode...
If my goal is to simply play an original NES console in HD on modern TV's, what is the advantage to spying on the PPU/CPU to upscale to digital video out like the Hi-Def NES kit does versus a plug-n-play upscaler like the retrotink5x?

EDIT: I realize the Hi-Def NES kit allows you to adjust settings from the console controller where as the retrotink needs it's own separate remote, so my question is excluding that advantage.
Joe
Posts: 650
Joined: Mon Apr 01, 2013 11:17 pm

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by Joe »

Spying on the bus gets you pixel-perfect output with no NTSC (or PAL) artifacts and your choice of RGB palette.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by Fiskbit »

I know very little about that device specifically, so there's some amount of speculation in this post. Someone else who knows more can correct me or fill in the gaps.

The PPU generates composite video natively and that video is full of artifacts, such as color artifacts on chroma or luma transitions and the 3-scanline zigzag pattern. I would imagine that the RetroTink device is faithfully preserving these artifacts in its conversion to HDMI, because otherwise it has to try to decode the original 256x240 picture (or 283x242 if you're including the border) from this signal. Based on a prior discussion on Discord, I believe that conversion can be done, but the device presumably needs to know it's operating on an NES signal and there is some difficulty when working with color emphasis (specifically when 2 bits are set, if I recall correctly).

There's also still a mismatch in frame rate because the console is running at its original speed of about 60.1 FPS while HDMI expects 60.0, so unless you're using something like gsync or freesync, you'll either have tearing in the form of a seam slowly running across the screen or an occasional dropped frame, on a period of 10 seconds.

The Hi-Def NES avoids these issues because it gets (mostly) clean digital pixel data from the PPU and it slows down the system to 60.0 FPS (which is fine for the casual gamer, but not the speedrunner). But there are costs as outlined in my prior post, and because the PPU has to be partially emulated, there is room for error there (and the NESRGB in particular doesn't display anything when rendering is disabled, which breaks color effects such as those used in the Micro Machines menus).

Since you're interested in HDMI specifically, note that getting the proper 8:7 pixel aspect ratio there is challenging. Without 4k, there aren't enough pixels for this, so you end up with pixels that are not uniform size and ugly 'shimmering ' when scrolling as pixels change size while they move across the screen. I think composite artifacts help cover this up, but if you have a clean signal, it's very noticeable. That said, a lot of people seem to want to go to a lot of trouble to make their authentic hardware look like an inauthentic emulator, and I think many of these people are happy to play with square 1:1 pixels, so perhaps this problem is moot.
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

Fiskbit wrote: Mon Mar 20, 2023 1:23 pm Since you're interested in HDMI specifically, note that getting the proper 8:7 pixel aspect ratio there is challenging. Without 4k, there aren't enough pixels for this, so you end up with pixels that are not uniform size and ugly 'shimmering ' when scrolling as pixels change size while they move across the screen. I think composite artifacts help cover this up, but if you have a clean signal, it's very noticeable. That said, a lot of people seem to want to go to a lot of trouble to make their authentic hardware look like an inauthentic emulator, and I think many of these people are happy to play with square 1:1 pixels, so perhaps this problem is moot.
I would probably just have it output 4k only if that made the project easier and/or cheaper. I don't think I care about artifacts but if spying on the PPU?CPU fixes that then I'd prefer doing that.
I personally don't care about losing the 0.1 FPS.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by Fiskbit »

Can you be more specific about what your end goal is? It sounds like you're OK with whatever limitations, so have you tried existing composite-to-HDMI converters and is there something that makes them unacceptable?

The way things are, no solution is perfect; they all introduce at least some kind of visual problem in some situations and, for digital video without freesync, either timing differences or tearing/hitching. Personally, I think the composite artifacts from the NES are a feature, not a bug, but as far as pixel-perfect solutions go, the one most interesting to me would be reproducing the original 256x240 or 283x242 pixel image from the composite output, as I hinted at above. This would require no modifications to the console, no emulation, and no weird passthrough cartridge that won't work well with a frontloader. Then with 4k and freesync, you'd be able to have it run at the correct aspect ratio and speed. Tepples, aka PinoBatch, wrote about this solution on the Discord a couple years ago:
PinoBatch: The composite output stage of the NES PPU performs phase modulation on a square wave. It uses both positively and negatively clocked ring counters to generate an effective 43 MHz time base from which it derives 12 phases.
PinoBatch: Each dot on NTSC contains 8 out of 12 phases, and each dot on PAL contains 10 out of 12 phases.
PinoBatch: Each color has a high level and a low level determined by bits 5-4 (0x, 1x, 2x, or 3x). A solid color will alternate 6 samples of the high level with 6 samples of the low level.
PinoBatch: Each dot on NTSC contains at least two samples of the high level and at least two samples of the low level, plus one or two transitions from high to low or vice versa.
PinoBatch: By sampling all eight phases in a dot, an ideal scaler could determine what that dot's phase is, what its high level is, and what its low level is.
PinoBatch: emphasis introduces an additional wrinkle.
PinoBatch: The high/low technique works with no emphasis bits, all emphasis bits, or one emphasis bit over gray.
PinoBatch: One emphasis bit over color requires some additional processing. The main color generator and the emphasis generator result in two overlaid square waves that combine somewhat nonlinearly. There are up to four levels and three edges in each dot, which are enough to derive the phase of both generators.
PinoBatch: If two out of three emphasis bits are on, each cycle of the emphasis generator is two phases high and ten low. This is where it gets real tricky, as the low part can cover an entire dot. Fortunately, emphasis changes much slower than color, or no faster than once every 12 pixels. Over the course of those 12 pixels, the scaler can synchronize to the phase of the emphasis generator.
PinoBatch: The one remaining case is emphasis over $0F. Low nibbles E and F temporarily disable the emphasis generator, causing the overall output to be the same as unemphasized $1D.
PinoBatch: I have not yet thought through the pathological case of two emphasis bits in a mostly-$0F picture. It should be so uncommon that I doubt anyone will notice momentary artifacts.
This approach hasn't been done yet, so I don't know what unexpected issues may be lurking. I mentioned earlier that delay in transitions on the analog signal vs the digital EXT output causes problems for NESRGB and Hi-Def NES, so maybe that could be a concern here, but since everything here would be subject to the same delay, it may be OK. Beyond that, I don't know.
forkram
Posts: 20
Joined: Mon Mar 13, 2023 7:04 pm
Location: Wisconsin

Re: Bringing a new HI-DEF NES kit to market; where to start

Post by forkram »

Fiskbit wrote: Mon Mar 20, 2023 2:58 pm Can you be more specific about what your end goal is?
You bet!
I want to play my original NES console, with original carts on an HDTV, not have it look like crap and not have noticeable lag from whatever the HD solution is; that's it.
I don't want to RGB mod and use an external upscaler.
That leaves my only option as the Hi-DEF NES kit, which is no longer made.
I know there is demand for wanting the same thing as I just watched an ebay auction for the kit end last night at $647.
=====================================================
So I want to recreate the kit; even if it has less adjustment options than the original.
The boards are easy to reverse engineer, but I don't know how to program the FPGA to take the CPU/PPU data and output the HD signal.
=====================================================
If I can get help with the programing portion (be it volunteered or paid for) in whatever way makes it the easiest to just get a 4k signal out* (no extra settings needed) and the hardware design makes it configurable enough to be able to make firmware updates to add those extra options later then I would make the program open source to allow the community to build on it.
*I'd settle for 720p only output as long as 4K was still doable with the same FPGA chip in the future.
--------------
Fiskbit wrote: Mon Mar 20, 2023 2:58 pm ..have you tried existing composite-to-HDMI converters and is there something that makes them unacceptable?
Yes, and maybe I bought the cheapest of the cheapest, but it stretches the picture out and it looks like crap. I have one Hisense Roku 4K tv that has component inputs still and it upscales the NES signal to 480p while keeping the 4:3 aspect ratio and it actually looks better than the stretched out composite-to-HDMI connection on the same TV. (I can't adjust the picture size manually in the tv settings outside some generic presets).
Side note on that TV is it is the cheapest 4K tv I have and is 12.5ms faster regarding lag input lag versus my expensive 4K Samsung
Post Reply