NES to USB Keyboard interface?

[Pokun]

I really like the Family Keyboard compatibility, that means more people would be able to play homebrew supporting it, besides making Family BASIC more accessible.

D0 is left disconnected so that Controller 2 can still operate (on FC, or if this is connected to the expansion port on an NES) but I was wondering if we want D0 connected for a controller passthrough for the interface, or perhaps having the keyboard emulate a controller.

A passthrough for any unused pins may be fine, but wouldn't using $4017-D0 for anything cause a fight with controller 2 if connected (which it normally is on most Famicoms)?

Emulating controllers for $4016-D1 and $4017-D1 (controller 3 and 4) however could be useful. A keyboard can easily fit two controllers using WASD and cursor keys.

Code: Select all

Controller 3 example:
WASD: d-pad
L: A
K: B
Q: SELECT
E: START

Controller 4 example:
Cursor keys: d-pad
Numpad radix character: A
Numpad 0: B
Numpad 7: SELECT
Numpad 9: START

That's how I always map controllers 1 and 2 in emulators as the layout is similar to the real controllers while all buttons are accessible with two hands.

[bitcores]

A passthrough for any unused pins may be fine, but wouldn't using $4017-D0 for anything cause a fight with controller 2 if connected (which it normally is on most Famicoms)?

From what I understand, because the controller/expansion port lines are pulled up on the console side by resistors to 5V and they signal binary 1s by giving path to ground, which pulls it down. So you could have both connected at the same time and they would be acting as the same controller (so you couldn't have a third player on one).
The point I was thinking was more for NES (or AV Famicom?) where the keyboard may be plugged into the second controller port. The keyboard could then be controller 2, or pass through a connector for controller 2.

But, the idea of having a keyboard act as controllers 3 and 4 by utilizing 4016D1 and 4017D1 through the expansion port sounds easily doable. I'm a bit worried about trying to combine it all into a single package, though. Unfortunately you can't really configure a Pico/RP2040 by dropping a text config file into it, it has to be compiled into the binary. That's why I'm planning to do mode select and keyboard layouts/scan code maps by jumpers. Adding more configs for keyboard controllers might be too much.
Instead we could just branch it into a separate package based on most of the same code. If we pin it out the same it could even be compatible so you just flash the one you want to use at the time.

[lidnariq]

From what I understand, because the controller/expansion port lines are pulled up on the console side by resistors to 5V and they signal binary 1s by giving path to ground, which pulls it down. So you could have both connected at the same time and they would be acting as the same controller (so you couldn't have a third player on one).

Tragically, the Famicom is not so well designed; the controller itself uses a CMOS 4021 with a push-pull output stage, and trying to override it is a bus conflict.

[Pokun]

Emulated controllers aren't that important of course, it's more like a bonus that could be added to it if it fits. The main point is to be able to use a modern keyboard with it.

I thought of something else, since Rainwarrior's protocol only uses two pins D3 and D4 (plus the strobe pin) it would be easy to adapt it for SNES (which is in much need of a keyboard since there were never a Super Family BASIC). SFC/SNES doesn't have D3 and D4 anymore but $4016-D0, $4016-D1, $4017-D0 and $4017-D1 are all accessible in their respective controller port much like NES (and unlike Famicom), so an adapter could just use those, sacrificing the controller passthrough though.
It also has a programmable I/O pin which could be used for output to the LEDs and stuff I guess. For Famicom and NES the LEDs would need to use the output pins (Output $4016-D1 or Output $4016-D2) in the Famicom or NES expansion port (missing in the NES controller ports and not even connected from the CPU on the SFC/SNES).

[bitcores]

lidnariq always appears to tell me when I am wrong. It's great.

From what I see the FC/NES and SFC/SNES controllers are technically compatible, so as long as the timing isn't an issue it should be compatible out of the box with SFC/SNES if it is connected the right way. Unfortunately I don't have any way to play SFC homebrew to make a test ROM (I've looked at doing a Pico cart for SFC, but I don't think there's any way around needing two Picos for it) so I can't test it.

I haven't really considered what to do about the status LEDs. The big problem is having that bi-directional communication because the console needs to be able to tell the keyboard what LEDs need to be lit, but it might just be easier in a per-application basis to have some indicator on the screen when the Lock keys are active, if they have any affect at all.

[lidnariq]

SNES needs more inputs from the address bus than a single Pico has free, but that should be the only limitation. To support a 2MiB ROM, one'd need 21 address lines, 8 data lines, and 1 (or 2) enables.

So you might be able to get away with "just" a single 8-bit shift register; 26 GPIO, minus 3 to drive the shift register, plus 8 from the shift register? We'd have to guess where the memory access cycles are by changes of address bus, though.

[Pokun]

From what I see the FC/NES and SFC/SNES controllers are technically compatible, so as long as the timing isn't an issue it should be compatible out of the box with SFC/SNES if it is connected the right way.

Yeah that's what I meant, the SNES also uses the exact same timing as the NES for D0 and D1 so I assume it should be fine. I don't know if the programmable I/O pin, which is a new thing for the SNES, uses different timing however.

The SNES would only be able to use a single controller with the keyboard connected, but with the 5p-multitap in the other controller port, 4 controllers + the keyboard would be possible. I guess all this stuff could be using too much power though.

I haven't really considered what to do about the status LEDs. The big problem is having that bi-directional communication because the console needs to be able to tell the keyboard what LEDs need to be lit, but it might just be easier in a per-application basis to have some indicator on the screen when the Lock keys are active, if they have any affect at all.

Yeah and since the NES controller ports only has a single output pin, which is already used for the strobe, it would not be possible to use the NES controller ports (which was one of the main points with Rainwarrior's protocol) if you want to control the LEDs.

Well Family BASIC does it without LEDs. It only has kana mode and changes the square cursor to the underscore character to indicate when kana mode is active, much like what most computers does when insert mode is active. Family BASIC's insert mode (which can only be activated by CTR+A as INS instead simply inserts a space) is not indicated in any way.

[rainwarrior]

Many keyboards do not have indicator LEDs anyway. If they're relevant to your game software you could just put an onscreen indicator for capslock/etc. Lots of people install a taskbar app to do just this because of their non-indicating keyboards. (Probably best if the USB host device tells it to keep the lights off, to avoid confusion?)

[bitcores]

I think for now I will go ahead with ignoring the Lock LEDs.

Right now I see three modes to have available; 4 byte serialized reporting, Family Basic and Subor (Subor looks like it is working the same as Family Basic except rearranged and with more rows). I'm not sure how the Subor Keyboard and Mouse are handled together in ROMs/Homebrew, though. Seeing the Mouse uses D0, doing a full matrix read of the keyboard would allow 3 bytes to be delivered through D0. I assume this is how it would work to deliver both keyboard and mouse data concurrently, but I don't know for sure.

I am thinking to have four keyboard layouts built in; US QWERTY 104-key, JP QWERTY 106-key, AZERTY ??, Chinese/custom??
This would allow most people to use the recompiled build instead of having to set up a build environment (which is really kinda annoying) and configure themselves.
--edit--
Actually, if the decision is to go with USB host only then this could all be handled on the USB host making it much easier to customize/configure without having to rebuild it, so that might be a point of consideration.
--end--

For communication to a USB host device (Pi variant probably) I am thinking of using i2c. It's just two lines and standard mode of 100kbit/s should be plenty.
There is a question of whether to keep direct input from PS/2 compatible devices that I have now, or to remove that once I have USB host device input done. Direct input would require people to provide USB connectors, PS/2 DIN connectors, or directly solder the device cables and I'm not sure if that is an option people would want to keep around when they could just connect a cheap SBC up with two wires and plug their (likely) USB devices into that.
Project could also be split into separate direct input and USB host branches.

I welcome any thoughts on these points.

[Pokun]

The three most common keyboard layouts are called "ISO" (ISO/IEC 9995-2), "ANSI" (ANSI-INCITS 154-1988) and "JIS" (JIS X 6002-1980). ANSI is the US one (also used in Netherlands) and ISO is the one used in most of Europe and other places. AZERTY is just a variant of ISO used in France. Germany has their own QWERTZ variant of ISO and in Scandinavia we use a QWERTY variant of ISO but with other unique differences in some special characters mainly to accommodate for Å, Ä/Æ and Ö/Ø characters, so ISO (105-key) is what you want.

ISO itself is a variant of ANSI with one extra unique key (OEM 7, scancode 2B) not found on ANSI and some rearrangement of keys to make room for a larger Return key (the OEM 5 key has been moved to between Left Shift and Z keys), so it's a 105-key layout (or 102-key until 1995).

JIS was a 106-key layout before 1995 (when ANSI was 101-key layout), but it became a 109-key layout after Microsoft introduced their 3 extra logo keys.

I'm not sure about Chinese keyboards. There is the "Asian 101" which is a sort of combination of ANSI, ISO and JIS with a very large Return key. There are some variants of Asian 101 like the "Asian 102".



If possible I think it would be neat if you kept PS/2, there are adapters for USB-keyboards that supports the PS/2 protocol. Though there are like 3 different scancode sets for PS/2 keyboards (the PC XT set, the PC AT set and the 3270 PC set) which may or may not be supported by various PS/2 (and AT/XT) keyboards.

[bitcores]

The three most common keyboard layouts are called "ISO" (ISO/IEC 9995-2), "ANSI" (ANSI-INCITS 154-1988) and "JIS" (JIS X 6002-1980). ANSI is the US one (also used in Netherlands) and ISO is the one used in most of Europe and other places. AZERTY is just a variant of ISO used in France. Germany has their own QWERTZ variant of ISO and in Scandinavia we use a QWERTY variant of ISO but with other unique differences in some special characters mainly to accommodate for Å, Ä/Æ and Ö/Ø characters, so ISO (105-key) is what you want.

The point is more that the scancodes map to the key arrangements, not the characters that are on the keys. So on a QWERTY keyboard the scancode for Q is the same scancode as A on an AZERTY keyboard.
Going straight from scancodes there has to be a mapping step, which is the layouts I am talking about supporting. Symbols are something that are heavily rearranged on non-US keyboards. On JP keyboards, colon and semicolon are on separate keys, so are apostrophe and quotation marks. How I should deal with that is something of an open question for me. For the USB host the solution could be combination of watching the scancodes and the ASCII output from the OS.

[Pokun]

Yeah but if you pick, say AZERTY, as the only ISO option it will only work well with AZERTY keyboards while other ISO users would be stuck with ANSI as the closest QWERTY variant (for not mentioning QWERTZ users) which I think has keys on different scancodes too and is missing a key.

A user-programmable option would be required anyway. On emulators of home computers I generally map the emulated keyboard so that its keys matches the positions of keys in most cases when possible as I generally remember what key is what anyway and it improves compatibility with software that expects the real hardware's keyboard layout. In some cases this makes it harder to type however so it is useful to be able to switch to mapping after characters as well.

[bitcores]

Ok, so here is what I am thinking.

Seeing there are so many possible layouts it would be hard to pick a few to have compiled in and selectable from jumpers to have good coverage for direct input usage, so instead I will built it targeting US 104 ANSI (which I think is a very common English layout at the least) as the reference build and have a number of available layouts to choose from or customize for people who want to compile it for their particular keyboards.

I'll handle all this on the USB host in a more easily customizable format that won't require recompiling of any of that nonsense, and this could be used with the reference build for the Pico and it wouldn't matter which layout it was built for. This is the more likely way people would use this anyway, I think.

[bitcores]

There has been some progress.

I discovered that the ps/2 mouse needs to be enabled to start sending data, so I need to work out bidirectional communication (basically rewrite the package I'm using for the keyboard). Because of this I decided to work on the USBHost side of the implementation.

I have gone with i2c and I'm using an Orange Pi PC as the host and am using python (with smbus2 and evdev modules) to capture and send the input. This could of course be rewritten into a compiled language but I'm not sure how much latency there is right now (even python could be faster than a frame on these things).
Mouse it set up with absolute positioning only so far because it is the simplest. I will need to redo how I buffer data from the USBHost for relative positioning (Pico will have to add position changes together until it is polled by the NES).

Anyway, code is going up in the repo and here's a short video showing it working.
https://www.youtube.com/shorts/P3b_20RxyJQ

--edit--
I'm going to do an edit update this time.

I think I have basically finished a full implementation of the USB host based on the specification rainwarrior laid out. Considering how robust the USB host mode is, it suggests that the complete rewrite of the PS/2 communication is absolutely necessary.
There is a point I want to get some decision or consensus on, though.
Until the USB host is ready or sends the first packets, what default state should the Pico send to the NES if the NES strobes for a read? Should it just send the 110 device identification? Should it say that there is a keyboard and mouse present and that it is in relative or absolute mode, even though it hasn't received that information from the USB host yet?
What should be the default initial state and what behavior should software written for this spec expect?

At this point, scroll wheel isn't additive because I didn't feel like writing signed 4 bit addition, but I feel like one scroll per frame it probably more than enough, unless you attach a mouse with free scrolling, but let me know if you think otherwise.
I think I'll get subor support in next, the mouse communication looks like the hardest part of that, and then I'll probably work on cleaning up and generally improving the codebase before going back to the direct input, PS/2 rewrite.
I have included two wiring guide images for Pico and Pico-Zero devices based on the way I have it arranged. If building one, you will need a diode to prevent reverse powering the NES if you draw power from the console, a 33~47uf capacitor on the NES power seems to be helpful, and enough Logic Level Converters to cover all the lanes you want to support (you need a minimum of five protected lines if wanting to support Family Basic mode, but only three if only support USB Host and serialized keyboard&mouse). I will very likely design an example PCB or two at a later date.

Finally, one more demo video https://www.youtube.com/watch?v=8wUuU8-6OOQ and the github repository again https://github.com/bitcores/pico-ps2famikb

[rainwarrior]

Until the USB host is ready or sends the first packets, what default state should the Pico send to the NES if the NES strobes for a read? Should it just send the 110 device identification? Should it say that there is a keyboard and mouse present and that it is in relative or absolute mode, even though it hasn't received that information from the USB host yet?
What should be the default initial state and what behavior should software written for this spec expect?

1. Absolute vs. relative as a default:

The communication is only one way, so the absolute vs. relative bit just allows the hardware to be one or the other, or it could have its own mode switch I guess. If the hardware you build can do both, choosing a default is probably situational.

My intent with the inclusion of an absolute mode was to make it easy to integrate mouse support that works seamlessly to move between the emulator and surrounding tools. So, in an emulated window, absolute is likely the best default, but in other situations relative is probably better.

I would expect the NES software to just check the bit after polling, and deal with both cases. If software requires one or the other, it could say so. In general, software should be able to deal with that bit changing on a per-frame basis.

In a hardware implementation, I guess if you have an absolute option, it might start up by default at 128,120 to begin in the middle of the screen? For the software, there's no guaranteed startup position for absolute anyway, since in the emulator case it would just be wherever your mouse already is.


2. Use of mouse/keyboard present bits:

The other issue, I guess you're saying it might not know if a keyboard or mouse are present right after powering on? I think it should always give the device ID whenever polled, if that's possible. Ideally software should know this host device is connected without having to delay after booting. Normally it's not a very good idea to hotswap controllers on an NES anyway, and I think this is still true for this host device, regardless of whether its USB keyboard/mouse are hot pluggable.

Otherwise, it seems fine if the mouse/keyboard present bits can change at runtime to indicate connect/disconnects. Software should generally be able to respond to this, The bit is just indicates whether either is worth polling/responding to, and that's easily something that can be decided on a per-frame basis. It could also be used to relay the device connection information to the user.

Alternatively, if it's easier to implement, there's not really anything wrong with the host always sending 1s for mouse/keyboard present regardless.

So, I think best practice for software is just that if it sees the 110 device signature at boot, it should expect that host device will be connected for the lifetime of the program, and be ready to poll and respond to either a mouse and keyboard, on any frame they report as present.

If desired, the software could do stuff like hide the mouse cursor if indicated as not present, or have an on-screen indicator for the keyboard, but that'd be up to the program.


If you think the device should be permanently attached, with most games just using the controller as pass-through, it might be nice to have an on/off switch that could disable the device signature.