The convoluted scancodes are probably only the indirect source of the problem in Mesen 2, as it's the OS's job to translate those. My guess is that because Print Screen (and possibly sometimes Pause) does not have a break code, it gets both a make and break event immediately when pushed down, and Mesen probably works based on collecting a state once per frame, so it will process the release event before it ever runs a frame to see it pressed?
Edit: Following up on that idea, I did some debugging and found that, at least on my system, Mesen 2 gets a KeyUp event but no KeyDown for Print Screen. Pause, however, had a normal down-repeat-up like other keys, though that may not be consistent across systems, since Pokun mentioned that not working.
Anyway, I made this test modification of the key mapping window to account for these two keys having momentary/incomplete events like this, which does fix the problem for the setup UI. However, it would need to do something similar at both the keyboard shortcuts layer, and the emulated input layer, to make them actually function after being mapped via the setup UI.
GitHub: momentary-key-test (code changes)
Even with the OS helping out a little, these keys are still annoying.
I also discovered, one of my keyboards has a volume wheel on the side. It actually gives both a down+up immediately, which also bypasses Mesen because it is only polling state to check (just adding these to the momentaries list in my code above solves it in the same way).
Edit: also went through some
international key testing. At least using my Windows 10 onscreen keyboard, the "Kana" key (International2) seems to have a down+repeat but no up, for some reason. The "Han/Zen" (LANG5) key seems to act as a toggle for two separate keys, giving both a down and up on those two opposing virtual keys every time it is pressed down. Probably neither of these is useful for the sake of emulation, they both seem entangled with the OS language input to me.