N64 3D Basics: How do you even display a single polygon?

[Drew Sebastino]

I know this is really sad, but I've looked for things like "3D programming basics" and they usually don't go more in depth than saying that you have to work on a 3 coordinate plane and then they call it a day, sometimes showing you some java code or something that doesn't help in the slightest. The only potentially useful thing I ever found was this:

Reality Coprocessor.pdf

(394.52 KiB) Downloaded 252 times

which at least helps with registers and some hardware information, but it isn't nearly as straightforward as "BG 2 X Scrolling Position". I think I also read something about how you actually need to program the video chip, and this is the supposed "microcode" I've always heard about? I don't even understand how that works. another thing is how do you write the coordinates for points and then fill it in with even just a simple color? Is there something like a giant space of memory the video chip uses that's kind of like OAM on the SNES, in that there are some thousands of slots for points that contain registers for their X, a Y, and a Z, coordinates? Also, how does the "frame buffer" have any relation to this? I though frame buffers were only useful for software rendering, like if you look at vram in a Super FX game. Is it like the video chip has a frame buffer, but you don't need to mess with it because it puts everything there? Can you even directly access the frame buffer for drawing over the picture I guess? Sorry if this sounds terribly uneducated, I just haven't found a single useful thing.

Edit: I actually found this http://n64.icequake.net/doc/n64intro/ka ... 2/4-3.html and it says this:

Sprite Microcode
•S2DEX is the microcode that makes it possible to use sprites in N64 development. This microcode makes it possible for you to manage drawn objects using the separate concepts of the sprite and the BG. This makes it easy for you to get used to the usual sprite game development methods.

Can you essentially turn the N64 into a 2D machine, where it accepts tiles and whatnot? (Again, I have no 3D hardware experience.)

Edit 2: How does resolution (or frame buffer size here, I'd imagine) affect performance?

[DoNotWant]

- I think I also read something about how you actually need to program the video chip, and this is the supposed "microcode" I've always heard about?

I think so. I guess it would be sort of like how you program GPUs today with shader languages.


- I don't even understand how that works. another thing is how do you write the coordinates for points and then fill it in with even just a simple color? Is there something like a giant space of memory the video chip uses that's kind of like OAM on the SNES, in that there are some thousands of slots for points that contain registers for their X, a Y, and a Z, coordinates?

I don't think the N64 is tile/sprite based like the SNES, it's polygons and textures. I don't know how
the gfx are stored in memory.


- Can you essentially turn the N64 into a 2D machine, where it accepts tiles and whatnot? (Again, I have no 3D hardware experience.)

Yeah, I think this guy got some SNES example(tiles).
https://github.com/PeterLemon/N64/


- Edit 2: How does resolution (or frame buffer size here, I'd imagine) affect performance?

Same as on the SNES? Bigger resolution -> more pixels -> more data needs to be transfered.
Same goes for color depth; More BPP -> more data per pixel -> same as above.

I hope I helped a little, sorry if something is inaccurate.

[Joe]

I think I also read something about how you actually need to program the video chip, and this is the supposed "microcode" I've always heard about?

The RSP (signal processor) is a chopped-up R4000-based CPU with SIMD instructions. It runs microcode. Most games use it to transform 3D models into a 2D projection for the RDP to render, though you can make it do anything you want with your own microcode.

The RDP (display processor) is a rasterizer that takes a series of command packets and spits out polygons. Usually it's fed by the RSP, though it can also take input directly from the CPU.

another thing is how do you write the coordinates for points and then fill it in with even just a simple color?

The document you've linked covers enough of the RDP instruction set to initialize the RDP and make it spit out a solid-colored rectangle, but you don't even have to use the RDP if you don't want to. (Namco Museum doesn't use the RSP or RDP at all - it renders everything in software!)

Is there something like a giant space of memory the video chip uses that's kind of like OAM on the SNES, in that there are some thousands of slots for points that contain registers for their X, a Y, and a Z, coordinates?

Nope. Instead, you put RDP commands in memory and tell the RDP to execute them. How you store your 3D data is up to you, but most N64 games use a format that looks pretty similar to the RDP instruction set (and use the RSP to translate it into actual RDP instructions).

Also, how does the "frame buffer" have any relation to this? I though frame buffers were only useful for software rendering, like if you look at vram in a Super FX game. Is it like the video chip has a frame buffer, but you don't need to mess with it because it puts everything there? Can you even directly access the frame buffer for drawing over the picture I guess?

The frame buffer is the section of RAM that contains the pixels to be output to the screen. It can be located anywhere in the N64's RAM. The RDP can render to arbitrary buffers in RAM, but most of the time you'll want it rendering to the frame buffer. Since it's just normal RAM, you can directly access the frame buffer (this is how Namco Museum works), but if you're already using the RDP to fill the frame buffer, it's easier to just have the RDP do all of the work instead of trying to share the work between CPU and RDP.

Can you essentially turn the N64 into a 2D machine, where it accepts tiles and whatnot? (Again, I have no 3D hardware experience.)

The RDP is more than capable enough to handle something like that. All you have to do is set the appropriate transform parameters. Of course, there's always the option of doing your rendering in software, like Namco Museum.

How does resolution (or frame buffer size here, I'd imagine) affect performance?

Bigger framebuffers are slower, since you have to render more pixels to fill the whole screen. In theory, you can make the framebuffer as big as you want, but horizontally you're limited by the pixel clock (~12.5MHz, ~640 pixels) and vertically you're limited by the TV standard (240p/480i for NTSC and PAL-M, 288p/576i for PAL).

The format of the framebuffer also affects speed, since it determines the size. A 32bpp framebuffer is twice as big as a 16bpp framebuffer, which makes it take about twice as long to fill.

[tokumaru]

I had no idea how the N64 worked... So it has a regular 2D frame buffer that you can fill however you want, but it has some extra hardware that can help you fill that with 3D graphics?

How do emulators that render the graphics with better resolution than the original hardware work then? If the 3D graphics are turned into pixels before being displayed, how is this kind of enhancement possible?

[Drew Sebastino]

One caveat to be careful is that anti-aliasing must be enabled when running in 320x240 16bpp mode. if you do not at least enable resampling anti-aliasing when running in this mode, you will get corrupted video. All other modes do not seem to have this restriction.

And of course, that would have probably have been the best for a 2D game...

I had no idea how the N64 worked... So it has a regular 2D frame buffer that you can fill however you want, but it has some extra hardware that can help you fill that with 3D graphics?

Seems pretty unorthodox to me. I would have thought it would have been a lot more hardware based in the way everything is rendered, like just about every 2D system in that you don't use a frame buffer for sprites or whatever.

The document you've linked covers enough of the RDP instruction set to initialize the RDP and make it spit out a solid-colored rectangle, but you don't even have to use the RDP if you don't want to. (Namco Museum doesn't use the RSP or RDP at all - it renders everything in software!)

Couldn't you literally just write a word to the frame buffer and see a pixel on the screen? I guess you'd have to set up the frame buffer, but that shouldn't be hard.

You know, aside from the part where the frame buffer goes to the screen, how is the video chip any more dedicated to 3D than the main CPU? (Apparently, the z-buffer, which apparently acts as a priority thing so stuff doesn't get drawn on stuff that's supposed to be closer to the camera, is bad on the N64 in that it slows down fill rate, so people even had to code their own. Does any game actually use CPU power to aid with the graphics?)

[tepples]

I had no idea how the N64 worked... So it has a regular 2D frame buffer that you can fill however you want, but it has some extra hardware that can help you fill that with 3D graphics?

How do emulators that render the graphics with better resolution than the original hardware work then?

By high-level-emulating the RDP and shortcutting the frame buffer when the plug-in feels it appropriate to do so. In the most advanced HLEs, textures in the texture buffer get hashed into high resolution textures, much as in the "high definition" SMS and GB emulators.

One caveat to be careful is that anti-aliasing must be enabled when running in 320x240 16bpp mode. if you do not at least enable resampling anti-aliasing when running in this mode, you will get corrupted video. All other modes do not seem to have this restriction.

And of course, that would have probably have been the best for a 2D game...

If you have a 2D game, 60 fps, and occasional sprite scaling, would it be a good idea to turn on pseudo-interlace and resampling AA so that 2D sprites can grow and shrink smoothly?

You know, aside from the part where the frame buffer goes to the screen, how is the video chip any more dedicated to 3D than the main CPU?

The Reality coprocessor consists of the RSP, a CPU whose instruction set appears to be designed for vertex shaders, and the RDP, a fixed-function texture mapper. Several other classic platforms use a "blitter" that takes commands from the CPU to draw to a frame buffer, such as Williams arcade boards and the Atari Lynx. Anyway, I'm assuming that it's possible to use the RDP without the RSP, in which case all your triangles have to be transformed and lit on the CPU. Am I right?

[Drew Sebastino]

occasional sprite scaling

But I don't want any sprite scaling. I think ant-aliasing looks horrible for 2D games in the way koitsu thinks anti-aliased font does.

You know, if graphics are essentially software rendered with a coprocessor, why are there limits on things like texture size? I'm really not sure how much is done in software and how much is done in hardware on the RSP.

[Drag]

if graphics are essentially software rendered with a coprocessor, why are there limits on things like texture size? I'm really not sure how much is done in software and how much is done in hardware on the RSP.

If there is a texture size limit, it would be because the RSP or the RDP only supports a limited texture size when texturing polygons. You'd have no texture size limit if you rendered everything in software, but you'd lose the speed boost of having hardware-accelerated graphics.

I was confused about how PSX graphics work, if I recall correctly, you give commands to a helper-chip (like "draw a triangle here, draw a sprite here, draw a solid rectangle here", etc) which drives the hardware that produces the actual 3D graphics on the framebuffer. If I've understood this thread correctly, the N64 is similar with the RSP (the "helper chip") and RDP, except you program the RSP yourself, instead of being stuck with whatever the built-in functionality is, like with the PSX.

Could you say the RSP is similar to the Super-FX chip on the SNES? The more I think about it, the more similar they seem.

[thefox]

I don't think there's much point to programming N64 at the register level. You'd be better off using the official C API provided by Nintendo (or some amateur library, usually with less features, if you worry about the legalities). By the time N64 and PSX came around, the hardware started to be a bit too complex to handle manually in a convenient way (and CPUs got faster, so the penalty of the abstractions was less of a problem).

If you dig up the official Nintendo 64 SDK, you'll find explanations of the overall system architecture and the API they provided.

[DoNotWant]

I don't think there's much point to programming N64 at the register level. You'd be better off using the official C API provided by Nintendo (or some amateur library, usually with less features, if you worry about the legalities). By the time N64 and PSX came around, the hardware started to be a bit too complex to handle manually in a convenient way (and CPUs got faster, so the penalty of the abstractions was less of a problem).

If you dig up the official Nintendo 64 SDK, you'll find explanations of the overall system architecture and the API they provided.

You wouldn't be able to legaly release a homebrew using PSY-Q or anything official tho.
Maybe libdragon is something to look into?
https://dragonminded.com/n64dev/libdragon/

[Sik]

Somebody may want to help with this:
http://forums.tigsource.com/index.php?topic=46690

(early incarnations involved a software renderer but it was switched to microcode later on)

How do emulators that render the graphics with better resolution than the original hardware work then? If the 3D graphics are turned into pixels before being displayed, how is this kind of enhancement possible?

They cheat like crazy. Nintendo didn't allow custom microcode until quite later on, so there are only a few revisions of the "official" microcode and most games use them. This means that what emulators do (until not long ago at least) is to compare the loaded microcode with one of the known ones and then simulate what that microcode was supposed to do. It isn't even proper emulation, and also why stuff involving reading back from the framebuffer tends to cause trouble.

Proper microcode emulation is rather recent, and as you can imagine much slower too.

[Joe]

So it has a regular 2D frame buffer that you can fill however you want, but it has some extra hardware that can help you fill that with 3D graphics?

That's correct.

One caveat to be careful is that anti-aliasing must be enabled when running in 320x240 16bpp mode.

And of course, that would have probably have been the best for a 2D game...

The accuracy of that document is questionable. There isn't a whole lot of good information about the N64.

Seems pretty unorthodox to me. I would have thought it would have been a lot more hardware based in the way everything is rendered, like just about every 2D system in that you don't use a frame buffer for sprites or whatever.

PCs were pretty major 2D systems in the eighties and nineties, and most 2D PC games used a frame buffer. (Some 3D games did too, with software rendering.)

Couldn't you literally just write a word to the frame buffer and see a pixel on the screen? I guess you'd have to set up the frame buffer, but that shouldn't be hard.

Yep. It's pretty easy to set that up.

You know, aside from the part where the frame buffer goes to the screen, how is the video chip any more dedicated to 3D than the main CPU?

The RSP has SIMD instructions that tend to be useful for 3D graphics manipulation (and MPEG decoding, apparently). The RDP can transform textures in ways that are useful for rendering 3D graphics, and has a Z-buffer.

Anyway, I'm assuming that it's possible to use the RDP without the RSP, in which case all your triangles have to be transformed and lit on the CPU. Am I right?

You are correct.

You know, if graphics are essentially software rendered with a coprocessor, why are there limits on things like texture size? I'm really not sure how much is done in software and how much is done in hardware on the RSP.

Transform and lighting is usually done in software using the RSP. Rasterizing is usually done in hardware using the RDP. Texturing the polygons is part of the rasterizing step, which means texture size is limited by the RDP's capabilities.

Could you say the RSP is similar to the Super-FX chip on the SNES? The more I think about it, the more similar they seem.

They're pretty similar, although the RSP doesn't do any rasterizing. (The RSP is even more similar to the 3D hardware in the SGI O2...)

[Drew Sebastino]

I'm not really sure why the N64 got a 97 MHz CPU, but a 62.5 MHz GPU when GPU speed seemed to be a bigger problem than CPU speed. I image that with a very graphically strenuous game, you could have a lot of things be hardware rendered with the RSP and also software render some other things with the CPU, like if you were to render characters with the CPU and the background with the GPU or something. The only problem I could see with that is how they would interact with each other so nothing appears over something it's not supposed to, as in you'd have to make a z-buffer that would work for both.

Yep. It's pretty easy to set that up.

Hmm... I'll try to make a quick little something. (and will probably fail)

Transform and lighting is usually done in software using the RSP. Rasterizing is usually done in hardware using the RDP. Texturing the polygons is part of the rasterizing step, which means texture size is limited by the RDP's capabilities.

You're telling me it's the RDP that applies the texture? I thought all it did was put the frame buffer on the screen. Do the RSP and the RDP work together to put stuff in the frame buffer, and then the RDP takes over and puts the image on the screen when everything's done? It's like the RSP says the coordinates for the polygons, and the RDP then fills the with the texture?

The RSP has SIMD instructions that tend to be useful for 3D graphics manipulation (and MPEG decoding, apparently). The RDP can transform textures in ways that are useful for rendering 3D graphics, and has a Z-buffer.

The Z-buffer apparently sucks though. Companies toward the end of the N64's life made their own Z-buffer using custom microcode for the RSP. The problem apparently was that Z-buffering slowed down drawing stuff to the frame buffer I think.

The accuracy of that document is questionable. There isn't a whole lot of good information about the N64.

True. It doesn't really sound like the people who made the document really knew what they were doing. Did Nintendo even make it? I just found it online. Speaking of finding stuff online, I found this just recently: http://patater.com/gbaguy/n64asm.htm (Hopefully, it's all right.)

They cheat like crazy.

Is there even really a good N64 emulator? I'm talking about like BSNES level on the N64. My wimpy Celeron (I like to say it has the processing power of celery) probably couldn't handle it.

I don't think there's much point to programming N64 at the register level.

There is if you want software rendering.

[tepples]

It's like the RSP says the coordinates for the polygons, and the RDP then fills the with the texture?

Correct. Rotating coordinates based on the camera is part of vertex shading, which is the RSP's job.

The Z-buffer apparently sucks though. Companies toward the end of the N64's life made their own Z-buffer using custom microcode for the RSP. The problem apparently was that Z-buffering slowed down drawing stuff to the frame buffer I think.

Z buffer hurts fill rate because it reads memory once for each pixel and writes once for each pixel that is not occluded. If you can avoid Z buffering, such as by sorting all the polygons on the RSP before handing them off to the RDP, you get that much more fill rate. Games on the original PlayStation, which lacked a Z buffer, would use bucket sort to get the polygons in a scene into back-to-front order.

[Drew Sebastino]

Correct. Rotating coordinates based on the camera is part of vertex shading, which is the RSP's job.

So it the "microcode" just the program that reads the data you give it and it turns it into something useable? Is it kind of like a metasprite routine on a 2D system?

Z buffer hurts fill rate because it reads memory once for each pixel and writes once for each pixel. If you can avoid Z buffering, such as by sorting all the polygons on the RSP before handing them off to the RDP, you get that much more fill rate. Games on the original PlayStation, which lacked a Z buffer, would use bucket sort to get the polygons in a scene into back-to-front order.

The PS1 didn't even have a Z-buffer? Sorting polygons by priority makes sense. Could you actually figure out not to draw polygons that would be covered up by something, like a wall obstructing an enemy? You could even have it to where the backside of characters aren't drawn. I imagine you'd need a hella good microcode or something to figure all that out. Maybe the main CPU could? If it isn't handling vertex positions, I'm not really sure why you'd need 97 MHz.