Kulor's Guide to Mode 7 Perspective Planes

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
creaothceann
Posts: 454
Joined: Mon Jan 23, 2006 7:47 am
Location: Germany
Contact:

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by creaothceann »

kulor wrote: Mon Aug 01, 2022 3:12 pm Image
Populate the sky with white blocks against a blue sky, and it might look like Minecraft.
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
iNCEPTIONAL
Posts: 891
Joined: Sun Jan 30, 2022 4:33 am

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by iNCEPTIONAL »

If you combine the version above (with the Mode 7 at the top and bottom of the screen) with the SNES' two window/shape masks to create the illusion of a couple of additional vertical walls (and they could even be semi-transparent)*, adjusting them in realtime to look like they match the Mode 7 scene as it moves and rotates, you could almost have the illusion of some very basic "Doom" levels, with the floor, ceiling and two simple walls.

*https://youtu.be/PKZxhsZjI20?t=629 (from around 10:29)
Last edited by iNCEPTIONAL on Tue Aug 02, 2022 2:17 pm, edited 1 time in total.
User avatar
Dwedit
Posts: 4659
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by Dwedit »

Use ?t=629 when you want to link to that particular time in the video.

The lawnmower man trick is very neat, and well done. The draw distance leaves a bit to be desired though, so you don't get that much time to react.

The perspective is always straight ahead though. One NES game that managed to pull off a similar look was Cosmic Epsilon (with a very low-res playfield), but the NES doesn't have windowing features necessary to do those walls.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
none
Posts: 102
Joined: Thu Sep 03, 2020 1:09 am

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by none »

I've had a look into this because I tried to figure out the "mystery curve". I didn't find the issue yet though.

I tried to do a perspective projection myself and I took another approach for comparison.

I think this approach might be easier to understand for some people so I'd like to try and explain it here.

I guess there are a lot of other additional ways to do this.

Basically you can look at this from the perspective of a ray tracer.

What we want to display is the plane on the ground that is defined by the plane equation:

Code: Select all

z = 0
A raytracer works by intersecting a ray from the camera with an object in world space.

We can cast two rays from the camera for each scanline, one at the left edge of the screen and one at the right edge.

Code: Select all

ray_origin = camera_position
ray_direction_left = view_matrix_3x3 * (-128, 256, scanline - 112)
ray_direction_right = view_matrix_3x3 * (128, 256, scanline - 112)
The formula for finding a point at the end of the ray, in general is

Code: Select all

point = ray_origin + ray_direction * t
where t is the distance from the origin (which is the camera) to the object we want to display (which is the plane).

we can find t by inserting the plane equation z = 0 into the ray equation

Code: Select all

z = ray_origin.z + ray_direction.z * t = 0 
=> ray.origin.z / ray_direction.z = t
and find the point of the intersection that way (which will always be at z = 0)

Code: Select all

=> point = ray_origin + ray_direction * (ray.origin.z / ray_direction.z)
Now we have the two points for the scanline.

Now there's another ingredient we need, and that's the rotation matrix that will give us our actual ray directions
I'll not go into details into how to figure that one out for sake of brevity.

Code: Select all

| cos a | -sin a cos b | sin a sin b  |
| sin a |  cos a cos b | -cos a sin b |
| 0     | sin b        | cos b        |
Here a is yaw and b is pitch.
We need to multiply with this rotation matrix to get a world space ray from a screen space ray.
I.e. we generate the ray directions in screen space (with y pointing into the screen), and then transform them into world space with a matrix transform.

Code: Select all

ray_direction.x = screenspace.x * cos(a) + 256 * -sin(a) * cos(b) + (scanline - 112) * sin(a) * sin(b)
ray_direction.y = screenspace.x * sin(a) + 256 * cos(a) * cos(b) + (scanline - 112) * -cos(a) * sin(b)
ray_direction.z = 256 * sin(b) + (scanline - 112) * cos(b)
There's a lot of stuff here that is actually the same for the left ray and the right ray so this doesn't need to be computed twice (it's constant across the scanline)

Code: Select all

constant.x = 256 * -sin(a) * cos(b) + (scanline - 112) * sin(a) * sin(b)
constant.y = 256 * cos(a) * cos(b) + (scanline - 112) * -cos(a) * sin(b)
constant.z = 256 * sin(b) + (scanline - 112) * cos(b)

ray_direction_left.x = -128 * cos(a) + constant.x
ray_direction_left.y = -128 * sin(a) + constant.y
ray_direction_left.z = constant.z

ray_direction_right.x = 128 * cos(a) + constant.x
ray_direction_right.y = 128 * sin(a) + constant.y
ray_direction_right.z = constant.z
That's enough to get the actual left and right point coordinates.

Code: Select all

distance = camera.z / constant.z

point_left.x = camera.x + (-128 * cos(a) + constant.x) * distance
point_left.y = camera.y + (-128 * sin(a) + constant.y) * distance

point_right.x = camera.x + (128 * cos(a) + constant.x) * distance
point_right.y = camera.y + (128 * sin(a) + constant.y) * distance
That's everything we need to generate the actual mode 7 parameters quite easily.

The trick is to place the center of rotation (m7x and m7y) exactly at point_left.
Also set the horizontal scroll offset (hofs) to point_left's x component.

Now the leftmost pixel is at the correct position and we can use the the matrix registers m7a and m7c to align the rest of the pixels in the scanline correctly.
It is easiest to think about those here not as a rotation and scale thing as it is usually explained, but rather as a "path" that the hw should trace across the plane from the left point to the right point.
We can make this path by computing a vector from the left point to the right point.

Code: Select all

offset = point_right - point_left
And we can just plug the components of that vector into the matrix registers m7a and m7c.

m7b and m7d, and m7vofs can all be set to zero.
Alternatively, m7vofs can be set to (point_left.y - scanline), then m7b and m7d can be set to any value without an effect on the result because the hw will multiply them with zero.



The full JS snippet

Code: Select all

function cos(a) {return(Math.cos(a))}
function sin(a) {return(Math.sin(a))}

var yaw = var1 * Math.PI / 180;
var pitch = var2 * Math.PI / 180;

var camera_x = 0;
var camera_y = 0;
var camera_z = 100;

var constant_x = 256 * -sin(yaw) * cos(pitch) + (scanline - 112) * sin(yaw) * sin(pitch);
var constant_y = 256 * cos(yaw) * cos(pitch) + (scanline - 112) * -cos(yaw) * sin(pitch);
var constant_z = 256 * sin(pitch) + (scanline - 112) * cos(pitch);

var distance = camera_z / constant_z;

var point_left_x = camera_x + (-128 * cos(yaw) + constant_x) * distance;
var point_left_y = camera_y + (-128 * sin(yaw) + constant_y) * distance;

var point_right_x = camera_x + (128 * cos(yaw) + constant_x) * distance;
var point_right_y = camera_y + (128 * sin(yaw) + constant_y) * distance;

var offset_x = point_right_x - point_left_x;
var offset_y = point_right_y - point_left_y;

m7a = offset_x;
m7b = 0;
m7c = offset_y;
m7d = 0;
m7x = point_left_x;
m7y = point_left_y;
m7hofs = point_left_x;
m7vofs = 0;

return [m7a, m7b, m7c, m7d, m7x, m7y, m7hofs, m7vofs];
Edit: Minor correction... m7vofs doesnt need to be set
none
Posts: 102
Joined: Thu Sep 03, 2020 1:09 am

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by none »

Here's an improved version of the above, this one takes into account that the precision is better for the b and d registers than for the x and y registers.
I had a look at how the PPU actually does the math, and with this version, the result should be about the same, but point_left_x and point_left_y each now have 5 bits of subpixel precision. This looks a lot better when moving the camera close to the ground.

Code: Select all

function cos(a) {return(Math.cos(a))}
function sin(a) {return(Math.sin(a))}

var yaw = var1 * 5 * Math.PI / 180;
var pitch = var2 * 5 * Math.PI / 180;

var camera_x = 0;
var camera_y = 0;
var camera_z = 40;

var constant_x = 256 * -sin(yaw) * cos(pitch) + (scanline - 112) * sin(yaw) * sin(pitch);
var constant_y = 256 * cos(yaw) * cos(pitch) + (scanline - 112) * -cos(yaw) * sin(pitch);
var constant_z = 256 * sin(pitch) + (scanline - 112) * cos(pitch);

var distance = camera_z / constant_z;

var point_left_x = camera_x + (-128 * cos(yaw) + constant_x) * distance;
var point_left_y = camera_y + (-128 * sin(yaw) + constant_y) * distance;

var point_right_x = camera_x + (128 * cos(yaw) + constant_x) * distance;
var point_right_y = camera_y + (128 * sin(yaw) + constant_y) * distance;

var offset_x = point_right_x - point_left_x;
var offset_y = point_right_y - point_left_y;

m7a = offset_x;
m7b = point_left_x * 64;
m7c = offset_y;
m7d = point_left_y * 64;
m7x = 256;
m7y = 4;
m7hofs = 0;
m7vofs = -scanline;

return [m7a, m7b, m7c, m7d, m7x, m7y, m7hofs, m7vofs];
User avatar
kulor
Posts: 33
Joined: Thu Mar 15, 2018 12:49 pm

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by kulor »

none wrote: Tue Aug 02, 2022 9:19 amI tried to do a perspective projection myself and I took another approach for comparison.

I think this approach might be easier to understand for some people so I'd like to try and explain it here.
Thanks for posting this, very nicely done! I made some minor adjustments to account for a horizon, make the variable controls work more like they do in mine, and have it move in a circular path if you click "do animation".

Code: Select all

function cos(a) {return(Math.cos(a))}
function sin(a) {return(Math.sin(a))}

var yaw = var3 * 5 * Math.PI / 180;
var pitch = (180 - var1) * -Math.PI / 180;

var camera_x = Math.sin(((framecount / 10) % 256) / 256 * 2 * Math.PI) * 512;
var camera_y = -Math.cos(((framecount / 10) % 256) / 256 * 2 * Math.PI) * 512;
var camera_z = -256 - var2 * 5;

var constant_x = 256 * -sin(yaw) * cos(pitch) + (scanline - 112) * sin(yaw) * sin(pitch);
var constant_y = 256 * cos(yaw) * cos(pitch) + (scanline - 112) * -cos(yaw) * sin(pitch);
var constant_z = 256 * sin(pitch) + (scanline - 112) * cos(pitch);

var distance = camera_z / constant_z;

var point_left_x = camera_x + (-128 * cos(yaw) + constant_x) * distance;
var point_left_y = camera_y + (-128 * sin(yaw) + constant_y) * distance;

var point_right_x = camera_x + (128 * cos(yaw) + constant_x) * distance;
var point_right_y = camera_y + (128 * sin(yaw) + constant_y) * distance;

var offset_x = point_right_x - point_left_x;
var offset_y = point_right_y - point_left_y;

if (distance > 0) {
	m7a = offset_x;
	m7b = 0;
	m7c = offset_y;
	m7d = 0;
	m7x = point_left_x;
	m7y = point_left_y;
	m7hofs = point_left_x;
	m7vofs = 0;
}
else {
	m7a = 0x0000;
	m7b = 0x0000;
	m7c = 0x0000;
	m7d = 0x0000;
	m7x = 128;
	m7y = 112;
	m7hofs = 0;
	m7vofs = 0;
}

return [m7a, m7b, m7c, m7d, m7x, m7y, m7hofs, m7vofs];
Yeah, that's definitely a much more elegant, cleaner/nicer solution than mine! No mysterious magic number curves either. :lol:
I double-checked it with my Unity reference scene and it's pretty much perfect. It looks like the normalized height of this camera would be 256, and the FOV would be 47.333[...] degrees. I would be curious where you defined that FOV though, or how it would be adjusted. I know for mine, I sorta started trying to parameterize FOV, but kinda gave up on it halfway through, and I suspect the mystery curve would need to be solved in order to be able to have a truly adjustable FOV...
At a glance, it kinda looks like what the GBA tutorial was trying to explain? I wasn't really able to derive this from it or the source code though, so it's interesting to see it in action. I'd be curious to see what handling sprite transforms looks like with this implementation.
Probably the only two arguments you could make for my implementation in favor of this would be, for one, it seems like mine breaks down a bit cleaner at lower altitudes, especially at odd yaw angles.
And for two, mine would require 4 HDMA tables across 2 HDMA channels, one each for m7a, m7b, m7c, m7d, and only requires writing m7y, m7vofs, and m7hofs once per frame. Yours looks like it would require 5 HDMA tables across 4 HDMA channels, one each for m7a, m7c, m7x, m7y, and m7hofs...but if your per-line values are significantly easier to calculate, then it may come out ahead regardless.

Well, looks like you ninja'd me and fixed both of those, so yours is definitely the far better solution. Really nice!
none
Posts: 102
Joined: Thu Sep 03, 2020 1:09 am

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by none »

FOV can be adjusted by changing the set of 3 "256" values around, sorry for not making that a variable. It is basically the forward component of the view vector. FOV should be atan(forward / 128) * 2.
I.e. if you set forward to 128 instead, you should get a FOV of exactly 90 degrees.

Idk about this version really being better, I like your solution too, I think the texture mapping / perspective correction derived approach is valid and it could be streamlined a little probably. i also still need HDMA for vofs.

In the end performance will depend a lot on how many values need to be interpolated across the y direction to make the values for the HDMA table. It bothers me a little that I have to do some things twice for the left and right ray (probably need to do the division twice because of fixed point precision issues)
lidnariq
Posts: 10932
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by lidnariq »

I split the tangent about multiple textures into this thread: viewtopic.php?t=24060
none
Posts: 102
Joined: Thu Sep 03, 2020 1:09 am

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by none »

I don't get the GBA tutorial either, but I don't think he's was doing the same thing that I was doing, because he's talking alot about it in terms of rotating and zooming the bg to make it fit.

About positioning the sprites, I'd go about it much the same way you did. Basically all the conversion from screenspace to worldspace needs to be done in reverse. For that we need the inverse of the matrix. This is a little sketchy with the terminology because in standard cg terminology it is the other way around, what I was calling the view / rotation matrix above is called the inverse view matrix, while the matrix that transforms from worldspace to screenspace is usually what is just called the view matrix.

Anyways, since computing an inverse matrix is kind of tricky in the general case it is easier to get this matrix by just applying all the transforms in reverse, in this case first rotating by the negative pitch, then the negative yaw. This is basically the matrix that you already mentioned in the op, it's just we use different coordinate systems.

Code: Select all

| cos(a) | sin(a) | 0 |  
| -sin(a) cos(b) | cos(a) cos(b) | -sin(b) |
| -sin(a) sin(b) | cos(a) sin(b) |  cos(b) |
The sprites screenspace coordinates can be obtained in different ways, but I'd do it this way

First, compute the vector from the sprite to the camera.

Code: Select all

sprite_camera_vector = camera - sprite
Then, multiply the vector by the inverse rotation matrix

Code: Select all

sprite_camera_vector' =  inverse(view_matrix_3x3) * sprite_camera_vector
Doing it this way avoids computing the fourth column for a 4x3 or 4x4 matrix.

Finally, you can just do the perspective division. With this coordinate system, negative y is the axis that points into the screen, so it's important to divide by -y.

Code: Select all



function cos(a) {return(Math.cos(a))}
function sin(a) {return(Math.sin(a))}

// setup

var FOV = 90;
var forward = 128 / Math.tan(FOV * (Math.PI * 2 / 360) / 2);

var yaw = var1 * 5 * Math.PI / 180;
var pitch = var2 * 5 * Math.PI / 180;

var camera_x = 0;
var camera_y = 0;
var camera_z = 50;

// sprite stuff

var sprite_x = -8;
var sprite_y = 90;
var sprite_z = 0;

sprite_x = camera_x - sprite_x;
sprite_y = camera_y - sprite_y;
sprite_z = camera_z - sprite_z;

var tf_sprite_x = sprite_x * cos(yaw) + sprite_y * sin(yaw);
var tf_sprite_y = sprite_x * -sin(yaw) * cos(pitch) + sprite_y * cos(yaw) * cos(pitch) + sprite_z * -sin(pitch);
var tf_sprite_z = sprite_x * -sin(yaw) * sin(pitch) + sprite_y * cos(yaw) * sin(pitch) + sprite_z * cos(pitch);

var ss_sprite_x = tf_sprite_x * forward / -tf_sprite_y + 128;
var ss_sprite_y = tf_sprite_z * forward / -tf_sprite_y + 112;

rectangle(ss_sprite_x - 8, ss_sprite_y - 8, 16, 16);

// mode 7 stuff

var constant_x = forward * -sin(yaw) * cos(pitch) + (scanline - 112) * sin(yaw) * sin(pitch);
var constant_y = forward * cos(yaw) * cos(pitch) + (scanline - 112) * -cos(yaw) * sin(pitch);
var constant_z = forward * sin(pitch) + (scanline - 112) * cos(pitch);

var distance = camera_z / constant_z;

var point_left_x = camera_x + (-128 * cos(yaw) + constant_x) * distance;
var point_left_y = camera_y + (-128 * sin(yaw) + constant_y) * distance;

var point_right_x = camera_x + (128 * cos(yaw) + constant_x) * distance;
var point_right_y = camera_y + (128 * sin(yaw) + constant_y) * distance;

var offset_x = point_right_x - point_left_x;
var offset_y = point_right_y - point_left_y;

m7a = offset_x;
m7b = point_left_x * 64;
m7c = offset_y;
m7d = point_left_y * 64;
m7x = 256;
m7y = 4;
m7hofs = 0;
m7vofs = -scanline;

return [m7a, m7b, m7c, m7d, m7x, m7y, m7hofs, m7vofs];
User avatar
kulor
Posts: 33
Joined: Thu Mar 15, 2018 12:49 pm

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by kulor »

I got ninja'd again!
none wrote: Tue Aug 02, 2022 3:45 pm i also still need HDMA for vofs.
Ahh that's true, I didn't see that hiding at the bottom there. So that would be 3 HDMA channels to my 2, but at least the table for that wouldn't need to be maintained at all.
Really I just like that there's no magical mystery curve. Still curious what that's about...
Thanks for the writeup on sprite positioning, I was thinking there might be some work or matrices shared between sprites and the plane that would make it much easier to compute vs. mine, where I'm doing these matrices solely for the sprites and not even touching them for the plane. Doesn't seem like that's the case though. Being able to get away with just a 3x3 matrix is interesting though, I bet that would be a more optimized way to handle it.
User avatar
kulor
Posts: 33
Joined: Thu Mar 15, 2018 12:49 pm

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by kulor »

Thinking some more about how you might optimize this, and...I think there might only be 2 actual multiplications per-scanline needed in my implementation?

Code: Select all

var topdist = dist(topa);
var btmdist = dist(btma);
var sl = lerp(1/topdist, 1/btmdist, scanline / 223);
var scale = (1/sl) * distanceToScale;
[...]
	m7a = Math.cos(a) * scale;
	m7b = Math.sin(a) * scale;
	m7c = -Math.sin(a) * scale;
	m7d = Math.cos(a) * scale;
topdist and btmdist only need to be measured once per frame/plane. For sl, you don't really need to lerp, you could just do (1/topdist - 1/btmdist) / 223 once per frame, then iteratively add that to get each scanline's value...maybe you'd lerp a couple different ones and iterate to get the in-betweens, just to keep error buildup down. scale is always 1/value * (8/7), so it's possible that entire thing could just be a lookup table, assuming you don't need dynamic FOV (go make an N64 game instead!). The only ones you really need to do are the cos(a) * scale and sin(a) * scale. Unless I'm missing one somewhere, I think that's it, everything else is just once per-frame.
User avatar
rainwarrior
Posts: 8399
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by rainwarrior »

kulor wrote: Tue Aug 02, 2022 8:01 pm Thinking some more about how you might optimize this, and...I think there might only be 2 actual multiplications per-scanline needed in my implementation?
I haven't gone over your calculations in detail, though I've been working on something similar myself.

Generally speaking, for it to be perspective correct you should need a divide on each line, though you might get away with doing it every few lines and interpolating. After working out and timing an algorithm for my own version, it felt like every 2nd line was reasonable. (It's not yet fully realized, though, so I can't share it yet.)

I looked a bit at F-Zero and Final Fantasy VI. I don't think either of them does interpolation, but with the compromise of using the same values for horizontal and vertical scale they can cut the computation in half. The computation is less than 1-scanline per HDMA line. However, the compromise means that the draw-distance/vertical and view-width/horizontal are entangled in a way that can't be changed independently.

That's what motivates me to interpolate every other line. I think I can manage separated scaling, but the computation is little more expensive per-line, so I want to interpolate to make up the difference.

I also looked a bit at Mario Kart and Pilotwings, which do have independent axis scaling... but then I realized they had the DSP1 to play with, so they had extra power to accommodate it.
kulor wrote: Tue Aug 02, 2022 8:01 pmit's possible that entire thing could just be a lookup table, assuming you don't need dynamic FOV (go make an N64 game instead!).
All 4 of the SNES games I mentioned can dynamically change the tilt. I don't think it's unreasonable to ask of the SNES... but if you allow a dynamic perspective, I don't think there's a practical way to use a table to bypass the divide.
User avatar
kulor
Posts: 33
Joined: Thu Mar 15, 2018 12:49 pm

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by kulor »

rainwarrior wrote: Tue Aug 02, 2022 9:49 pm All 4 of the SNES games I mentioned can dynamically change the tilt. I don't think it's unreasonable to ask of the SNES... but if you allow a dynamic perspective, I don't think there's a practical way to use a table to bypass the divide.
Tilt would be pitch, not FOV. Dynamic FOV would be something like, being able to zoom in with a sniper rifle in a game like Goldeneye or MDK, or I seem to recall Ocarina of Time animates camera position with FOV to get this kind of "background zooming out" effect...it's a bit hard to explain, if I can find it I'll give it a link here. I've never seen an SNES game with an adjustable FOV, that was only something that started cropping up with real polygon engines in 5th gen stuff.
Drag
Posts: 1447
Joined: Mon Sep 27, 2004 2:57 pm
Contact:

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by Drag »

kulor wrote: Wed Aug 03, 2022 7:23 am I seem to recall Ocarina of Time animates camera position with FOV to get this kind of "background zooming out" effect
That's called a "dolly zoom", if you or anyone else were interested. :D
User avatar
kulor
Posts: 33
Joined: Thu Mar 15, 2018 12:49 pm

Re: Kulor's Guide to Mode 7 Perspective Planes

Post by kulor »

Drag wrote: Wed Aug 03, 2022 8:11 am That's called a "dolly zoom", if you or anyone else were interested. :D
Yeah that's totally it!
I was scouring OOT cutscenes trying to find a really obvious example of it, and this was the first one I found. I'm pretty sure they're subtly tweening FOV all over the place in pretty much every cutscene in this game though.
Post Reply