nesdev.com's "not" longest thread ever.

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
Post Reply
User avatar
tokumaru
Posts: 12106
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: What does it mean to "hack" a game?

Post by tokumaru »

psycopathicteen wrote:Almost half a frame is spent doing what appears to be a metasprite drawing routine, and it just barely misses the end of the frame.
Strange how a number of SNES games suffer from this... I wonder if they all used some sort of badly unoptimized library.
MottZilla wrote:I wished someone could do that for Mega Man The Wily Wars on the Genesis as it suffers some pathetic slow down in many places.
Is it possible that they blindly/automatically translated the 6502 code into 68000 ASM instead of properly reimplementing the logic?
User avatar
koitsu
Posts: 4203
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: What does it mean to "hack" a game?

Post by koitsu »

Espozo wrote:About leaning mode 0 at your friend's house at three hundred in the morning, how old were you? I assume there weren't any tutorials online? (Or online, for that mater.) Since you've kind of given me a background history of yourself, I am going to ask when and why did you drop out of highschool? What is your current Job? You don't have to tell me anything if you don't want to.
This would have been in 1993, so that would have made 16 at the time (I was born in 1977). I skipped 2nd grade so I was a year younger than everyone else in school.

At the time there were no tutorials of any kind. The Internet (as we know it today) did not exist then. I was online at that point, and working at Oregon State University, but websites and HTTP were just barely being introduced and were very bare-bones. The web became more commonplace in the mid-to-late 90s (I still remember the first time I saw a URL shown during a TV ad -- it was from Sony).

SNES development during that era consisted mainly of discussions on Usenet (NNTP-based), a.k.a. "newsgroups", or what today is known as "Google Groups", and on private mailing lists. There was a snesdev mailing list maintained by Charles Doty (I think?) back then, and many of us contributed. Here's some proof I put on Twitter last year (be sure to look closely at the Date: header in the pictures).

There was only one document at the time (ca. 1992) on how the SNES worked, what its registers were, and how they behaved. It was a document written by Dax & Corsair (two Amiga guys). There were typos galore, with lots of bits defined wrong, and was really quite awful by today's standards (if you Google something like "corsair dax snes" you'll find some of the docs, but not all of them). That and my own hacking is what prompted me to write my SNES documentation, which is what a lot of people used for probably the next 10 years or so (people tell me they STILL use it though, just because of how I write/phrase things, with my intention being ease-of-understanding). People have found lots of mistakes in my stuff as well, and now a lot of people (most?) have access to the official developers docs anyway. The reason I got into SNES hacking was because the SNES used the same CPU as my Apple IIGS -- the day I found that out I became completely enthralled with trying to write code on the SNES. Games like Actraiser blew my mind (we had nothing like this on the IIGS).

School is somewhat of a weird subject for me (I absolutely HATE school even to this day. But strongly advocate people stay in school). I did "okay" in grammar school and junior high/middle school, getting usually Bs. I was never an A-grade student. When high school came around, especially my junior year, I really stopped caring about a lot of the subjects I was required to take. The subjects just didn't interest me. You could see it on my report cards: Ds in English, mathematics, social studies, maybe a C in sociology, while getting B+s in subjects like computer science and Chinese (both subjects I enjoyed). I failed biology twice (it was a semester course, and I failed both times -- it didn't help that they changed teachers after the first semester), and I failed "college-level" trigonometry as well as calculus.

By the time I was a senior, I was skipping pretty much all my classes except the ones that interested me (only 2 or 3 at that point). My lack of attendance eventually got me into trouble, where after 3 months (don't ask me why it took them so long to find me -- I was at school the entire time, I never left campus) the in-school suspension fellow was given a photograph of me and went looking for me + found me, and I had in-school suspension for (I think?) 3 days. You aren't allowed to leave barring restroom usage, and you have to use the time to do school work (and any new school work they go and fetch for you). Friends of mine kept coming to the in-school door and sliding notes under it that read things like "FREE YOSHI" and all this stuff, which embarassed the hell out of me and made the in-school guy go "WTF?"

The in-school programme gives you this "questionnaire" where you're asked a bunch of self-reflective questions about why you're in in-school, what you could do better, etc.. During those 3 days I wrote a 10-page (double-sided) response to that questionnaire, and at the end of the 3 days dropped the tome on the guys' desk, left, and went right back to doing what I had been (skipping most of my classes + going to the ones I found interesting). About 2 weeks later, the same guy came to the CS lab -- with the vice principal in tow -- and they asked if I'd come to the VPs office for a chat. I figured I was going to be expelled (my mother would have killed me). Instead, the in-school guy slides my 10-page work across the table to me and says "This is the most eloquent, well-thought-out, wonderfully introspective thing I have ever read from someone during all my years doing in-school suspension. The VP thought so too. We don't understand why someone of your calibre would skip class." I explained why, and they seemed to understand, but I don't think they *truly* understood. My opinion was that the in-school guy had probably, in all his years, just gotten people writing "SUCK A DICK" or "I FUCKED YOUR MOM" over and over rather than actually getting something coherent and intelligent. After the 30-minute chat, the let me leave, and I went back to what I was already doing. I saw both of them throughout the rest of the year (in the hall, usually when I was skipping class) and neither said or did anything.

At the same time I was going to school, I was working (voluntarily) at Oregon State University as a UNIX systems administrator (for about 4 years), which is eventually what lead to my career (although these days I do a lot more than just that, as I have a pretty strong background in networking as well; here's my CV/resume). That was where I got my feet wet. And during my senior year, I also lived with a Chinese family (locally) for 6 months where I wasn't permitted to speak English. Also, my high school computer science department used Novell Netware 3.1; and one year (I think my senior year?) Novell offered free CNE training (and certification, assuming we passed) to those of us in the lab who were interested (at the time I was writing DOS applications in a mix of Pascal and x86 assembly to do certain Netware transactions over IPX), so 4 of us did that and we all passed. (I have no idea where my certificate is now -- I really couldn't care less, and you couldn't pay me a billion dollars to work on Novell now, haha)

So -- when graduation came around, I had excess credits in electives (computer science, photography/art, Chinese, you name it), and had barely managed to get all of my pre-requisite course credits (I think they gave me the credit for biology, barely, because I took it twice). The only problem was: I wasn't going to be allowed to graduate. You see, my junior year, the 509J school district introduced two mandatory courses into the curriculum: "Living On Your Own" and "Personal Finance". I went to both of these classes maybe 4 or 5 times at most, and they were year-long. I thought they were absolutely ridiculous -- I was already living on my own (i.e. I wasn't living with my parents during those 6 months), and I knew how to balance my chequebook (my mother ran an engraving company and was the company's bookkeeper). In Oregon at the time you could "challenge" a course -- they'd give you the final test and if you passed it you didn't have to take the class. However, 509J deemed both of these classes MANDATORY, so you couldn't challenge them. My mother and I both contested the matter with the school district and we lost. Thus, I said fuck it, and didn't attend them.

So when my senior year ended, my school informed me that because I was 2 credits short, I would be allowed to go through the graduation ceremony (and go to prom), but what they'd hand me up at the podium would be an empty piece of paper. To me that's pointless effort, and I never cared about prom or graduation, so I did neither of them. When the next school year started (which would be my second senior year per se), I went back to high school for about 4 months before dropping out entirely (and was spending most of my time at OSU working). PSU (Portland State University) was also doing a take-university-courses-by-mail thing at the time (this would have been mid-1995), so I tried doing that for a few months but I was horrible at it -- the books and curriculum had no context, so it was just like throwing random pieces of test paperwork at me without any background. I quit.

After that, still in 1995 (I can tell because the stuff I uploaded to Twitter has my Email being @mediacity.com), I ended up moving to California to live with my girlfriend, and worked at an ISP (MediaCity) doing technical support -- until the systems administrator got wind of the fact that I had 4 years of UNIX experience and so I became a kind of combination tech support + junior UNIX SA. Since then all I've done is industry work, with a brief stint in 1997 where I worked for my mother's engraving company doing vinyl application + graphic design + sign installation.

You couldn't pay me to go back to school. I absolutely hate it. But as I said, I tell kids to stay in school -- my lack of education has never hindered me (every job I've applied for, except one, I've gotten), but the early-to-mid-90s were a very unique time to be alive when it came to computing and the introduction of the Internet, so I just got lucky in that regard.

About the only thing today I'd want educationally would be to take some mathematics courses, specifically to refresh my memory of algebra. For example, until sometime in 2011, I had completely forgotten how to do long-hand division (computers spoil us, and I don't think that's a positive thing).

P.S. -- There's a hilarious twist to my lack of graduation: my sister (younger by 7 or 8 years) went to the same high school I did, and during (I think?) her freshman year, the 509J district made both of the aforementioned mandatory courses optional. I've always wanted to go back there and be like "I see you changed your mind... so can I have my diploma now? 21 years later?" ;-)
tepples
Posts: 22345
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: What does it mean to "hack" a game?

Post by tepples »

koitsu wrote:About the only thing today I'd want educationally would be to take some mathematics courses, specifically to refresh my memory of algebra. For example, until sometime in 2011, I had completely forgotten how to do long-hand division (computers spoil us, and I don't think that's a positive thing).
I still remember how to do long division, but only in binary, and only because of NESdev. See getSlope1 in the source code of Thwaite and RHDE.
psycopathicteen
Posts: 3001
Joined: Wed May 19, 2010 6:12 pm

Re: What does it mean to "hack" a game?

Post by psycopathicteen »

tokumaru wrote:
psycopathicteen wrote:Almost half a frame is spent doing what appears to be a metasprite drawing routine, and it just barely misses the end of the frame.
Strange how a number of SNES games suffer from this... I wonder if they all used some sort of badly unoptimized library.
That's what I've thought for a long time, but SGnG's sprite drawing routines are different from R-Type. Maybe I'll look through some more games, and see if any use the same code library.
tepples
Posts: 22345
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: What does it mean to "hack" a game?

Post by tepples »

And while you're at it, see why Ikari Warriors and other arcade ports by Micronics are so damn slow.
User avatar
Drew Sebastino
Formerly Espozo
Posts: 3496
Joined: Mon Sep 15, 2014 4:35 pm
Location: Richmond, Virginia

Re: What does it mean to "hack" a game?

Post by Drew Sebastino »

I'm just curious, psychopathicteen, but how are you finding all this stuff? Do you just look at ram like previously suggested? And what are people calling "long division?" just something like 3681/86529? Also, It seemed like sprite drawing seemed to be a very common problem with a lot of SNES games. Is it really that hard to write to hioam? I always thought that you would have a word that would be used for the X position for metasprites. In your metasprite creating routine, I imagine you would just see if the 9th bit of the X position word was filled with 1, and if so, just write 1 to hioam. I don't know specifically how you write it, but it really doesn't seem like it should take that much CPU time.

While we are talking about Super R-Type, I will say that, while mainly fixed, R-Type III actually does have some slowdown, but only really on the mode 7 parts in the first stage. Because it really isn't hard to do mode 7, (just write to 2 different registers for rotating and scalling) I'm guessing it slows down because of some sort of fancy collision detection going on while it's rotating.

Also, about Micronics, they seem very bad at programming. (An NES game running at 12 frames per second? What are they doing, realtime 3D?)

koitsu, I read what you wrote, and I am not lying when I say I found it genuinely interesting. I love how when you were in in school suspention, they didn't let you use the restroom. (What did they expect you to do, take a dump all over the floor?) Why did you have to stay with a Chinese family for 6 months? And the last question. How did you get to be on this website? Did you create it? Did you know all the other moderators before this website was created?
tepples
Posts: 22345
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: What does it mean to "hack" a game?

Post by tepples »

Espozo wrote:And what are people calling "long division?" just something like 3681/86529?
The procedure described in this article. Or as Capt. James T. Kirk might put it: KHAAAN!
Also, It seemed like sprite drawing seemed to be a very common problem with a lot of SNES games. Is it really that hard to write to hioam?
Bit packing is sort of tedious. And unless you use the technique I call "soft OBC1", you have to maintain two different index registers: one for normal OAM and one for high OAM. This means you don't have one left for the metasprite table.
Also, about Micronics, they seem very bad at programming. (An NES game running at 12 frames per second? What are they doing, realtime 3D?)
Walk animations in my own games Thwaite and RHDE run as slow as 5 frames per second, but I'm under the impression that the player doesn't really notice because the characters are so small. Besides, games for 8-bit home computers such as Rocky's Boots and Knight Lore and The Trap Door were running at 12 fps or slower, though a lot of this was because of software 2D compositing.
koitsu wrote:You aren't allowed to leave barring restroom usage
I love how when you were in in school suspention, they didn't let you use the restroom. (What did they expect you to do, take a dump all over the floor?)
"You aren't allowed to leave except for restroom usage".
Why did you have to stay with a Chinese family for 6 months? And the last question. How did you get to be on this website? Did you create it?
NESdev.com was created by Memblers and originally hosted on Parodius Networking, a hosting service for retro-gaming sites operated by koitsu. Parodius closed in mid-2012.
Sik
Posts: 1589
Joined: Thu Aug 12, 2010 3:43 am

Re: What does it mean to "hack" a game?

Post by Sik »

tepples wrote:
Sik wrote:
tepples wrote:Super Mario All-Stars also uses uncompressed tiles, apparently because it needs to stream a bunch in during vblank to simulate MMC3 bank switching.
good question, do all the graphics for a level (well, a section) fit in VRAM?
SMB2 and SMB3 use animated tiles (moving grass, spinning ? blocks, etc.). Even if they did all fit in VRAM, the game would need to rewrite the nametable to animate them, and that's probably more complex code-wise than just uploading new CHR.
Oh, I was talking about Final Fight when I made that question ( ・・) (but what you describe doesn't sound like every tile in those games...)
User avatar
Drew Sebastino
Formerly Espozo
Posts: 3496
Joined: Mon Sep 15, 2014 4:35 pm
Location: Richmond, Virginia

Re: What does it mean to "hack" a game?

Post by Drew Sebastino »

tepples wrote:"You aren't allowed to leave except for restroom usage".
Ops... :oops: (Some of the Nazis at my school don't let you go to the bathroom without having points off your next test.)
tepples wrote:Quote:Also, It seemed like sprite drawing seemed to be a very common problem with a lot of SNES games. Is it really that hard to write to hioam?Bit packing is sort of tedious. And unless you use the technique I call "soft OBC1", you have to maintain two different index registers: one for normal OAM and one for high OAM. This means you don't have one left for the metasprite table.
What is bit packing? Is it just operations like AND? If so, what's bad about it? Also, what is the "soft OBC1" technique?
tepples wrote:Walk animations in my own games Thwaite and RHDE run as slow as 5 frames per second
That's just walk animation though, right? I imagine the games still run at 60fps?
User avatar
bazz
Posts: 476
Joined: Fri Sep 02, 2011 8:34 pm
Contact:

Re: What does it mean to "hack" a game?

Post by bazz »

Y'all seem to really know what it takes to create an optimized SNES game.. (not sarcastic) Have you thought of collecting this knowledge into a basin / wiki to organize it. It can include guide lines and tips, no? It would beat the bits and pieces of gold being randomly scattered across these forum posts.

For instance, what is this Soft-OBC1 routine .. Maybe we can start a thing on http://wiki.superfamicom.org

OK, I'll take some liberty in my own hands right now.. and add a SNES Optimizations page..
User avatar
bazz
Posts: 476
Joined: Fri Sep 02, 2011 8:34 pm
Contact:

Re: What does it mean to "hack" a game?

Post by bazz »

I've added an Optimizations section at the end of http://wiki.superfamicom.org
Someone please populate the "Soft OBC-1" cause I have no idea what the frick yer talking about.
O! and if you have any other 1337 optimization knowledge stuffz, feel free to add your goods.
I'm talking about bit-stuffing, byte squeezing, frame-saving- miniscule-sized smallest-cycle efficient greatness :D theheheh

Or even advices page feel free to share your wisdom how to efficiently perform a certain crucial function of your game. I think a lot of people would love to hear about it.
psycopathicteen
Posts: 3001
Joined: Wed May 19, 2010 6:12 pm

Re: What does it mean to "hack" a game?

Post by psycopathicteen »

It's not just the hioam routine that is slow, it's the normal oam routine that is slow too. R-Type has this nonsensical code:

Code: Select all

lda SpriteY
xba
lda SpriteX
rep #$20
sta oam,x
sep #$20
Why didn't they just do this:

Code: Select all

lda SpriteX
sta oam,x
lda SpriteY
sta oam+1,x
Even then you can ask, after adding the metasprite displacement, camera adjustments and bounds checking, why did they store it back to a register just to pick it back up again, and move it to shadow oam?
User avatar
rainwarrior
Posts: 8062
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: What does it mean to "hack" a game?

Post by rainwarrior »

psycopathicteen wrote:Why didn't they...
The answer to this is usually because it worked well enough and they were under time and budget constraints to finish the game. In any shipped game there are a million little things that could have gone better. You simply can't make a game without making some mistakes or poor decisions along the way. There was probably a huge game-breaking bug that was more important to solve than saving a few cycles in a sprite routine that is already at least doing its job. Stuff like this persists in the shipping game because it never became a priority to fix.
User avatar
koitsu
Posts: 4203
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: What does it mean to "hack" a game?

Post by koitsu »

Espozo wrote:koitsu, I read what you wrote, and I am not lying when I say I found it genuinely interesting. I love how when you were in in school suspention, they didn't let you use the restroom. (What did they expect you to do, take a dump all over the floor?) Why did you have to stay with a Chinese family for 6 months? And the last question. How did you get to be on this website? Did you create it? Did you know all the other moderators before this website was created?
I should have put "barring restroom usage" in parenthesis or used a comma. Barring means to exclude, i.e. "we weren't allowed to leave the room, except for bathroom breaks". There are places, however, that don't allow people to use the restroom outside of specifically assigned windows/breaks (they're almost all in industry, not academia), and I find that whole concept offensive and just downright inhumane. My opinion is amplified by the fact that I have moderate-to-severe mixed-type IBS, so having access to a toilet is something I classify as a necessity, not a nicety.

I didn't "have" to stay with a Chinese family -- it was something I opted (chose) to do. It was an immersion experience, basically. My Mandarin had gotten to the point where I was quite fluent but wanted further practise, and I felt a good way to do that would be to live in an environment where by default nobody spoke English. I knew the family personally for 6 or 7 years prior to living with them. The other (minor) reason was location: I grew up in a rural area ~2 miles outside the city, while the family lived in the city and only a couple blocks from the high school (can't link to that now because they moved the high school back in early 2000). By the time I did the immersion, I was biking to school and then to OSU (to work) and then back home every day (vs. riding the bus to school, walking to OSU after school, then walking home every night), so it also doubled a more optimal place to be given where the high school and university were.

How I got to be on this website... well, I suppose running Parodius Networking for nearly 18 years or so has something to do with it. :-) The guy who originally created all of this (the nesdev website concept) is Memblers, and it all used to be hosted by me as http://nesdev.parodius.com/ until I closed down Parodius in August 2012. There is a #nesdev IRC channel on EFnet that (at the time) was used for a lot of collaboration and communication, so I hung out there. When Memblers wanted to do a website, I was happy to host the content (since we mainly focused on hosting things relating to emulation, video game development, or electronica). I stopped joining the IRC channel at some point (maybe 2007? I forget) and just hung out here.

Sort of along those same lines (and earlier parts of my posts here), there's an old thread talking about why I stopped doing nesdev, in case you wanted to read that. The very last bullet-point is the one that sticks, the rest are just things I experienced which greatly diminished my interest in doing nesdev. But like I said earlier, I still do some development through the form of romhacks (specifically hacking existing games so they could have their text translated from Japanese into English), so I guess I haven't completely left. I haven't done SNES stuff since the mid-to-late 90s, but most of it is still in my brain. I really do think the SNES is the easier/better console to develop for (from a programmer's perspective); once you deal with the NES and its nuances, almost all the things on the SNES are like a dream. I think the one that stands out the most (to me) is how on the SNES you can get the state of all the joypad buttons + directions in a single 16-bit read statement (lda $4218) while on the NES you can only read 1 bit (button/direction) at a time, thus have to do stuff like this. We're talking 5 or 6 cycles vs. 132+ on the NES.
tepples
Posts: 22345
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: What does it mean to "hack" a game?

Post by tepples »

Espozo wrote:What is bit packing? Is it just operations like AND?
It's shifts and ANDs and ORs and a separate index, which can get expensive with only two index registers.
If so, what's bad about it?
The index of a particular sprite in high OAM differs from the same sprite's index in low OAM. Specifically, they differ by a factor of sixteen. The 65816 has two index registers: X and Y. If X is used for the metasprite table and Y for the index into low OAM, you need to swap Y back and forth between pointing to low OAM and pointing to high OAM.
Also, what is the "soft OBC1" technique?
It's my term for storing sprites' high OAM bits (X bit 8 and size bit) in a separate 512-byte table that uses the same indices (4 bytes per sprite) as the 512-byte low OAM table. Then a separate optimized post-processing loop uses two index registers to combine the high bits of appropriate sprites into high OAM. See ppu_pack_oamhi in src/main.s in my Super NES project template.
Espozo wrote:
tepples wrote:Walk animations in my own games Thwaite and RHDE run as slow as 5 frames per second
That's just walk animation though, right? I imagine the games still run at 60fps?
It's a mix of 50-60 Hz (every frame), 25-30 Hz (every other frame), 10 Hz (every 5 or 6 frames), and 5 Hz (every 10 or 12 frames). The things that the player has the most direct control (mostly cursor movement) run every frame, but other tasks run on a "time-sharing" system where one is run each frame. For example, in Thwaite, only half the explosions are collision-tested and drawn each frame (hence 30 fps), and only half the smoke particles are drawn each frame. This alternation between even and odd also gives the explosions and smoke a constant semitransparent flicker. And in RHDE, only one unit moves each frame, and it takes 0.2 seconds to cycle through all units owned by both players (hence 5 fps). A player can give a movement command to his current unit at any time, with a visible effect (movement of the destination cursor) reflected immediately before the unit actually starts moving.
Post Reply