What counts as a "big" optimization?

[psycopathicteen]

I've ran into this situation before when I point out examples of inefficient code in games, and I get responses like "but that only takes 5% of a frame, it's probably not worth the time optimizing." I know it's a small amount of time, but it's still enough to quickly add up, if you make several other optimizations of the same size. I think people spend too much time trying to find the "king optimization" that would completely obliterate slowdown, or measuring which optimization is bigger, instead of actually making the optimizations.

[tepples]

I've ran into this situation before when I point out examples of inefficient code in games, and I get responses like "but that only takes 5% of a frame, it's probably not worth the time optimizing." I know it's a small amount of time, but it's still enough to quickly add up, if you make several other optimizations of the same size. I think people spend too much time trying to find the "king optimization" that would completely obliterate slowdown, or measuring which optimization is bigger

Measuring which part of a program has more potential for speedup is called profiling. Starting from a good profile is essential in order to direct your optimization labor to the places where it'd be most fruitful. And sometimes you don't need a "king optimization" as much as a "cabinet of optimizations". This refers to whatever set of 1%, 2%, and 5% improvements can fit in a four-hour part-time day, including tests that the optimized code performs equivalently to the unoptimized code.

[rainwarrior]

I've ran into this situation before when I point out examples of inefficient code in games, and I get responses like "but that only takes 5% of a frame, it's probably not worth the time optimizing." I know it's a small amount of time, but it's still enough to quickly add up, if you make several other optimizations of the same size. I think people spend too much time trying to find the "king optimization" that would completely obliterate slowdown, or measuring which optimization is bigger, instead of actually making the optimizations.

If you're referring to me, I am usually merely trying to point out that it's perfectly normal to find thousands of instances of slightly sub-optimal code in a completed game. There is always something else to optimize, and the smaller the kind of optimization you start looking for, the more instances you can find. There is no such thing as perfect optimization, there is always a faster or smaller way to do it if you look long enough.

As always, unless I'm paying you, I don't have a say in what's worth your time. If you want to go and make a thousand micro-optimizations, nobody here is stopping you. If you want to prove that you can do this to save a game like Super R-Type from slowdown, go ahead and do it, but until you put up a patch I think you're just blowing smoke. The reason these optimizations weren't done in the first place are the same reason you haven't done them. It's a lot of time and effort, and eventually you need to either ship the game or call it off.

As tepples was getting at, the usual approach to optimization in the professional world goes something like:
1. Work on completing the game.
2. Notice a performance issue that you think needs to be fixed.
3. Profile the code to measure what is contributing the most to your performance problem.
4. Use the profiling and your knowledge of the game to identify the best optimization candidate (expected performance impact vs. expected work).
5. Optimize your best candidate.
6. If performance is not good enough, go back to 3 and select the next best candidate.
7. If performance is good enough, go back to 1.

Your approach seems to just skip the profiling step, and try to tackle optimization by just arbitrarily picking routines and optimizing them. This is a backwards and blind approach. If you're not profiling, you're not optimizing. You need to measure first, optimize, then measure again after to make sure you've made a difference. If you try to approach this without assessing the impact you're making, you will waste tremendous amounts of time optimizing things that just don't contribute enough to performance.

And yeah, sometimes it is worth trying to make a lot of micro-optimizations, but it is usually the very last resort. It is probably the most tedious, difficult, and time consuming way to try and optimize something.


One of the best things I ever read on this topic was Michael Abrash's "Black Book" of graphics programming.
It's available here for free: http://www.drdobbs.com/parallel/graphic ... /184404919

A lot of the book deals with now-outdated hardware stuff, but there is a whole lot of good stuff on optimization and effective programming. In particular I recommend reading the first chapter which presents a simple CRC program, and then proceeds to optimize it again and again, laying out the whole approach.
Chapter 1: http://twimgs.com/ddj/abrashblackbook/gpbb1.pdf

[psycopathicteen]

I do profile code, but I am not obsessive compulsive about counting cycles, nor do I try fixing every tiny optimization I can find. I use a lot of mental short cuts when I'm optimizing though. If I know something is repeated a lot, it's usually worth it.

[rainwarrior]

I am not obsessive compulsive about counting cycles

It is usually more effective to let a computer obsess about counting cycles. Profiling tools can do the measuring for you.

The other thing is, how "big" an optimization is not so much a function of how much code is changed as it is how often that code is run. Saving 5 cycles in a loop that runs 2000 times per frame is better than saving 500 cycles that runs once, and probably takes you less time to rewrite, too.

It's a lot easier to tell which is which if you have good profiling tools.

[tepples]

I wonder how the popular debugging emulators could be enhanced to gather profile data.

[rainwarrior]

A profiling emulator might be nice, but you can do a lot with what we've already got.

Run a trace for the frames you want to profile, and write a script to process the trace log. It could count cycles by execution address, or sort out function calls and create a hierarchical profile, etc. Maybe you could have it generate nicely formatted HTML output, like The Crinkler, which aside from being a compressing-linker is also a great executable size profiler.

Ready-made profiling tools are great, but I've also written all sorts of ad-hoc profiling tools as I needed them on most of the game projects I've been a part of. Often you need some really specific information that has stuff to do with your game data's organization that a generic CPU profiler wouldn't know about. A lot of times the easiest thing to do is dump logs and process/data-mine them after.

[Movax12]

NintendulatorDX can do profiling.

[mikejmoffitt]

A really cheap way I've profiled some Genesis code I've written is to change the background color for different segments of code. The game then paints the backdrop different tones based on what it's doing. If my physics routines paint the backdrop pink, and I see a ton of pink, well, then I know my physics might need some work.

[Movax12]

Yeah you can do this on NES too, commonly setting PPU MASK ($2001) bit0 to 1 for greyscale during the routine in question. You could also use emphasis.

[tokumaru]

Color emphasis is good because you get to use different tints to time different things, but grayscale is easier to see.

[rainwarrior]

Greyscale + tint at the same time works really well, especially since greyscale bumps black up to grey, which can then be tinted.

Unfortunately, FCEUX is not up to the task for displaying it. It's currently got a kludgey solution that only really permits one emphasis/greyscale change per frame. Works well in Nintendulator though.

[Dwedit]

There's also the more subtle option of disabling the leftmost 8 pixels...

[tokumaru]

There's also the more subtle option of disabling the leftmost 8 pixels...

Unless you have lots of blank areas in the background.

[psycopathicteen]

Optimizing is also much easier if you're optimizing your own game with the original code vs optimizing somebody else's finished game by tracing it in a debugger and ROM hacking it.

If I had the source code for SC4 and wanted to get rid of the slowdown during the mode 7 tunnel, I could quickly look up the code for mode 7 hdma, bone confetti, and whip physics.