What calling conventions do you use?

[UnDisbeliever]

It seams like everyone is using different calling conventions in their SNESdev projects. The 65c816 instruction set and SNES memory map makes designing a best-fit calling convention difficult. I've resorted to having multiple calling conventions with the same project to efficiently handle all of my conflicting needs.

This is an important decision in any SNESdev project and can be difficult to change in large projects without modifying a lot of code and risking crashes. I have not seen many discussions or documents about SNES assembly code calling conventions.

I want to write a document about 65c816 calling conventions after the 2025 SNESdev game jam and I am curious as to what calling conventions you are using.


By calling convention I'm talking about:

• Register sizes (Accumulator and Index)
• DB Data Bank register values
• DP Direct Page register values
• What do you do for subroutines and macros with a different register-size, DB or DP requirements?
• What register do you typically use to pass entity/actor/object/sprite (whatever you call it) values on? X, Y or DP?
• What do you do when you run out of registers for function arguments?
• Are any registers or arguments callee saved?

There's probably a lot I'm missing here, feel free to extend the list.



I currently have 4 active SNES projects, all with slightly different calling conventions.


untech-engine:
Inconsistent call convention. Expected register sizes/values are documented in the subroutine/macro code. It's been a while since I delved into this engine, I honestly don't know if this is accurate:

Mainloop:

• Accumulator varies. Usually 16-bit, occasionally 8-bit
• Typically 16-bit Index
• DP = 0 or entity pointer
• DB = 0x7e to access a lot of Work-RAM. Long addressing or direct-page is used to access the MMIO/PPU registers.

Setup:

• 8 bit Accumulator, 16 bit Index
• DB = $7e, DP = 0
• DB = $80, DP = 0
• DB = $7e, DP = $2100 (subroutine name suffixed with "_dp2100")

VBlank:

• 16 bit Accumulator, 8 bit Index
• DB = $80, DP = 0
• DB = $80, DP = $4300 (subroutine name suffixed with "_dp4200")
• DB = $7e, DP = $2100 (subroutine name suffixed with "_dp2100")

Extra arguments are stored in zeropage temporary variables

Registers are not callee saved. Sometimes subroutines will keep X or Y unchanged for optimisation purposes. Those are tagged with `KEEP` in the function comments.


snes test roms:
Typically: 8 bit A, 16 bit Index, DB = 0x80, DP = 0
A few tests use: 8 bit A, 16 bit Index, DB = 0x7e, DP = 0


Unnamed SNES Game: documented

Contexts:

• Main Loop: 8 bit A, 8 bit Index, DB = 0x7e, DP = 0
• Entity Loop: 8 bit A, 8 bit Index, DB = 0x7e, DP = 0, Y = entityId
• MetaSprite rendering: 8 bit A, 16 bit Index, DB = 0x7e, DP = 0
• Setup (force-blank): 8 bit A, 16 bit Index, DB = 0x80, DP = 0
• VBlank: 8 bit A, 16 bit Index, DB = 0x80, DP = 0

The entityId is stored in Y, so I can use jump/call function tables with (addr,x) and access ROM data with the far,x addressing mode without clobbering entityId.

Y is callee saved if it is the entityId.

Extra arguments are stored in zeropage temporary variables.

Some functions do not honor my calling convention and the function name must be private (with a double-undersocre prefix) and/or suffixed appropriately (ie, "__clobbers_y", "__keep_y", "__mem16_idx8", etc).


Terrific Audio Driver ca65/64tass API: Also documented

• 65816 native mode
• Decimal mode off
• 8 bit Memory
• 16 bit Index, unless the subroutine is tagged "I unknown" (the queue and query functions)
• D Direct Page = 0
• DB Data Bank Register accesses low-RAM ($7e or $00..=$3f or $80..=$bf). With one exception - the LoadAudioData callback.

[Oziphantom]

calling conventions are for compilers. A huge reason people argued for compilers even when they sucked was because a programmer would tend to stick to register or value does X and thus waste registers and clocks, while a compiler would reuse things more. I.e the Compiler sucked but its was still better than 75% of the programmers on the team anyway.

You do what ever you need for the situation at hand. The 65816 is not designed for "common calling", "compiled languages" and doesn't have any convenience functionality to handle it, and at 3.58mhz you don't have the luxury of it either.
Lets take an Entity and then you want to have it build out its Meta Sprite. Well the Entity function uses DP, but when you get the Meta function you can't make DP the meta sprite def as you need the Entity data around to get the frame and facing etc so now to hold the Frame Desc info you need an Index, but then you need another Index into the sprite mirror, so the Entity code will have one convention, the Meta sprite another and the Sprite functions another, as you go "down the chain". Combined with the multi dimensional hydra that the 65816 address space becomes and the half RAM + register + data in one bank, code in another bank, full RAM access on long,x and only x and then 00,x and 0000,y asymmetry and "there are no gods here"

For rare call utility options you would probably "data post call".

If you function uses DP moving for speed, stack manipulations for building, then those are out for that function.

What a utility function needs will change for how your entity functions work. For example in most of my DMA functions I make them "don't care" and they will save and restore reg size. But for a common thing called in a loop, I will optimise them away and keep it "what the function" needs.

For Memory layout though,
Bank 00 common data and common functions.
Then have a Data bank that is assumed to be the DBR.

After that you are just drawing lines in sand that will make your life more awkward.

But a general rule, A is whatever it needs to be, and XY 16bit unless the clocks are being shaved.

[NovaSquirrel]

Both my main project and 256K game do things like this:

• Accumulator size: 16-bit almost always, though I'll switch to 8-bit for hardware register access, or parts of the program that primarily work with 8-bit data.
• Index size: 16-bit with few exceptions.
• X is "this" and is set to the first byte of the structure, allowing for "direct page, x" to access struct fields.
• I execute code out of $80-$BF and synchronize the data bank with the program bank unless I want to do a lot of accesses to a big ROM table or something in $7E.
• Direct page is $0000 nearly always, switching to $2100 or $4300 to speed up vblank code.
• Accumulator is used to pass function parameters if possible; if I need more, I will likely use Y and/or use direct page starting at zero and increasing for as many variables are needed.
• Subroutines preserve data bank, direct page, and register sizes, but direct page locations used as variables are usually caller-saved. I may put variables on the stack to avoid this.
• X is preserved, but Y may not be; if the purpose of the subroutine means that it would need to be preserved, that preservation usually goes in the subroutine.
• Carry used for true/false or success/fail output, with set=success and clear=fail.