ld65 complains if I try to reuse zp addresses in different source files

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

spaceharrier
Posts: 40
Joined: Wed Jan 19, 2022 9:52 am

ld65 complains if I try to reuse zp addresses in different source files

Post by spaceharrier »

This is most likely a PEBCAK issue (problem exists between keyboard and chair)

I have separate source files, and am attempting to reuse labels in zero page (since the operations happening don't overlap in zp usage)

So, for example, in one source file, I might declare:

.segment "ZEROPAGE": zeropage
.org $0000
foo: .res 1
bar: .res 1

and in the second source file I might declare

.segment "ZEROPAGE": zeropage
.org $0000
boo: .res 1
far: .res 1

when I link my object files, ld65 does not like this.

Is there some special song and dance I can do to resolve this?
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

spaceharrier wrote: Mon Feb 14, 2022 8:02 pmwhen I link my object files, ld65 does not like this.
"does not like this" is very vague. Can you tell us what the actual error message is?
Is there some special song and dance I can do to resolve this?
When you use .ORG in ca65, that changes the value of the PC to the specified value, but anything you put after that will still consume space in the segment you're in (i.e. even though the addresses are the same, the assembler will consume space in the segment for each variable you define). If you want variables to overlap, you need to do the same thing we do for PRG-ROM bankswitching (which is essentially overlapping ROM pages), which is defining a separate segment for each "layer".

That can really get out of hand with variables though, as it's not feasible to have a separate segment for each function's local variables, for example. What I do in this case is create a really big segment that can accommodate all the overlapping variables I'll ever need, and use that instead of the normal RAM segments.

Also, don't ever forget to put the assembler back in relocatable code mode after using .ORG.

EDIT: I use this special segment for ALL of my variables, because I want full control over where they'll be in memory (to avoid page crossing, to overlap more than just local memory for functions, etc.), but if you want to keep using the normal RAM segments in relocatable mode, you can, but it may be a good idea to exclude from them the range(s) where you want to declare overlapping stuff, so you don't accidentally declare anything permanent in those areas.
spaceharrier
Posts: 40
Joined: Wed Jan 19, 2022 9:52 am

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by spaceharrier »

tokumaru wrote: Mon Feb 14, 2022 8:20 pm "does not like this" is very vague. Can you tell us what the actual error message is?
An example of output error ld65 gives me is as follows:

Code: Select all

ld65: Warning: Address size mismatch for 'readjoy_safe': Exported from joy.o, source\joy.s:79 as 'zeropage', import in start.o, source/joy.inc:12 as 'absolute'
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

spaceharrier wrote: Mon Feb 14, 2022 8:34 pm

Code: Select all

ld65: Warning: Address size mismatch for 'readjoy_safe': Exported from joy.o, source\joy.s:79 as 'zeropage', import in start.o, source/joy.inc:12 as 'absolute'
Are you exporting "readjoy_safe" as ZP in one file and importing it as ABS in another? This is what the message says, and sounds easy enough to fix (just make sure the export and the import are the same size), but I don't see how that relates to the problem described in your first post...
spaceharrier
Posts: 40
Joined: Wed Jan 19, 2022 9:52 am

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by spaceharrier »

Those symbols are not at all related to anything in the zero page and I am not importing them as ZP symbols, however I was not aware that you had to put the assembler back into relocatable mode after using .org, and I would guess some cascaded failure is related to my failed use of .org

putting it back in relocatable mode after using .org seems to have worked, and the debugger is showing ZP address reuse. Thanks.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

spaceharrier wrote: Mon Feb 14, 2022 9:01 pmputting it back in relocatable mode after using .org seems to have worked, and the debugger is showing ZP address reuse. Thanks.
You will still run out of ZP space earlier than expected, I think. Like I said before, even though it looks like the assembler is reusing addresses, it's still consuming space in the ZP segment for each declared variable, so after you declare 256 bytes worth of variables there, regardless of their addresses, the ZP segment will be full. Give this a try: declare a bunch of dummy variables the way you did in the first post and see if the ZP segment overflows once you have over 256 bytes worth of variables, even if they don't get anywhere near address $FF.
spaceharrier
Posts: 40
Joined: Wed Jan 19, 2022 9:52 am

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by spaceharrier »

Yeah, the linker does complain about overflowing in that instance. I'll try your advice of adding extra segments in my linker config file. SInce the granularity is on a file-by-file basis for how I'm organizing my code, it shouldn't be too terribly messy for things.

edit: I also noticed that if I increase the size of my zp segment in my config file, it gets rid of the overflow warning and I still see address reuse as intended. The only drawback to that is that debugging labels are not produced in overlapped areas.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

spaceharrier wrote: Tue Feb 15, 2022 9:51 amI also noticed that if I increase the size of my zp segment in my config file, it gets rid of the overflow warning and I still see address reuse as intended.
Yes, that works, but if you use the same segment for absolute variables (.ORG) and relocatable ones, the absolute ones will create gaps in the memory that the relocatable mode will not be able to use. If you plan on declaring variables using both absolute and relocatable modes, it's better to use separate segments for each mode.
The only drawback to that is that debugging labels are not produced in overlapped areas.
I think that the debugging labels are in fact exported, but when emulators find two symbols pointing to the same address, they can't tell which one is correct for each particular instruction, so it has to pick only one name and stick to it...

EDIT: Here's what happens if you mix the two modes:

Code: Select all

.segment "ZEROPAGE"
foo: .res 2 ;foo = $00, consumes $00 and $01
.org $0080
bar: .res 2 ;bar = $80, consumes $02 and $03
.org $0080
baz: .res 1 ;baz = $80, consumes $04
.reloc
qux: .res 2 ;qux = $05, consumes $05 and $06
Addresses $02, $03 and $04 became unusable for relocatable variables.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by rainwarrior »

spaceharrier wrote: Mon Feb 14, 2022 8:02 pm I have separate source files, and am attempting to reuse labels in zero page (since the operations happening don't overlap in zp usage)

So, for example, in one source file, I might declare:

.segment "ZEROPAGE": zeropage
.org $0000
foo: .res 1
bar: .res 1

and in the second source file I might declare

.segment "ZEROPAGE": zeropage
.org $0000
boo: .res 1
far: .res 1
First, I advise against using org at all in ca65. It's the linker's (ld65) job to place things at specific addresses, and org overrides this in a way that is very seldom helpful. The primary reason org exists in ca65 is to assist compatibility when porting old code from other assemblers that did not have a linker.

Second, if you want to use a label from one module in another, use the import/export/global and importzp/exportzp/globalzp directives. Once exported, you can import that label in as many other modules as you like.

Code: Select all

; file 1
.segment "ZEROPAGE": zeropage
foo: .res 1
bar: .res 1
.exportzp foo
.exportzp bar
; exportzp makes zeropage labels importable in other files

; file 2
.importzp foo
.importzp bar
; file 2 now has access to foo and bar which were exported from file 1
Reference: Not sure if the reordered bar/foo in your example was intentional, but if you do want to change the name of label you can make aliases.

Code: Select all

; file 2
.importzp foo
.importzp bar
bar2 = foo
foo2 = bar
; now bar2 and foo2 exist as alternate names, visible only within file 2
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

rainwarrior wrote: Thu Feb 17, 2022 10:29 pmFirst, I advise against using org at all in ca65.
If you don't mind me asking, what is your proposed solution for declaring overlapping variables (such as for local variables or for modules that never run at the same time) that doesn't use ORG and doesn't require the creation of a bazillion different segments?
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by rainwarrior »

tokumaru wrote: Thu Feb 17, 2022 10:40 pm
rainwarrior wrote: Thu Feb 17, 2022 10:29 pmFirst, I advise against using org at all in ca65.
If you don't mind me asking, what is your proposed solution for declaring overlapping variables (such as for local variables or for modules that never run at the same time) that doesn't use ORG and doesn't require the creation of a bazillion different segments?
Honestly, if I had that problem I'd probably want to solve it with segments. I don't really understand the opposition to using segments for this? Segments are easy to create, and carry a lot of relevant semantic info and link-time protection.

However, okay you don't like segments. Aliases also work fine for this. I just explained how to make aliases in my post above. I guess block out the relevant area in one place, then break it up with aliases everywhere else? This has less semantic checking than using segments, but still more than org.

Code: Select all

; file 1
block: .res 16
.export block
; file 2
.import block
var = block
var2 = var+2 ; .res substitute with previous variable + size
; file 3
.import block
var = block+0
var2 = block+7 ; .res substitute as absolute offset into shared block
I don't think I'd consider using org for it. Yes, it's one way to force things to be at a specific address, but it also intentionally disables any possible linker analysis/protection. If it works for you, go ahead, but I still don't recommend it.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

rainwarrior wrote: Thu Feb 17, 2022 11:24 pmHonestly, if I had that problem I'd probably want to solve it with segments. I don't really understand the opposition to using segments for this? Segments are easy to create, and carry a lot of relevant semantic info and link-time protection.
I don't have anything against segments, but if you have an area in ZP that you use for local variables, function arguments and the like, it's just not feasible to create one segment for every single function in your program... Not only because of the pain that would be to edit the cfg file all the time, but mainly because of the limitation of 254 different segments per object file. If you're already using lots of segments for ROM banks and such, it'd not be incredibly hard to reach 254 if every function needed its own segment.
Aliases also work fine for this.
But then you lose all the versatility of using .res for reserving space, and instead have to do it the error-prone way of adding a number to the previous variable (var2 = var+2). Want to reorganize variables? Then you need to change all the references to the preceding variables as well, not to mention that the number you add on each line is the size of the variable in the *previous* line (the last variable doesn't even have its size explicitly stated unless you add a dummy variable after it just for that). This is way messier and error-prone than a simple .org based solution, and I'm really surprised that you find that the cleaner alternative.
If it works for you, go ahead, but I still don't recommend it.
I like having control over where my variables will end up, not gonna lie. I'm all for letting the assembler do it's thing with ROM, but with RAM I may want to explicitly prevent large arrays from crossing pages, or I may need specific alignments in order to optimize the indexing, and so on. I don't normally use multiple object files either (all the exporting and importing is a pain in the ass IMO), but even when doing that I believe I'd prefer to have all global variables assembled in a single module where they could be carefully organized and then exported to the other modules. Segments may work well for overlapping global variables, which aren't so numerous, but for local variables .org is a much simpler solution.

But ca65 is versatile enough to allow vastly different approaches to everything, and everyone is free to do what they think works best for them and is easier to manage.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by rainwarrior »

tokumaru wrote: Fri Feb 18, 2022 1:27 amit's just not feasible to create one segment for every single function in your program
I can't imagine wanting to do that? I don't really understand the need for this organizational problem in the first place.

At the function level, I usually just use a handful of static temporaries. Occasionally I might use a function-scoped alias if the function is more complicated than average.
tokumaru wrote: Fri Feb 18, 2022 1:27 amThis is way messier and error-prone than a simple .org based solution, and I'm really surprised that you find that the cleaner alternative.
I have to assume you have fundamentally different programming practices than I do. At the function level, I might alias temporaries, but those temporaries already have defined sizes, so there's not really a need for a mechanism like .res to keep track of variable sizes, I suppose.

I do use the aliasing method I suggested for module-level organization sometimes. For instance, in huffmunch it abstracts the library's memory needs away from the rest of a program using it.

Yes, this would be annoying to reorder them a hundred times... but also I don't have to do that. I only had to order these variables once in the 3 years since i wrote it, and there's only six of them.

For this, a segment doesn't seem to be an appropriate solution, as it would make it tricker to use as a drop-in library. Similarly, .org would also make it hard for a user to integrate it. This way all the user needs to do is provide a 9 byte block that can be overwritten during a decompression event.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

rainwarrior wrote: Fri Feb 18, 2022 11:16 amI can't imagine wanting to do that? I don't really understand the need for this organizational problem in the first place.
The idea is to keep local memory allocation versatile.
I have to assume you have fundamentally different programming practices than I do.
We probably do.
At the function level, I might alias temporaries, but those temporaries already have defined sizes, so there's not really a need for a mechanism like .res to keep track of variable sizes, I suppose.
But what about static memory allocation for call trees? Since all functions have to draw from In the same pool of fixed-size temporaries, you need a good amount of bytes, words and whatever other data sizes you use in order to serve them all without having to clumsily repurpose pairs of bytes as words or having arrays start or end in the middle of a word. It's not very versatile.

The main problem of non-trivial call trees is deciding where one function's variables end and another's begin. It's easy at first, but once you need to add more variables, you have to hunt down all nested function calls to be sure you're not claiming memory that's already in use. And if you're using fixed-size temporaries, this means that there's no clear separation between each function's variables, they're all intertwined, making collisions much harder to catch.

You may not yet have run into a case where this became a problem for you personally, but you can't confidently say that this approach is versatile enough to handle anything you throw at it.

To handle the matter of call trees, in each function I reserve a block of memory large enough before its locals so they go after all the locals of the functions it calls. Then I write one assert (conveniently hidden behind a macro) for each function called to guarantee that the variables are indeed starting after those used in nested calls. This way I get a warning if a change results in a collision and I can go straight to where it happened and fix it. This is only possible because the local variables are all sequential and there are clear boundaries between each one's memory.
Yes, this would be annoying to reorder them a hundred times... but also I don't have to do that. I only had to order these variables once in the 3 years since i wrote it, and there's only six of them.
I guess it's like you said... If that works for you, that's cool, but I wouldn't feel comfortable doing this for locals everywhere considering all the drawbacks I mentioned before.
Similarly, .org would also make it hard for a user to integrate it. This way all the user needs to do is provide a 9 byte block that can be overwritten during a decompression event.
Indeed, when you're making something for others to use you shouldn't be relying on your own development ecosystem, you should keep things as generic as possible. In this case this is indeed what I'd do as well.

But when you're writing a game with hundreds of functions that can change at any time, relying on manually created aliases without any sort of added protection for collisions is way too error-prone for me.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: ld65 complains if I try to reuse zp addresses in different source files

Post by tokumaru »

My reason for using .org (along with supporting macros) is precisely to increase automation and safety for an aspect that the assembler doesn't have native provisions for (static memory allocation for call trees). I would not promote something that sounds increase the chance for errors.
Post Reply