Some Command-Line Disassembly Practices

[segaloco]

RAM or even just ROM. Mappers/bank switching can be seen as an overlay system. Overlays were super common in the mainframe and mini eras: https://en.m.wikipedia.org/wiki/Overlay_(programming)

[Pokun]

Those Wikipedia articles seems to distinguish between overlaying, bank switching and paging/swapping though.

• Overlaying seems to be for uploading code to RAM for execution (such is done from disk media).

• Bank switching seems to be when you switch out chunks of memory in the directly accessible address space (can be ROM or RAM).

• Paging or swapping seems to be about swapping out memory media like mass storage devices (such as an FDD) in order to extend memory in a non-contiguous way, though it's a bit vague and I don't fully understand the definition.

Bank switching seems to be similar to overlaying in that a portion of directly accessible memory is replaced by another, but bank switching seems to mean connecting another piece of memory (such as when using memory mapping hardware) to the address space while overlaying is mainly done in software by copying the data over and therefore requires RAM (or perhaps PROM that is programmable at runtime).

I've seen "paging" been used as a synonym of "bank switching" in some homebrew communities, but I think they probably misunderstood the meaning of paging and Wikipedia specifically says the two terms shouldn't be confused with each other.

[Fiskbit]

Paging and bank switching do not seem particularly different to me, and I'm a bit surprised that Wikipedia's distinction is that paging allows extra memory to go out to a storage device. That suggests to me that paging is just bank switching where the memory is RAM and the system has a separate storage device, which isn't really a property of the switching mechanism itself. There are many things that make x86 paging distinct from the bank switching we do (usually storing the paging information in RAM instead of special registers that can be directly accessed, page access properties, having just two address spaces (linear vs physical) rather than separate address spaces for different types of memory, etc), but the ability to move RAM data out to disk to use that RAM for something else is not something I would have considered to be part of the mechanism itself.

I suppose it could come down to having page access properties; the ability to mark a page as not present is what enables the system to bring in the memory that should be there without cooperation from user software. That's something you don't have with bank switching as I've ever seen it and feels like a satisfying definition that meets Wikipedia's criteria.

[segaloco]

Well if you want to get really technical paging doesn't necessarily imply swapping and vice versa. You do them together in MMUs running timesharing systems, but I think of paging strictly in terms of the memory map is broken into fixed sized pages and then manipulated by a memory unit as such. That memory unit then may be doing things like swapping page contents to secondary memory and back, but swapping predates paged VM systems, the UNIX kernel on the PDP-11 also did whole process swapping, but it was segmented memory rather than paged. The distinction gets fuzzy though because what is fixed segments but 6-8 very large pages. Still you handle page faults differently than on say a VAX or other such true VM systems.

Overlaying goes back to the earliest IBM mainframes as a strategy not unlike VM to have processes and programs larger than the addressable memory space, really the main distinction is that the overlay memory range is fixed rather than pages being dynamically managed at runtime. This means non-PIC and comes out pretty much just like bank swapping, just done in memory with a memory copy rather than in hardware with some bank selection latch. Like banking too the OS knew very little about the overlay, if there even was an OS, typically the program itself would come to an overlay swap and you'd toss on the relevant magtape, paper tape, or if you were lucky, a platter was already mounted with the various overlays.

[creaothceann]

Paging and bank switching do not seem particularly different to me, and I'm a bit surprised that Wikipedia's distinction is that paging allows extra memory to go out to a storage device. That suggests to me that paging is just bank switching where the memory is RAM and the system has a separate storage device, which isn't really a property of the switching mechanism itself.

Bank switching is (afaik) just the mapping hardware adding more bits to a CPU address. It results in a rigid layout where only some parts of the address space can be changed, but it's essentially instant.

Paging uses virtual memory. Any logical page can be loaded into any physical RAM page, or swapped out to a page file. It's more flexible (offering a flat address space), but much more expensive: every read/write access has to be checked; there has to be a page table and a translation lookaside buffer; accessing swapped-out pages generates page faults - exceptions (slow) that require a kernel context switch (also slow); a rarely-used page has to be written to storage and the requested page has to be loaded into RAM.

[Fiskbit]

Many of the properties you're discussing are not core to paging itself, but to implementations of paging. Paging could have used one register for every page table entry (PTE) and had instant lookup time just like bank switching does, but this isn't practical because the tiny size of one page compared to the size of the address space results in an extremely large number of pages, necessitating RAM. This is not true for the number of banks in bank switching. It is because PTEs are stored in RAM rather than registers that lookups are slow and caching of full translations in the TLB and of individual entries is a necessity. There is also at least one example of paging using registers instead of RAM for PTEs: In x86's protected mode with PAE enabled, the 4 page directory pointer table entries are loaded into 4 registers so that despite having 3-level paging, only 2 entries have to be looked up in RAM, and this is made possible by the number of entries at this level being very small.

Both paging and banking are about remapping address bits, but for most of paging's life, the reality was that the CPU address space was large compared to the amount of memory available. When that wasn't true in the 32-bit era, we got things like x86's 36-bit PSE and PAE features that enabled mapping larger amounts of physical memory into a smaller 32-bit address space, which looks a lot more like traditional bank switching.

What apparently makes paging unique is that it has these permission bits that allow a supervisor to manage the address space for its user(s) without those users having to do anything, which is what enables page faults that allow for easy, seamless paging of memory from disk.

[Pokun]

I see, so bank switching is only about remapping address lines while paging is a more sophisticated memory extension system that possibly employs bank switching as a part of it in the background but is abstracted away from the user.

[segaloco]

The permission stuff is very important to optimal OS execution. The less the OS has to do to gate virtual and physical memory addresses between user and supervisor space (and co-resident processes) the better. One of the key goals of any good kernel is process isolation. An MMU with strong security features is indispensable. For simple situations like overlays and banking in a real-time application, there's a lot less concern with whether the approved user can do whatever they want in memory or not, you're not trying to keep from wrecking other processes.

[Oziphantom]

I feel that Overlay is something a program does to it self. In that a way for me to make a program larger than memory is to develop and plan overlays. I don't expect an OS to handle overlays for me and on all the platforms I've used overlays it has been something we set up in the compiler toolchain to generate and make for us. I also feel that Overlay is a code only concept and then data/files is not overlaid rather replaced or merged.

While Banking is a hardware way to expand the memory address and is something offered by the hardware on the machine to which it is something the program controls or it is used by the OS to facilitate expanded memory. I.e the NES is program controlled, a bare metal C128 is program controlled while using the 128 OS banking is OS controlled.

While Paging is either a hardware or software feature of the machine/OS and is performed by the host transparency. I don't ever get into the details of paging in my code, I just request memory and let the machine slide the page into view for me without me knowing the page wasn't there. However if I'm bare metal then I would see Paging as Banking for my direct control.

[segaloco]

Yeah once you throw an OS in the mix all bets are off, you're now (mostly) writing for the virtual machine the OS userland presents. Unless you're messing around with mmap directly you're not liable to run into many scenarios where knowing your memory hardware is critical information.

[segaloco]

So this isn't more FDS stuff, I'll follow up on more FDS-specific analysis stuff some other time, but just getting into a nice groove with some of my toolchain lately and figured this could be helpful for others. So I mostly use vi(1) for my day to day disassembly stuff, with VSCode seeing occasional use when I'm on a computer that happens to have it (I do like the graphical search/replace stuff for certain operations).

Anywho, working in vi(1) is a bit tricky if you're not used to modal editors. Essentially vi(1) operates in two modes, command mode and insert mode. When launched, you are in command mode. A number of commands (e.g. a - append, i - insert, c - change) place you in edit mode, in which the text you type is entered in the current editor buffer until you exit input mode (usually ESC). A nice thing about vi(1) is that it has shell escapes in which you can run shell commands and view their output on your terminal (basically hop out to a shell for a sec, see what you get back). Even better, you can run shell commands in which a selection of text in the current file is provided as standard input and the resulting standard output overwrites that data. In other words, you can select a series of text and, for instance, direct it to sed(1) to leverage an existing set of editor scripts (although bad example, if you can do it in sed(1), you can do it in command mode in vi(1)).

A useful pattern I've gotten into lately is using this mechanism to interpret garbage disassembly into data bytes rapidly while analyzing files.

For instance, say I get a disassembler output like:

Code: Select all

LA012:
	rti
	rti
	ora	($01, x)
	ora	($02, x)
	jmp	$1212
	rts
	rti
	
LA01D:
	lda	LA012, x
	sta	$00
	rts

Obviously the data between LA012 and LA01D is garbage, you could decide to go use od(1) to analyze what hex data you're looking at there, and then edit that hex data in over this, but this is a bit of a pain. I wrote a little script, dump65[1], that takes a starting and ending offset, a datatype to export, and a file to read data from, and it exports that chunk of data as ca65-compatible data directives. In this case, I would want to use this tool to generate the data for LA012.

Well, the process in vi(1) with this script available would basically be:

1. Use the m (mark) command to mark the first and last opcode of the section to replace.
2. Identify the hex origin of the binary file involved, this will need to be subtracted from the labels/output addresses to figure out offsets relative to the start of the binary file (as opposed to offsets relative to the loading segment).
3. Subtracting this origin from the label before and after the data chunk gives you the offsets. So presuming the above is from a binary segment named prg18.bin that starts at 0xA000, the coordinate options to the dump65 tool are "-s 0x12 -e 0x1D".
4. Finally, this is all the information needed to do the replacement. At this point you would run a shell escape (!) to replace the input with the results from dump65.

The result of all of this is running the following:

Code: Select all

:'a,'b!dump65 -s 0x12 -e 0x1D prg18.bin

And the result

Code: Select all

LA012:
	.byte	$40
	.byte	$40
	.byte	$01
	.byte	$01
	.byte	$01
	.byte	$02
	.byte	$4C
	.byte	$12
	.byte	$12
	.byte	$60
	.byte	$40

LA01D:
	lda	LA012, x
	sta	$00
	rts

And I didn't once need to leave vi(1) for any of this. Working out this workflow for data chunks has significantly sped up my process without having to rerun the disassembler multiple times with config files or otherwise regenerate a bunch of stuff all the time. If one really wanted to get fancy, one could write a script that takes a chunk bounded by labels on the stdin along with an origin for the file and then automatically based on file names and such does the rest of the extraction, that way it becomes simply pass a range to the tool, it does all the rest of the figuring and prints back the decoded stuff. This could then be hooked up to some sort of macro perhaps, the possibilities are quite broad when tying together pipeable tools.

Anywho, just wanted to share this as it is a good example of how to very efficiently step through some of the garbage a disassembler might spit back at you. Speaking of such garbage, I've got an active issue here with the cc65 folks to address some relative symbol generation stuff that currently prevents da65 output from immediately being ca65 input: https://github.com/cc65/cc65/issues/2717 I'm hoping that getting fixed up will make automated analysis tools that do this sort of stuff more attainable. A lovely follow on tool would be one that accumulates labels and determines if they're contextually code or data (i.e. if they're control flow targets or indexed memory) and tries to automatically do this on data bits. Of course that would all be based on assumptions, but could make the bootstrapping of a disassembly project easier by offering some predictions on what might be data.

Anywho, I hope this highlights some of what simple, focused tools make possible. As time goes on I feel so much more productive than I ever felt using bulky graphical tools like IDA Pro and Ghidra. Sure they've got some nice features, but you're also stuck in their ecosystem until you export your assembly file, using a bunch of individual UNIX tools makes it a lot easier to have full control over every step of the analytical process, even if I have to fill in some of the blanks myself. Plus, this is applicable to anything, some esoteric ISA that a graphical tool might not include is a pain to get integrated there, but since my process only depends on a bare bones CLI disassembler, my barrier to entry on any given platform is much lower. Can IDA Pro or Ghidra handle MAC-8 analysis? No way, but my toolchain literally just needs a disassembler and I'm good to go.

[1] - https://gitlab.com/segaloco/misc/-/blob ... pts/dump65

[segaloco]

Here's a script I put together for more rapidly editing OAM references into symbols compatible with my disassembly standards:

Code: Select all

#!/bin/sh
#
# oamsub - generate ed(1) substitute commands for OAM references
#
# Copyright 2025 Matthew Gilmore
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# 
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors
# may be used to endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

OAM_BUFFER_SYM="oam_buffer"
OBJ_SIZE_VAR="OBJ_SIZE"
VAR_POS_V="OBJ_POS_V"
VAR_CHR_NO="OBJ_CHR_NO"
VAR_ATTR="OBJ_ATTR"
VAR_POS_H="OBJ_POS_H"
PAGEBASE_HEX=200
OBJ_COUNT=64

STR_SUB="g/\\\$%.4X/s//%s+(%s*%d)+%s/g\n"
PAGEBASE_DEC=`echo "ibase=16;$PAGEBASE_HEX" | bc`
OBJ_ID=0
for i in `echo "for (i=0;i<$OBJ_COUNT;i++)i" | bc`
do
        ADDR_POS_V=`echo "$PAGEBASE_DEC+($i*4)+0" | bc`
        ADDR_CHR_NO=`echo "$PAGEBASE_DEC+($i*4)+1" | bc`
        ADDR_ATTR=`echo "$PAGEBASE_DEC+($i*4)+2" | bc`
        ADDR_POS_H=`echo "$PAGEBASE_DEC+($i*4)+3" | bc`
        printf $STR_SUB "$ADDR_POS_V"  "$OAM_BUFFER_SYM" "$OBJ_SIZE_VAR" $i "$VAR_POS_V"
        printf $STR_SUB "$ADDR_CHR_NO" "$OAM_BUFFER_SYM" "$OBJ_SIZE_VAR" $i "$VAR_CHR_NO"
        printf $STR_SUB "$ADDR_ATTR"   "$OAM_BUFFER_SYM" "$OBJ_SIZE_VAR" $i "$VAR_ATTR"
        printf $STR_SUB "$ADDR_POS_H"  "$OAM_BUFFER_SYM" "$OBJ_SIZE_VAR" $i "$VAR_POS_H"
        OBJ_ID=`expr $OBJ_ID + 1`
done

printf "w\n"
printf "q\n"

This is a script to generate a set of substitution commands which can then be used with ed(1) to replace OAM memory references in da65 disassembly output with symbolic representations. By default, this generates the references with my own symbol names as I use in my various disassemblies. The output looks something like:

Code: Select all

g/\$0200/s//oam_buffer+(OBJ_SIZE*0)+OBJ_POS_V/g
g/\$0201/s//oam_buffer+(OBJ_SIZE*0)+OBJ_CHR_NO/g
g/\$0202/s//oam_buffer+(OBJ_SIZE*0)+OBJ_ATTR/g
g/\$0203/s//oam_buffer+(OBJ_SIZE*0)+OBJ_POS_H/g
g/\$0204/s//oam_buffer+(OBJ_SIZE*1)+OBJ_POS_V/g
g/\$0205/s//oam_buffer+(OBJ_SIZE*1)+OBJ_CHR_NO/g
g/\$0206/s//oam_buffer+(OBJ_SIZE*1)+OBJ_ATTR/g
g/\$0207/s//oam_buffer+(OBJ_SIZE*1)+OBJ_POS_H/g
g/\$0208/s//oam_buffer+(OBJ_SIZE*2)+OBJ_POS_V/g
g/\$0209/s//oam_buffer+(OBJ_SIZE*2)+OBJ_CHR_NO/g
g/\$020A/s//oam_buffer+(OBJ_SIZE*2)+OBJ_ATTR/g
g/\$020B/s//oam_buffer+(OBJ_SIZE*2)+OBJ_POS_H/g
.
.
.
w
q

The end result is the output of this script can be piped into ed(1) with any file and it should result in the raw addresses $0200, $0201, etc. being replaced with symbolic constants like "oam_buffer+(OBJ_SIZE*0)+OBJ_POS_V", "oam_buffer+(OBJ_SIZE*0)+OBJ_CHR_NO", etc.

All of the symbolic names as well as the base pointer for the OAM buffer are variables at the start of the script that can be adjusted for other OAM buffer addresses and symbolic names for the various parts of an OAM entry. For now you have to edit them in the file, in the future I may make certain values into CLI arguments instead. To use this script succinctly on a file, you can do the following:

Code: Select all

oamsub | ed <file>

And the generated substitute commands will be executed by ed(1) on the provided <file>. Typical ed(1) precautions apply, the script ends with w then q commands, so there is no confirmation. Don't point this at a file you expect to revert unless you make a backup or are using source control first.

Like many of my scripts this will eventually find its way into one of my tool repositories to act as a post-processing step on my automated disassembly tools.