| 9120e874 Andy McCall |
1 |
# code_map: where the 65C816 operand widths are still a guess |
| 2 |
|
| 3 |
Status: **known limitation, deliberately not "fixed"**. This note exists so the |
| 4 |
next person to look at it does not mistake it for an oversight, and so that the |
| 5 |
tempting fix is on record as the wrong one. |
| 6 |
|
| 7 |
## The gap |
| 8 |
|
| 9 |
`cm_propagate()` in `src/code_map.c` predicts the effective processor status of |
| 10 |
the next instruction from the current opcode. Exactly five instructions change |
| 11 |
the 65C816 register widths: |
| 12 |
|
| 13 |
|
| 14 |
|
| 15 |
| `REP #imm` | its status bits, exactly | the bits to clear are in the operand | |
| 16 |
| `SEP #imm` | its status bits, exactly | the bits to set are in the operand | |
| 17 |
| `XCE` | only if C and E are right | swaps carry and the emulation flag | |
| 18 |
| `PLP` | **no** | restores a status byte pulled off the stack | |
| 19 |
| `RTI` | **no** | restores the status the interrupt pushed | |
| 20 |
|
| 21 |
`XCE` is exact only when **both** of its inputs are right, and neither is |
| 22 |
guaranteed. |
| 23 |
|
| 24 |
*The carry* is not tracked through any data-dependent change. `CLC` and `SEC` |
| 25 |
are modelled, and `REP`/`SEP`/`XCE` naturally update it as part of what they do, |
| 26 |
but `ADC`, `SBC`, the compares, and the shifts and rotates all move the carry |
| 27 |
without the estimate noticing. So an `LSR A` that really clears the carry, |
| 28 |
followed by `XCE`, makes the model predict a switch into emulation mode that |
| 29 |
does not happen, forcing the widths to 8-bit for every following line that has |
| 30 |
no anchor of its own. That needs no `PLP` or `RTI` at all; |
| 31 |
`tests/test_code_map.c` pins it under `KNOWN LIMITATION`. |
| 32 |
|
| 33 |
*The emulation flag* is not recorded by anchors at all — they store the |
| 34 |
effective status byte but **not** `E`. The running `E` is seeded from the live |
| 35 |
`regs.e`, so landing on an anchor re-syncs that line's widths but not `E`, and |
| 36 |
propagating past the anchor folds the stale `E` back in |
| 37 |
(`if (e) status |= INDEX|MEMORY`) — which can mis-size the next unanchored line, |
| 38 |
wherever the stale `E` and the real one imply different widths. |
| 39 |
|
| 40 |
A bad `E` is not unrecoverable, though. `XCE` writes `E` from the estimate's |
| 41 |
carry, which is exactly what the real instruction does, so an `XCE` decoded with |
| 42 |
an *accurate* carry — one taken from an anchor, or set explicitly by a `CLC`/ |
| 43 |
`SEC` the estimate did track — puts `E` back in step. What is not promised is |
| 44 |
that such a sequence appears: an ordinary anchor does not repair `E`, so nothing |
| 45 |
guarantees recovery. |
| 46 |
|
| 47 |
Most X16 software sets `E` once during startup and leaves it alone, which is why |
| 48 |
this is latent rather than routine. What defeats that reassurance is not a |
| 49 |
deliberate mode switch by the guest: it is enough that the walk *crosses* an |
| 50 |
`XCE` — the one from startup, one on a path the program never takes, or a `$FB` |
| 51 |
byte that is really data — with a carry the estimate did not track. `run_e` is |
| 52 |
written in exactly one place, `case 0xFB`, so without an `XCE` in the walked |
| 53 |
stream the estimate keeps the seeded `E`. |
| 54 |
|
| 55 |
For `PLP` and `RTI` the file leaves the running estimate untouched. Operand |
| 56 |
widths for lines decoded after one of them can therefore be wrong, and with |
| 57 |
them the instruction boundaries that follow. |
| 58 |
|
| 59 |
## Why it is not fixed |
| 60 |
|
| 61 |
The value is not recoverable by this analysis, which sees only the instruction |
| 62 |
stream. What `PLP` restores |
| 63 |
depends on what was pushed, which depends on the path taken to get there; what |
| 64 |
`RTI` restores is the status at the moment an interrupt was taken, which is not |
| 65 |
in the code at all. |
| 66 |
|
| 67 |
Two avenues were considered and rejected: |
| 68 |
|
| 69 |
- **Walk back from `PLP` to the matching `PHP`.** Needs the routine boundaries |
| 70 |
and a guarantee that no other stack traffic intervened. Neither is available, |
| 71 |
and a wrong pairing produces a confident wrong width rather than an admitted |
| 72 |
unknown. |
| 73 |
- **Record the status pushed at each interrupt and consult it for `RTI`.** The |
| 74 |
emulator does see every interrupt, and `src/cpu/irq_ctx.c` already tracks |
| 75 |
entries and exits — but it records the vector, the interrupted PC and the |
| 76 |
stack pointer, not the pushed status. Adding it is easy; *using* it is not. |
| 77 |
`cm_propagate()` runs over arbitrary addresses while drawing a disassembly |
| 78 |
window, and has no way to know which interrupt frame a particular `RTI` on |
| 79 |
screen will return through. The innermost live frame is not necessarily the |
| 80 |
right one. So the result would still be a guess, just a more expensive one. |
| 81 |
|
| 82 |
Both need a notion of "known vs unknown" per status bit, so that a failed |
| 83 |
attempt reports uncertainty instead of substituting another guess. That does not |
| 84 |
exist, and adding it is a larger change than the defect warrants. |
| 85 |
|
| 86 |
## Why the gap is bounded |
| 87 |
|
| 88 |
It costs nothing unless *all* of these hold at once: |
| 89 |
|
| 90 |
- the machine is a 65C816 (on the 65C02 operand widths never vary, so the |
| 91 |
estimate cannot change a length); |
| 92 |
- the line being drawn has no currently believed anchor — usually because that |
| 93 |
code has never executed, but also if its anchor was evicted with its bank |
| 94 |
context (the cache is capped, see `CM_MAX_CONTEXTS`) or is no longer believed |
| 95 |
because the opcode byte under it changed; |
| 96 |
- the status the estimate carries into that line is not the status the code |
| 97 |
there really runs with. |
| 98 |
|
| 99 |
That third condition is the general one, and it is *not* only the unmodelled |
| 100 |
instructions. It happens whenever: |
| 101 |
|
| 102 |
- one of the unmodelled cases lies between the last anchor and the line — a |
| 103 |
`PLP`, an `RTI`, a stale carry feeding an `XCE`, or a stale `E`; |
| 104 |
- **the walk crossed a branch or a call.** Propagation is address-linear: it |
| 105 |
applies whatever opcode sits at the next *address*. Given |
| 106 |
`BRA +2 / REP #$20 / LDA #`, execution jumps over the `REP` and the `LDA` is |
| 107 |
two bytes, while the walk applies the `REP` and sizes it three. The same goes |
| 108 |
for a `JSR` whose callee changes `M`/`X` and returns. `REP` and `SEP` are |
| 109 |
applied exactly *as instructions*; that does not make the resulting estimate |
| 110 |
right for a path the program never took; |
| 111 |
- **the walk started somewhere the seed does not describe.** |
| 112 |
`code_map_disasm_forward` seeds from `regs.status`/`regs.e`, which is the |
| 113 |
state at the *current PC*, not at `start`. That is the DAP `disassemble` |
| 114 |
path, which takes an arbitrary address, so disassembling never-executed code |
| 115 |
that runs at a different width is wrong from the first line — with no anchor |
| 116 |
and no unmodelled instruction anywhere in sight. |
| 117 |
|
| 118 |
Note what is **not** on that list: native mode. The real CPU in emulation mode |
| 119 |
does force 8-bit widths and is always right — but the fold-in here |
| 120 |
(`if (e) status |= INDEX|MEMORY`) uses the *estimated* `E`, not the machine's, |
| 121 |
and no anchor repairs that estimate. `run_e` is seeded once per forward run and |
| 122 |
thereafter written only by `XCE`, from the estimate's own carry; an anchor |
| 123 |
overrides the status but never the `E`. Once the estimate diverges it will size |
| 124 |
operands as though native while the machine is really in emulation, until an |
| 125 |
`XCE` with an accurate carry happens to put it back. A test pins exactly that. |
| 126 |
|
| 127 |
The mechanism that would recover the value is largely the same mechanism that |
| 128 |
makes the gap irrelevant: code that has executed has anchors. The correspondence |
| 129 |
is not perfect — a handler paused before its own, not-yet-executed `RTI` does |
| 130 |
have a live interrupt frame — but associating a particular on-screen `RTI` with |
| 131 |
a particular frame is exactly the part that is not solved. |
| 132 |
|
| 133 |
Every line this can affect already reports `recorded == false`, so a UI can say |
| 134 |
so. **The strength of the recovery differs by which input was wrong**, and none |
| 135 |
of it is unconditional: |
| 136 |
|
| 137 |
- a wrong **status** estimate can be corrected once an accurate, believed anchor |
| 138 |
is reached. Three caveats, all real: a same-opcode stale anchor is still |
| 139 |
believed and hands back its *old* status; there is no guarantee an accurate |
| 140 |
anchor exists ahead at all; and even an accurate one only fixes its **own** |
| 141 |
line — an anchor on a `PLP` or `RTI` supplies the status going *into* it, |
| 142 |
while the byte it restores is still unmodelled, so the next line can be wrong |
| 143 |
immediately; |
| 144 |
- a wrong **E** is weaker again, since anchors do not store `E` at all. An |
| 145 |
anchor fixes the width of its own line, and propagation past it folds the |
| 146 |
stale `E` straight back in, which can mis-size the next unanchored line |
| 147 |
wherever the two imply different widths. It is recoverable in principle — an |
| 148 |
`XCE` decoded with an accurate carry writes a correct `E` — but no ordinary |
| 149 |
anchor does it, so nothing guarantees it happens. |
| 150 |
|
| 151 |
In both cases the decode is not allowed to swallow a recorded start (unless it |
| 152 |
is itself recorded — see the overlap policy in `cm_fill`), so a wrong guess |
| 153 |
cannot silently run past hard evidence about *boundaries*, even where it keeps |
| 154 |
getting *widths* wrong. |
| 155 |
|
| 156 |
## What is pinned by tests |
| 157 |
|
| 158 |
`tests/test_code_map.c` covers the bound rather than the missing value, for all |
| 159 |
three cases: |
| 160 |
|
| 161 |
- `PLP`: "reports a line decoded from a guessed width as unrecorded", |
| 162 |
"re-syncs onto the next anchor after a bad width guess", "recovers the real |
| 163 |
width from the anchor it landed on". |
| 164 |
- `RTI`: "leaves the width estimate alone across RTI instead of guessing", |
| 165 |
"reports a line decoded across RTI as a guess, not as known", "reaches the |
| 166 |
anchor after RTI and recovers the real width". |
| 167 |
- `XCE` with a stale carry: "KNOWN LIMITATION: a stale carry into XCE |
| 168 |
mispredicts the widths", "an anchor recovers the width of its own line after a |
| 169 |
bad XCE", and — pinning the *weaker* bound described above — "KNOWN |
| 170 |
LIMITATION: past that anchor a stale E mis-sizes again". |
| 171 |
|
| 172 |
The first `RTI` check is deliberately written to fail if someone substitutes a |
| 173 |
plausible width without also building the known/unknown machinery. It is a |
| 174 |
tripwire, not a claim that the current behaviour is ideal — if the widths are |
| 175 |
ever tracked properly, that check should be replaced, not deleted quietly. |
| 176 |
|
| 177 |
--- |
| 178 |
|
| 179 |
# Related: a stale anchor that kept its opcode byte |
| 180 |
|
| 181 |
**Known limitation, pinned by tests, not fixed.** |
| 182 |
|
| 183 |
An anchor is believed while the opcode byte it recorded is still in memory. That |
| 184 |
one-byte check catches overlay loads, a second program over the first, and |
| 185 |
self-modifying code without needing a hook on every write — but replacement code |
| 186 |
that happens to repeat the same opcode byte at the same address keeps the old |
| 187 |
anchor, and with it the old recorded **status**. |
| 188 |
|
| 189 |
On a 65C816 that stale status can imply a different operand width, so the line |
| 190 |
decodes wider than the new code really is. Because a recorded line is exempt |
| 191 |
from the interior-anchor clamp (see the overlap policy in `cm_fill`), it can |
| 192 |
then swallow a genuinely fresh anchor inside it: |
| 193 |
|
| 194 |
``` |
| 195 |
old: $8000 A9 xx xx LDA #$xxxx recorded with 16-bit A -> 3 bytes |
| 196 |
new: $8000 A9 xx LDA #$xx 8-bit A, and $8002 freshly executed |
| 197 |
``` |
| 198 |
|
| 199 |
The opcode is still `$A9`, so the stale anchor survives and wins. |
| 200 |
|
| 201 |
**It cuts both ways.** The example above decodes too *wide* and swallows a fresh |
| 202 |
anchor, which is the loud failure. The opposite is quieter and just as |
| 203 |
reachable: an anchor recorded when the accumulator was 8 bits sizes its `LDA #` |
| 204 |
at two bytes, so if the replacement code is about to run with a 16-bit |
| 205 |
accumulator the row stops a byte early and the *next* row starts inside the real |
| 206 |
instruction's operand. Nothing is swallowed and the rows still tile perfectly, |
| 207 |
so there is no visible seam — the disassembly is simply misaligned from there |
| 208 |
until an accurate anchor is reached. Both directions are pinned by tests. |
| 209 |
|
| 210 |
Why it is accepted rather than fixed: |
| 211 |
|
| 212 |
- It needs the replacement code to repeat the same opcode byte at the same |
| 213 |
address, which the one-byte check then cannot see through. |
| 214 |
- The damage is self-correcting *if* an accurate anchor is reached that this |
| 215 |
line — or another stale one — has not swallowed. That is not guaranteed: right |
| 216 |
after an overlay load the new code typically has no valid anchors at all, and |
| 217 |
the stale status then propagates to the end of the window. (An anchor whose |
| 218 |
opcode no longer matches re-syncs nothing; it is rejected by `cm_anchor_ok()` |
| 219 |
and simply stops being evidence.) |
| 220 |
- The alternative — always clamping — costs the far more common case, the |
| 221 |
`.byte $2C` skip idiom, where two overlapping starts are both genuinely real |
| 222 |
and clamping renders a whole known instruction as a fragment. |
| 223 |
|
| 224 |
**Scope: the wrong-*width* flavour is 65C816-only; the swallowing is not.** On a |
| 225 |
65C02 operand widths never vary, so a stale anchor cannot decode wider than the |
| 226 |
new code. But it does not have to: a stale anchor whose opcode byte survived is |
| 227 |
still believed, is still exempt from the interior-anchor clamp, and can still |
| 228 |
cover a fresh start inside it. An old `BIT abs` at `$8000` overlaid by new code |
| 229 |
that also begins `$2C` will swallow a genuine instruction start at `$8001` on |
| 230 |
any CPU. `tests/test_code_map.c` covers both flavours. |
| 231 |
|
| 232 |
**The stale case is not distinguishable from a legitimate one.** The recorded |
| 233 |
state above is byte-for-byte identical to a routine that genuinely executed at |
| 234 |
two different accumulator widths — an anchor at `$8000` with a 16-bit status and |
| 235 |
an anchor at `$8002` with an 8-bit one, over memory starting `$A9`. In that |
| 236 |
benign case emitting the 3-byte `LDA` is exactly correct. |
| 237 |
|
| 238 |
That rules out the obvious fix. A per-address recording *epoch* ("prefer the |
| 239 |
newer anchor") does not resolve this: applied to the benign case it would clamp |
| 240 |
a legitimately recorded 3-byte `LDA` back to 2 bytes purely because the other |
| 241 |
width ran more recently, reintroducing the defect this branch fixed. It trades |
| 242 |
one wrong answer for another. |
| 243 |
|
| 244 |
Widening the staleness check — storing two or more bytes per anchor instead of |
| 245 |
one — is also not the clean win it looks like. It would catch *this* example |
| 246 |
(the operand bytes differ), but it is the same class of heuristic, so it only |
| 247 |
narrows the window rather than closing it; replacement code with identical bytes |
| 248 |
stays invisible either way. It also misfires on one-byte instructions, which are |
| 249 |
extremely common (`NOP`, `INX`, `RTS`, …): for those the second byte compared is |
| 250 |
not part of the instruction at all but the *next* instruction's opcode, so |
| 251 |
changing the following instruction would invalidate a perfectly good anchor. |
| 252 |
Storing the instruction's real length would avoid that — it does not need a full |
| 253 |
`disasm()` call, an opcode length table plus the small M/X-dependent set would |
| 254 |
do — but it is per-instruction classification plus another byte of storage per |
| 255 |
address, and it still does not close the identical-bytes hole. |
| 256 |
|
| 257 |
What does resolve it is invalidation keyed on the **write** that replaced the |
| 258 |
code, because that is the event which actually differs between the two cases. |
| 259 |
That does not necessarily mean a hook on every CPU write: the dominant source of |
| 260 |
stale anchors is a host-side load, and those already funnel through the KERNAL |
| 261 |
LOAD/MACPTR interception in `main.c` (the bytes themselves land in `ieee.c`), so |
| 262 |
even a conservative `code_map_reset()` when an intercepted load completes would |
| 263 |
be sound for that case and needs no range or bank bookkeeping. Targeted range |
| 264 |
invalidation keeps more coverage but is more work. Self-modifying code and |
| 265 |
in-guest decompressors would still need the general write path in `memory.c`. |
| 266 |
Either way it is a change to the recording side rather than the disassembly |
| 267 |
side. |
| 268 |
|
| 269 |
(It would *not* help the PLP/RTI width gap described earlier in this document. |
| 270 |
That one is about status bytes restored from the stack, which is unrelated to |
| 271 |
code being replaced.) |
| 272 |
|
| 273 |
`tests/test_code_map.c` pins the current behaviour under checks labelled |
| 274 |
`KNOWN LIMITATION`, so it is visible rather than folklore. If it is ever fixed |
| 275 |
those checks will fail — update them, do not delete them. |
| 276 |
|
| 277 |
One caveat on that tripwire: it fires for a fix *inside* code_map (a wider |
| 278 |
staleness check, a stored instruction length, an epoch), but **not** for the |
| 279 |
write-invalidation fix recommended above. The unit test models the overlay with |
| 280 |
its own `poke()`, which writes the harness's memory arrays directly, and |
| 281 |
code_map exposes no write-notification entry point for it to call. Anyone |
| 282 |
adding one should route `poke()` through it, or these checks will stay green |
| 283 |
while this note quietly becomes a description of behaviour that no longer |
| 284 |
exists. |
| 285 |
|
| 286 |
--- |
| 287 |
|
| 288 |
# Related: a window-straddling instruction is decoded through one bank |
| 289 |
|
| 290 |
**Not fixed. Not covered by a passing test.** Found while adding banking |
| 291 |
coverage for `cm_x16bank_for()`. |
| 292 |
|
| 293 |
`cm_fill()` reads the raw bytes of an instruction one at a time, each through |
| 294 |
the window backing that byte's own address, so an instruction straddling the |
| 295 |
`$BFFF`/`$C000` boundary shows the right bytes. `cm_decode()` does not: it hands |
| 296 |
`disasm()` a *single* `x16Bank`, derived from the instruction's **start** |
| 297 |
address, and `disasm()` uses that one bank for every operand read. |
| 298 |
|
| 299 |
So for a `STA $1234` whose opcode is at `$BFFF` in RAM bank 5 and whose operand |
| 300 |
lives at `$C000`/`$C001` in ROM bank 2, the operand is read as *ROM bank 5*. |
| 301 |
With the test fixture that means `bytes[]` correctly reads `8D 34 12` while |
| 302 |
`text` reads `sta $eaea` and `eff_addr` is `$EAEA`. This is a *debugger-side* |
| 303 |
mis-read only: the CPU's own fetch goes through the real banking hardware and |
| 304 |
is correct, so the program runs fine — it is the disassembly shown to the user |
| 305 |
that is wrong. The same wrong bytes come back from the host-side read on a real |
| 306 |
machine's memory image (`ROM[rambank * 16384 + …]`; for a bank number ≥ 32 it is |
| 307 |
open bus, or the cartridge's bytes when one supplies that bank). |
| 308 |
|
| 309 |
Note the width propagation had the *same* bug and **is** fixed: `cm_propagate()` |
| 310 |
now reads a REP/SEP operand through the window backing the operand's own |
| 311 |
address. That one is code_map's own read, so fixing it touched nothing shared. |
| 312 |
|
| 313 |
The test in `tests/test_code_map.c` asserts the displayed **bytes** only, and |
| 314 |
says so in a comment; it does not assert the text, because the text is wrong. |
| 315 |
|
| 316 |
Fixing it properly means giving `disasm()` a per-byte, window-aware reader (or |
| 317 |
having `cm_decode()` detect a straddle and re-render the operand from the bytes |
| 318 |
`cm_fill()` already read correctly). `disasm()` is shared with the classic SDL |
| 319 |
debugger and the DAP server, so that is a wider change than the code_map |
| 320 |
defects this work covers. |
| 321 |
|
| 322 |
Scope: one instruction per window boundary, and only when code is laid across |
| 323 |
one. There are two such boundaries, not one — `$9FFF`/`$A000` as well as |
| 324 |
`$BFFF`/`$C000` — since an opcode below `$A000` follows the live bank while its |
| 325 |
operand in the RAM window needs the caller's RAM bank. Rare, but it renders as |
| 326 |
confidently wrong text rather than as an admitted unknown, which is the worst |
| 327 |
failure mode this file has. |