| 1 |
# UnapiNet bridge protocol v2 |
| 2 |
|
| 3 |
Status: **implemented — this is the reference.** This document was agreed |
| 4 |
as a proposal before it turned into code; what follows is the protocol |
| 5 |
the device and the driver (`unapinet.com`, Z80) actually speak, and the |
| 6 |
golden-byte tests pin every reply struct's layout to it. As promised while it was |
| 7 |
under review, the v1 document it replaces is folded in below as a |
| 8 |
historical appendix — still the byte-level reference for the |
| 9 |
parameter-block layouts, which v2 keeps. Functional testing added one |
| 10 |
clause the proposal had left unstated: UDP_OPEN falls back to an |
| 11 |
ephemeral port when the host refuses the requested one (see UDP below). |
| 12 |
|
| 13 |
The driver and this device ship from the same source project and are |
| 14 |
updated together. v1 has four users, all personally reachable; there is |
| 15 |
no installed base to migrate. v2 is therefore a clean break, with |
| 16 |
detection designed so that any old/new mismatch fails cleanly at |
| 17 |
installation time (analysis, and one caveat, under "Migration and |
| 18 |
failure analysis" below). |
| 19 |
|
| 20 |
## Design rules |
| 21 |
|
| 22 |
v1's problems are catalogued at the end of the v1 appendix; they reduce to |
| 23 |
one sentence: *every command grew its own conventions*. v2 replaces them |
| 24 |
with four rules: |
| 25 |
|
| 26 |
1. **Every reply begins with a status byte.** `0x00` = success. Any other |
| 27 |
value is the MSX-UNAPI TCP/IP error code the driver should hand to its |
| 28 |
caller, verbatim. The device speaks UNAPI error codes natively; the |
| 29 |
driver stops translating. |
| 30 |
2. **An error reply is exactly one byte** — the status. No partial |
| 31 |
payloads: if the status is non-zero, there is nothing else to read. |
| 32 |
3. **Parameter blocks are validated for exact size.** Too short, too long, |
| 33 |
or inconsistent with a declared payload length → `ERR_INV_PARAM`, and |
| 34 |
the command has no effect. |
| 35 |
4. **No valid status byte is 0xFF.** Reading the data port past the end |
| 36 |
of a reply (or with no reply pending) returns `0xFF` — the same value |
| 37 |
an absent device yields from the bus pull-ups, and never a valid |
| 38 |
status (success is `0x00`, and no UNAPI error code is `0xFF`). A |
| 39 |
driver reading a status position and seeing `0xFF` knows it is |
| 40 |
desynchronized or talking to nothing — never "OK". (v1 returned `0x00` |
| 41 |
there, which is indistinguishable from success — Wouter's observation.) |
| 42 |
Bytes *inside* a reply are length-delimited and unconstrained: an IP |
| 43 |
octet, a payload byte or the DNS failed-lookup marker may legitimately |
| 44 |
be `0xFF`. |
| 45 |
|
| 46 |
### Error codes on the wire |
| 47 |
|
| 48 |
The status byte uses the MSX-UNAPI TCP/IP error space. The device emits |
| 49 |
this subset (the values match the error table the driver already defines; |
| 50 |
the device side is what v2 added): |
| 51 |
|
| 52 |
|
| 53 |
|
| 54 |
| 0 | ERR_OK | success | |
| 55 |
| 1 | ERR_NOT_IMP | unknown opcode; the ICMP commands on a host without working ICMP (caps bit4 clear) | |
| 56 |
| 2 | ERR_NO_NETWORK | the host socket layer refused: socket()/bind()/listen() failure, a synchronous connect() failure, a failed or short sendto() (UDP) | |
| 57 |
| 3 | ERR_NO_DATA | UDP_RECV / ICMP_RECV with an empty queue | |
| 58 |
| 4 | ERR_INV_PARAM | malformed parameter block (rule 3), empty hostname, one longer than 253 bytes or one with an embedded NUL, a TCP_SEND length beyond `MAX_TRANSFER`, an undefined flag bit set | |
| 59 |
| 5 | ERR_QUERY_EXISTS| DNS_QUERY while a lookup is already running | |
| 60 |
| 9 | ERR_NO_FREE_CONN| TCP_OPEN / UDP_OPEN with all handles in use | |
| 61 |
| 11 | ERR_NO_CONN | handle out of range (0 included, save the close-all forms of TCP_CLOSE / UDP_CLOSE / TCP_ABORT); or no open connection on it — except TCP_STATE / TCP_RECV, which answer for any in-range handle (see notes) | |
| 62 |
| 12 | ERR_CONN_STATE | a connection exists but its state forbids the command — including a TCP send whose socket write fails, which resets the connection and answers 12 | |
| 63 |
| 13 | ERR_BUFFER | TCP_SEND with the 64 KiB send queue full (recoverable: retry later; nothing is partially queued) | |
| 64 |
|
| 65 |
Rule 3 applies to **every** opcode, parameterless ones included: a |
| 66 |
malformed or stray parameter block answers `{4}` regardless of the |
| 67 |
command, and the per-command "err" lists below do not repeat that. Where |
| 68 |
a list names 4 explicitly, a *well-formed* block can still be |
| 69 |
semantically invalid there. One check precedes rule 3: opcode dispatch — |
| 70 |
an unknown opcode answers `{1}` whether or not stray parameter bytes |
| 71 |
accompany it (there is no expected size to validate against). In both |
| 72 |
cases the parameter buffer is cleared, as after every command. |
| 73 |
|
| 74 |
Validation order is fixed: form first (rule 3, `{4}`, together with the |
| 75 |
semantic `{4}`s — an invalid value inside a well-formed block); then |
| 76 |
capability (`{1}`, the ICMP commands); then handle (`{11}`); then |
| 77 |
connection state (`{12}`, and DNS's busy `{5}`); then the data check |
| 78 |
(`{3}`, the RECV queues). The host operation runs last, so its errors |
| 79 |
(`{2}`, `{13}` — plus the TCP write-failure `{12}`, the one |
| 80 |
post-operation use of that code) imply every earlier check passed. In |
| 81 |
the OPEN commands the free-slot check (`{9}`) precedes the host socket |
| 82 |
work (`{2}`). |
| 83 |
|
| 84 |
### The status register is gone — the command port is write-only |
| 85 |
|
| 86 |
The driver's only use of the v1 status register is "did my command produce |
| 87 |
a result?" — with rule 1 that information is the reply's first byte, so |
| 88 |
the register is redundant as a mechanism, and v2 removes it entirely |
| 89 |
rather than keeping a mirror nobody needs (Wouter's suggestion). The |
| 90 |
device registers the command port for writes only; reading it decodes to |
| 91 |
the open bus (`0xFF` on a standard MSX), exactly as if no device were |
| 92 |
present. The status byte lives in one place — the reply — so there is |
| 93 |
nothing to poll, nothing that can disagree with it, and nothing extra to |
| 94 |
document. The three v1 situations that raised `STATUS_ERROR` in the |
| 95 |
register become ordinary single-byte error replies (unknown opcode → 1, |
| 96 |
empty hostname → 4, DNS busy → 5). |
| 97 |
|
| 98 |
## What does not change — and two things that do |
| 99 |
|
| 100 |
Unchanged from v1: ports and decoding (base 0x28: command, base+1: |
| 101 |
parameter/result — the command port's read side is gone, see above), the |
| 102 |
synchronous transaction model, parameter |
| 103 |
accumulation, the parameter-buffer cap, opcode numbers (except 0x10, |
| 104 |
retired — below), parameter block layouts, handles (1-based, 1..4, TCP |
| 105 |
and UDP independent), `handle 0 = close all transient` in the CLOSE |
| 106 |
commands (v2 extends the same form to TCP_ABORT — see the handles |
| 107 |
notes), MAX_TRANSFER/MAX_RECV_BUF/MAX_SEND_BUF, the graceful-close |
| 108 |
semantics, and the receive-side backpressure. Byte order stays as it is: |
| 109 |
IPs as network-order octets, 16-bit quantities little-endian. |
| 110 |
|
| 111 |
Two v1 statements are explicitly superseded — a reader must NOT carry |
| 112 |
them over from the v1 document: |
| 113 |
|
| 114 |
* v1's transaction model says reading past the end returns `0x00` and |
| 115 |
gates each command on `status register == STATUS_DATA (2)`. In v2, |
| 116 |
past-end reads return `0xFF` (rule 4) and the register does not exist — |
| 117 |
the port reads as open bus (`0xFF`), so a v1-style `== 2` gate can |
| 118 |
never pass. |
| 119 |
* v1's DNS_QUERY takes a NUL-terminated name (terminator effectively |
| 120 |
optional, trailing bytes ignored). v2 drops the terminator entirely: |
| 121 |
the parameter block *is* the hostname. This is the only change at the |
| 122 |
block-form level; no other layout moved. (One *value* check is also |
| 123 |
new — TCP_OPEN's undefined flag bits answer `{4}` where v1 ignored |
| 124 |
them. And one value gains meaning: v1 destroyed a datagram unread on |
| 125 |
UDP_RECV maxlen 0; v2 makes that TCPIP_UDP_RCV's deliberate DE=0 |
| 126 |
discard, reporting the source and size while copying nothing.) |
| 127 |
|
| 128 |
The recovery rule is kept and completed: writing a **parameter byte** |
| 129 |
while a result is pending discards the unread result (as in v1), and |
| 130 |
writing a **command opcode** while a result is pending abandons it too — |
| 131 |
the new reply replaces it. A driver never has to drain a reply it has |
| 132 |
lost interest in. |
| 133 |
|
| 134 |
## DETECT (0x00) — replaces PING and QUERY_CAP |
| 135 |
|
| 136 |
Wouter suggested merging the two, and the merge solves three things at |
| 137 |
once: the confusing "PING" name (it is not ICMP), the version byte nobody |
| 138 |
reads, and the magic bump v2 needs anyway. |
| 139 |
|
| 140 |
`DETECT`, opcode 0x00, no parameters. Success reply, 5 bytes: |
| 141 |
|
| 142 |
|
| 143 |
|
| 144 |
| 0 | status 0x00 | |
| 145 |
| 1 | magic **0x55** (was 0xAB in v1) | |
| 146 |
| 2 | protocol version: **2** | |
| 147 |
| 3 | capabilities: bit0 DNS, bit1 TCP active, bit2 TCP passive, bit3 UDP, bit4 ICMP, rest reserved 0 | |
| 148 |
| 4 | reserved 0 | |
| 149 |
|
| 150 |
The driver accepts the device if and only if bytes 0-2 are exactly |
| 151 |
`00 55 02`. Bytes 3-4 are data, not part of the acceptance rule: a driver |
| 152 |
must tolerate unknown capability bits (and any value in byte 4), or it |
| 153 |
will reject future devices for advertising more. Byte 2 is the upgrade |
| 154 |
path: a hypothetical v3 reports 3 there, a v2 driver rejects it cleanly, |
| 155 |
and a v3 driver may accept both if v3 is a superset. |
| 156 |
|
| 157 |
Capabilities are honest and enforced: bit4 (ICMP) is set only when the |
| 158 |
host can actually ping — platform support compiled in *and* the ICMP |
| 159 |
channel opening successfully at device start (today that means: Windows, |
| 160 |
and `IcmpCreateFile()` succeeded); it is latched once at start. If bit4 |
| 161 |
is clear, the ICMP commands answer `ERR_NOT_IMP`. Bits 0-3 are always set |
| 162 |
on this device today (byte 3 reads 0x0F or 0x1F); they are informational, |
| 163 |
and no command is gated on them. |
| 164 |
|
| 165 |
One corner is worth a recipe: a crashed predecessor may have left stray |
| 166 |
parameter bytes buffered, and a strict DETECT would then answer `{4}` — |
| 167 |
yet DETECT is also the stream-resynchronization command, so it must work |
| 168 |
from any state. Wouter's resolution: keep DETECT strict, and have the |
| 169 |
driver issue it **twice unconditionally**, reading only the second |
| 170 |
reply. The first either succeeds and is abandoned (issuing a command |
| 171 |
discards a pending result — the recovery rule) or answers `{4}` and |
| 172 |
clears the stray block; either way the second DETECT runs on a clean |
| 173 |
buffer. No conditional on either side. |
| 174 |
|
| 175 |
Opcode 0x10 (QUERY_CAP) is retired and answers `ERR_NOT_IMP` like any |
| 176 |
other unknown opcode. |
| 177 |
|
| 178 |
## Command reference (v2 replies) |
| 179 |
|
| 180 |
Parameter layouts are unchanged from v1 (see the v1 appendix for byte |
| 181 |
detail); this table gives the v2 reply, status byte included. Where a row |
| 182 |
shows `{0, Struct}`, the v2 wire struct is the v1 struct with the status |
| 183 |
byte **prepended** as its new first member — sizes grow by one, and the |
| 184 |
count shown already includes it — except ICMP_RECV, where the status |
| 185 |
**replaces** v1's `hasData` marker byte (still 12). "err:" lists the |
| 186 |
codes each command can emit besides 0 and besides the universal `{4}` of |
| 187 |
rule 3. |
| 188 |
|
| 189 |
|
| 190 |
|
| 191 |
| 0x00 | DETECT | {0, 0x55, 2, caps, 0} (5) | — | |
| 192 |
| 0x01 | DNS_QUERY | {0, 0} async started, or {0, 1, ip4} (6) resolved immediately | 4, 5 | |
| 193 |
| 0x02 | DNS_STATUS | {0, state, ...} — see DNS below | — | |
| 194 |
| 0x03 | TCP_OPEN | {0, handle} (2) | 2, 4, 9 | |
| 195 |
| 0x04 | TCP_SEND | {0} (1) | 4, 11, 12, 13| |
| 196 |
| 0x05 | TCP_RECV | {0, len.lo, len.hi, payload...} (3+len) | 11 | |
| 197 |
| 0x06 | TCP_CLOSE | {0} (1) | 11 | |
| 198 |
| 0x07 | TCP_STATE | {0, TcpStateResult} (13) | 11 | |
| 199 |
| 0x08 | TCP_ABORT | {0} (1) | 11 | |
| 200 |
| 0x09 | UDP_OPEN | {0, handle} (2) | 2, 9 | |
| 201 |
| 0x0A | UDP_CLOSE | {0} (1) | 11 | |
| 202 |
| 0x0B | UDP_STATE | {0, size.lo, size.hi} (3) | 11 | |
| 203 |
| 0x0C | UDP_SEND | {0} (1) | 2, 11 | |
| 204 |
| 0x0D | GET_LOCALIP | {0, ip4} (5) | — | |
| 205 |
| 0x0E | NET_STATE | {0, 2} (2) | — | |
| 206 |
| 0x0F | UDP_RECV | {0, srcip4, port.2, len.2, payload} (9 + min(len, maxlen)) | 3, 11 | |
| 207 |
| 0x11 | ICMP_SEND | {0} (1) | 1 | |
| 208 |
| 0x12 | ICMP_RECV | {0, srcip4, ttl, id.2, seq.2, len.2} (12)| 1, 3 | |
| 209 |
|
| 210 |
### Handles, slots, and what survives a close |
| 211 |
|
| 212 |
* Handles are 1..4. **Handle 0 is out of range** (`{11}`) everywhere |
| 213 |
except TCP_CLOSE, UDP_CLOSE and TCP_ABORT, where it means "close (or |
| 214 |
abort) all transient" — and succeeds (`{0}`) even when nothing is |
| 215 |
open. The ABORT form is new in v2: UNAPI defines it (TCPIP_TCP_ABORT, |
| 216 |
B = 0), but v1 silently no-opped it — the device rejected handle 0 and |
| 217 |
the driver never looked at the reply, so the caller got ERR_OK with |
| 218 |
nothing aborted. |
| 219 |
* `ERR_NO_CONN` (11) means the handle is out of range — or, for the |
| 220 |
commands that act on a live connection (SEND, CLOSE, ABORT, and all of |
| 221 |
UDP), that no open connection sits on it. **TCP_STATE and TCP_RECV |
| 222 |
answer for any in-range handle, open or closed**: a closed connection |
| 223 |
keeps its final state, its close reason and its undrained receive data |
| 224 |
until TCP_OPEN reuses the slot. That is deliberate v1 behavior, kept on |
| 225 |
purpose — the tail of a connection that died stays readable (TCP_ABORT |
| 226 |
included), and the driver can still see *why* it died. What 11 removes |
| 227 |
is v1's ambiguity: a zeroed struct or a zero length no longer doubles |
| 228 |
as "bad handle". |
| 229 |
* A **never-used** in-range slot answers TCP_STATE with closeReason 1 |
| 230 |
(NeverUsed) and every other field zero, and TCP_RECV with length 0 — same frame |
| 231 |
as v1, minus the bad-handle overload. |
| 232 |
* Slot allocation: lowest-numbered free slot first. A closed slot holding |
| 233 |
a sticky tail **is** free — it does not count toward `{9}`, and reusing |
| 234 |
it destroys the tail. A slot consumed by a failed TCP_OPEN (`{2}`) is |
| 235 |
released, not left in ConnectFailed (that state belongs to the |
| 236 |
*asynchronous* connect path only). |
| 237 |
|
| 238 |
### TCP |
| 239 |
|
| 240 |
* **TCP_SEND** may send in Established and CloseWait; any other existing |
| 241 |
connection state answers `{12}`; no connection at all answers `{11}`. |
| 242 |
A length beyond `MAX_TRANSFER` (4096) answers `{4}`; length 0 is a |
| 243 |
valid no-op (`{0}`). `{13}` is all-or-nothing: nothing was queued, the |
| 244 |
same command may be retried verbatim later. A socket write failure |
| 245 |
resets the connection (closeReason 4, ConnectionReset) and answers |
| 246 |
`{12}`. |
| 247 |
* **TCP_RECV**: `maxlen` is a ceiling request — a value beyond 4096 is |
| 248 |
clamped, not an error (contrast TCP_SEND, where length states a fact |
| 249 |
about the payload that follows and must be exact). `maxlen` 0 answers |
| 250 |
`{0, 0, 0}` and consumes nothing. Length 0 with a live connection is |
| 251 |
normal stream behavior; poll TCP_STATE to tell "live but idle" from |
| 252 |
"finished". |
| 253 |
* **TCP_CLOSE** is graceful: FinWait1, FIN once everything queued has |
| 254 |
been sent, handle freed when the peer closes too — or after 30 s. |
| 255 |
Buffered received data stays readable throughout (and after, per the |
| 256 |
sticky-slot rule). Closing a Listen handle is valid and frees it. A |
| 257 |
second CLOSE while the close is in progress is idempotent (`{0}`). |
| 258 |
**TCP_ABORT** drops immediately (closeReason 3); handle 0 aborts |
| 259 |
every transient connection, mirroring UNAPI's TCPIP_TCP_ABORT. |
| 260 |
* **TCP_OPEN**: the `timeout` field stays in the layout, is accepted with |
| 261 |
any value, and is ignored (reserved). Undefined flag bits (2-7) must be |
| 262 |
0; a set one answers `{4}`. Remote IP / port are not semantically |
| 263 |
validated — an active open to 0.0.0.0 or port 0 is handed to the host |
| 264 |
and fails there (`{2}` or, asynchronously, ConnectFailed). Passive |
| 265 |
open: a non-zero remote IP filters the peer; a non-matching peer is |
| 266 |
accepted and immediately dropped, and the listener keeps listening; on |
| 267 |
a match the handle *becomes* the connection (Listen → Established — |
| 268 |
the listener is gone, one connection per passive open). |
| 269 |
|
| 270 |
### UDP |
| 271 |
|
| 272 |
* **Every UDP socket is transient.** The UNAPI residency concept has no |
| 273 |
wire representation in UdpOpenParams, so UDP_CLOSE 0 closes every open |
| 274 |
UDP socket. (The device keeps an internal resident flag for future use; |
| 275 |
nothing can set it today.) |
| 276 |
* **UDP_CLOSE discards the socket's queued datagrams** — unlike TCP, |
| 277 |
nothing UDP stays readable after close. |
| 278 |
* **UDP_SEND**: an oversized datagram is not a form error — the host |
| 279 |
refuses it and the reply is `{2}` (as is a short kernel write). A |
| 280 |
failed send leaves the socket open and usable. Length 0 sends an empty |
| 281 |
datagram (`{0}`). |
| 282 |
* **UDP_RECV**: the head datagram is consumed whole. `len` in the reply |
| 283 |
is the datagram's size **as received**, and the payload that follows |
| 284 |
is the first `min(len, maxlen)` bytes — the excess, if any, is |
| 285 |
discarded with the datagram. This is exactly TCPIP_UDP_RCV's contract: |
| 286 |
its BC output is "the datagram data size as it was received, which may |
| 287 |
be larger than the number of bytes actually retrieved", and its DE=0 |
| 288 |
("no data will be copied at all") is a *deliberate* discard — so |
| 289 |
`maxlen` 0 is valid: header only, datagram gone. Contrast TCP_RECV, |
| 290 |
where `maxlen` 0 consumes nothing — a stream has no unit to destroy, |
| 291 |
a datagram is all-or-nothing. An empty datagram (0 data bytes) is a |
| 292 |
normal datagram: UDP makes it visible to the receiver, so it queues |
| 293 |
and is delivered with `len` 0. Datagrams were already truncated to |
| 294 |
2 KiB at receive time and queue at most 16 deep (v1 behavior, |
| 295 |
unchanged). |
| 296 |
* **UDP_STATE** answers the head datagram's size, which is 0 both for |
| 297 |
an empty queue and for an empty datagram at its head — "UDP_RECV until |
| 298 |
`{3}`" is the idiom that tells them apart, and it is TCPIP_UDP_STATE's |
| 299 |
own advice (its datagram count must not be relied upon). |
| 300 |
* **UDP_OPEN**: local port 0xFFFF requests an ephemeral port; local port |
| 301 |
0 reaches the host `bind()` where it also yields an ephemeral port — |
| 302 |
the two coincide by different routes. A requested port the host |
| 303 |
refuses to bind falls back to an ephemeral one too, silently: the |
| 304 |
host OS itself may own the very port the MSX asks for — Windows' |
| 305 |
time service holds UDP 123, which is exactly what an SNTP client |
| 306 |
requests — a collision that cannot exist on the real-hardware UNAPI |
| 307 |
stacks this bridge stands in for, and a client's local port carries |
| 308 |
no meaning for the peer. (The first v2 build answered `{2}` there, |
| 309 |
in the name of honesty; functional testing vetoed it — SNTP broke on |
| 310 |
every Windows host whose time service was running — so v1's silent |
| 311 |
fallback is restored, and this time written down.) `{2}` remains the |
| 312 |
answer when the ephemeral bind fails as well. |
| 313 |
|
| 314 |
### ICMP |
| 315 |
|
| 316 |
* With caps bit4 clear, ICMP_SEND and ICMP_RECV answer `{1}` — v1 |
| 317 |
accepted the request, dropped it silently, and let ICMP_RECV report |
| 318 |
"no data" forever; the capability bit becomes enforced, not |
| 319 |
decorative. |
| 320 |
* The echo-size field keeps v1's clamp to 512. On a capable host a |
| 321 |
well-formed ICMP_SEND cannot fail: `{0}` acknowledges queueing to the |
| 322 |
worker, not delivery — an unreachable destination simply never yields |
| 323 |
a reply. |
| 324 |
* Replies queue FIFO, at most 16 deep; when the queue is full the |
| 325 |
**oldest** reply is dropped — a reply nobody polled for is worth less |
| 326 |
than the fresh one the current program is waiting for (v1 dropped the |
| 327 |
newest; that was code, not design). Each successful ICMP_RECV consumes |
| 328 |
exactly one. |
| 329 |
The device does not correlate replies with requests — matching |
| 330 |
identifier/sequence is the driver's job. |
| 331 |
|
| 332 |
### Miscellaneous |
| 333 |
|
| 334 |
* **GET_LOCALIP** answers 0.0.0.0 when the host lookup fails (still a |
| 335 |
success reply — v1 parity). On a multi-homed host, which address is |
| 336 |
reported is implementation-defined. |
| 337 |
* **NET_STATE** keeps its constant answer (`{0, 2}`); it exists because |
| 338 |
the driver calls it, and inventing semantics for it is not this |
| 339 |
redesign's job. |
| 340 |
* In `UnapiNetWire.hh`, every result struct gains `uint8_t status = 0;` |
| 341 |
as its first member — the hardcoding Wouter suggested: the correct |
| 342 |
value is baked into the type, `T{}` produces a valid success frame, |
| 343 |
and the golden-bytes tests verify it like any other field. |
| 344 |
|
| 345 |
## DNS |
| 346 |
|
| 347 |
`DNS_QUERY` — parameters: the hostname itself. The whole parameter |
| 348 |
block is the name; there is no terminator (the block length already |
| 349 |
delimits it — v1's NUL was redundant, as Wouter noted). The |
| 350 |
device applies no syntax or charset rules to the hostname — beyond the |
| 351 |
dotted-quad fast path below, the bytes go to the host resolver as-is. |
| 352 |
The two bounds are transport, not syntax: a block longer than |
| 353 |
**253 bytes** (the DNS presentation-form name limit) is "too long" under |
| 354 |
rule 3 → `{4}`, and a block with an **embedded NUL** is malformed → |
| 355 |
`{4}` — the host resolver API speaks C strings and cannot carry the |
| 356 |
name past the NUL, so no lookup could ever see it; rejecting the block |
| 357 |
beats resolving a silently truncated prefix. A block that |
| 358 |
parses as a strict dotted-quad (`a.b.c.d`, four decimal octets 0-255) |
| 359 |
resolves immediately → `{0, 1, ip4}`, and arms the sticky Complete state |
| 360 |
exactly as an asynchronous success does. Anything else starts the |
| 361 |
resolver thread → `{0, 0}`. An empty hostname (an empty block), one |
| 362 |
beyond the length cap, or one with an embedded NUL answers `{4}`; a |
| 363 |
lookup already running answers `{5}`; all of these leave the DNS state |
| 364 |
and any running lookup untouched. |
| 365 |
|
| 366 |
`DNS_STATUS` (no parameters) reports the lookup, and its reply is data, |
| 367 |
not a command error — a well-formed DNS_STATUS always succeeds: |
| 368 |
|
| 369 |
|
| 370 |
|
| 371 |
| {0, 0} | idle — no lookup since reset | |
| 372 |
| {0, 1} | in progress | |
| 373 |
| {0, 2, ip4} (6) | complete | |
| 374 |
| {0, 0xFF, sub} (3) | lookup failed — a success reply carrying bad news; `sub` = UNAPI DNS_S sub-error (today only 3, "no such host") | |
| 375 |
|
| 376 |
The completed/failed state is sticky until the next `DNS_QUERY` or a |
| 377 |
device reset, so the driver may re-read it freely. Keeping the failure |
| 378 |
inside a successful reply preserves rule 2 (*command* errors are |
| 379 |
single-byte) while still carrying the sub-code the driver's callers |
| 380 |
display to users. |
| 381 |
|
| 382 |
## Reset |
| 383 |
|
| 384 |
A device reset (power-on, MSX reset) restores the ground state: every |
| 385 |
TCP and UDP connection is closed and freed — resident ones included — |
| 386 |
all receive buffers, send queues, datagram queues and ICMP replies are |
| 387 |
discarded, any pending reply and accumulated parameter bytes are |
| 388 |
dropped, and the DNS state returns to idle. The ICMP capability stays as |
| 389 |
latched at device start; reset does not re-probe it. |
| 390 |
|
| 391 |
## Migration and failure analysis |
| 392 |
|
| 393 |
Both halves ship together; the four v1 users get the new `unapinet.com` |
| 394 |
with the new openMSX build. The interesting cases are the mismatches: |
| 395 |
|
| 396 |
* **v1 driver, v2 device.** The driver sends opcode 0x00 (its PING) and |
| 397 |
expects the register to read `STATUS_DATA` (2) and the data port to |
| 398 |
yield `0xAB`. On the v2 device the register read hits an unregistered |
| 399 |
port and returns `0xFF`, and the data port yields `0x00 0x55 ...`. Both |
| 400 |
checks fail on the first byte → the driver prints "extension not |
| 401 |
found" and exits. Nothing misbehaves quietly. |
| 402 |
* **v2 driver, v1 device.** DETECT goes out as opcode 0x00, which v1 |
| 403 |
treats as PING and answers `0xAB`. The v2 driver requires the first |
| 404 |
byte to be `0x00` — `0xAB` fails immediately. Same clean exit. |
| 405 |
* **No device.** Every read returns `0xFF` from the pull-ups, which v2 |
| 406 |
defines as never-valid at a status position. (This is why past-end |
| 407 |
reads also return `0xFF`: "absent device" and "desynchronized driver" |
| 408 |
become the same, safe, diagnosis.) |
| 409 |
|
| 410 |
One caveat for completeness: the analysis above covers detection, i.e. |
| 411 |
installation time — the only supported pairing path. A *resident* v1 TSR |
| 412 |
carried into a v2 device by other means (a savestate taken on an old |
| 413 |
openMSX and restored on a new one) is not served, but it fails safe: |
| 414 |
v1's per-call gate is `status register == 2`, and on a v2 device that |
| 415 |
read returns `0xFF` from the unregistered port, so every call reports |
| 416 |
failure instead of half-working. Re-running the driver's installer is |
| 417 |
the answer; the spec simply does not promise more. |
| 418 |
|
| 419 |
## How the open points resolved |
| 420 |
|
| 421 |
The proposal closed with three points for review; for the record: |
| 422 |
|
| 423 |
1. The magic value (0x55) and the capability bit assignments stood |
| 424 |
unchallenged, and are final. |
| 425 |
2. NET_STATE keeps its constant answer. Real semantics — or retirement — |
| 426 |
remain a question for after v2, deliberately not bundled into this |
| 427 |
redesign. |
| 428 |
3. The err column and the validation order became the contract the TSR |
| 429 |
was written against, as intended, and the golden-byte tests pin the |
| 430 |
layout of every reply struct (the two-byte replies DNS and NET_STATE |
| 431 |
assemble inline are covered by the spec table alone). The one place |
| 432 |
where reality pushed back is recorded above: UDP_OPEN's ephemeral |
| 433 |
fallback. |
| 434 |
|
| 435 |
--- |
| 436 |
|
| 437 |
## Appendix — the v1 protocol (historical) |
| 438 |
|
| 439 |
The v1 document follows — verbatim below its old title, headings demoted |
| 440 |
one level — as it stood the day v2 replaced it. It stays authoritative |
| 441 |
for the parameter-block layouts, which v2 did not move (DNS_QUERY's |
| 442 |
dropped terminator is the one exception), and its closing catalogue of |
| 443 |
v1 inconsistencies is the list this redesign set out to fix. Where it |
| 444 |
contradicts v2 — past-end reads, the status register, the detection |
| 445 |
handshake, the error conventions — everything above supersedes it. Its |
| 446 |
code references describe the v1-era sources. |
| 447 |
|
| 448 |
This document describes the private wire protocol between the Z80 MSX-UNAPI |
| 449 |
TCP/IP driver (`unapinet.com`) and the `UnapiNet` device in openMSX. It is |
| 450 |
*not* the MSX-UNAPI specification: UNAPI defines what MSX software sees when |
| 451 |
it calls the driver; everything below is how the driver talks to the bridge |
| 452 |
behind the scenes. The driver and the device are developed and released |
| 453 |
together - the compatibility notes below rely on that - but only the device |
| 454 |
lives in the openMSX tree. |
| 455 |
|
| 456 |
The C++ implementation is `src/network/UnapiNet.cc`; the byte-exact layouts |
| 457 |
of every parameter and result block are the structs in |
| 458 |
`src/network/UnapiNetWire.hh`. Where a struct name appears below, that |
| 459 |
struct *is* the layout. |
| 460 |
|
| 461 |
Protocol version described here: **v1** (bridge version byte 4, as reported |
| 462 |
by `QUERY_CAP`). Known inconsistencies of v1, and the plan to fix them in |
| 463 |
v2, are collected at the end. |
| 464 |
|
| 465 |
### I/O ports |
| 466 |
|
| 467 |
The device claims two consecutive I/O ports, base 0x28 (`share/extensions/ |
| 468 |
unapinet.xml`, `<io base="0x28" num="2"/>`). Decoding is on bit 0 of the |
| 469 |
port number: |
| 470 |
|
| 471 |
|
| 472 |
|
| 473 |
| 0x28 | command (opcode) | status register | |
| 474 |
| 0x29 | parameter byte | result byte (consuming) | |
| 475 |
|
| 476 |
### Transaction model |
| 477 |
|
| 478 |
Every operation is one synchronous transaction: |
| 479 |
|
| 480 |
1. The MSX writes the parameter block, one byte at a time, to port 0x29. |
| 481 |
2. The MSX writes the command opcode to port 0x28. The device processes the |
| 482 |
command *inside that I/O write*: by the time the OUT instruction |
| 483 |
completes, the result is ready. The MSX never has to wait or poll for |
| 484 |
command completion. (Operations that take real time - DNS lookups, |
| 485 |
connection establishment, ICMP echo - return immediately and are polled |
| 486 |
with their own dedicated commands; see "Asynchronous operations".) |
| 487 |
3. The MSX reads the status register (port 0x28). `0x02` (`STATUS_DATA`) |
| 488 |
means the result is ready to be read. |
| 489 |
4. The MSX reads the result, one byte per IN, from port 0x29. Each read |
| 490 |
consumes one byte; reading past the end returns 0x00. |
| 491 |
|
| 492 |
Bookkeeping rules, all enforced by the device: |
| 493 |
|
| 494 |
* The parameter buffer is always cleared after a command executes, whether |
| 495 |
it succeeded or not. |
| 496 |
* Writing a parameter byte while a result is still pending discards the |
| 497 |
unread result (a new transaction implicitly abandons the old one). This is |
| 498 |
the recovery path if the MSX program is interrupted mid-read: the next |
| 499 |
transaction starts clean. |
| 500 |
* The parameter buffer is capped (64 KiB + 16). Excess bytes are dropped; |
| 501 |
the command's own size check then reports an error. A runaway MSX program |
| 502 |
cannot exhaust host memory. |
| 503 |
|
| 504 |
#### Status register values |
| 505 |
|
| 506 |
|
| 507 |
|
| 508 |
| 0x00 | STATUS_OK | idle; no result pending | |
| 509 |
| 0x01 | STATUS_ERROR | protocol-level error (see below) | |
| 510 |
| 0x02 | STATUS_DATA | a result is pending on the data port | |
| 511 |
|
| 512 |
`STATUS_ERROR` is raised in exactly three situations: unknown opcode, |
| 513 |
`DNS_QUERY` with an empty host name, and `DNS_QUERY` while a lookup is |
| 514 |
already running. Every other error is reported *inside* a normal result |
| 515 |
block. In practice the driver only ever performs one check on this |
| 516 |
register: "did the command I just issued produce a result?" |
| 517 |
(`status == STATUS_DATA`). |
| 518 |
|
| 519 |
### Detection handshake |
| 520 |
|
| 521 |
The driver detects the device at installation time: |
| 522 |
|
| 523 |
1. `PING` (0x00, no parameters) must answer `STATUS_DATA` with the single |
| 524 |
magic byte **0xAB**. Anything else means the extension is not present. |
| 525 |
2. `QUERY_CAP` (0x10, no parameters) answers two bytes: a capability |
| 526 |
summary (0x0F = PING + DNS + TCP + UDP) and the bridge version (4). |
| 527 |
|
| 528 |
*v1 quirk:* the current driver only performs step 1; nothing reads the |
| 529 |
version byte today. |
| 530 |
|
| 531 |
### Data types |
| 532 |
|
| 533 |
* **IPv4 address** - 4 bytes, the octets of `a.b.c.d` in order |
| 534 |
(network/big-endian; `Endian::UA_B32` in the structs). |
| 535 |
* **port / length / id / seq** - 2 bytes, low byte first (little-endian, |
| 536 |
Z80-natural; `Endian::UA_L16`). |
| 537 |
* **handle** - 1 byte. TCP and UDP handles are independent ranges, both |
| 538 |
1-based on the wire: 1..4 (`MAX_TCP` = `MAX_UDP` = 4). In results, handle |
| 539 |
0 means "could not open". |
| 540 |
|
| 541 |
### Command reference |
| 542 |
|
| 543 |
|
| 544 |
|
| 545 |
| 0x00 | PING | none | 1 byte: 0xAB | |
| 546 |
| 0x01 | DNS_QUERY | hostname bytes, NUL-terminated | see "Asynchronous operations" | |
| 547 |
| 0x02 | DNS_STATUS | none | see "Asynchronous operations" | |
| 548 |
| 0x03 | TCP_OPEN | `TcpOpenParams` (11) | 1 byte: handle, 0 = failed | |
| 549 |
| 0x04 | TCP_SEND | `TcpSendParamHeader` (3) + payload | 1 byte: 0 = accepted, 1 = error, 2 = send buffer full | |
| 550 |
| 0x05 | TCP_RECV | `TcpRecvParams` (3) | `TcpRecvResultHeader` (2) + payload | |
| 551 |
| 0x06 | TCP_CLOSE | 1 byte: handle (0 = all transient) | 1 byte: 0 = OK, 1 = error | |
| 552 |
| 0x07 | TCP_STATE | 1 byte: handle | `TcpStateResult` (12) | |
| 553 |
| 0x08 | TCP_ABORT | 1 byte: handle | 1 byte: 0 = OK, 1 = error | |
| 554 |
| 0x09 | UDP_OPEN | `UdpOpenParams` (2) | 1 byte: handle, 0 = failed | |
| 555 |
| 0x0A | UDP_CLOSE | 1 byte: handle (0 = all transient) | 1 byte: 0 = OK, 1 = error | |
| 556 |
| 0x0B | UDP_STATE | 1 byte: handle | `UdpStateResult` (2) | |
| 557 |
| 0x0C | UDP_SEND | `UdpSendParamHeader` (9) + payload | 1 byte: 0 = OK, 1 = error | |
| 558 |
| 0x0D | GET_LOCALIP | none | 4 bytes: host IPv4 | |
| 559 |
| 0x0E | NET_STATE | none | 1 byte: always 2 ("open") | |
| 560 |
| 0x0F | UDP_RECV | `UdpRecvParams` (3) | `UdpRecvResultHeader` (8) + payload | |
| 561 |
| 0x10 | QUERY_CAP | none | 2 bytes: caps (0x0F), version (4) | |
| 562 |
| 0x11 | ICMP_SEND | `IcmpSendParams` (11) | 1 byte: 0 = queued, 1 = error | |
| 563 |
| 0x12 | ICMP_RECV | none | 1 byte: 0 = none, or `IcmpRecvResult` (12) | |
| 564 |
|
| 565 |
Any other opcode raises `STATUS_ERROR`. |
| 566 |
|
| 567 |
#### TCP |
| 568 |
|
| 569 |
**TCP_OPEN** - `TcpOpenParams`: remote IP (4), remote port (2), local |
| 570 |
port (2), timeout (2, *reserved: never read*), flags (1; bit 0 = passive |
| 571 |
open, bit 1 = resident). An active open starts a non-blocking `connect()` |
| 572 |
and returns the handle immediately; the driver polls `TCP_STATE` until the |
| 573 |
state leaves SynSent. A passive open (bit 0) listens on the local port; |
| 574 |
remote IP, if non-zero, filters which peer may connect. |
| 575 |
|
| 576 |
**TCP_SEND** - header {handle (1), length (2)} followed by exactly `length` |
| 577 |
payload bytes. The driver keeps to 4096 bytes (`MAX_TRANSFER`) per command |
| 578 |
by convention, but the bridge does not enforce that bound on the send side |
| 579 |
(a v1 quirk; only TCP_RECV clamps to it). Result 2 ("send buffer full") is |
| 580 |
a recoverable condition: the bridge queues up to 64 KiB (`MAX_SEND_BUF`) |
| 581 |
that the kernel has not yet accepted, and the driver should retry later. |
| 582 |
Result 1 is a real failure (bad handle/state/length, or a connection error |
| 583 |
while queueing). |
| 584 |
|
| 585 |
**TCP_RECV** - {handle (1), maxlen (2)}. Answers {actualLen (2)} followed |
| 586 |
by `actualLen` bytes taken from the connection's receive buffer, at most |
| 587 |
4096 per command. `actualLen` 0 means "nothing buffered right now" - poll |
| 588 |
`TCP_STATE` to distinguish an empty buffer on a live connection from a |
| 589 |
finished one. *v1 quirk:* a bad handle also answers `actualLen` 0. |
| 590 |
|
| 591 |
**TCP_STATE** - answers `TcpStateResult`: state (1), bytes available to |
| 592 |
read (2), close reason (1), remote IP (4), remote port (2), local |
| 593 |
port (2). An invalid or missing handle answers all-zero fields with |
| 594 |
closeReason = 1 (NeverUsed). |
| 595 |
|
| 596 |
State values (TCP-standard names): |
| 597 |
0 Closed, 1 Listen, 2 SynSent, 3 SynRecv, 4 Established, 5 FinWait1, |
| 598 |
6 FinWait2, 7 CloseWait, 8 Closing, 9 LastAck, 10 TimeWait. |
| 599 |
|
| 600 |
Close reasons: 0 None, 1 NeverUsed, 2 ClosedByUser, 3 Aborted, |
| 601 |
4 ConnectionReset, 6 ConnectFailed. |
| 602 |
|
| 603 |
**TCP_CLOSE vs TCP_ABORT** - CLOSE is graceful: the connection enters |
| 604 |
FinWait1, the FIN goes out once everything the MSX queued has actually |
| 605 |
been sent, and the handle is freed when the peer closes too - or after a |
| 606 |
30 s timeout, so a silent peer cannot pin a handle forever. Received data |
| 607 |
still buffered stays readable meanwhile. ABORT drops the connection |
| 608 |
immediately. Handle 0 in TCP_CLOSE (and UDP_CLOSE) closes every open |
| 609 |
non-resident connection at once - the driver uses it on program exit; |
| 610 |
resident connections (TCP_OPEN flags bit 1) survive it. |
| 611 |
|
| 612 |
#### UDP |
| 613 |
|
| 614 |
**UDP_OPEN** - local port (2), 0xFFFF = ephemeral. **UDP_STATE** answers |
| 615 |
the size of the first queued datagram (2 bytes; 0 = queue empty, or bad |
| 616 |
handle - same v1 quirk as TCP_RECV). **UDP_SEND** - header {handle (1), |
| 617 |
destination IP (4), destination port (2), length (2)} + payload; result 1 |
| 618 |
also covers a short kernel write. **UDP_RECV** - {handle (1), maxlen (2)}; |
| 619 |
answers {source IP (4), source port (2), actualLen (2)} + payload. |
| 620 |
Datagram semantics apply: if the first queued datagram is larger than |
| 621 |
`maxlen`, the excess of *that datagram* is discarded. |
| 622 |
|
| 623 |
#### ICMP |
| 624 |
|
| 625 |
**ICMP_SEND** - {destination IP (4), TTL (1), identifier (2), |
| 626 |
sequence (2), size (2, clamped to 512)}. No payload follows: the bridge |
| 627 |
fabricates the echo data itself. The request is queued to a worker thread; |
| 628 |
the result byte only acknowledges the queueing. **ICMP_RECV** polls for |
| 629 |
replies: 1 byte 0 = none yet, otherwise {1, source IP (4), TTL (1), |
| 630 |
identifier (2), sequence (2), dataLen (2)} - header only, the echo payload |
| 631 |
is not returned. *Current limitation:* the ICMP worker is implemented with |
| 632 |
the Windows ICMP API; on other host platforms ICMP_RECV never reports |
| 633 |
data. |
| 634 |
|
| 635 |
### Asynchronous operations |
| 636 |
|
| 637 |
**DNS.** `DNS_QUERY` takes the hostname as its parameter block, |
| 638 |
NUL-terminated (the terminator and anything after it are ignored). If the |
| 639 |
name parses as a dotted-quad IP it answers immediately with 5 bytes |
| 640 |
{1, ip}. Otherwise it starts a resolver thread and answers a single byte 0 |
| 641 |
("started"); the driver then polls `DNS_STATUS`, which answers one byte |
| 642 |
0/1 (idle / in progress), 5 bytes {2, ip} on success, or 2 bytes |
| 643 |
{0xFF, 3} on failure (the only failure reported is "no such host", UNAPI |
| 644 |
error 3). The completed status is sticky: it stays readable until the next |
| 645 |
`DNS_QUERY` or a device reset. Issuing `DNS_QUERY` while a lookup is |
| 646 |
running raises `STATUS_ERROR` without disturbing the running lookup. |
| 647 |
|
| 648 |
**TCP connect** is polled via `TCP_STATE` (SynSent → Established or |
| 649 |
Closed + ConnectFailed). |
| 650 |
|
| 651 |
**Receive path.** A background thread moves incoming TCP data into a |
| 652 |
per-connection buffer of up to 64 KiB (`MAX_RECV_BUF`), from which |
| 653 |
`TCP_RECV` reads. When the buffer is full the bridge stops reading from |
| 654 |
the socket, the kernel buffer fills, and TCP flow control throttles the |
| 655 |
peer - no TCP data is ever dropped. UDP is best-effort, as datagrams are: |
| 656 |
they are read into a 2 KiB buffer (larger ones are truncated at receive |
| 657 |
time) and queued per socket up to 16 deep; beyond that, new arrivals are |
| 658 |
read and discarded. |
| 659 |
|
| 660 |
### Known v1 inconsistencies, and the v2 plan |
| 661 |
|
| 662 |
These grew one command at a time and are preserved here so the next reader |
| 663 |
does not have to rediscover them: |
| 664 |
|
| 665 |
1. **Three error conventions coexist.** OPEN commands answer a handle |
| 666 |
where 0 = failure; most other commands answer 0 = OK / 1 = error (and |
| 667 |
TCP_SEND adds 2 = buffer full); `STATUS_ERROR` covers only the three |
| 668 |
protocol-level cases listed above. |
| 669 |
2. **"No data" and "bad handle" are indistinguishable** in TCP_RECV, |
| 670 |
UDP_STATE and UDP_RECV. |
| 671 |
3. **Parameter blocks are validated as "at least this long"**, not exactly; |
| 672 |
trailing garbage is silently accepted. |
| 673 |
4. **`QUERY_CAP` is implemented but never issued** by the driver, so the |
| 674 |
version byte is dead on the wire. |
| 675 |
5. **`NET_STATE` answers a constant.** |
| 676 |
6. **`TcpOpenParams.timeout` is never read.** |
| 677 |
7. **`MAX_TRANSFER` is enforced on TCP_RECV but not on TCP_SEND** - the |
| 678 |
4096-byte bound on sends is a driver-side convention the bridge |
| 679 |
silently trusts. |
| 680 |
|
| 681 |
The planned v2, coordinated with the driver (both halves ship from the |
| 682 |
same repository): every reply starts with a status byte (0 = OK, otherwise |
| 683 |
the UNAPI error code the driver should return, `ERR_BUFFER` included); |
| 684 |
handles stop doubling as error codes; parameter blocks are validated for |
| 685 |
exact size; the PING magic is bumped so an old driver fails detection |
| 686 |
cleanly instead of misbehaving quietly; and the driver reads `QUERY_CAP` |
| 687 |
for real. With a status byte on every reply the status register becomes |
| 688 |
redundant (the driver's only use of it is the `== STATUS_DATA` check after |
| 689 |
each command); whether to repurpose or retire it is an open design point. |