OXYGENHUB
◆ main openmsx / src / network / unapinet_protocol-v2.md
37.9 KiB · Markdown · last changed chore(): initial commit 1 month ago renderedsource blame history raw
e8555c89 Andy McCall 1
# UnapiNet bridge protocol v2
2
3
Status: **implemented — this is the reference.** This document was agreed
4
as a proposal before it turned into code; what follows is the protocol
5
the device and the driver (`unapinet.com`, Z80) actually speak, and the
6
golden-byte tests pin every reply struct's layout to it. As promised while it was
7
under review, the v1 document it replaces is folded in below as a
8
historical appendix — still the byte-level reference for the
9
parameter-block layouts, which v2 keeps. Functional testing added one
10
clause the proposal had left unstated: UDP_OPEN falls back to an
11
ephemeral port when the host refuses the requested one (see UDP below).
12
13
The driver and this device ship from the same source project and are
14
updated together. v1 has four users, all personally reachable; there is
15
no installed base to migrate. v2 is therefore a clean break, with
16
detection designed so that any old/new mismatch fails cleanly at
17
installation time (analysis, and one caveat, under "Migration and
18
failure analysis" below).
19
20
## Design rules
21
22
v1's problems are catalogued at the end of the v1 appendix; they reduce to
23
one sentence: *every command grew its own conventions*. v2 replaces them
24
with four rules:
25
26
1. **Every reply begins with a status byte.** `0x00` = success. Any other
27
   value is the MSX-UNAPI TCP/IP error code the driver should hand to its
28
   caller, verbatim. The device speaks UNAPI error codes natively; the
29
   driver stops translating.
30
2. **An error reply is exactly one byte** — the status. No partial
31
   payloads: if the status is non-zero, there is nothing else to read.
32
3. **Parameter blocks are validated for exact size.** Too short, too long,
33
   or inconsistent with a declared payload length → `ERR_INV_PARAM`, and
34
   the command has no effect.
35
4. **No valid status byte is 0xFF.** Reading the data port past the end
36
   of a reply (or with no reply pending) returns `0xFF` — the same value
37
   an absent device yields from the bus pull-ups, and never a valid
38
   status (success is `0x00`, and no UNAPI error code is `0xFF`). A
39
   driver reading a status position and seeing `0xFF` knows it is
40
   desynchronized or talking to nothing — never "OK". (v1 returned `0x00`
41
   there, which is indistinguishable from success — Wouter's observation.)
42
   Bytes *inside* a reply are length-delimited and unconstrained: an IP
43
   octet, a payload byte or the DNS failed-lookup marker may legitimately
44
   be `0xFF`.
45
46
### Error codes on the wire
47
48
The status byte uses the MSX-UNAPI TCP/IP error space. The device emits
49
this subset (the values match the error table the driver already defines;
50
the device side is what v2 added):
51
52
| Code | Name            | Emitted when                                    |
53
|------|-----------------|-------------------------------------------------|
54
| 0    | ERR_OK          | success                                          |
55
| 1    | ERR_NOT_IMP     | unknown opcode; the ICMP commands on a host without working ICMP (caps bit4 clear) |
56
| 2    | ERR_NO_NETWORK  | the host socket layer refused: socket()/bind()/listen() failure, a synchronous connect() failure, a failed or short sendto() (UDP) |
57
| 3    | ERR_NO_DATA     | UDP_RECV / ICMP_RECV with an empty queue         |
58
| 4    | ERR_INV_PARAM   | malformed parameter block (rule 3), empty hostname, one longer than 253 bytes or one with an embedded NUL, a TCP_SEND length beyond `MAX_TRANSFER`, an undefined flag bit set |
59
| 5    | ERR_QUERY_EXISTS| DNS_QUERY while a lookup is already running      |
60
| 9    | ERR_NO_FREE_CONN| TCP_OPEN / UDP_OPEN with all handles in use      |
61
| 11   | ERR_NO_CONN     | handle out of range (0 included, save the close-all forms of TCP_CLOSE / UDP_CLOSE / TCP_ABORT); or no open connection on it — except TCP_STATE / TCP_RECV, which answer for any in-range handle (see notes) |
62
| 12   | ERR_CONN_STATE  | a connection exists but its state forbids the command — including a TCP send whose socket write fails, which resets the connection and answers 12 |
63
| 13   | ERR_BUFFER      | TCP_SEND with the 64 KiB send queue full (recoverable: retry later; nothing is partially queued) |
64
65
Rule 3 applies to **every** opcode, parameterless ones included: a
66
malformed or stray parameter block answers `{4}` regardless of the
67
command, and the per-command "err" lists below do not repeat that. Where
68
a list names 4 explicitly, a *well-formed* block can still be
69
semantically invalid there. One check precedes rule 3: opcode dispatch —
70
an unknown opcode answers `{1}` whether or not stray parameter bytes
71
accompany it (there is no expected size to validate against). In both
72
cases the parameter buffer is cleared, as after every command.
73
74
Validation order is fixed: form first (rule 3, `{4}`, together with the
75
semantic `{4}`s — an invalid value inside a well-formed block); then
76
capability (`{1}`, the ICMP commands); then handle (`{11}`); then
77
connection state (`{12}`, and DNS's busy `{5}`); then the data check
78
(`{3}`, the RECV queues). The host operation runs last, so its errors
79
(`{2}`, `{13}` — plus the TCP write-failure `{12}`, the one
80
post-operation use of that code) imply every earlier check passed. In
81
the OPEN commands the free-slot check (`{9}`) precedes the host socket
82
work (`{2}`).
83
84
### The status register is gone — the command port is write-only
85
86
The driver's only use of the v1 status register is "did my command produce
87
a result?" — with rule 1 that information is the reply's first byte, so
88
the register is redundant as a mechanism, and v2 removes it entirely
89
rather than keeping a mirror nobody needs (Wouter's suggestion). The
90
device registers the command port for writes only; reading it decodes to
91
the open bus (`0xFF` on a standard MSX), exactly as if no device were
92
present. The status byte lives in one place — the reply — so there is
93
nothing to poll, nothing that can disagree with it, and nothing extra to
94
document. The three v1 situations that raised `STATUS_ERROR` in the
95
register become ordinary single-byte error replies (unknown opcode → 1,
96
empty hostname → 4, DNS busy → 5).
97
98
## What does not change — and two things that do
99
100
Unchanged from v1: ports and decoding (base 0x28: command, base+1:
101
parameter/result — the command port's read side is gone, see above), the
102
synchronous transaction model, parameter
103
accumulation, the parameter-buffer cap, opcode numbers (except 0x10,
104
retired — below), parameter block layouts, handles (1-based, 1..4, TCP
105
and UDP independent), `handle 0 = close all transient` in the CLOSE
106
commands (v2 extends the same form to TCP_ABORT — see the handles
107
notes), MAX_TRANSFER/MAX_RECV_BUF/MAX_SEND_BUF, the graceful-close
108
semantics, and the receive-side backpressure. Byte order stays as it is:
109
IPs as network-order octets, 16-bit quantities little-endian.
110
111
Two v1 statements are explicitly superseded — a reader must NOT carry
112
them over from the v1 document:
113
114
* v1's transaction model says reading past the end returns `0x00` and
115
  gates each command on `status register == STATUS_DATA (2)`. In v2,
116
  past-end reads return `0xFF` (rule 4) and the register does not exist —
117
  the port reads as open bus (`0xFF`), so a v1-style `== 2` gate can
118
  never pass.
119
* v1's DNS_QUERY takes a NUL-terminated name (terminator effectively
120
  optional, trailing bytes ignored). v2 drops the terminator entirely:
121
  the parameter block *is* the hostname. This is the only change at the
122
  block-form level; no other layout moved. (One *value* check is also
123
  new — TCP_OPEN's undefined flag bits answer `{4}` where v1 ignored
124
  them. And one value gains meaning: v1 destroyed a datagram unread on
125
  UDP_RECV maxlen 0; v2 makes that TCPIP_UDP_RCV's deliberate DE=0
126
  discard, reporting the source and size while copying nothing.)
127
128
The recovery rule is kept and completed: writing a **parameter byte**
129
while a result is pending discards the unread result (as in v1), and
130
writing a **command opcode** while a result is pending abandons it too —
131
the new reply replaces it. A driver never has to drain a reply it has
132
lost interest in.
133
134
## DETECT (0x00) — replaces PING and QUERY_CAP
135
136
Wouter suggested merging the two, and the merge solves three things at
137
once: the confusing "PING" name (it is not ICMP), the version byte nobody
138
reads, and the magic bump v2 needs anyway.
139
140
`DETECT`, opcode 0x00, no parameters. Success reply, 5 bytes:
141
142
| Offset | Value                                              |
143
|--------|----------------------------------------------------|
144
| 0      | status 0x00                                        |
145
| 1      | magic **0x55** (was 0xAB in v1)                    |
146
| 2      | protocol version: **2**                            |
147
| 3      | capabilities: bit0 DNS, bit1 TCP active, bit2 TCP passive, bit3 UDP, bit4 ICMP, rest reserved 0 |
148
| 4      | reserved 0                                         |
149
150
The driver accepts the device if and only if bytes 0-2 are exactly
151
`00 55 02`. Bytes 3-4 are data, not part of the acceptance rule: a driver
152
must tolerate unknown capability bits (and any value in byte 4), or it
153
will reject future devices for advertising more. Byte 2 is the upgrade
154
path: a hypothetical v3 reports 3 there, a v2 driver rejects it cleanly,
155
and a v3 driver may accept both if v3 is a superset.
156
157
Capabilities are honest and enforced: bit4 (ICMP) is set only when the
158
host can actually ping — platform support compiled in *and* the ICMP
159
channel opening successfully at device start (today that means: Windows,
160
and `IcmpCreateFile()` succeeded); it is latched once at start. If bit4
161
is clear, the ICMP commands answer `ERR_NOT_IMP`. Bits 0-3 are always set
162
on this device today (byte 3 reads 0x0F or 0x1F); they are informational,
163
and no command is gated on them.
164
165
One corner is worth a recipe: a crashed predecessor may have left stray
166
parameter bytes buffered, and a strict DETECT would then answer `{4}` —
167
yet DETECT is also the stream-resynchronization command, so it must work
168
from any state. Wouter's resolution: keep DETECT strict, and have the
169
driver issue it **twice unconditionally**, reading only the second
170
reply. The first either succeeds and is abandoned (issuing a command
171
discards a pending result — the recovery rule) or answers `{4}` and
172
clears the stray block; either way the second DETECT runs on a clean
173
buffer. No conditional on either side.
174
175
Opcode 0x10 (QUERY_CAP) is retired and answers `ERR_NOT_IMP` like any
176
other unknown opcode.
177
178
## Command reference (v2 replies)
179
180
Parameter layouts are unchanged from v1 (see the v1 appendix for byte
181
detail); this table gives the v2 reply, status byte included. Where a row
182
shows `{0, Struct}`, the v2 wire struct is the v1 struct with the status
183
byte **prepended** as its new first member — sizes grow by one, and the
184
count shown already includes it — except ICMP_RECV, where the status
185
**replaces** v1's `hasData` marker byte (still 12). "err:" lists the
186
codes each command can emit besides 0 and besides the universal `{4}` of
187
rule 3.
188
189
| Op   | Name        | Success reply (bytes)                    | err          |
190
|------|-------------|------------------------------------------|--------------|
191
| 0x00 | DETECT      | {0, 0x55, 2, caps, 0} (5)                | —            |
192
| 0x01 | DNS_QUERY   | {0, 0} async started, or {0, 1, ip4} (6) resolved immediately | 4, 5 |
193
| 0x02 | DNS_STATUS  | {0, state, ...} — see DNS below          | —            |
194
| 0x03 | TCP_OPEN    | {0, handle} (2)                          | 2, 4, 9      |
195
| 0x04 | TCP_SEND    | {0} (1)                                  | 4, 11, 12, 13|
196
| 0x05 | TCP_RECV    | {0, len.lo, len.hi, payload...} (3+len)  | 11           |
197
| 0x06 | TCP_CLOSE   | {0} (1)                                  | 11           |
198
| 0x07 | TCP_STATE   | {0, TcpStateResult} (13)                 | 11           |
199
| 0x08 | TCP_ABORT   | {0} (1)                                  | 11           |
200
| 0x09 | UDP_OPEN    | {0, handle} (2)                          | 2, 9         |
201
| 0x0A | UDP_CLOSE   | {0} (1)                                  | 11           |
202
| 0x0B | UDP_STATE   | {0, size.lo, size.hi} (3)                | 11           |
203
| 0x0C | UDP_SEND    | {0} (1)                                  | 2, 11        |
204
| 0x0D | GET_LOCALIP | {0, ip4} (5)                             | —            |
205
| 0x0E | NET_STATE   | {0, 2} (2)                               | —            |
206
| 0x0F | UDP_RECV    | {0, srcip4, port.2, len.2, payload} (9 + min(len, maxlen)) | 3, 11 |
207
| 0x11 | ICMP_SEND   | {0} (1)                                  | 1            |
208
| 0x12 | ICMP_RECV   | {0, srcip4, ttl, id.2, seq.2, len.2} (12)| 1, 3         |
209
210
### Handles, slots, and what survives a close
211
212
* Handles are 1..4. **Handle 0 is out of range** (`{11}`) everywhere
213
  except TCP_CLOSE, UDP_CLOSE and TCP_ABORT, where it means "close (or
214
  abort) all transient" — and succeeds (`{0}`) even when nothing is
215
  open. The ABORT form is new in v2: UNAPI defines it (TCPIP_TCP_ABORT,
216
  B = 0), but v1 silently no-opped it — the device rejected handle 0 and
217
  the driver never looked at the reply, so the caller got ERR_OK with
218
  nothing aborted.
219
* `ERR_NO_CONN` (11) means the handle is out of range — or, for the
220
  commands that act on a live connection (SEND, CLOSE, ABORT, and all of
221
  UDP), that no open connection sits on it. **TCP_STATE and TCP_RECV
222
  answer for any in-range handle, open or closed**: a closed connection
223
  keeps its final state, its close reason and its undrained receive data
224
  until TCP_OPEN reuses the slot. That is deliberate v1 behavior, kept on
225
  purpose — the tail of a connection that died stays readable (TCP_ABORT
226
  included), and the driver can still see *why* it died. What 11 removes
227
  is v1's ambiguity: a zeroed struct or a zero length no longer doubles
228
  as "bad handle".
229
* A **never-used** in-range slot answers TCP_STATE with closeReason 1
230
  (NeverUsed) and every other field zero, and TCP_RECV with length 0 — same frame
231
  as v1, minus the bad-handle overload.
232
* Slot allocation: lowest-numbered free slot first. A closed slot holding
233
  a sticky tail **is** free — it does not count toward `{9}`, and reusing
234
  it destroys the tail. A slot consumed by a failed TCP_OPEN (`{2}`) is
235
  released, not left in ConnectFailed (that state belongs to the
236
  *asynchronous* connect path only).
237
238
### TCP
239
240
* **TCP_SEND** may send in Established and CloseWait; any other existing
241
  connection state answers `{12}`; no connection at all answers `{11}`.
242
  A length beyond `MAX_TRANSFER` (4096) answers `{4}`; length 0 is a
243
  valid no-op (`{0}`). `{13}` is all-or-nothing: nothing was queued, the
244
  same command may be retried verbatim later. A socket write failure
245
  resets the connection (closeReason 4, ConnectionReset) and answers
246
  `{12}`.
247
* **TCP_RECV**: `maxlen` is a ceiling request — a value beyond 4096 is
248
  clamped, not an error (contrast TCP_SEND, where length states a fact
249
  about the payload that follows and must be exact). `maxlen` 0 answers
250
  `{0, 0, 0}` and consumes nothing. Length 0 with a live connection is
251
  normal stream behavior; poll TCP_STATE to tell "live but idle" from
252
  "finished".
253
* **TCP_CLOSE** is graceful: FinWait1, FIN once everything queued has
254
  been sent, handle freed when the peer closes too — or after 30 s.
255
  Buffered received data stays readable throughout (and after, per the
256
  sticky-slot rule). Closing a Listen handle is valid and frees it. A
257
  second CLOSE while the close is in progress is idempotent (`{0}`).
258
  **TCP_ABORT** drops immediately (closeReason 3); handle 0 aborts
259
  every transient connection, mirroring UNAPI's TCPIP_TCP_ABORT.
260
* **TCP_OPEN**: the `timeout` field stays in the layout, is accepted with
261
  any value, and is ignored (reserved). Undefined flag bits (2-7) must be
262
  0; a set one answers `{4}`. Remote IP / port are not semantically
263
  validated — an active open to 0.0.0.0 or port 0 is handed to the host
264
  and fails there (`{2}` or, asynchronously, ConnectFailed). Passive
265
  open: a non-zero remote IP filters the peer; a non-matching peer is
266
  accepted and immediately dropped, and the listener keeps listening; on
267
  a match the handle *becomes* the connection (Listen → Established —
268
  the listener is gone, one connection per passive open).
269
270
### UDP
271
272
* **Every UDP socket is transient.** The UNAPI residency concept has no
273
  wire representation in UdpOpenParams, so UDP_CLOSE 0 closes every open
274
  UDP socket. (The device keeps an internal resident flag for future use;
275
  nothing can set it today.)
276
* **UDP_CLOSE discards the socket's queued datagrams** — unlike TCP,
277
  nothing UDP stays readable after close.
278
* **UDP_SEND**: an oversized datagram is not a form error — the host
279
  refuses it and the reply is `{2}` (as is a short kernel write). A
280
  failed send leaves the socket open and usable. Length 0 sends an empty
281
  datagram (`{0}`).
282
* **UDP_RECV**: the head datagram is consumed whole. `len` in the reply
283
  is the datagram's size **as received**, and the payload that follows
284
  is the first `min(len, maxlen)` bytes — the excess, if any, is
285
  discarded with the datagram. This is exactly TCPIP_UDP_RCV's contract:
286
  its BC output is "the datagram data size as it was received, which may
287
  be larger than the number of bytes actually retrieved", and its DE=0
288
  ("no data will be copied at all") is a *deliberate* discard — so
289
  `maxlen` 0 is valid: header only, datagram gone. Contrast TCP_RECV,
290
  where `maxlen` 0 consumes nothing — a stream has no unit to destroy,
291
  a datagram is all-or-nothing. An empty datagram (0 data bytes) is a
292
  normal datagram: UDP makes it visible to the receiver, so it queues
293
  and is delivered with `len` 0. Datagrams were already truncated to
294
  2 KiB at receive time and queue at most 16 deep (v1 behavior,
295
  unchanged).
296
* **UDP_STATE** answers the head datagram's size, which is 0 both for
297
  an empty queue and for an empty datagram at its head — "UDP_RECV until
298
  `{3}`" is the idiom that tells them apart, and it is TCPIP_UDP_STATE's
299
  own advice (its datagram count must not be relied upon).
300
* **UDP_OPEN**: local port 0xFFFF requests an ephemeral port; local port
301
  0 reaches the host `bind()` where it also yields an ephemeral port —
302
  the two coincide by different routes. A requested port the host
303
  refuses to bind falls back to an ephemeral one too, silently: the
304
  host OS itself may own the very port the MSX asks for — Windows'
305
  time service holds UDP 123, which is exactly what an SNTP client
306
  requests — a collision that cannot exist on the real-hardware UNAPI
307
  stacks this bridge stands in for, and a client's local port carries
308
  no meaning for the peer. (The first v2 build answered `{2}` there,
309
  in the name of honesty; functional testing vetoed it — SNTP broke on
310
  every Windows host whose time service was running — so v1's silent
311
  fallback is restored, and this time written down.) `{2}` remains the
312
  answer when the ephemeral bind fails as well.
313
314
### ICMP
315
316
* With caps bit4 clear, ICMP_SEND and ICMP_RECV answer `{1}` — v1
317
  accepted the request, dropped it silently, and let ICMP_RECV report
318
  "no data" forever; the capability bit becomes enforced, not
319
  decorative.
320
* The echo-size field keeps v1's clamp to 512. On a capable host a
321
  well-formed ICMP_SEND cannot fail: `{0}` acknowledges queueing to the
322
  worker, not delivery — an unreachable destination simply never yields
323
  a reply.
324
* Replies queue FIFO, at most 16 deep; when the queue is full the
325
  **oldest** reply is dropped — a reply nobody polled for is worth less
326
  than the fresh one the current program is waiting for (v1 dropped the
327
  newest; that was code, not design). Each successful ICMP_RECV consumes
328
  exactly one.
329
  The device does not correlate replies with requests — matching
330
  identifier/sequence is the driver's job.
331
332
### Miscellaneous
333
334
* **GET_LOCALIP** answers 0.0.0.0 when the host lookup fails (still a
335
  success reply — v1 parity). On a multi-homed host, which address is
336
  reported is implementation-defined.
337
* **NET_STATE** keeps its constant answer (`{0, 2}`); it exists because
338
  the driver calls it, and inventing semantics for it is not this
339
  redesign's job.
340
* In `UnapiNetWire.hh`, every result struct gains `uint8_t status = 0;`
341
  as its first member — the hardcoding Wouter suggested: the correct
342
  value is baked into the type, `T{}` produces a valid success frame,
343
  and the golden-bytes tests verify it like any other field.
344
345
## DNS
346
347
`DNS_QUERY` — parameters: the hostname itself. The whole parameter
348
block is the name; there is no terminator (the block length already
349
delimits it — v1's NUL was redundant, as Wouter noted). The
350
device applies no syntax or charset rules to the hostname — beyond the
351
dotted-quad fast path below, the bytes go to the host resolver as-is.
352
The two bounds are transport, not syntax: a block longer than
353
**253 bytes** (the DNS presentation-form name limit) is "too long" under
354
rule 3 → `{4}`, and a block with an **embedded NUL** is malformed →
355
`{4}` — the host resolver API speaks C strings and cannot carry the
356
name past the NUL, so no lookup could ever see it; rejecting the block
357
beats resolving a silently truncated prefix. A block that
358
parses as a strict dotted-quad (`a.b.c.d`, four decimal octets 0-255)
359
resolves immediately → `{0, 1, ip4}`, and arms the sticky Complete state
360
exactly as an asynchronous success does. Anything else starts the
361
resolver thread → `{0, 0}`. An empty hostname (an empty block), one
362
beyond the length cap, or one with an embedded NUL answers `{4}`; a
363
lookup already running answers `{5}`; all of these leave the DNS state
364
and any running lookup untouched.
365
366
`DNS_STATUS` (no parameters) reports the lookup, and its reply is data,
367
not a command error — a well-formed DNS_STATUS always succeeds:
368
369
| Reply              | Meaning                                   |
370
|--------------------|-------------------------------------------|
371
| {0, 0}             | idle — no lookup since reset              |
372
| {0, 1}             | in progress                               |
373
| {0, 2, ip4} (6)    | complete                                  |
374
| {0, 0xFF, sub} (3) | lookup failed — a success reply carrying bad news; `sub` = UNAPI DNS_S sub-error (today only 3, "no such host") |
375
376
The completed/failed state is sticky until the next `DNS_QUERY` or a
377
device reset, so the driver may re-read it freely. Keeping the failure
378
inside a successful reply preserves rule 2 (*command* errors are
379
single-byte) while still carrying the sub-code the driver's callers
380
display to users.
381
382
## Reset
383
384
A device reset (power-on, MSX reset) restores the ground state: every
385
TCP and UDP connection is closed and freed — resident ones included —
386
all receive buffers, send queues, datagram queues and ICMP replies are
387
discarded, any pending reply and accumulated parameter bytes are
388
dropped, and the DNS state returns to idle. The ICMP capability stays as
389
latched at device start; reset does not re-probe it.
390
391
## Migration and failure analysis
392
393
Both halves ship together; the four v1 users get the new `unapinet.com`
394
with the new openMSX build. The interesting cases are the mismatches:
395
396
* **v1 driver, v2 device.** The driver sends opcode 0x00 (its PING) and
397
  expects the register to read `STATUS_DATA` (2) and the data port to
398
  yield `0xAB`. On the v2 device the register read hits an unregistered
399
  port and returns `0xFF`, and the data port yields `0x00 0x55 ...`. Both
400
  checks fail on the first byte → the driver prints "extension not
401
  found" and exits. Nothing misbehaves quietly.
402
* **v2 driver, v1 device.** DETECT goes out as opcode 0x00, which v1
403
  treats as PING and answers `0xAB`. The v2 driver requires the first
404
  byte to be `0x00` — `0xAB` fails immediately. Same clean exit.
405
* **No device.** Every read returns `0xFF` from the pull-ups, which v2
406
  defines as never-valid at a status position. (This is why past-end
407
  reads also return `0xFF`: "absent device" and "desynchronized driver"
408
  become the same, safe, diagnosis.)
409
410
One caveat for completeness: the analysis above covers detection, i.e.
411
installation time — the only supported pairing path. A *resident* v1 TSR
412
carried into a v2 device by other means (a savestate taken on an old
413
openMSX and restored on a new one) is not served, but it fails safe:
414
v1's per-call gate is `status register == 2`, and on a v2 device that
415
read returns `0xFF` from the unregistered port, so every call reports
416
failure instead of half-working. Re-running the driver's installer is
417
the answer; the spec simply does not promise more.
418
419
## How the open points resolved
420
421
The proposal closed with three points for review; for the record:
422
423
1. The magic value (0x55) and the capability bit assignments stood
424
   unchallenged, and are final.
425
2. NET_STATE keeps its constant answer. Real semantics — or retirement —
426
   remain a question for after v2, deliberately not bundled into this
427
   redesign.
428
3. The err column and the validation order became the contract the TSR
429
   was written against, as intended, and the golden-byte tests pin the
430
   layout of every reply struct (the two-byte replies DNS and NET_STATE
431
   assemble inline are covered by the spec table alone). The one place
432
   where reality pushed back is recorded above: UDP_OPEN's ephemeral
433
   fallback.
434
435
---
436
437
## Appendix — the v1 protocol (historical)
438
439
The v1 document follows — verbatim below its old title, headings demoted
440
one level — as it stood the day v2 replaced it. It stays authoritative
441
for the parameter-block layouts, which v2 did not move (DNS_QUERY's
442
dropped terminator is the one exception), and its closing catalogue of
443
v1 inconsistencies is the list this redesign set out to fix. Where it
444
contradicts v2 — past-end reads, the status register, the detection
445
handshake, the error conventions — everything above supersedes it. Its
446
code references describe the v1-era sources.
447
448
This document describes the private wire protocol between the Z80 MSX-UNAPI
449
TCP/IP driver (`unapinet.com`) and the `UnapiNet` device in openMSX. It is
450
*not* the MSX-UNAPI specification: UNAPI defines what MSX software sees when
451
it calls the driver; everything below is how the driver talks to the bridge
452
behind the scenes. The driver and the device are developed and released
453
together - the compatibility notes below rely on that - but only the device
454
lives in the openMSX tree.
455
456
The C++ implementation is `src/network/UnapiNet.cc`; the byte-exact layouts
457
of every parameter and result block are the structs in
458
`src/network/UnapiNetWire.hh`. Where a struct name appears below, that
459
struct *is* the layout.
460
461
Protocol version described here: **v1** (bridge version byte 4, as reported
462
by `QUERY_CAP`). Known inconsistencies of v1, and the plan to fix them in
463
v2, are collected at the end.
464
465
### I/O ports
466
467
The device claims two consecutive I/O ports, base 0x28 (`share/extensions/
468
unapinet.xml`, `<io base="0x28" num="2"/>`). Decoding is on bit 0 of the
469
port number:
470
471
| Port | Write            | Read                          |
472
|------|------------------|-------------------------------|
473
| 0x28 | command (opcode) | status register               |
474
| 0x29 | parameter byte   | result byte (consuming)       |
475
476
### Transaction model
477
478
Every operation is one synchronous transaction:
479
480
1. The MSX writes the parameter block, one byte at a time, to port 0x29.
481
2. The MSX writes the command opcode to port 0x28. The device processes the
482
   command *inside that I/O write*: by the time the OUT instruction
483
   completes, the result is ready. The MSX never has to wait or poll for
484
   command completion. (Operations that take real time - DNS lookups,
485
   connection establishment, ICMP echo - return immediately and are polled
486
   with their own dedicated commands; see "Asynchronous operations".)
487
3. The MSX reads the status register (port 0x28). `0x02` (`STATUS_DATA`)
488
   means the result is ready to be read.
489
4. The MSX reads the result, one byte per IN, from port 0x29. Each read
490
   consumes one byte; reading past the end returns 0x00.
491
492
Bookkeeping rules, all enforced by the device:
493
494
* The parameter buffer is always cleared after a command executes, whether
495
  it succeeded or not.
496
* Writing a parameter byte while a result is still pending discards the
497
  unread result (a new transaction implicitly abandons the old one). This is
498
  the recovery path if the MSX program is interrupted mid-read: the next
499
  transaction starts clean.
500
* The parameter buffer is capped (64 KiB + 16). Excess bytes are dropped;
501
  the command's own size check then reports an error. A runaway MSX program
502
  cannot exhaust host memory.
503
504
#### Status register values
505
506
| Value | Name         | Meaning                                          |
507
|-------|--------------|--------------------------------------------------|
508
| 0x00  | STATUS_OK    | idle; no result pending                          |
509
| 0x01  | STATUS_ERROR | protocol-level error (see below)                 |
510
| 0x02  | STATUS_DATA  | a result is pending on the data port             |
511
512
`STATUS_ERROR` is raised in exactly three situations: unknown opcode,
513
`DNS_QUERY` with an empty host name, and `DNS_QUERY` while a lookup is
514
already running. Every other error is reported *inside* a normal result
515
block. In practice the driver only ever performs one check on this
516
register: "did the command I just issued produce a result?"
517
(`status == STATUS_DATA`).
518
519
### Detection handshake
520
521
The driver detects the device at installation time:
522
523
1. `PING` (0x00, no parameters) must answer `STATUS_DATA` with the single
524
   magic byte **0xAB**. Anything else means the extension is not present.
525
2. `QUERY_CAP` (0x10, no parameters) answers two bytes: a capability
526
   summary (0x0F = PING + DNS + TCP + UDP) and the bridge version (4).
527
528
*v1 quirk:* the current driver only performs step 1; nothing reads the
529
version byte today.
530
531
### Data types
532
533
* **IPv4 address** - 4 bytes, the octets of `a.b.c.d` in order
534
  (network/big-endian; `Endian::UA_B32` in the structs).
535
* **port / length / id / seq** - 2 bytes, low byte first (little-endian,
536
  Z80-natural; `Endian::UA_L16`).
537
* **handle** - 1 byte. TCP and UDP handles are independent ranges, both
538
  1-based on the wire: 1..4 (`MAX_TCP` = `MAX_UDP` = 4). In results, handle
539
  0 means "could not open".
540
541
### Command reference
542
543
| Op   | Name        | Parameters (struct)            | Result (struct)                  |
544
|------|-------------|--------------------------------|----------------------------------|
545
| 0x00 | PING        | none                           | 1 byte: 0xAB                     |
546
| 0x01 | DNS_QUERY   | hostname bytes, NUL-terminated | see "Asynchronous operations"    |
547
| 0x02 | DNS_STATUS  | none                           | see "Asynchronous operations"    |
548
| 0x03 | TCP_OPEN    | `TcpOpenParams` (11)           | 1 byte: handle, 0 = failed       |
549
| 0x04 | TCP_SEND    | `TcpSendParamHeader` (3) + payload | 1 byte: 0 = accepted, 1 = error, 2 = send buffer full |
550
| 0x05 | TCP_RECV    | `TcpRecvParams` (3)            | `TcpRecvResultHeader` (2) + payload |
551
| 0x06 | TCP_CLOSE   | 1 byte: handle (0 = all transient) | 1 byte: 0 = OK, 1 = error    |
552
| 0x07 | TCP_STATE   | 1 byte: handle                 | `TcpStateResult` (12)            |
553
| 0x08 | TCP_ABORT   | 1 byte: handle                 | 1 byte: 0 = OK, 1 = error        |
554
| 0x09 | UDP_OPEN    | `UdpOpenParams` (2)            | 1 byte: handle, 0 = failed       |
555
| 0x0A | UDP_CLOSE   | 1 byte: handle (0 = all transient) | 1 byte: 0 = OK, 1 = error    |
556
| 0x0B | UDP_STATE   | 1 byte: handle                 | `UdpStateResult` (2)             |
557
| 0x0C | UDP_SEND    | `UdpSendParamHeader` (9) + payload | 1 byte: 0 = OK, 1 = error    |
558
| 0x0D | GET_LOCALIP | none                           | 4 bytes: host IPv4               |
559
| 0x0E | NET_STATE   | none                           | 1 byte: always 2 ("open")        |
560
| 0x0F | UDP_RECV    | `UdpRecvParams` (3)            | `UdpRecvResultHeader` (8) + payload |
561
| 0x10 | QUERY_CAP   | none                           | 2 bytes: caps (0x0F), version (4) |
562
| 0x11 | ICMP_SEND   | `IcmpSendParams` (11)          | 1 byte: 0 = queued, 1 = error    |
563
| 0x12 | ICMP_RECV   | none                           | 1 byte: 0 = none, or `IcmpRecvResult` (12) |
564
565
Any other opcode raises `STATUS_ERROR`.
566
567
#### TCP
568
569
**TCP_OPEN** - `TcpOpenParams`: remote IP (4), remote port (2), local
570
port (2), timeout (2, *reserved: never read*), flags (1; bit 0 = passive
571
open, bit 1 = resident). An active open starts a non-blocking `connect()`
572
and returns the handle immediately; the driver polls `TCP_STATE` until the
573
state leaves SynSent. A passive open (bit 0) listens on the local port;
574
remote IP, if non-zero, filters which peer may connect.
575
576
**TCP_SEND** - header {handle (1), length (2)} followed by exactly `length`
577
payload bytes. The driver keeps to 4096 bytes (`MAX_TRANSFER`) per command
578
by convention, but the bridge does not enforce that bound on the send side
579
(a v1 quirk; only TCP_RECV clamps to it). Result 2 ("send buffer full") is
580
a recoverable condition: the bridge queues up to 64 KiB (`MAX_SEND_BUF`)
581
that the kernel has not yet accepted, and the driver should retry later.
582
Result 1 is a real failure (bad handle/state/length, or a connection error
583
while queueing).
584
585
**TCP_RECV** - {handle (1), maxlen (2)}. Answers {actualLen (2)} followed
586
by `actualLen` bytes taken from the connection's receive buffer, at most
587
4096 per command. `actualLen` 0 means "nothing buffered right now" - poll
588
`TCP_STATE` to distinguish an empty buffer on a live connection from a
589
finished one. *v1 quirk:* a bad handle also answers `actualLen` 0.
590
591
**TCP_STATE** - answers `TcpStateResult`: state (1), bytes available to
592
read (2), close reason (1), remote IP (4), remote port (2), local
593
port (2). An invalid or missing handle answers all-zero fields with
594
closeReason = 1 (NeverUsed).
595
596
State values (TCP-standard names):
597
0 Closed, 1 Listen, 2 SynSent, 3 SynRecv, 4 Established, 5 FinWait1,
598
6 FinWait2, 7 CloseWait, 8 Closing, 9 LastAck, 10 TimeWait.
599
600
Close reasons: 0 None, 1 NeverUsed, 2 ClosedByUser, 3 Aborted,
601
4 ConnectionReset, 6 ConnectFailed.
602
603
**TCP_CLOSE vs TCP_ABORT** - CLOSE is graceful: the connection enters
604
FinWait1, the FIN goes out once everything the MSX queued has actually
605
been sent, and the handle is freed when the peer closes too - or after a
606
30 s timeout, so a silent peer cannot pin a handle forever. Received data
607
still buffered stays readable meanwhile. ABORT drops the connection
608
immediately. Handle 0 in TCP_CLOSE (and UDP_CLOSE) closes every open
609
non-resident connection at once - the driver uses it on program exit;
610
resident connections (TCP_OPEN flags bit 1) survive it.
611
612
#### UDP
613
614
**UDP_OPEN** - local port (2), 0xFFFF = ephemeral. **UDP_STATE** answers
615
the size of the first queued datagram (2 bytes; 0 = queue empty, or bad
616
handle - same v1 quirk as TCP_RECV). **UDP_SEND** - header {handle (1),
617
destination IP (4), destination port (2), length (2)} + payload; result 1
618
also covers a short kernel write. **UDP_RECV** - {handle (1), maxlen (2)};
619
answers {source IP (4), source port (2), actualLen (2)} + payload.
620
Datagram semantics apply: if the first queued datagram is larger than
621
`maxlen`, the excess of *that datagram* is discarded.
622
623
#### ICMP
624
625
**ICMP_SEND** - {destination IP (4), TTL (1), identifier (2),
626
sequence (2), size (2, clamped to 512)}. No payload follows: the bridge
627
fabricates the echo data itself. The request is queued to a worker thread;
628
the result byte only acknowledges the queueing. **ICMP_RECV** polls for
629
replies: 1 byte 0 = none yet, otherwise {1, source IP (4), TTL (1),
630
identifier (2), sequence (2), dataLen (2)} - header only, the echo payload
631
is not returned. *Current limitation:* the ICMP worker is implemented with
632
the Windows ICMP API; on other host platforms ICMP_RECV never reports
633
data.
634
635
### Asynchronous operations
636
637
**DNS.** `DNS_QUERY` takes the hostname as its parameter block,
638
NUL-terminated (the terminator and anything after it are ignored). If the
639
name parses as a dotted-quad IP it answers immediately with 5 bytes
640
{1, ip}. Otherwise it starts a resolver thread and answers a single byte 0
641
("started"); the driver then polls `DNS_STATUS`, which answers one byte
642
0/1 (idle / in progress), 5 bytes {2, ip} on success, or 2 bytes
643
{0xFF, 3} on failure (the only failure reported is "no such host", UNAPI
644
error 3). The completed status is sticky: it stays readable until the next
645
`DNS_QUERY` or a device reset. Issuing `DNS_QUERY` while a lookup is
646
running raises `STATUS_ERROR` without disturbing the running lookup.
647
648
**TCP connect** is polled via `TCP_STATE` (SynSent → Established or
649
Closed + ConnectFailed).
650
651
**Receive path.** A background thread moves incoming TCP data into a
652
per-connection buffer of up to 64 KiB (`MAX_RECV_BUF`), from which
653
`TCP_RECV` reads. When the buffer is full the bridge stops reading from
654
the socket, the kernel buffer fills, and TCP flow control throttles the
655
peer - no TCP data is ever dropped. UDP is best-effort, as datagrams are:
656
they are read into a 2 KiB buffer (larger ones are truncated at receive
657
time) and queued per socket up to 16 deep; beyond that, new arrivals are
658
read and discarded.
659
660
### Known v1 inconsistencies, and the v2 plan
661
662
These grew one command at a time and are preserved here so the next reader
663
does not have to rediscover them:
664
665
1. **Three error conventions coexist.** OPEN commands answer a handle
666
   where 0 = failure; most other commands answer 0 = OK / 1 = error (and
667
   TCP_SEND adds 2 = buffer full); `STATUS_ERROR` covers only the three
668
   protocol-level cases listed above.
669
2. **"No data" and "bad handle" are indistinguishable** in TCP_RECV,
670
   UDP_STATE and UDP_RECV.
671
3. **Parameter blocks are validated as "at least this long"**, not exactly;
672
   trailing garbage is silently accepted.
673
4. **`QUERY_CAP` is implemented but never issued** by the driver, so the
674
   version byte is dead on the wire.
675
5. **`NET_STATE` answers a constant.**
676
6. **`TcpOpenParams.timeout` is never read.**
677
7. **`MAX_TRANSFER` is enforced on TCP_RECV but not on TCP_SEND** - the
678
   4096-byte bound on sends is a driver-side convention the bridge
679
   silently trusts.
680
681
The planned v2, coordinated with the driver (both halves ship from the
682
same repository): every reply starts with a status byte (0 = OK, otherwise
683
the UNAPI error code the driver should return, `ERR_BUFFER` included);
684
handles stop doubling as error codes; parameter blocks are validated for
685
exact size; the PING magic is bumped so an old driver fails detection
686
cleanly instead of misbehaving quietly; and the driver reads `QUERY_CAP`
687
for real. With a status byte on every reply the status register becomes
688
redundant (the driver's only use of it is the `== STATUS_DATA` check after
689
each command); whether to repurpose or retire it is an open design point.