0.5.0 advertises cumulatively, so a platform lacking a device cannot conform above the level that added it issue #3
open andymccall opened this · 0 comments
andymccall
Symptom
Conformance against the NEC PC-FX fork (Mednafen): 19 passed, 3 failed — the three being test_pointer_injection, test_pointer_readback, test_pointer_get_is_get.
The PC-FX has no pointing device. Nothing is broken; the contract has no way to say so.
Why both available options misreport
A level is advertised only when all of its callbacks are present. So for a platform missing one device:
choice
/status.contract
/pointer
consequence
Stub set_pointer/get_pointer → 400
0.5.0
400
Honest about /pad. But the suite gates pointer tests on minor >= 4, so they run and fail.
Leave the callbacks NULL
0.3.0
501
Pointer honestly absent — but /pad support that genuinely exists is now invisible, and /pad tests skip.
A single cumulative integer cannot express "has pad, has no pointer".
This is structural, not PC-FX-specific. The X16 has a pointer and advertises 0.5.0 honestly today — but the first level adding a device the X16 lacks puts it in exactly the same position.
Options
A. Per-capability declaration in /status (preferred by both the PC-FX and X16 backends). e.g.
Conformance gates each test on the capability rather than on a version number. contract stays the compatibility gate; capabilities answers "what does this machine actually have". A server knows the fact — it is just not currently able to state it.
B. Treat a documented "no such device" 400 as a skip. Cheaper, but weaker: it also hides a genuinely broken backend, which is precisely what conformance exists to catch. "I do not implement this" is a fact the server should assert, not something a client infers from an error code.
Both backends favour A.
Notes
Whatever lands should keep contract MAJOR-gated for clients, since that is the documented compatibility rule.
Reproduce: build the Mednafen fork with --enable-rrdc, launch with MEDNAFEN_CONTROLPORT, then RRDC_PORT=<port> python3 -m pytest conformance -v.
Imported from GitHub #7, opened by andymccall on 2026-07-29.
Symptom
Conformance against the NEC PC-FX fork (Mednafen): 19 passed, 3 failed — the three being
test_pointer_injection,test_pointer_readback,test_pointer_get_is_get.The PC-FX has no pointing device. Nothing is broken; the contract has no way to say so.
Why both available options misreport
A level is advertised only when all of its callbacks are present. So for a platform missing one device:
/status.contract/pointerset_pointer/get_pointer→ 4000.5.0/pad. But the suite gates pointer tests onminor >= 4, so they run and fail.0.3.0/padsupport that genuinely exists is now invisible, and/padtests skip.A single cumulative integer cannot express "has pad, has no pointer".
This is structural, not PC-FX-specific. The X16 has a pointer and advertises 0.5.0 honestly today — but the first level adding a device the X16 lacks puts it in exactly the same position.
Options
A. Per-capability declaration in
/status(preferred by both the PC-FX and X16 backends). e.g.Conformance gates each test on the capability rather than on a version number.
contractstays the compatibility gate;capabilitiesanswers "what does this machine actually have". A server knows the fact — it is just not currently able to state it.B. Treat a documented "no such device" 400 as a skip. Cheaper, but weaker: it also hides a genuinely broken backend, which is precisely what conformance exists to catch. "I do not implement this" is a fact the server should assert, not something a client infers from an error code.
Both backends favour A.
Notes
contractMAJOR-gated for clients, since that is the documented compatibility rule.--enable-rrdc, launch withMEDNAFEN_CONTROLPORT, thenRRDC_PORT=<port> python3 -m pytest conformance -v.Imported from GitHub #7, opened by andymccall on 2026-07-29.