Summary
uts/realtime/unit/helpers/mock_websocket.md is the contract the realtime unit specs are written against, and the specs use a great deal that it does not declare. Most of it is one omission repeated 193 times, which is cheap to fix. The rest is eight tests calling five members that do not exist at all.
This costs an SDK that already has a working harness nothing — we implemented the missing pieces by inference and lost no tests. It costs the next SDK, deriving from the contract alone, eight unimplementable tests and a lot of guessing.
Line references are against d9a04ca; paths are relative to uts/realtime/unit/.
What the contract declares
interface MockWebSocket (helpers/mock_websocket.md:27-49) exposes events, send_to_client, send_to_client_and_close, simulate_disconnect(error?), send_ping_frame, await_next_message_from_client, await_connection_attempt, await_client_close and reset.
interface PendingConnection (:70) exposes url, protocol, timestamp and five respond_with_* methods, including respond_with_success(connected_message: ProtocolMessage) (:76), respond_with_dns_error() (:79) and respond_with_error(error_message: ProtocolMessage, then_close: bool = true) (:80).
enum MockEventType (:51-59) and struct MockEvent (:61-64, fields type, timestamp, data) are the event model.
1. Four names the suite depends on that the contract never declares
| Name |
Usage |
Status |
mock_ws.active_connection |
193 times across 28 of the 56 spec files |
Not in interface MockWebSocket. Appears in mock_websocket.md only inside two usage examples, :181 and :411 |
MockEvent.connection and MockEvent.message |
connection/connection_failures_test.md:171, connection/connection_recovery_test.md:623; .message at connection_recovery_test.md:445, :631, :635 |
MockEvent has type, timestamp and data only. The intended spelling is presumably .data |
MockWebSocketClient |
26 times in 3 files — channels/channel_annotations.md (15), channels/channel_update_delete_message.md (9), channels/channel_history.md (2) |
Defined nowhere; every other file uses MockWebSocket. None of these three files calls install_mock either |
create_realtime_client(...) |
connection/connection_open_failures_test.md:208, connection/connection_failures_test.md:313, channels/channel_detach.md:434 |
Defined nowhere; every other file writes Realtime(options: …) |
active_connection is the significant one. It is the single most-used mock member in the suite and it is not part of the interface. channels/channel_detach.md:429 goes further and assigns to it:
mock_ws = MockWebSocket(
onConnectionAttempt: (conn) =>
mock_ws.active_connection = conn # :429
conn.respond_with_connected() # :430 — see below
)
so an implementer cannot even tell whether it is meant to be a read-only accessor the mock maintains or a field the test populates. (We made it read-only, maintained by the mock. It works, but it is a guess.)
2. Five calls on members that do not exist, or that do something else
| Test |
Written |
The contract offers |
realtime/unit/RTL5l/detach-attached-when-disconnected-1 (channels/channel_detach.md:418) |
conn.respond_with_connected() at :430 |
respond_with_success(connected_message: ProtocolMessage) (:76). respond_with_connected is the only occurrence in the whole uts/ tree, and it is called with no argument, so no CONNECTED would reach the client even if the method existed |
realtime/unit/RTN13d/ping-deferred-disconnected-1 (connection/connection_ping_test.md:556) |
mock_ws.active_connection.close_from_server() at :607 |
simulate_disconnect() (:34). close_from_server appears nowhere else in uts/ |
realtime/unit/RTN16f/recover-initializes-msgserial-0 and realtime/unit/RTN16j/recover-channel-serials-0 (connection/connection_recovery_test.md) |
mock_ws.events.filter(e => e.type == "ws_frame" AND e.direction == "client_to_server") at :444 and :629-630 |
MESSAGE_FROM_CLIENT. Neither a ws_frame event type nor a direction field exists — three undefined names across two tests |
realtime/unit/RTN17f/fallback-on-error-0 (connection/fallback_hosts_test.md:121) |
conn.respond_with_error("Host unresolvable") at :138, under the comment # Primary domain: unresolvable (simulated) at :137 |
respond_with_error takes a ProtocolMessage, not a string, and it establishes the connection before having the server send an ERROR. The condition the comment describes is respond_with_dns_error() (:79) |
realtime/unit/RTN17i/prefer-primary-domain-0 (connection/fallback_hosts_test.md:17) |
mock_ws.active_connection.close() at :74 |
Neither declared nor equivalent to anything that is. simulate_disconnect() is presumably meant |
Separately, the contract's own example at mock_websocket.md:275 calls conn.respond_with_success() with no argument, against a signature that requires one — worth making the connected_message parameter optional, since a good number of tests only need the connection established.
Proposed fix
Two halves, and the first is much the cheaper:
- Declare what the suite already uses. Add
active_connection: PendingConnection? to interface MockWebSocket and say whether the mock maintains it or the test sets it; give MockEvent the connection and message accessors the tests read (or repoint those five call sites onto .data); and make respond_with_success's argument optional. This alone makes the great majority of the suite derivable from the contract.
- Repoint the eight tests in section 2 onto methods that exist, and replace
MockWebSocketClient and create_realtime_client with the spellings the other 53 files use.
Found while deriving uts/realtime/unit for ably-python (all 54 specs, 481 tests).
Summary
uts/realtime/unit/helpers/mock_websocket.mdis the contract the realtime unit specs are written against, and the specs use a great deal that it does not declare. Most of it is one omission repeated 193 times, which is cheap to fix. The rest is eight tests calling five members that do not exist at all.This costs an SDK that already has a working harness nothing — we implemented the missing pieces by inference and lost no tests. It costs the next SDK, deriving from the contract alone, eight unimplementable tests and a lot of guessing.
Line references are against
d9a04ca; paths are relative touts/realtime/unit/.What the contract declares
interface MockWebSocket(helpers/mock_websocket.md:27-49) exposesevents,send_to_client,send_to_client_and_close,simulate_disconnect(error?),send_ping_frame,await_next_message_from_client,await_connection_attempt,await_client_closeandreset.interface PendingConnection(:70) exposesurl,protocol,timestampand fiverespond_with_*methods, includingrespond_with_success(connected_message: ProtocolMessage)(:76),respond_with_dns_error()(:79) andrespond_with_error(error_message: ProtocolMessage, then_close: bool = true)(:80).enum MockEventType(:51-59) andstruct MockEvent(:61-64, fieldstype,timestamp,data) are the event model.1. Four names the suite depends on that the contract never declares
mock_ws.active_connectioninterface MockWebSocket. Appears inmock_websocket.mdonly inside two usage examples,:181and:411MockEvent.connectionandMockEvent.messageconnection/connection_failures_test.md:171,connection/connection_recovery_test.md:623;.messageatconnection_recovery_test.md:445,:631,:635MockEventhastype,timestampanddataonly. The intended spelling is presumably.dataMockWebSocketClientchannels/channel_annotations.md(15),channels/channel_update_delete_message.md(9),channels/channel_history.md(2)MockWebSocket. None of these three files callsinstall_mockeithercreate_realtime_client(...)connection/connection_open_failures_test.md:208,connection/connection_failures_test.md:313,channels/channel_detach.md:434Realtime(options: …)active_connectionis the significant one. It is the single most-used mock member in the suite and it is not part of the interface.channels/channel_detach.md:429goes further and assigns to it:so an implementer cannot even tell whether it is meant to be a read-only accessor the mock maintains or a field the test populates. (We made it read-only, maintained by the mock. It works, but it is a guess.)
2. Five calls on members that do not exist, or that do something else
realtime/unit/RTL5l/detach-attached-when-disconnected-1(channels/channel_detach.md:418)conn.respond_with_connected()at:430respond_with_success(connected_message: ProtocolMessage)(:76).respond_with_connectedis the only occurrence in the wholeuts/tree, and it is called with no argument, so no CONNECTED would reach the client even if the method existedrealtime/unit/RTN13d/ping-deferred-disconnected-1(connection/connection_ping_test.md:556)mock_ws.active_connection.close_from_server()at:607simulate_disconnect()(:34).close_from_serverappears nowhere else inuts/realtime/unit/RTN16f/recover-initializes-msgserial-0andrealtime/unit/RTN16j/recover-channel-serials-0(connection/connection_recovery_test.md)mock_ws.events.filter(e => e.type == "ws_frame" AND e.direction == "client_to_server")at:444and:629-630MESSAGE_FROM_CLIENT. Neither aws_frameevent type nor adirectionfield exists — three undefined names across two testsrealtime/unit/RTN17f/fallback-on-error-0(connection/fallback_hosts_test.md:121)conn.respond_with_error("Host unresolvable")at:138, under the comment# Primary domain: unresolvable (simulated)at:137respond_with_errortakes aProtocolMessage, not a string, and it establishes the connection before having the server send an ERROR. The condition the comment describes isrespond_with_dns_error()(:79)realtime/unit/RTN17i/prefer-primary-domain-0(connection/fallback_hosts_test.md:17)mock_ws.active_connection.close()at:74simulate_disconnect()is presumably meantSeparately, the contract's own example at
mock_websocket.md:275callsconn.respond_with_success()with no argument, against a signature that requires one — worth making theconnected_messageparameter optional, since a good number of tests only need the connection established.Proposed fix
Two halves, and the first is much the cheaper:
active_connection: PendingConnection?tointerface MockWebSocketand say whether the mock maintains it or the test sets it; giveMockEventtheconnectionandmessageaccessors the tests read (or repoint those five call sites onto.data); and makerespond_with_success's argument optional. This alone makes the great majority of the suite derivable from the contract.MockWebSocketClientandcreate_realtime_clientwith the spellings the other 53 files use.Found while deriving
uts/realtime/unitfor ably-python (all 54 specs, 481 tests).