Four `realtime/unit` tests pass whether or not the SDK implements the spec point they test — RTP18a is currently passing against an SDK that does not implement it

[owenpearson]

Summary

Four tests in uts/realtime/unit pass identically whether or not the SDK implements the spec point they are filed under. Each is a fixture problem, not a spec-text problem, and each has a cheap fix — in two cases the correct pattern is already in the same file.

RTP18a is the one worth fixing first: it is currently passing against at least one SDK that does not implement the requirement.

Line references are against d9a04ca; paths are relative to uts/.

---

1. RTP18a — the second sync is a superset of the first, so neither outcome is distinguishable

realtime/unit/RTP18a/new-sync-discards-previous-1, realtime/unit/presence/presence_sync.md:230.

RTP18a requires that a new sync sequence identifier discard any sync already in progress. The fixture:

# :244-245  — pre-populate
map.put(PresenceMessage(action: ENTER, clientId: "alice", connectionId: "c1", ...))
map.put(PresenceMessage(action: ENTER, clientId: "bob",   connectionId: "c2", ...))

# :248  — first sync starts; only alice arrives
map.startSync()
map.put(PresenceMessage(action: PRESENT, clientId: "alice", ...))

# :253  — the behaviour under test: a new sync starts before the first ends
map.startSync()

# :256-257  — both members arrive
map.put(PresenceMessage(action: PRESENT, clientId: "alice", ...))
map.put(PresenceMessage(action: PRESENT, clientId: "bob",   ...))

Assertions at :265-269: leave_events.length == 0, map.values().length == 2, both members present.

The second sync delivers a superset of the first, and that is why the test is blind:

	after the second sync
Compliant (first sync discarded)	residual set re-snapshotted as {alice, bob}; both seen; nothing left over → 0 leaves, 2 members
Non-compliant (first sync's residual carries across)	residual {bob}; bob is seen in the second sync; nothing left over → 0 leaves, 2 members

Both paths produce exactly the asserted values.

This has a live cost. ably-python does not implement RTP18a — PresenceMap.start_sync() is a no-op while a sync is already in progress — and this test passes against it.

Fix: have the second sync omit a member the first delivered, and assert that member is gone. The neighbouring RTP18c test at :296-320 in the same file already does exactly this.

2. RTN19a2 — the paired tests assert identical values

realtime/unit/RTN19a2/new-serial-failed-resume-1, realtime/unit/channels/channel_publish.md:1996.

RTN15c7 requires the msgSerial counter to reset on a failed resume, and RTN19a2's paired test covers the successful case where it is preserved. But:

• same-serial-on-resume-0 (:1900) asserts at :1987-1988 that the resent messages carry original_serial_1 / original_serial_2 — i.e. 0 and 1.
• new-serial-failed-resume-1 (:1996) asserts at :2091-2092 that the resent messages carry msgSerial == 0 and == 1.

Those are the same two values. The test's own first-transport check at :2073-2074 confirms the originals were 0 and 1. An SDK that ignores RTN15c7's reset entirely — always preserving serials — passes both tests.

Fix: make the original serials something other than 0 and 1, so the reset is observable. Publish and let the server ACK two messages first (consuming serials 0 and 1), then publish the two messages the test actually tracks — they now carry serials 2 and 3. After the failed resume:

	resent serials
Compliant (RTN15c7 reset)	0, 1
Non-compliant (counter preserved)	2, 3

The paired same-serial-on-resume-0 test can take the same setup and assert 2, 3 in both places, which makes the two tests genuinely complementary rather than identical.

3. RTL2i — an optional attribute asserted unconditionally

realtime/unit/RTL2i/has-backlog-flag-true-0, realtime/unit/channels/channel_state_events.md:418.

:463 asserts captured_change.hasBacklog == true. But the file's own requirement table at :420-423 says:

Spec	Requirement
RTL2i	ChannelStateChange may expose hasBacklog property

and features.md:691 (RTL2i) says "may optionally expose", with :1744 (TH6) saying the attribute "may contain" a value. A conforming SDK that declines the option cannot pass this test.

The sibling has-backlog-flag-false-1 gets it right — its assertion is the disjunction == false OR IS null.

Fix: make the true case conditional in the same way, or promote the attribute to mandatory in features.md. (For what it is worth, we think mandatory is the better answer — the flag is cheap and useful — but that is a features.md change, not a UTS one.)

4. RTL13b — the advance window is sized below the jittered retry it waits for

realtime/unit/RTL13b/repeated-failure-cycle-2, realtime/unit/channels/channel_server_initiated_detach.md:366.

# :403-404
realtimeRequestTimeout: 100,
channelRetryTimeout: 200

# :432-436  cycle 1
ADVANCE_TIME(150)   -> suspended
ADVANCE_TIME(250)   -> attaching

# :439-441  cycle 2
ADVANCE_TIME(150)   -> suspended
ADVANCE_TIME(250)

features.md:1000-1002 (RTB1) makes the SUSPENDED-channel retry channelRetryTimeout × backoff × jitter, with RTB1a's backoff sequence [1, 4/3, 5/3, 2, …] and RTB1b's jitter uniform on 0.8–1.

• Cycle 1's retry lands in 160–200 ms — the 250 ms window clears it every time.
• Cycle 2's retry lands in 213–267 ms — the 250 ms window clears it only about seven times in ten.

So the test is flaky by construction against a conforming SDK. (Against an SDK that schedules the flat timeout with no backoff or jitter, it is an exact-boundary case at 200 ms instead — which is how it came to light.)

Fix: size the windows for RTB1's jittered upper bound, and widen the gap between realtimeRequestTimeout and channelRetryTimeout so the two timers cannot be confused.

---

Found while deriving uts/realtime/unit for ably-python (all 54 specs, 481 tests).