Wait for the push that carries a missing event, not for a clock

[pathscale]

The drain loop deferred on a gap in the index event stream by sleeping 500ms. This replaces the clock with the event, and makes what is left of the number a per-table parameter rather than a constant.

What was wrong

A batch can only apply a contiguous run of the event stream. When collection finds a hole it defers, and the loop then slept a flat 500ms before retrying. That sleep was standing in for an event it could have waited on directly.

It was measured downstream as the dominant cost of shutting a daemon down. A settled store stepped from 0.42s at 700 indexed files to 1.45s at 800 and then flattened, rising after that only at the underlying linear rate of about 0.06s per 100 files. A page drain cannot step by a second and then flatten; the riser was two of these sleeps. A mid-write shutdown cost the same as a settled one, which is what says the hole is the ordinary event-order inversion rather than an unfinished write.

It was worst during close, where it is futile by construction: closing refuses every push, so the event the hole waits for can never be queued.

The change

Queue::await_more_operations registers on the queue's own wake-up and returns the instant an operation is pushed, which is the only way the missing event can arrive. Two things end it early:

• A non-empty queue. The push may have landed while the collection that found the hole was still running, in which case the event is already here and waiting for it waits for something that has happened. The waiter is registered before the queue is read, so a push landing between the two wakes it rather than being missed.

• Any state other than Running. This one is provable rather than a heuristic: a push takes the lifecycle lock and is refused unless the state is Running, and begin_close takes the same lock, so once Closing has been observed here no further push can ever be accepted. The missing event cannot arrive. A yield keeps the remaining escalation off the worker's only pool thread without pretending to wait.

The number is now a parameter

What is left is a backstop against a wake lost to a race with no later push to deliver another. It is PersistenceConfig::event_gap_wait_cap(), a provided method defaulting to DEFAULT_EVENT_GAP_WAIT_CAP (100ms), so no implementor breaks and a table whose producers are genuinely slower than the default assumes can say so, instead of one constant deciding for every table at once. It is read once when the table starts: a parameter that could change under a running drain loop would be a worse thing to explain than one fixed at startup.

GIVE_UP_AFTER_ATTEMPTS counts deferrals, not time, and its doc claimed "about a minute" from 500ms x 120. That stopped being true here, so it now states the relationship instead: the cap times the count is the worst case before a stalled gap fails the table.

Effect

Measured on a consumer daemon whose shutdown this dominated: 19-22s to 1010ms, three runs at 1014 / 1011 / 1010, zero "did not unload" reports. The remaining second is the real drain.

Verification

cargo check --all-targets clean, cargo clippy --workspace --all-targets -- -D warnings clean, cargo fmt applied. cargo test --lib persistence::task is 20 passed / 0 failed, including only_an_exhausted_collection_waits_for_more_operations, which pins the predicate the loop asks before waiting, and draining_scattered_writes_stays_linear.

The first commit reduces the constant; the second replaces the mechanism. Reviewable separately.