From ae3815b543d2f8c1c925c25d4232422d53c1e560 Mon Sep 17 00:00:00 2001 From: Kinflou Date: Tue, 3 Nov 2026 16:06:59 +0800 Subject: [PATCH] wire: char and bounds per format; conformance spec 0.3 (core#98) --- docs/docs/reference/conformance.md | 29 +++++++++++---- .../docs/reference/wire/official-encodings.md | 29 +++++++++++++++ docs/docs/reference/wire/packed.md | 35 ++++++++++++++++--- 3 files changed, 82 insertions(+), 11 deletions(-) diff --git a/docs/docs/reference/conformance.md b/docs/docs/reference/conformance.md index b80c67d..4dff347 100644 --- a/docs/docs/reference/conformance.md +++ b/docs/docs/reference/conformance.md @@ -13,7 +13,7 @@ differ, they win. ## Version -**Spec version: `0.2`** (`runner::SPEC_VERSION`). It covers the case format +**Spec version: `0.3`** (`runner::SPEC_VERSION`). It covers the case format (`vectors/*.json`, `profiles/*.json`), the ops, and the value and type encodings. @@ -23,8 +23,19 @@ encodings. Under it, ops and reply fields are only ever added. Changes, newest first (the full changelog is in the -[runtime README](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/README.md#the-spec-version-02)): - +[runtime README](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/README.md#the-spec-version-03)): + +- **0.3:** bounds and `char` + ([core#98](https://github.com/ComlineProject/core/issues/98)). + - New type descriptions: + - `"char"`; + - `{"bounded_string": {"max": N}}` for `char[max N]`; + - `{"bounded_array": {"of": T, "max": N}}` for `T[max N]`; + - `{"recursive": {"max_depth": D, "of": T}}`, with `"self"` inside `T` for the recursive type. + - A new value, `{"char": "é"}`. + - An `encode` case may carry a `type`. An `encode` rejection is a value + over its type's bounds: it asks only to encode, and encoding must fail. + - No profile facts change, and the ops protocol stays at 1. - **0.2:** handshake profiles state `schema_agreement` and `encoding_agreement` as JSON booleans (`"value": true`). A fact stated `false` names no cases, and a handshake profile may leave a fact out, which @@ -67,7 +78,7 @@ A case is one operation and the bytes it must produce: | `op` | the operation, below | | `input` | the operation's input | | `bytes` | the encoded form, lowercase hex | -| `expect` | optional: `"reject"` means decoding `bytes` must fail; the input then names only what selects the decoder (the framing) | +| `expect` | optional: `"reject"` means decoding `bytes` must fail; the input then names only what selects the decoder (the framing). An `encode` rejection is a value over its `type`'s bounds, which encoding must refuse; its `bytes` are empty | | `compare` | optional: `"json"` compares JSON text as JSON (RFC 8259) instead of byte for byte | - **Bytes** are lowercase hex. @@ -86,7 +97,7 @@ A case is one operation and the bytes it must produce: | `envelope` | `kind`: `ok` with `payload`, `err` with `id` and `body`, or `fault` | encode and decode | | `request` | `framing`, `call_id`, `method`, `request_id`, `format`, `params`, and `one_way` when `true` | encode and decode | | `response` | `framing`, `request_id`, and an envelope's fields | encode and decode | -| `encode` | `format`, `value` | encode | +| `encode` | `format`, `value`, and `type` when its bounds matter | encode; a rejection's encode must fail | | `decode` | `format`, `type`, `value` | decode: `bytes` read as `type` give `value` | | `handshake_check` | one end's `ir_hash`, `wire_format`, `framing`, `capabilities` | decode only: `bytes`, the peer's `comline@1` frame, is accepted (giving the input back) when the IR hash, format and framing agree, and refused when one differs. Capability bits may differ | | `stream_prefix` | `len`, `max` | encode and decode the 4-byte length prefix of a `len`-byte frame under the limit `max` | @@ -96,7 +107,11 @@ A framing is proven with `request` and `response` cases; a format with `encode` and `decode` cases. Values and types are written in a typed encoding (`{"u8": 255}`, -`{"struct": {"name": "Point", "fields": [...]}}`, and so on). The full list is +`{"struct": {"name": "Point", "fields": [...]}}`, and so on). Bounds have +their own type descriptions: `{"bounded_string": {"max": N}}`, +`{"bounded_array": {"of": T, "max": N}}`, and +`{"recursive": {"max_depth": D, "of": T}}`, where the outermost value is +level 1 and level D + 1 is refused. The full list is in the [runtime README](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/README.md#the-corpus). @@ -111,7 +126,7 @@ no prefix. See [Transports](../guide/extending/transports.md). - **A rejection** (`expect: "reject"`) asks to decode its `bytes`, which must fail. A `stream_prefix` rejection also asks to encode its input, which must - fail. + fail. An `encode` rejection asks only to encode its input, which must fail. - **Any other case** asks to encode its input (unless it is a `decode` or a `handshake_check`) and to decode its `bytes` (unless it is a `name_hash` or an `encode`). diff --git a/docs/docs/reference/wire/official-encodings.md b/docs/docs/reference/wire/official-encodings.md index 20fa5dd..7db0198 100644 --- a/docs/docs/reference/wire/official-encodings.md +++ b/docs/docs/reference/wire/official-encodings.md @@ -54,6 +54,35 @@ The tables below are generated from those profiles, as core embeds them. +## `char` and bounds + +Each format's choices for a single `char`, and for values with a bound +(`char[max N]`, `T[max N]`, `@max_depth`, [core#98](https://github.com/ComlineProject/core/issues/98)). +Bounds don't change the bytes, only what a decoder accepts. A value over its +bound is refused while decoding, like malformed input, and an encoder refuses +to write it. + +| Format | One `char` | `T[max N]` | `char[max N]` | +|---|---|---|---| +| [`packed@1`](packed.md#char-and-bounds) | its UTF-8 bytes, 1 to 4, no prefix, checked by the strict UTF-8 check | a count prefix above N refused before allocating | a byte length above 4 × N refused before reading; then counted exactly during the UTF-8 check | +| `msgpack@1` | a one-character string (`fixstr`, 2 to 5 bytes) | a count prefix above N refused before allocating | a byte length above 4 × N refused; then counted exactly | +| `json@1` | a one-character string, `"a"` | refused at item N + 1, which isn't built | counted after unescaping; the frame limit caps the allocation | + +- **Characters, not bytes.** A string's maximum counts Unicode scalar values, + and text is never normalized: what's sent is what's counted. +- **Depth:** a type that can contain itself, under `@max_depth = D`, is + refused at level D + 1 (the outermost value is level 1), in every format. +- **Proven** by each format's vectors in the + [runtime's corpus](https://github.com/ComlineProject/runtime/tree/main/runtime/conformance/vectors): + - a single `char` of 1 to 4 bytes, and bytes that aren't one character; + - each bound at its maximum and one over; + - multi-byte characters near the limit, both under by characters and over by bytes, and the reverse; + - the depth limit, on decode and encode. +- **No profile fact** covers bounds: every format checks them the same way. + A changed maximum is a breaking change in the version comparison. It also + changes the protocol's wire-shape hash in the `comline@2` compatibility + facts. + ## The guarantees they keep A protocol keeps a [guarantee](../../guide/idl/protocol.md#giving-up-a-guarantee) diff --git a/docs/docs/reference/wire/packed.md b/docs/docs/reference/wire/packed.md index 218b776..cea1e48 100644 --- a/docs/docs/reference/wire/packed.md +++ b/docs/docs/reference/wire/packed.md @@ -17,9 +17,10 @@ declared, it's this one. | `u8` `u16` `u32` `u64` `u128` | 1, 2, 4, 8, 16 bytes, little-endian | | `s8` `s16` `s32` `s64` `s128` | 1, 2, 4, 8, 16 bytes, little-endian two's complement | | `f32` `f64` | 4, 8 bytes, IEEE 754 little-endian | -| `string` | length as a varint, then that many bytes of UTF-8 (invalid UTF-8 is an error) | -| `T[]` | element count as a varint, then the elements | -| `T[N]` | the N elements; the length is in the schema | +| `string`, `char[max N]`, `char[N]` | length in bytes as a varint, then that many bytes of UTF-8 (invalid UTF-8 is an error). An array of `char` is always text, so `char[N]` sends its length too | +| `char` | its UTF-8 bytes alone, 1 to 4, no length: the first byte says how many follow | +| `T[]`, `T[max N]` | element count as a varint, then the elements | +| `T[N]` | the N elements; the length is in the schema (except `char[N]`, above) | | struct, error body, a call's arguments | the fields in declaration order | | optional field | `00`, or `01` followed by the value | | enum | the variant's index in declaration order, as a varint | @@ -35,6 +36,31 @@ bytes are an error. A length larger than the bytes left is an error before anything is allocated, so a hostile length can't make the decoder reserve memory. +## `char` and bounds + +**A single `char`** is one Unicode scalar value, sent as its UTF-8 bytes +with no prefix: `61` for `a`, `c3 a9` for `é`, `e2 82 ac` for `€`, +`f0 9f 98 80` for `😀`. The decoder reads the first byte's width and runs +the strict UTF-8 check. It refuses: +- a continuation byte or `c0`, `c1`, `f5`..`ff` in first place; +- an overlong form; +- a surrogate; +- a value past U+10FFFF; +- a character cut short. + +**Bounds** (`char[max N]`, `T[max N]`, `@max_depth`) don't change the bytes, +only what a decoder accepts. A value over its bound is refused like +malformed input, and an encoder refuses to write one. + +| Bound | Decoding refuses | +|---|---| +| `T[max N]` | a count above N, before anything is allocated | +| `char[max N]` | a byte length above 4 × N before reading the string, since N characters take at most 4 × N bytes; then, during the UTF-8 check, a string of more than N characters. Characters are counted, never bytes, and never normalized | +| `@max_depth = D` | a value of the recursive type nested more than D deep (the outermost is level 1), before reading any deeper | + +`"é€😀a"` fits `char[max 4]`: 4 characters in 10 bytes. `"ééééé"` doesn't: +5 characters, though only 10 bytes. + ## Example ```ids @@ -97,5 +123,6 @@ Changing a package's `wire.format` is itself a major change. codec walks. `u64`, `s64` and wider are `bigint`. - **Conformance:** byte-exact vectors in the [runtime's wire corpus](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/vectors/format-packed.json) - pin each type's bytes and each decoding error. Both runtimes pass them; see + pin each type's bytes and each decoding error, a single `char`, and each + bound at and over its limit. Both runtimes pass them; see [Runtimes side by side](../languages/index.md#runtimes-side-by-side).