Wire: char and bounds per format, conformance spec 0.3 (core#98)

docs/docs/reference/conformance.md

@@ -13,7 +13,7 @@ differ, they win.
 
 ## Version
 
-**Spec version: `0.2`** (`runner::SPEC_VERSION`). It covers the case format
+**Spec version: `0.3`** (`runner::SPEC_VERSION`). It covers the case format
 (`vectors/*.json`, `profiles/*.json`), the ops, and the value and type
 encodings.
 
@@ -23,8 +23,19 @@ encodings.
   Under it, ops and reply fields are only ever added.
 
 Changes, newest first (the full changelog is in the
-[runtime README](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/README.md#the-spec-version-02)):
-
+[runtime README](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/README.md#the-spec-version-03)):
+
+- **0.3:** bounds and `char`
+  ([core#98](https://github.com/ComlineProject/core/issues/98)).
+  - New type descriptions:
+    - `"char"`;
+    - `{"bounded_string": {"max": N}}` for `char[max N]`;
+    - `{"bounded_array": {"of": T, "max": N}}` for `T[max N]`;
+    - `{"recursive": {"max_depth": D, "of": T}}`, with `"self"` inside `T` for the recursive type.
+  - A new value, `{"char": "é"}`.
+  - An `encode` case may carry a `type`. An `encode` rejection is a value
+    over its type's bounds: it asks only to encode, and encoding must fail.
+  - No profile facts change, and the ops protocol stays at 1.
 - **0.2:** handshake profiles state `schema_agreement` and
   `encoding_agreement` as JSON booleans (`"value": true`). A fact stated
   `false` names no cases, and a handshake profile may leave a fact out, which
@@ -67,7 +78,7 @@ A case is one operation and the bytes it must produce:
 | `op` | the operation, below |
 | `input` | the operation's input |
 | `bytes` | the encoded form, lowercase hex |
-| `expect` | optional: `"reject"` means decoding `bytes` must fail; the input then names only what selects the decoder (the framing) |
+| `expect` | optional: `"reject"` means decoding `bytes` must fail; the input then names only what selects the decoder (the framing). An `encode` rejection is a value over its `type`'s bounds, which encoding must refuse; its `bytes` are empty |
 | `compare` | optional: `"json"` compares JSON text as JSON (RFC 8259) instead of byte for byte; see [Comparing as JSON](#comparing-as-json) |
 
 - **Bytes** are lowercase hex.
@@ -86,7 +97,7 @@ A case is one operation and the bytes it must produce:
 | `envelope` | `kind`: `ok` with `payload`, `err` with `id` and `body`, or `fault` | encode and decode |
 | `request` | `framing`, `call_id`, `method`, `request_id`, `format`, `params`, and `one_way` when `true` | encode and decode |
 | `response` | `framing`, `request_id`, and an envelope's fields | encode and decode |
-| `encode` | `format`, `value` | encode |
+| `encode` | `format`, `value`, and `type` when its bounds matter | encode; a rejection's encode must fail |
 | `decode` | `format`, `type`, `value` | decode: `bytes` read as `type` give `value` |
 | `handshake_check` | one end's `ir_hash`, `wire_format`, `framing`, `capabilities` | decode only: `bytes`, the peer's `comline@1` frame, is accepted (giving the input back) when the IR hash, format and framing agree, and refused when one differs. Capability bits may differ |
 | `stream_prefix` | `len`, `max` | encode and decode the 4-byte length prefix of a `len`-byte frame under the limit `max` |
@@ -110,7 +121,11 @@ or a JSON-RPC frame). An encode passes when its output equals `bytes` as JSON:
   data.
 
 Values and types are written in a typed encoding (`{"u8": 255}`,
-`{"struct": {"name": "Point", "fields": [...]}}`, and so on). The full list is
+`{"struct": {"name": "Point", "fields": [...]}}`, and so on). Bounds have
+their own type descriptions: `{"bounded_string": {"max": N}}`,
+`{"bounded_array": {"of": T, "max": N}}`, and
+`{"recursive": {"max_depth": D, "of": T}}`, where the outermost value is
+level 1 and level D + 1 is refused. The full list is
 in the
 [runtime README](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/README.md#the-corpus).
 
@@ -125,7 +140,7 @@ no prefix. See [Transports](../guide/extending/transports.md).
 
 - **A rejection** (`expect: "reject"`) asks to decode its `bytes`, which must
   fail. A `stream_prefix` rejection also asks to encode its input, which must
-  fail.
+  fail. An `encode` rejection asks only to encode its input, which must fail.
 - **Any other case** asks to encode its input (unless it is a `decode` or a
   `handshake_check`) and to decode its `bytes` (unless it is a `name_hash` or an
   `encode`).

docs/docs/reference/wire/official-encodings.md

@@ -54,6 +54,35 @@ The tables below are generated from those profiles, as core embeds them.
 
 <!-- encodings:end -->
 
+## `char` and bounds
+
+Each format's choices for a single `char`, and for values with a bound
+(`char[max N]`, `T[max N]`, `@max_depth`, [core#98](https://github.com/ComlineProject/core/issues/98)).
+Bounds don't change the bytes, only what a decoder accepts. A value over its
+bound is refused while decoding, like malformed input, and an encoder refuses
+to write it.
+
+| Format | One `char` | `T[max N]` | `char[max N]` |
+|---|---|---|---|
+| [`packed@1`](packed.md#char-and-bounds) | its UTF-8 bytes, 1 to 4, no prefix, checked by the strict UTF-8 check | a count prefix above N refused before allocating | a byte length above 4 × N refused before reading; then counted exactly during the UTF-8 check |
+| `msgpack@1` | a one-character string (`fixstr`, 2 to 5 bytes) | a count prefix above N refused before allocating | a byte length above 4 × N refused; then counted exactly |
+| `json@1` | a one-character string, `"a"` | refused at item N + 1, which isn't built | counted after unescaping; the frame limit caps the allocation |
+
+- **Characters, not bytes.** A string's maximum counts Unicode scalar values,
+  and text is never normalized: what's sent is what's counted.
+- **Depth:** a type that can contain itself, under `@max_depth = D`, is
+  refused at level D + 1 (the outermost value is level 1), in every format.
+- **Proven** by each format's vectors in the
+  [runtime's corpus](https://github.com/ComlineProject/runtime/tree/main/runtime/conformance/vectors):
+  - a single `char` of 1 to 4 bytes, and bytes that aren't one character;
+  - each bound at its maximum and one over;
+  - multi-byte characters near the limit, both under by characters and over by bytes, and the reverse;
+  - the depth limit, on decode and encode.
+- **No profile fact** covers bounds: every format checks them the same way.
+  A changed maximum is a breaking change in the version comparison. It also
+  changes the protocol's wire-shape hash in the `comline@2` compatibility
+  facts.
+
 ## The guarantees they keep
 
 A protocol keeps a [guarantee](../../guide/idl/protocol.md#giving-up-a-guarantee)

docs/docs/reference/wire/packed.md

@@ -17,9 +17,10 @@ declared, it's this one.
 | `u8` `u16` `u32` `u64` `u128` | 1, 2, 4, 8, 16 bytes, little-endian |
 | `s8` `s16` `s32` `s64` `s128` | 1, 2, 4, 8, 16 bytes, little-endian two's complement |
 | `f32` `f64` | 4, 8 bytes, IEEE 754 little-endian |
-| `string` | length as a varint, then that many bytes of UTF-8 (invalid UTF-8 is an error) |
-| `T[]` | element count as a varint, then the elements |
-| `T[N]` | the N elements; the length is in the schema |
+| `string`, `char[max N]`, `char[N]` | length in bytes as a varint, then that many bytes of UTF-8 (invalid UTF-8 is an error). An array of `char` is always text, so `char[N]` sends its length too |
+| `char` | its UTF-8 bytes alone, 1 to 4, no length: the first byte says how many follow |
+| `T[]`, `T[max N]` | element count as a varint, then the elements |
+| `T[N]` | the N elements; the length is in the schema (except `char[N]`, above) |
 | struct, error body, a call's arguments | the fields in declaration order |
 | optional field | `00`, or `01` followed by the value |
 | enum | the variant's index in declaration order, as a varint |
@@ -35,6 +36,31 @@ bytes are an error. A length larger than the bytes left is an error before
 anything is allocated, so a hostile length can't make the decoder reserve
 memory.
 
+## `char` and bounds
+
+**A single `char`** is one Unicode scalar value, sent as its UTF-8 bytes
+with no prefix: `61` for `a`, `c3 a9` for `é`, `e2 82 ac` for `€`,
+`f0 9f 98 80` for `😀`. The decoder reads the first byte's width and runs
+the strict UTF-8 check. It refuses:
+- a continuation byte or `c0`, `c1`, `f5`..`ff` in first place;
+- an overlong form;
+- a surrogate;
+- a value past U+10FFFF;
+- a character cut short.
+
+**Bounds** (`char[max N]`, `T[max N]`, `@max_depth`) don't change the bytes,
+only what a decoder accepts. A value over its bound is refused like
+malformed input, and an encoder refuses to write one.
+
+| Bound | Decoding refuses |
+|---|---|
+| `T[max N]` | a count above N, before anything is allocated |
+| `char[max N]` | a byte length above 4 × N before reading the string, since N characters take at most 4 × N bytes; then, during the UTF-8 check, a string of more than N characters. Characters are counted, never bytes, and never normalized |
+| `@max_depth = D` | a value of the recursive type nested more than D deep (the outermost is level 1), before reading any deeper |
+
+`"é€😀a"` fits `char[max 4]`: 4 characters in 10 bytes. `"ééééé"` doesn't:
+5 characters, though only 10 bytes.
+
 ## Example
 
 ```ids
@@ -97,5 +123,6 @@ Changing a package's `wire.format` is itself a major change.
   codec walks. `u64`, `s64` and wider are `bigint`.
 - **Conformance:** byte-exact vectors in the
   [runtime's wire corpus](https://github.com/ComlineProject/runtime/blob/main/runtime/conformance/vectors/format-packed.json)
-  pin each type's bytes and each decoding error. Both runtimes pass them; see
+  pin each type's bytes and each decoding error, a single `char`, and each
+  bound at and over its limit. Both runtimes pass them; see
   [Runtimes side by side](../languages/index.md#runtimes-side-by-side).