Extensibility: third-party formats, framings, transports and dispatchers #120

How developers extend Comline with their own wire formats, framings, transports and dispatchers, and what separates an official extension from a third-party one. Proposed, open for discussion; nothing implemented. The full write-up is the proposal page in ComlineProject/docs#48.

Where things stand

• Transports: open. Rust's Transport and TS's equivalent take anything that moves bytes.
• Wire formats: open at runtime (WireFormat in Rust, Codec in TS). contract/wire.rs already asks third-party formats to namespace their names.
• Framings: the runtime trait is open, but generated code isn't. Generators bake the framing in and know only datagram and jsonrpc. Core checks @framing against that list, so a third-party framing can't be named in a schema. An unknown default_framing silently becomes datagram (cli#81).
• Dispatchers: the generated <Proto>Dispatcher implements Dispatch, and wrapping it works, but that isn't documented as an extension point.
• Generators: compiled into the CLI (Generators outside the CLI build: loading, discovery, compatibility #8). Process plugins (Proposal D) are blocked by the playground.
• Nothing lets a schema say "this protocol uses com.acme.frame@1, and here is what that means".

Proposal

1. Official encodings use the same public mechanism as third-party ones. "Official" means maintained here and shipped by default, not built in through a private path, the same principle as std being a package.
2. An encoding is a definition: a versioned name (bare for official, namespaced otherwise), a profile (Compatibility rules: what a version bump means #119, encoding profiles) and vectors in the corpus format. Implementations are per language and separate. The corpus format and the profile schema become a public, versioned spec. The conformance runner becomes a library a third party runs in their own CI.
3. Core learns about a third-party encoding from a dependency. A package declares the encodings it defines (name plus profile) in config.idp. A dependent package can then use those names in @framing and in its declared formats. The profile is trusted like the dependency's schemas: declared, and proven in the author's CI. An encoding with no profile gets the strictest one.
4. Generated code gets the implementation from the consumer's config. comline.toml maps each name to a type per target, e.g. "com.acme.frame@1" = "acme_frame::AcmeFraming". The generator needs no plugin to wire it. A declared but unmapped encoding is a generation error, never a fallback. The per-package format trait (generation#67) is implemented for each mapped name instead of being sealed.
5. A format serializes the generated data model (serde in Rust, Shapes in TS). Needing more from the schema, such as tags or ordinals, makes it a language feature, not a plug-in.
6. Transports and dispatchers stay runtime-only: no name, profile or declaration, since they can't affect compatibility. They get documented patterns (middleware by wrapping Dispatch).
7. Generators stay separate (Generators outside the CLI build: loading, discovery, compatibility #8 / Proposal D). Nothing above needs runtime generator loading.

Promotion path: a third-party encoding becomes official by contributing its profile, vectors and implementations to the official runtimes. It keeps its namespaced name, because renaming would change the handshake. Bare names are reserved for encodings that start out official.

Open questions

• Trust: is "declared by a dependency" enough, or should core require the dependency to ship vectors too, and could a CI tool run a third-party implementation against them?
• Where the name → type mapping lives: comline.toml per target (consumer-side), or also a default shipped by the encoding's package per language.
• Whether a third-party encoding package can also carry its implementations (a crate, an npm package) and have generated lib-mode output depend on them automatically.
• Whether the same mechanism should cover transports later (for example, a handshake capability naming the transport).

Work (after discussion)

• Public spec: the corpus case format and the profile schema, versioned.
• The conformance runner as a reusable library (Rust; TS follows).
• config.idp: a package declares the encodings it defines (builds on config.idp: declare the encodings a package uses, and framing instances #117).
• Core: @framing and declared formats accept names declared by dependencies.
• comline.toml: a per-target name → type mapping; the generators wire mapped types (builds on generation#67).
• Handshake as a declared axis (comline@1, none, custom), with @handshake and the call-site skip renamed *_unverified: Handshake as a declared, pluggable axis; *_unchecked skip paths runtime#18
• Giving up a guarantee: @unverified = "reason"; core works out the set; off by default; consumers accept per protocol, recording the set: @unverified: an explicit, opt-in marker for giving up a peer guarantee #121
• Docs: a guide to writing a format, a framing, a handshake, a transport and a dispatcher wrapper.

Related: #8, #117, #119, generation#67, cli#81, runtime#14.