Repository navigation
Generators outside the CLI build: loading, discovery, compatibility #8
Description
Rescoped 2026-10-18. The generator architecture has changed since this was filed.
coreno longer carries a generator (codelib_genwas removed). This issue now covers the part that's still open: generators that aren't compiled into the CLI. The original plan is in this issue's edit history.
Current architecture
ComlineProject/generation(codegen, cratecomline-codegen) is the shared layer: theRegistry, the resolution kernel that generators share (they resolve once and render per target), and the generation modes (code/lib/dylib).- Each language is its own crate that registers itself:
comline-codegen-rust(ComlineProject/comline-rust) andcomline-codegen-typescript(ComlineProject/comline-typescript), each withregister(&mut Registry). - The CLI links them statically behind cargo features (
gen-rust,gen-typescript). Adding a language means rebuilding the CLI. - The compiled schema has a stable on-disk format: CBOR with name-tagged fields and a core-owned IR format stamp (core#84/chore: drop dead rmp_serde meta (de)serializer and stale allow(unused) #85).
What's still open
A third-party generator, or a newer version of one, can't be used without rebuilding the CLI.
- Choose a loading model. The stable CBOR IR makes a process-based generator a natural fit: the CLI runs an external executable and passes the compiled schema plus the generation request, and the generator writes files. That avoids a Rust ABI. The alternatives are WASM components (sandboxed, portable) and
dylib/abi_stable(fastest, but tied to Rust and to the platform). - Discovery: how
comline.toml's[[generate.target]]names an external generator (path, package, or registry name). - Compatibility: a generator declares which IR format stamps it accepts, and the CLI refuses a mismatch with a clear message.
- Guide for writing a generator outside this organisation.
Related: publishing bindings for third-party dependencies is still design-only.
Activity
Kinflou commentedon Aug 29, 2026
Body updated:
core/src/codelib_gen/{mod,rust}.rsis a real (if minimal, hardcoded, synchronous)rustgenerator now, not atodo!(). The oldercode_gen/lib_gen/rust_abi_stabledesign inComlineProject/generationis unmigrated (imports removedbasic_storage) and reviving it is part of this issue. The consumercomline.toml[generate] mode = code|lib|dylibsits on top of this.[-]Create Code Generation Plugin System[/-][+]Generators outside the CLI build: loading, discovery, compatibility[/+]Kinflou commentedon Oct 18, 2026
Related: #120 proposes how third-party encodings (formats and framings) plug in. It deliberately doesn't depend on loading generators at runtime: a consumer maps each encoding to a type per target in
comline.toml, so built-in generators can wire third-party encodings. The two questions meet only when a third-party language wants a third-party encoding.Kinflou commentedon Oct 31, 2026
Decided 2026-10-31:
.wasm. The CLI runs a.wasmunder wasmtime; the playground runs it through a browser WASI shim.liboutput depends on it automatically, and a consumer can override it incomline.toml.The extension API and the generator protocol are part of the 1.0 promise (docs
conduct/readiness.md). An implementation plan comes next.Kinflou commentedon Oct 31, 2026
Status (2026-11-01):
.wasmgenerators (generate: external generators (native and wasm), built-ins through the protocol cli#104), and the playground's WASI path (Pin the encodings cascade; run external WASI generators through a shim (A4) playground#103).Kinflou commentedon Oct 31, 2026
Done: the stdio protocol (ComlineProject/generation#85), generator binaries (ComlineProject/comline-rust#79, ComlineProject/comline-typescript#91), the CLI running external native and
.wasmgenerators (ComlineProject/cli#104), the playground runner (ComlineProject/playground#103, #106), and the guide and protocol reference (ComlineProject/docs#105). Naming a generator by registry package waits for the registry (ComlineProject/package-registry#1).