Migrate the generated TypeScript bindings from ts-proto to protobuf-es v2 #3632

Summary

Migrate the generated TypeScript bindings of the control plane API from ts-proto to protobuf-es v2 (@bufbuild/protobuf v2). Use Connect-ES v2 (@connectrpc/connect v2) for the clients.

Current state

app/controlplane/api/buf.gen.yaml generates the TypeScript bindings with ts-proto:

- remote: buf.build/community/stephenh-ts-proto:v1.151.1
  out: ./gen/frontend
  opt:
    - outputClientImpl=grpc-web
    - esModuleInterop=true
    - useOptionals=messages

• The output goes to app/controlplane/api/gen/frontend.
• The output contains the control plane, attestation, workflow contract, errors and jsonfilter protos.
• The output also contains third-party imported protos. Examples: protoc-gen-openapiv2/options/annotations.ts, protoc-gen-openapiv2/options/openapiv2.ts, buf/validate/... and google/api/....
• The generated code depends on @improbable-eng/grpc-web, browser-headers and protobufjs/minimal.

This is the only buf.gen.yaml in the repository that emits TypeScript. The other files (app/artifact-cas/api, pkg/attestation/crafter/api, pkg/credentials/api, the plugin SDK and the conf packages) emit Go only. So the scope is this one file and its output folder.

Motivation

• ts-proto with the improbable-eng grpc-web transport is in maintenance mode.
• protobuf-es v2 and Connect-ES v2 are the stack that Buf maintains. They have a smaller runtime, first-class ESM and TypeScript support, and they pass the Protobuf conformance tests.
• Some downstream consumers already generate their own protos with protobuf-es v2 into the same source tree. Today they get two different generated files for the same imported proto. For example, protoc-gen-openapiv2/options/annotations.ts from ts-proto sits next to annotations_pb.ts from protobuf-es. The consumers must filter files out when they sync the bindings. One generator for all bindings removes this problem.

Proposed implementation

1. Replace the generator plugin

protobuf-es v2 also generates the service descriptors. A separate connect-es plugin is not necessary.

- remote: buf.build/bufbuild/es:v2.16.0
  out: ./gen/frontend
  opt: target=ts
  include_imports: true

include_imports: true generates the imported dependencies (openapiv2 options, buf/validate, google/api) as *_pb.ts files. Consumers then get a self-contained set of files.

2. Changes in the generated code

• File names change to <name>_pb.ts.
• Messages are plain types. Create them with create(FooSchema, {...}).
• Serialize with toBinary, fromBinary, toJson and fromJson from @bufbuild/protobuf.
• Enums are TypeScript enums.
• Oneofs use the { case, value } form.
• Use @bufbuild/protobuf/wkt for Timestamp (timestampDate, timestampFromDate).

3. Clients

• Create clients with createClient(Service, transport) from @connectrpc/connect v2.
• Use createGrpcWebTransport from @connectrpc/connect-web v2. The transport stays gRPC-web, so the server needs no change.
• Remove the @improbable-eng/grpc-web, browser-headers and protobufjs dependencies.

import { createClient } from "@connectrpc/connect";
import { createGrpcWebTransport } from "@connectrpc/connect-web";
import { WorkflowService } from "./controlplane/v1/workflow_pb";

const transport = createGrpcWebTransport({ baseUrl: "https://cp.example.com" });
const client = createClient(WorkflowService, transport);

4. Transition

• This is a breaking change for every consumer of gen/frontend.
• If a transition period is necessary, emit the protobuf-es output to a new folder first (for example gen/frontend-es). Keep the ts-proto output in gen/frontend until the consumers migrate. Then remove the ts-proto plugin.

5. Regeneration and CI

• Run make api from the repository root (it runs buf generate in app/controlplane/api) and commit the new output.
• Remove the old ts-proto files from the output folder. buf generate does not delete stale files.
• The test.yml workflow runs make api and fails if the tree is dirty. The regenerated files must be in the same pull request.

Acceptance criteria

• app/controlplane/api/buf.gen.yaml uses buf.build/bufbuild/es v2 with target=ts and include_imports: true.
• The generated output has no imports of @improbable-eng/grpc-web, browser-headers or protobufjs.
• The generated output contains the imported third-party protos as *_pb.ts files.
• make api produces no diff in CI.
• The release notes state the breaking change for consumers of gen/frontend.

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri