A lightweight, DSL-driven LLM gateway for routing, patching provider quirks, and enforcing consistent APIs across channels
open-next-router (ONR) is a lightweight, DSL-driven LLM gateway that routes requests, applies compatibility patches, and normalizes behavior across providers and channels.
- Atomic, nginx-like DSL: runtime behavior is explicitly declared in DSL loaded from
config/onr.conf(typically includingconfig/providers/*.conf, routing, auth headers, transforms, SSE parsing, usage extraction). - Fast provider onboarding and patching: fix provider quirks by editing a
.conffile instead of changing and redeploying code. - Hot reload: reload
onr.yaml/keys.yaml/models.yaml/ provider DSL files via SIGHUP; provider DSL can also auto-reload by file watch (opt-in). - No hidden magic: compatibility is opt-in via directives (e.g.
req_map,resp_map,sse_parse,json_del,set_header) rather than implicit heuristics. - Streaming-aware normalization: handle SSE framing and provider-specific streaming semantics while keeping a stable client-facing API.
- Operational visibility: one-line request logs with optional usage/cost extraction help you debug channels and control spend.
All runtime behavior (routing, auth headers, request/response transforms, SSE parsing, usage extraction, etc.) is explicitly described
by DSL loaded from config/onr.conf, which typically includes files under config/providers/*.conf.
# Minimal: route + auth
# config/providers/acme.conf
syntax "next-router/0.1";
provider "acme" {
defaults {
upstream_config {
base_url = "https://api.example.com";
}
auth {
auth_bearer;
}
}
match api = "chat.completions" {
upstream {
set_path "/v1/chat/completions";
}
response {
resp_passthrough;
}
}
}# Extended: opt-in compatibility transforms (examples)
# config/providers/anthropic.conf
syntax "next-router/0.1";
provider "anthropic" {
defaults {
upstream_config {
base_url = "https://api.anthropic.com";
}
auth {
auth_header_key "x-api-key";
}
request {
set_header "anthropic-version" "2023-06-01";
}
}
# OpenAI /v1/chat/completions -> Anthropic /v1/messages (non-stream)
match api = "chat.completions" stream = false {
request {
req_map openai_chat_to_anthropic_messages;
json_del "$.stream_options";
}
upstream {
set_path "/v1/messages";
}
response {
resp_map anthropic_to_openai_chat;
}
}
# OpenAI /v1/chat/completions -> Anthropic /v1/messages (streaming)
match api = "chat.completions" stream = true {
request {
req_map openai_chat_to_anthropic_messages;
json_del "$.stream_options";
}
upstream {
set_path "/v1/messages";
}
response {
sse_parse anthropic_to_openai_chunks;
}
}
}More examples: config/providers/ • Full reference: DSL_SYNTAX.md
Clone with the SDK, documentation, and language-server submodules:
git clone --recurse-submodules https://github.com/r9s-ai/open-next-router.git
cd open-next-routerFor an existing clone, initialize them before running Go workspace commands:
git submodule update --init --recursiveThe client/sdk submodule is maintained in r9s-ai/onr-sdk.
Install latest runtime release as a systemd service:
curl -fsSL https://raw.githubusercontent.com/r9s-ai/open-next-router/main/tools/install_onr_service.sh | sudo bash -s -- \
--mode service \
--api-key 'change-me'Health check:
curl -sS http://127.0.0.1:3300/v1/models -H "Authorization: Bearer change-me"Other install modes:
- Docker:
--mode docker - Docker Compose:
--mode docker-compose
- Prepare configs
- Copy
config/onr.example.yaml->onr.yaml - Copy
config/keys.example.yaml->keys.yaml - Copy
config/models.example.yaml->models.yaml
- Run
cd open-next-router
go run ./cmd/onr --config ./onr.yaml- Reload (nginx-like)
After editing onr.yaml / keys.yaml / models.yaml / provider DSL files, you can reload runtime configs by sending SIGHUP:
go run ./cmd/onr --config ./onr.yaml -s reloadThis uses server.pid_file (default: /var/run/onr.pid).
Optional: enable provider DSL auto-reload by file watch (disabled by default):
providers:
dir: "./config/providers"
auto_reload:
enabled: true
debounce_ms: 300- Test config (nginx-like)
Test configs without starting the server:
# default config path
go run ./cmd/onr -t
# specify config file (flag)
go run ./cmd/onr -t -c ./onr.yaml
# specify config file (positional)
go run ./cmd/onr -t ./onr.yaml- Bundle provider DSL into a single file
Validate the provider DSL source first, then write a self-contained merged file:
# bundle a specific DSL source directly
go run ./cmd/onr-pack --providers ./config/onr.conf --out ./dist/providers.conf
# validate only; do not write output
go run ./cmd/onr-pack --providers ./config/onr.conf --check-only
# validate with extra named checks
go run ./cmd/onr-pack --providers ./config/onr.conf --check-only --check required-usage
# print version
go run ./cmd/onr-pack versionThe command validates the provider DSL before writing. If validation fails, no output file is generated. Use --check-only to validate only and print the validation result without writing a bundled file.
--check enables opt-in checks after the normal DSL validation. It may be repeated or comma-separated. Known checks:
required-usage: fails OpenAI-compatiblechat.completionsroutes whose upstream path looks like*/chat/completionswhen no matchingmetrics { usage_extract ... }is configured.all: runs all known extra checks.
onr-usage-extract is a local debugging utility for testing one explicit usage_mode against a response. It prints the matched canonical usage facts as a JSON array. Matched rules with the same normalized dimension, unit, and complete attributes set are combined into one fact whose quantity is their sum; facts with different attributes remain separate. A combined fact keeps the extraction metadata of the first matched rule.
go run ./cmd/onr-usage-extract \
--usage-mode-file ./config/modes/usage_modes.conf \
--usage-mode anthropic_messages \
--usage '{"usage":{"input_tokens":10,"output_tokens":4,"cache_read_input_tokens":2}}'Pass mode DSL directly with --usage-mode-dsl, or load it from --usage-mode-file; these flags are mutually exclusive. --usage-mode selects a named block and may be omitted only when the supplied DSL defines exactly one usage_mode. The response can be supplied with --usage, --usage-file, or stdin.
For an SSE response, pipe the raw text/event-stream body and add --stream. The command keeps the existing DSL event filtering, multi-line data: framing, blank-line flush, [DONE], usage_root merge, and fallback behavior.
curl -sS https://example.invalid/stream | go run ./cmd/onr-usage-extract \
--stream \
--usage-mode-file ./config/modes/usage_modes.conf \
--usage-mode anthropic_messages_streamRules that explicitly use source=request or source=derived can be supplied with --request / --request-file and --derived / --derived-file respectively.
- Setup Git hooks with prek
# install git hooks (force-replace if pre-commit hooks already exist)
prek install -f -t pre-commit -t commit-msg
# run all hooks manually
prek run --all-filesCreate runtime config files first:
cd open-next-router
cp config/onr.example.yaml onr.yaml
cp config/keys.example.yaml keys.yaml
cp config/models.example.yaml models.yaml
docker compose up --build- Call
curl -sS http://127.0.0.1:3300/v1/chat/completions \
-H "Authorization: Bearer change-me" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hello"}]}' ┌─────────────────────────────────────────┐
│ open-next-router │
│ (Gin server) │
└─────────────────────────────────────────┘
│
│ Auth: Bearer / x-api-key
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ Request Pipeline │
│ │
│ ┌──────────────────────────────┐ ┌───────────────────────┐ ┌───────────┐ │
│ │ request_id + access log │ │ provider selection │ │ DSL exec │ │
│ │ optional traffic dump │ │ 1) x-onr-provider │ │ (phases) │ │
│ └──────────────────────────────┘ │ 2) models.yaml │ └───────────┘ │
│ └───────────────────────┘ │
│ │
│ DSL phases (explicit, nginx-like): │
│ - upstream_config: base_url (and per-channel overrides) │
│ - auth: auth header shape, optional OAuth exchange + bearer injection │
│ - request: header/query/json patching, request mapping (compat mode) │
│ - upstream: set_path/set_query, proxy settings │
│ - response: passthrough / resp_map / sse_parse (streaming normalization) │
│ - error: error_map │
│ - metrics/pricing: usage_extract, cost estimation (optional) │
│ │
│ │
│ ┌──────────────────────────┐ │
│ │ upstream │ │
│ │ provider base_url + path │ │
│ └──────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────┐
│ Upstream APIs │
│ OpenAI-compatible and native provider │
│ APIs (Anthropic, Gemini, etc.) │
└──────────────────────────────────────────┘
Observability:
- [ONR] one-line request log
• base (always): ts, status, latency, client_ip, method, path
• fields (always): request_id, latency_ms
• routing (when available): api, provider, provider_source, model, stream
• upstream (when available): upstream_status, finish_reason
• usage (when available): usage_stage, input_tokens, output_tokens, total_tokens, cache_read_tokens, cache_write_tokens, billable_input_tokens
• usage extras (when produced by `usage_fact`): flattened fields such as `cache_write_ttl_5m_tokens`, `cache_write_ttl_1h_tokens`, `server_tool_web_search_calls`
• cost (when enabled/available): cost_total, cost_input, cost_output, cost_cache_read, cost_cache_write, cost_multiplier, cost_model, cost_channel, cost_unit
- usage_stage=upstream: usage returned by upstream
- usage_stage=estimate_*: best-effort estimation when upstream usage is missing/zero
- Optional traffic dump (file-based)
• META
• ORIGIN
• UPSTREAM
• PROXY
Config reload:
- Send SIGHUP to reload `onr.yaml` / `keys.yaml` / `models.yaml` / `config/onr.conf` and its included DSL files (nginx-like)
- Optional: enable `providers.auto_reload.enabled=true` to watch the resolved provider DSL source directory and auto-reload included DSL files
- Recommended:
Authorization: Bearer <ACCESS_KEY_FROM_KEYS_YAML> - Compatible headers:
x-api-key/x-goog-api-key onr.yamlcan omitauthentirely when usingkeys.yamlaccess_keys- Optional legacy mode:
auth.api_key(master key inonr.yaml)
If your client can only set a single API key and cannot add custom headers, you can use a URI-like token key:
No-sig mode (editable):
onr:v1?k=<ACCESS_KEY>&{query_without_k}
or
onr:v1?k64=<base64url(ACCESS_KEY)>&{query_without_k64}
Supported query params:
k/k64: access key (required by default)p: provider (optional)m: model override (optional; always enforced)uk: BYOK upstream key (optional; when set, ONR uses it directly to call upstream)
Optional config to allow BYOK token without k/k64 (default: false):
auth:
token_key:
allow_byok_without_k: trueGenerate a token key:
make build
onr-admin token create \
--config ./onr.yaml \
--access-key-name client-a \
--provider openai \
--model gpt-4o-miniMore details: see docs/ACCESS_KEYS_CN.md.
You can put plaintext keys in keys.yaml (not recommended for public repos).
keys.yaml supports encrypted values in this format:
ENC[v1:aesgcm:<base64(nonce+ciphertext)>]
To decrypt ENC[...] values, set ONR_MASTER_KEY (32 bytes or base64-encoded 32 bytes).
To generate an encrypted value:
export ONR_MASTER_KEY='...'
echo -n 'sk-xxxx' | onr-admin crypto encryptFor each key entry, you can override the value via environment variable:
- If
nameis set:ONR_UPSTREAM_KEY_<PROVIDER>_<NAME> - Otherwise:
ONR_UPSTREAM_KEY_<PROVIDER>_<INDEX>(1-based)
Example:
providers.openai.keys[0].name: key1->ONR_UPSTREAM_KEY_OPENAI_KEY1
Vertex AI provider keys can use a local GCP service account JSON file instead of value:
providers:
vertex:
keys:
- name: "vertex-sa"
credential_file: "/etc/onr/gcp/vertex-sa.json"
location: "global" # or us-central1
# For regional Vertex endpoints, set base_url_override explicitly.
# base_url_override: "https://us-central1-aiplatform.googleapis.com"location defaults to global when a credential file is configured. The location value is exposed to
provider DSL as $channel.location and is used by the bundled Vertex path templates. ONR does not derive a
regional Vertex host from location; for regional Vertex endpoints, set base_url_override explicitly, for
example https://us-central1-aiplatform.googleapis.com.
Keep the credential file readable only by the ONR runtime user, for example chmod 600 /etc/onr/gcp/vertex-sa.json.
keys.yaml can also contain access keys for clients:
access_keys:
- name: "client-a"
value: "ak-xxx"
comment: "iOS app"Env override:
- If
nameis set:ONR_ACCESS_KEY_<NAME>(e.g.ONR_ACCESS_KEY_CLIENT_A) - Otherwise:
ONR_ACCESS_KEY_<INDEX>(1-based)
onr-admin command usage is documented in:
onr-admin/USAGE.md
The repository now uses two Go modules:
.(onr runtime/server + onr-admin CLI)onr-core(shared library for external ecosystem and internal reuse)
For local multi-module development, use the repository root go.work.
Quick checks:
# onr
go test ./...
# onr-core
(cd onr-core && go test ./...)
# onr-admin (included in root module)
go test ./...onr-core is released with dedicated submodule tags: onr-core/vX.Y.Z.
go get github.com/r9s-ai/open-next-router/onr-core@v1.2.3You can configure an outbound HTTP proxy per upstream provider in onr.yaml:
upstream_proxies:
by_provider:
openai: "http://127.0.0.1:7890"
anthropic: "http://127.0.0.1:7891"Supported proxy URL schemes:
http:///https://socks5:///socks5h://(optional user/pass:socks5://user:pass@host:port)
Or override via environment variables:
ONR_UPSTREAM_PROXY_OPENAI=http://127.0.0.1:7890ONR_UPSTREAM_PROXY_ANTHROPIC=http://127.0.0.1:7891
When provider DSL auth uses OAuth directives, ONR can persist exchanged access tokens to local files.
oauth:
token_persist:
enabled: true
dir: "./run/oauth"Environment overrides:
ONR_OAUTH_TOKEN_PERSIST_ENABLED=true|falseONR_OAUTH_TOKEN_PERSIST_DIR=./run/oauth
- Override:
x-onr-provider: <provider>
In addition to OpenAI-style endpoints, open-next-router supports a subset of Gemini native endpoints:
POST /v1beta/models/{model}:generateContentPOST /v1beta/models/{model}:streamGenerateContent(SSE;alt=ssewill be added if missing)GET /v1beta/models(Gemini-style output)
Example (force provider selection via header):
curl -sS http://127.0.0.1:3300/v1beta/models/gemini-2.0-flash:generateContent \
-H "Authorization: Bearer change-me" \
-H "x-onr-provider: gemini" \
-H "Content-Type: application/json" \
-d '{"contents":[{"role":"user","parts":[{"text":"hello"}]}]}'You can bind a model to one or more providers. If a model is bound to multiple providers, open-next-router selects the provider using round-robin (per model).
Selection priority:
x-onr-providerheader (force)models.yamlrouting (per model round-robin)
Enable file-based traffic dump to capture request/response for debugging.
Configuration (config or env):
traffic_dump.enabled/ONR_TRAFFIC_DUMP_ENABLEDtraffic_dump.dir/ONR_TRAFFIC_DUMP_DIRtraffic_dump.file_path/ONR_TRAFFIC_DUMP_FILE_PATH(template supports{{.request_id}})traffic_dump.max_bytes/ONR_TRAFFIC_DUMP_MAX_BYTEStraffic_dump.mask_secrets/ONR_TRAFFIC_DUMP_MASK_SECRETStraffic_dump.sections/ONR_TRAFFIC_DUMP_SECTIONS(comma-separated allowlist; empty means all sections)
Captured sections (default: all; configurable via traffic_dump.sections):
=== META ====== ORIGIN REQUEST ====== UPSTREAM REQUEST ====== UPSTREAM RESPONSE ====== PROXY RESPONSE ====== STREAM ===
System logs are emitted to stderr in single-line text with a fixed prefix, and optional trailing KV fields.
Configuration (config or env):
logging.level(debug|info|warn|error)ONR_LOG_LEVEL
Example:
[ONR] 2026/02/27 - 12:34:56 | INFO | startup | startup config loaded | config_path=./onr.yaml providers_path=./config/onr.conf providers_source_is_file=true keys_file=./keys.yaml models_file=./models.yaml
[ONR] 2026/02/27 - 12:34:56 | INFO | startup | startup runtime flags | access_log_enabled=true access_log_target=stdout traffic_dump_enabled=false providers_auto_reload_enabled=false
[ONR] 2026/02/27 - 12:34:56 | INFO | server | open-next-router listening | listen_url=http://127.0.0.1:3300
Startup summary includes key runtime status fields:
config_pathproviders_path/providers_source_is_file/keys_file/models_filetraffic_dump_enabled/traffic_dump_dir/traffic_dump_max_bytesaccess_log_enabled/access_log_targetproviders_auto_reload_enabled/providers_auto_reload_debounce_mslisten_url(server listening log)
Built-in access log rotation is optional and applies to file output (logging.access_log_path).
Configuration (config or env):
logging.access_log_rotate.enabled/ONR_ACCESS_LOG_ROTATE_ENABLEDlogging.access_log_rotate.max_size_mb/ONR_ACCESS_LOG_ROTATE_MAX_SIZE_MBlogging.access_log_rotate.max_backups/ONR_ACCESS_LOG_ROTATE_MAX_BACKUPSlogging.access_log_rotate.max_age_days/ONR_ACCESS_LOG_ROTATE_MAX_AGE_DAYSlogging.access_log_rotate.compress/ONR_ACCESS_LOG_ROTATE_COMPRESS
Notes:
- When
logging.access_log_rotate.enabled=true,logging.access_log_pathmust be non-empty. - Rotation triggers on day boundary (local time) or when the file size threshold is exceeded.
Partnership with https://llmapis.com - Discover more AI tools and resources