[Bug]: gRPC router workers can never become ready when model is set #4240

Steps to reproduce

1. Create a service with a router replica group and a gRPC worker replica group, and set model on the service:

type: service
name: qwen38-27b-agentic-smg

groups:
  - replicas: 1
    python: "3.12"
    commands:
      - pip install smg
      - smg launch --host 0.0.0.0 --port 8000 --enable-igw --policy cache_aware --model-path Qwen/Qwen3.8-27B
    router:
      type: sglang
    resources:
      gpu: 0

  - replicas: 1
    image: vllm/vllm-openai:qwen38
    commands:
      - pip install smg-grpc-servicer
      - python3 -m vllm.entrypoints.grpc_server --host 0.0.0.0 --port 8000 --model Qwen/Qwen3.8-27B-FP8
    resources:
      gpu: H100:80GB

port: 8000
model:
  type: chat
  name: Qwen/Qwen3.8-27B
  format: openai
  prefix: /v1

2. Apply it and wait for both replicas to reach running.
3. Send a request to the service endpoint.

Actual behaviour

The request fails with SMG's own error:

{"error":{"type":"Not Found","code":"model_not_found",
 "message":"No worker available for model 'Qwen/Qwen3.8-27B'","param":null}}

The worker never registers with the router. dstack ps shows both replicas running, the worker's engine is fully up (VllmEngineServicer initialized, Standard gRPC health service initialized), and the router is reachable — but the router log contains zero POST /workers, and Tree memory: string_trees=0 models.

Chain

Setting model injects a default HTTP probe. _probes() has no router/worker distinction, so it is applied to every replica group:

That probe cannot succeed against a gRPC worker, because probes are HTTP-only:

Server log, replica 1 being the gRPC worker and replica 0 the router:

probe(d861b2)qwen38-27b-agentic-smg-0-1-0: probe failed: RemoteProtocolError('illegal request line')
probe(b4838d)qwen38-27b-agentic-smg-0-0-0: probe status code: 404

illegal request line is an HTTP client hitting a gRPC port. With the probe failing, is_job_ready() stays false and JobModel.ready stays 0 for both replicas.

The router-worker sync then loads only ready jobs:

so _get_router_job() finds nothing, sync_router_workers_for_run_model() returns immediately, and no worker is ever pushed to the router. This is visible as a sub-25ms sync:

DEBUG Processing service_router_worker_sync item 9afc133b-...
DEBUG Processed  service_router_worker_sync item 9afc133b-... in 0.010

Nothing is logged when this happens — the early return is silent, so from the outside the service looks healthy.

Why the documented example is unaffected

vLLM PD disaggregation uses gRPC workers and works, because it sets neither model nor probes. _probes() then returns [], and is_job_ready() is all() over an empty sequence, i.e. True.

SGLang PD disaggregation sets model but also an explicit /health probe, which works because SGLang workers speak HTTP.

The broken combination is model + gRPC workers, which neither example covers.

Note this is easy to hit unintentionally: presets generate model by default, so a service derived from a preset gets the HTTP probe without the user writing one.

Expected behaviour

A gRPC worker replica group should be able to become ready. Options, roughly in order of preference:

• A gRPC probe type. ProbeConfig.type is Literal["http"] with the comment # expect other probe types in the future; vLLM's gRPC server already registers the standard grpc.health.v1.Health service, so a gRPC health probe would work out of the box.
• Per-group probes, so a router group and a worker group can be probed differently. Today probes is service-wide, but with a router the two groups routinely speak different protocols.
• At minimum, do not apply the model default probe to non-router replica groups, and log when the router-worker sync returns early because no ready router job was found.

dstack version

0.21.3 (master, e828c9e14)

Additional information

Workaround, confirmed working: set probes: [] explicitly. The field is documented as [] = explicit empty, which bypasses the model default. Both replicas then flip to ready=1 and the sync starts reaching the router. The cost is losing dstack-side health detection; for a router deployment SMG does its own worker health checking and circuit breaking, so this is tolerable.

The other workaround is to omit model, matching the vLLM PD example.