[Tinker compat] Concurrent sample requests are serialized, so vLLM never batches them (~25 tok/s at any concurrency) #297

• I have searched existing issues, and this is a new bug report.

Bug Description

Concurrent sample() requests through the Tinker API are processed one at a time, so vLLM never batches them. /tinker/asample goes through schedule_task(), the serial compute queue (task_queue/mixin.py#L3-L6, #L183-L185: "Tasks are processed one at a time").

Throughput stays flat as concurrency grows. RL rollouts depend on that batching.

How to Reproduce

Submit N sc.sample(prompt, num_samples=1, SamplingParams(max_tokens=128)) calls at once. With Qwen3-1.7B:

Additional Information

Suggestion: let sampler requests bypass the serial queue and go straight to the async vLLM engine, or coalesce queued sample tasks into one vLLM call. The serial queue still makes sense for the trainer.

Environment: twinkle-kit 0.5.1, vLLM 0.20.2 (max_num_seqs: 16, enforce_eager: true), single RTX 4090, tinker 0.16.1.