Add the measured single-GPU vLLM rollout backend by DaoyuanLi2816 · Pull Request #95 · DaoyuanLi2816/mini-verl

DaoyuanLi2816 · 2026-08-29T14:23:21Z

Summary

add a pinned, miniVERL-managed vLLM 0.28.0 rollout backend for the direct-GKD profile
bind the selected rollout backend into immutable plans and expose it through miniverl plan / miniverl run
synchronize content-bound PEFT adapters, request raw token IDs, and tear down the complete localhost process group before teacher scoring and training updates
keep PG-k1 on hf_cached because the external BF16 rollout engine did not pass the sampled-token log-probability conformance threshold against the NF4 actor

The single-RTX-4080 WSL2 spike compared the two candidate engines before implementation:

SGLang 0.5.18 could not complete the qualified path without a system CUDA compiler/JIT dependency and was rejected cleanly.
vLLM 0.28.0 completed health, raw-token, dynamic-LoRA, repeated refresh, teardown, and managed miniVERL end-to-end probes with the WSL2 compatibility flags recorded in the launcher.
With persistent prefix caching disabled, vLLM measured 238.27 tok/s at response length 256 and 284.18 tok/s at 512, versus 122.44 and 124.70 tok/s for hf_cached (1.95x and 2.28x).
Greedy outputs agreed on 32/32 tokens. Sampled-token log-probability error did not pass the PG gate (max 1.803, mean 0.309, p99 1.592), so the backend fails closed for log-probability-dependent training.

The formal checksummed scorecard and benchmark record remain a separate evidence PR. No frozen task result or scientific conclusion changes here.

git diff --check
ruff check .
ruff format --check .
mypy src/miniverl
actionlint 1.7.12
pytest -q -m "not gpu and not network" --cov=miniverl: 2469 passed, 9 skipped, 83% coverage
pytest -q -m gpu: 10 passed
pytest -q -m network: 16 passed
mkdocs build --strict
generated documentation artifact checks, Markdown links, and text-integrity checks
python -m build and python -m twine check dist/*
clean core wheel install and source/package inventory checks
frozen calculator JSON SHA-256 remains 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc

Add the measured single-GPU vLLM rollout backend

30503de

DaoyuanLi2816 merged commit 80c207a into main Aug 29, 2026
16 checks passed

DaoyuanLi2816 deleted the v0.11.0-vllm-rollout branch August 29, 2026 14:29