8000 Add typed Rollout Runtime v2 and batched KV-cache HF backend by DaoyuanLi2816 · Pull Request #92 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]
Skip to content

Add typed Rollout Runtime v2 and batched KV-cache HF backend - #92

Merged
DaoyuanLi2816 merged 2 commits into
mainfrom
v0.11.0-rollout-core
Aug 29, 2026
Merged

Add typed Rollout Runtime v2 and batched KV-cache HF backend#92
DaoyuanLi2816 merged 2 commits into
mainfrom
v0.11.0-rollout-core

Conversation

@DaoyuanLi2816
@DaoyuanLi2816 DaoyuanLi2816 commented Aug 29, 2026 •
edited
Loading
Copy link
Copy Markdown
Owner

Summary

  • add a torch-free Rollout Runtime v2 contract for generation requests/results, grouped sample identity, actor policy identity, synchronization, lifecycle and backend metrics
  • preserve hf_reference as the default for every existing recipe and add opt-in hf_cached
  • implement padded prefill plus incremental KV-cache decode, finished-row masking, per-row RNG, greedy/stochastic sampling, sampled-token log-probabilities, EOS/text/max-token provenance and existing OOM bisection
  • bind prompt-generation manifests and requests to the live parameter, model, tokenizer, adapter, backend, profile and execution-plan identity
  • expose generation-only compile_backend without changing scoring or training forwards

samples_per_prompt is typed now but values above one still fail during config validation. Transactional grouped semantics remain PR 3.

Correctness

On a locally constructed tiny Qwen3 CUDA model, hf_cached matched per-request hf_reference for greedy and seeded stochastic generation:

  • output token ids: exact
  • stop reason: exact
  • FP32 sampled-token log-probabilities: maximum tolerance 2e-4
  • physical partitions [4] and [2,2]: identical token ids, log-probabilities within 2e-4
  • base-package imports remain torch-free

RTX 4080 profile

Same WSL2 RTX 4080, Qwen3-0.6B revision and preregistered workload identity as the frozen reference baseline. This is an implementation profile, not a committed scientific result.

Cell hf_reference compiled hf_cached Speedup Peak reserved
p128 / r256 / n1 / greedy 95.866 tok/s 318.873 tok/s 3.33x 1.18 GiB
p128 / r512 / n1 / greedy 93.158 tok/s 327.649 tok/s 3.52x 1.42 GiB

Eight consecutive cached generations held reserved memory constant at
1,520,435,200 bytes; teardown residual allocated memory was 8,520,704
bytes. The first Inductor compile took over two minutes, so compilation is
explicit and documented rather than a legacy default. Eager cached generation
was correct but measured only about 1.38x on the r256 greedy cell. A static-cache
attempt regressed to about 1.13x and was not retained.

Validation

  • ruff check .
  • ruff format --check .
  • mypy src/miniverl
  • 2410 passed, 9 skipped, 24 deselected for non-GPU/non-network tests; coverage 83%
  • 10 passed, 2433 deselected for GPU tests
  • 16 passed, 2427 deselected for network tests
  • mkdocs build --strict
  • Markdown links, text integrity, release-state and actionlint checks
  • wheel/sdist build and Twine check

PR verification artifacts:

  • wheel SHA-256: 5b96e5fb5919c184298378498183242f63a257bc40d07049018b3cd6ac5d5ca8
  • sdist SHA-256: bee73b417f05bb8df2b67f59943aeb0fada123281e60cc0dc4b22d4d3b801774

Frozen calculator SHA-256 remains
53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc.
The frozen hf_reference result SHA-256 remains
b25daee7ee726b7a7be18d7dbe26590fb61325bf212cfd9e7110b69f5fa8889c.

@DaoyuanLi2816
DaoyuanLi2816 merged commit 38748a2 into main Aug 29, 2026
16 checks passed
@DaoyuanLi2816
DaoyuanLi2816 deleted the v0.11.0-rollout-core branch August 29, 2026 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@DaoyuanLi2816
0