8000 Publish RTX 4080 rollout and competitor evidence by DaoyuanLi2816 · Pull Request #96 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content

Publish RTX 4080 rollout and competitor evidence - #96

Merged
DaoyuanLi2816 merged 6 commits into
mainfrom
v0.11.0-rollout-evidence
Aug 29, 2026
Merged

DaoyuanLi2816 merged 6 commits into
mainfrom
v0.11.0-rollout-evidence

Conversation

@DaoyuanLi2816
Copy link
Copy Markdown
Owner

Summary

  • publish the exact-wheel 24-cell RTX 4080 hf_cached and vLLM rollout measurements
  • add machine-readable raw/result schemas and a deterministic aggregate publisher
  • select managed vLLM 0.28.0 for the measured direct-GKD development path
  • preserve the initial vLLM startup timing across later adapter refreshes
  • update English, Chinese, PyPI and benchmark documentation with the measured outcome

Measured outcome

  • vLLM delivered 108.5–117.7 output tokens/s and was 1.65–4.55x faster than hf_cached in every 256/512-token cell
  • vLLM peaked at 11,820 MiB total GPU memory, confirmed eight unique policy refreshes without monotonic growth, and closed its localhost port
  • the 32-token greedy probe agreed exactly; sampled-token log-probability max error was 0.0194 against the 0.01 NF4 limit, so PG-k1 remains fail-closed
  • hf_cached did not meet the preregistered 2x speed gate over hf_reference; v0.11.0 publication remains blocked
  • SGLang 0.5.18 stopped before throughput measurement at the recorded WSL2 CUDA-toolchain blockers

Raw SHA-256:

  • hf_cached: a9f8f0b0275d940f30497a0d88d76da0e112ffbe2bc1b37a42c6ed313b852242
  • vLLM: abf14d73a7289f9619515943b4c8d7dafe7cd187fefe928655739ac7ed1ab16c
  • aggregate: 32be86856263ccdf787986c4ec54570d323af8c238dc1664e00a9ce41ca393c4
  • frozen calculator: 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc

Validation

  • git diff --check
  • ruff check .
  • ruff format --check .
  • mypy src/miniverl
  • actionlint 1.7.12 with the workflow-pinned archive checksum
  • pytest -q -m "not gpu and not network" --cov=miniverl: 2468 passed, 14 skipped, 24 deselected, 83% coverage
  • pytest -q -m gpu: 10 passed, 2496 deselected
  • pytest -q -m network: 15 passed, 1 skipped, 2490 deselected
  • extracted-sdist suite: 2465 passed, 17 skipped, 24 deselected
  • mkdocs build --strict
  • browser visual gate: 44 rendered SVG instances across four viewports
  • Markdown links, PyPI README byte check and aggregate byte comparison
  • package build and twine check
  • clean torch-free core wheel install and doctor smoke

The performance gate failure is intentional release-blocking evidence. This PR does not tag or publish v0.11.0.

@DaoyuanLi2816
DaoyuanLi2816 merged commit 8125f6f into main Aug 29, 2026
16 checks passed
@DaoyuanLi2816
DaoyuanLi2816 deleted the v0.11.0-rollout-evidence branch August 29, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0