8000 v0.15.0: run resolved verl configs directly on one GPU by DaoyuanLi2816 · Pull Request #120 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@ jobs:
python scripts/release_state.py --check
python scripts/check_known_good_environment.py
python scripts/publish_gpu_qualification_schema.py --check
python scripts/publish_verl_direct_schema.py --check
python scripts/publish_direct_examples.py --check
python scripts/publish_direct_compatibility.py --check
python scripts/check_markdown_links.py
python scripts/check_text_integrity.py
miniverl --help
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,13 @@ jobs:
--output "$full_dir/v014-product.json" \
--work "$full_work/v014-product" \
"${offline[@]}"
"$QUALIFICATION_PYTHON" scripts/run_v015_direct_qualification.py \
--commit "$GITHUB_SHA" \
--wheel "$QUALIFICATION_WHEEL" \
--wheel-sha256 "$QUALIFICATION_WHEEL_SHA256" \
--output "$full_dir/v015-direct.json" \
--work "$full_work/v015-direct" \
"${offline[@]}"
test -z "$(git status --porcelain)"
"$QUALIFICATION_PYTHON" scripts/benchmark_rollout_runtime_v2_selected.py \
--backend hf_cached \
Expand All @@ -231,7 +238,8 @@ jobs:
--hf-reference benchmarks/results/rollout-runtime-v2-hf-reference.json \
--v012-rl "$full_dir/v012-rl.json" \
--v013-ppo "$full_dir/v013-ppo.json" \
--v014-product "$full_dir/v014-product.json"
--v014-product "$full_dir/v014-product.json" \
--v015-direct "$full_dir/v015-direct.json"
- name: Upload full qualification records
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: ${{ success() && inputs.qualification_level == 'full' }}
Expand Down
38 changes: 37 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,40 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

## [0.15.0] - 2026-09-10

### Direct verl configurations

- Run resolved upstream PPO/GRPO-style YAML directly with `miniverl run FILE`.
The immutable v4 compiler separates semantic acceptance, missing local
bindings and exact-model hardware feasibility. Preserve v1/v2/v3 identities.
- Classify all leaves of seven complete pinned upstream cases; add five stress
variants, including explicit unsupported sample-selection behavior.
- Add typed model/data/reward bindings, checksum-approved Python reward code,
snapshot resolution, physical microbatch one and temporal role placement.
- Retain source bytes, field decisions, bindings, local hardware plan and IR
identity through execution and checkpoint resume.

### Experiment semantics

- Support upstream token/sequence reductions, with scalar, gradient and
optimizer conformance; retain logical denominators across physical batches.
- Filter overlong prompts before sampling, preserve drop-last traversal, derive
epoch schedules from retained rows, and execute scheduled reward validation.
- Support repeated critic-free actor epochs, independent critic snapshots and
minibatches, and rollout-iteration AdamW learning-rate schedules.
- Add an independent frozen-base KL reference and the upstream discriminative
classifier path using chat rendering and raw logits.
- Preserve new reduction, validation and critic settings in fail-closed v4
handoff templates. Report unmeasured success as N/A and identify RL as on-policy.

### Release integrity

- Gate publication on installed-candidate direct PPO and GRPO runs, exact
actor/critic/optimizer replay, source-byte retention and inspected handoff.
- Keep all frozen scientific JSON/JSONL, published tags and model revisions
unchanged. This release adds systems compatibility, not a new quality result.

## [0.14.0] - 2026-09-09

### Installed local RL workflows
Expand Down Expand Up @@ -1234,7 +1268,9 @@ Same-tokenizer only; one trajectory per forward pass; `swap` unavailable for
quantized models; only Qwen3 and Qwen2 architectures tested; single-seed GPU
results. The full list is in `docs/limitations.md`.

[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.14.0...HEAD
[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.15.0...HEAD
[0.15.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.14.0...v0.15.0
[0.14.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.13.0...v0.14.0
[0.13.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.12.0...v0.13.0
[0.12.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.11.0...v0.12.0
[0.11.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.10.1...v0.11.0
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ cff-version: 1.2.0
title: "miniVERL: Run verl experiment semantics on one GPU"
message: "If you use miniVERL in your work, please cite it as below."
type: software
version: 0.14.0
date-released: 2026-09-09
version: 0.15.0
date-released: 2026-09-10
license: Apache-2.0
repository-code: "https://github.com/DaoyuanLi2816/mini-verl"
url: "https://github.com/DaoyuanLi2816/mini-verl"
Expand Down
21 changes: 17 additions & 4 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,25 @@ Current maintainer handoff for **miniVERL** (`mini-verl` package, `miniverl`
CLI). `release-state.yaml` is the canonical version source; this page indexes
current product and evidence state rather than repeating release history.

Last updated: 2026-09-09.
Last updated: 2026-09-10.

Canonical release state: stable `v0.14.0` (`60afc63da1257e3e22aa6056607a63a90d4d90f1`), development `0.14.1.dev0`.
Canonical release state: releasing `v0.15.0`.

## Release state

- v0.15.0 is preparing the direct-config release. Implementation and installed
development-wheel PPO/GRPO rehearsals pass; formal candidate qualification
and OIDC publication are pending.
- v4 semantically accepts all seven complete pinned upstream configurations.
Five derived stress cases add two accepted and three explicit rejections.
Exact production-model execution is not claimed by static compilation.
- The installed direct rehearsal completed four actor updates in both cases,
four PPO critic updates, reward validation, exact tensor replay and handoff.
- Local full regression: 2,725 passed, 82.01% branch coverage, plus 18 pinned
upstream conformance cases and four documentation viewports.

### Prior stable release (historical)

- v0.14.0 productization is published: upstream configuration coverage,
packaged PPO/GRPO walkthroughs, strict run inspection and system-level recovery.
- Upstream drift check: GitHub's latest stable release is still `v0.9.0`
Expand All @@ -35,8 +48,8 @@ Canonical release state: stable `v0.14.0` (`60afc63da1257e3e22aa6056607a63a90d4d

## Current product boundary

Development `0.14.1.dev0` has a closed typed profile compiler: it retains v1/v2 and adds
`verl-rl-v0.9-single-gpu-v3`, a typed resolved RL subset of official verl
Release `0.15.0` has a closed typed profile compiler: it retains v1/v2/v3 and adds
`verl-rl-v0.9-single-gpu-v4`, direct execution of resolved RL inputs from official verl
`v0.9.0` at commit `483b8a009ba3a97563edee3a19887e4862b8094a`.
It compiles PPO/GAE into phased one-GPU actor, critic, reference and reward
roles. The critic is independently trainable and owns its optimizer, schedule,
Expand Down
Loading
0