8000 Synchronize v0.14.0 publication and begin 0.14.1 development by DaoyuanLi2816 · Pull Request #119 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

## [0.14.0] - 2026-09-08
## [0.14.0] - 2026-09-09

### Installed local RL workflows

Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "miniVERL: Run verl experiment semantics on one GPU"
message: "If you use miniVERL in your work, please cite it as below."
type: software
version: 0.14.0
date-released: 2026-09-08
date-released: 2026-09-09
license: Apache-2.0
repository-code: "https://github.com/DaoyuanLi2816/mini-verl"
url: "https://github.com/DaoyuanLi2816/mini-verl"
Expand Down
38 changes: 19 additions & 19 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,40 +4,38 @@ Current maintainer handoff for **miniVERL** (`mini-verl` package, `miniverl`
CLI). `release-state.yaml` is the canonical version source; this page indexes
current product and evidence state rather than repeating release history.

Last updated: 2026-09-08.
Last updated: 2026-09-09.

Canonical release state: releasing `v0.14.0`.
Canonical release state: stable `v0.14.0` (`60afc63da1257e3e22aa6056607a63a90d4d90f1`), development `0.14.1.dev0`.

## Release state

- Active work: `v0.14.0-p0-productization`, based on verified main `cdc608f`.
The next release focuses on upstream configuration coverage, packaged PPO/GRPO
walkthroughs, run inspection and system-level recovery. Implementation is
settled; release metadata is finalized for exact-candidate qualification.
- v0.14.0 productization is published: upstream configuration coverage,
packaged PPO/GRPO walkthroughs, strict run inspection and system-level recovery.
- Upstream drift check: GitHub's latest stable release is still `v0.9.0`
(published 2026-08-14); existing compiler targets remain appropriate.
- Implemented: v3 prompt-minibatch units and explicit physical lowerings;
six upstream scripts/seven resolved cases with complete field accounting;
installed PPO/GRPO templates and adaptation ledgers; strict run inspection;
PPO value/behavior identity guards and checkpoint-bound log recovery.
- Development rehearsal: both wheel-installed workflows completed on RTX 4080;
PPO 4 actor/4 critic updates and GRPO 2 actor updates, with exact tensor resume
and inspected handoff. This dirty-tree rehearsal is not release evidence.
- Pending: final exact-commit qualification, publication and state sync.
- Local regression: 2,655 non-GPU/non-network tests passed at 83% branch
- Exact-release qualification run `34318236880` passed on attempt 1, followed
by non-publishing gate `34325442958` and OIDC release `34328973021`.
PPO 4 actor/4 critic updates and GRPO 2 actor updates passed exact tensor
replay and inspected handoff from the installed candidate wheel.
- Local regression: 2,656 non-GPU/non-network tests passed at 83% branch
coverage; 10 RTX 4080 GPU and 16 network tests passed. Four pinned upstream
conformance tests and four-viewport documentation browser checks passed.

- Stable release: `v0.13.0` at
`8e7bc3b5ff1145ae502cca893b7fffca3e12b6a8`.
- Release candidate line: `0.14.0`.
- Stable release: `v0.14.0` at
`60afc63da1257e3e22aa6056607a63a90d4d90f1`.
- Current development line: `0.14.1.dev0`.
- Stable docs: <https://daoyuanli2816.github.io/mini-verl/>.
- Development docs: <https://daoyuanli2816.github.io/mini-verl/dev/>.
- Historical build log: [v0.1-v0.9 archive](docs/history/project-state-v0.1-v0.9.md).

## Current product boundary

Release `0.14.0` has a closed typed profile compiler: it retains v1/v2 and adds
Development `0.14.1.dev0` has a closed typed profile compiler: it retains v1/v2 and adds
`verl-rl-v0.9-single-gpu-v3`, a typed resolved RL subset of official verl
`v0.9.0` at commit `483b8a009ba3a97563edee3a19887e4862b8094a`.
It compiles PPO/GAE into phased one-GPU actor, critic, reference and reward
Expand Down Expand Up @@ -83,6 +81,8 @@ v0.8 environment/PPO artifact bridge is migration-only.

| Evidence | Status |
| --- | --- |
| Installed PPO product workflow | Qwen3-0.6B, 8 trajectories, 4 actor/4 critic updates, 2.7266 GiB peak reserved, 15.688 s reported train duration; exact tensor replay and handoff passed |
| Installed GRPO product workflow | Qwen3-0.6B, 8 trajectories, 2 actor updates, 1.502 GiB peak reserved, 8.923 s reported train duration; exact tensor replay and handoff passed |
| Qwen3-0.6B/1.7B developer workload | 32 prompts, 8 current-policy updates, 3.1914 GiB peak reserved on one RTX 4080; matched interruption/resume was byte-identical |
| Qwen3 sampled-k1 PG | 32 prompts, 8 updates, exact interruption/resume, 3.1914 GiB peak reserved; no quality comparison |
| SmolLM2-360M/1.7B direct GKD | 32 prompts, 8 updates, 1.4961 GiB peak reserved; exact resume, PEFT reload and materialized export passed |
Expand All @@ -96,11 +96,11 @@ calculator schema-v2 source stays at SHA-256

## Maintainer entry points

- [Current local runtime](docs/verl-opd-runtime.md)
- [Installed PPO/GRPO workflow](docs/local-rl-workflow.md)
- [Single-GPU verl RL](docs/verl-rl-runtime.md)
- [Current scale-out contract](docs/verl-opd-scaleout.md)
- [Current scale-out contract](docs/verl-rl-runtime.md#scale-out-handoff)
- [Compatibility policy](docs/compatibility.md)
- [Measured workload](docs/verl-opd-reference-workload.md)
- [Limitations](docs/limitations.md)
- Active roadmap: [issue #39](https://github.com/DaoyuanLi2816/mini-verl/issues/39)
and [issue #64](https://github.com/DaoyuanLi2816/mini-verl/issues/64)
- Active research roadmap: [issue #39](https://github.com/DaoyuanLi2816/mini-verl/issues/39).
Additional compiler lowerings are indexed by the [upstream corpus](docs/verl-compatibility-corpus.md).
48 changes: 24 additions & 24 deletions PYPI.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.14.0/docs/banner.svg" alt="miniVERL — lower verl experiment semantics onto one CUDA GPU" width="880">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/banner.svg" alt="miniVERL — lower verl experiment semantics onto one CUDA GPU" width="880">
</p>

<div align="center">
Expand All @@ -8,15 +8,15 @@
[![Build](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml/badge.svg)](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml)
[![PyPI](https://img.shields.io/pypi/v/miniverl.svg)](https://pypi.org/project/miniverl/)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](https://www.python.org)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/LICENSE)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE)

</div>

<p align="center">
<a href="https://pypi.org/project/miniverl/"><strong>PyPI</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/"><strong>Stable docs</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/dev/">Development docs</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/README.zh-CN.md">中文</a>
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/main/README.zh-CN.md">中文</a>
</p>

**miniVERL runs a validated subset of verl experiment semantics on one NVIDIA
Expand Down Expand Up @@ -60,13 +60,13 @@ miniverl export-verl --run runs/local-ppo --target-verl v0.9.0 --out ppo-handoff
miniverl bridge doctor ppo-handoff --json
```

The [five-minute workflow](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/for-verl-users.md) explains each artifact and
The [five-minute workflow](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/for-verl-users.md) explains each artifact and
offers the matching GRPO commands. These are small length-reward exercises;
their reward is directly inspectable in `rewards.jsonl`.

`[train]` supplies the ML stack; `[cuda]` additionally supplies quantization.
Select PyTorch's CUDA build with the [PyTorch installer](https://pytorch.org/get-started/locally/).
The [single-GPU guide](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/single-gpu-guide.md) covers memory planning and the
The [single-GPU guide](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md) covers memory planning and the
maintainer-measured RTX 4080 environment.

## What a run gives you
Expand All @@ -85,8 +85,8 @@ maintainer-measured RTX 4080 environment.
## How it works

<picture>
<source media="(max-width: 640px)" srcset="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.14.0/docs/verl-local-runtime-mobile.svg">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.14.0/docs/verl-local-runtime.svg" alt="A resolved verl config compiles into a validated single-GPU plan; actor, critic, reference, teacher and reward roles run in phases and produce portable artifacts plus a readiness report.">
<source media="(max-width: 640px)" srcset="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-local-runtime-mobile.svg">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-local-runtime.svg" alt="A resolved verl config compiles into a validated single-GPU plan; actor, critic, reference, teacher and reward roles run in phases and produce portable artifacts plus a readiness report.">
</picture>

One GPU is treated as a temporal scheduler for logical roles. RL generates
Expand All @@ -100,10 +100,10 @@ the learning signal.

| Goal | First command | Primary artifact | Next step |
| --- | --- | --- | --- |
| **Run local RL** | `miniverl import-verl --profile verl-rl-v0.9-single-gpu-v3 --example grpo --out local.yaml` | native recipe + compatibility report | [RL quickstart](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/verl-rl-runtime.md) |
| **Run local OPD** | `miniverl plan --profile verl-opd-v0.8-single-gpu-v1 --config verl-opd.yaml --out plan.json` | immutable execution plan | [OPD quickstart](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/opd-quickstart.md) |
| **Fit your GPU** | `miniverl plan --config verl-opd.yaml --probe` | measured placement plan | [Hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/hardware-planning.md) |
| **Hand off artifacts** | `miniverl export-verl --run runs/my-run --target-verl v0.9.0 --out scaleout` | actor, critic, Parquet + config bundle | [Compatibility contract](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/compatibility.md) |
| **Run local RL** | `miniverl import-verl --profile verl-rl-v0.9-single-gpu-v3 --example grpo --out local.yaml` | native recipe + compatibility report | [RL quickstart](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-rl-runtime.md) |
| **Run local OPD** | `miniverl plan --profile verl-opd-v0.8-single-gpu-v1 --config verl-opd.yaml --out plan.json` | immutable execution plan | [OPD quickstart](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/opd-quickstart.md) |
| **Fit your GPU** | `miniverl plan --config verl-opd.yaml --probe` | measured placement plan | [Hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/hardware-planning.md) |
| **Hand off artifacts** | `miniverl export-verl --run runs/my-run --target-verl v0.9.0 --out scaleout` | actor, critic, Parquet + config bundle | [Compatibility contract](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md) |

Native recipes also support SFT, DPO and offline KD, plus calculator, JSON
navigation, read-only SQLite and custom tool environments.
Expand All @@ -121,7 +121,7 @@ navigation, read-only SQLite and custom tool environments.
| Direct GKD / sampled-k1 OPD | supported | pinned verl v0.8 profiles with teacher targets |
| Ray, FSDP/FSDP2, Megatron, TP/PP/DP > 1 | distributed-only | these change physical scale, not the local objective |

The [upstream compatibility corpus](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/verl-compatibility-corpus.md) resolves
The [upstream compatibility corpus](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-compatibility-corpus.md) resolves
real verl examples and records every field's outcome. The v3 profile preserves
prompt-based minibatch units; v1/v2 remain available for existing recipes.

Expand All @@ -131,31 +131,31 @@ The published Qwen3-0.6B/1.7B OPD workload consumed 32 distinct prompts and
completed eight current-policy updates at 3.1914 GiB peak reserved VRAM on an
RTX 4080. A matched interruption after update four reproduced byte-identical
trajectories, adapter and optimizer tensors. The SmolLM2-360M/1.7B workload
completed the same shape at 1.4961 GiB. [System records](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/verl-opd-reference-workload.md)
completed the same shape at 1.4961 GiB. [System records](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-opd-reference-workload.md)
contain configs, hashes and phase timings.

Rollout Runtime v2 measured `hf_cached` and managed vLLM across 24 RTX 4080
cells spanning response length, sampling mode and `n=1/4`. See the
[runtime report](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/benchmarks/rollout-runtime-v2.md). Evidence for the new
[runtime report](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarks/rollout-runtime-v2.md). Evidence for the new
RL family is published through the exact-wheel release qualification rather
than presented as a task-quality comparison.

## Research record

The scientific reports preserve their original outcomes and frozen inputs:
[calculator protocol](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/benchmarking.md),
[RecoveryBench](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/recoverybench/recoverybench-v1.md),
[Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/alignment-lab/alignment-lab-v1.md), and the
[External Alignment Gate](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/alignment-external/alignment-external-v1.md).
[calculator protocol](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarking.md),
[RecoveryBench](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/recoverybench/recoverybench-v1.md),
[Alignment Lab](https://github.com/DaoyuanLi2816/min 8000 i-verl/blob/main/docs/alignment-lab/alignment-lab-v1.md), and the
[External Alignment Gate](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-external/alignment-external-v1.md).
They are scoped studies, separate from runtime qualification.

## Compatibility boundary

miniVERL targets one local process and one NVIDIA CUDA GPU. The compiler
distinguishes exact or conformant semantics, local physical lowering,
distributed-only settings, feasible work not yet implemented, and technically
unsupported inputs. The [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/compatibility.md) is the
complete matrix; [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/limitations.md) collects architecture,
unsupported inputs. The [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md) is the
complete matrix; [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md) collects architecture,
measurement, security and generalization boundaries in one place. miniVERL is
an independent Apache-2.0 project.

Expand All @@ -168,7 +168,7 @@ python -m pip install -e ".[dev,train]"
pytest -q -m "not gpu and not network"
```

See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/CONTRIBUTING.md), [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/CHANGELOG.md),
[CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/CITATION.cff), the [guide for verl users](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/docs/for-verl-users.md),
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/SECURITY.md) and the
[Apache-2.0 license](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.14.0/LICENSE).
See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CONTRIBUTING.md), [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CHANGELOG.md),
[CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CITATION.cff), the [guide for verl users](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/for-verl-users.md),
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/SECURITY.md) and the
[Apache-2.0 license](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE).
32 changes: 23 additions & 9 deletions docs/generated/quality.json
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
{
"schema_version": 2,
"release": "0.14.0",
"status": "candidate",
"status": "released",
"quality_floor": "2,000+ tests and 80%+ branch coverage at v0.14.0",
"local_validation": {
"scope": "P0 implementation and CLI compatibility regression on the maintainer workstation",
"commit": "39de76ad4399318d4b19e82f7a4f0d90316054c7",
"commit_relationship": "implementation commit; release metadata and citation wording follow in the candidate commit",
"measured_at": "2026-09-09T05:55:00Z",
"commit": "f128c98456929cdc53f9f094f5ecf8a791690aed",
"commit_relationship": "final PR head; the squash release commit has the same tree",
"measured_at": "2026-09-09T06:05:10Z",
"platform": "Windows 11",
"python": "CPython 3.10.11",
"coverage_mode": "branch",
"cpu_non_gpu_non_network": {
"passed": 2655,
"passed": 2656,
"skipped": 14,
"deselected": 24,
"branch_coverage_percent": 83.0,
Expand All @@ -24,10 +24,24 @@
"docs_visual": {"viewports": 4, "rendered_svg_instances": 44}
},
"release_validation": {
"commit": "pending",
"gpu_coverage": "pending exact-release-SHA full qualification",
"commit": "60afc63da1257e3e22aa6056607a63a90d4d90f1",
"gpu_coverage": "attempt-1 full qualification; installed PPO and GRPO command workflows, exact tensor resume, v0.9 handoff and historical regression matrix passed on one WSL2 RTX 4080",
"scope": "exact release candidate wheel",
"conclusion": "pending",
"required": "attempt-1 full RTX 4080 qualification, installed PPO/GRPO workflows, exact-byte release dry run and public artifact verification"
"conclusion": "passed",
"workflows": {
"ci": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34318072752",
"build": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34318072732",
"docs": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34328973019",
"pinned_verl_bridge": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34318072653",
"gpu_full_qualification": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34318236880",
"release_dry_run": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34325442958",
"release": "https://github.com/DaoyuanLi2816/mini-verl/actions/runs/34328973021"
},
"publication": {
"pypi": "https://pypi.org/project/miniverl/0.14.0/",
"github_release": "https://github.com/DaoyuanLi2816/mini-verl/releases/tag/v0.14.0",
"wheel_sha256": "f44e607acb2b20608ac217891862374743f7dd443d934664dd8c0349250fbf26",
"sdist_sha256": "fe9584bb9450a183372151dc1dea463d268a01cc3b6181afb4aaf54046825324"
}
}
}
Loading
0