8000 Bind runs to immutable execution plans by DaoyuanLi2816 · Pull Request #69 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

### Immutable plan/run workflow

- Added deterministic `plan --out plan.json` artifacts that bind the source
YAML, ordered overrides, accepted compatibility matrix, immutable model
revisions, scanned Parquet hashes/schema/rows, exact native config and
weight-free physical recommendations.
- Added `run --plan` with fail-closed plan, source, data and minor-version
validation. The plan digest is carried by run manifests, teacher caches and
checkpoints; direct `run --config` remains supported.

### Verl config UX

- Added safe trailing Hydra-style overrides and repeatable plain/JSON
Expand Down
8 changes: 8 additions & 0 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ fails closed if one of those values drifts. A generated report classifies all
semantically conformant, 22 locally reinterpreted, 18 informational and 3
unsupported. No algorithm or distributed boundary widened.

The second slice adds deterministic immutable OPD plans. `plan --out` hashes
the source config and every Parquet file, scans schema/content/row identity,
requires immutable model commits and seals the accepted compatibility result,
native config and weight-free recommendations. `run --plan` revalidates those
bytes without recompiling; the canonical plan digest propagates into the run
manifest, teacher cache and every checkpoint. Tokenizer structural identity is
truthfully deferred until tokenizers are loaded by the later bounded probe.

## v0.8.1 product surface

The landing pages now lead with the documented one-GPU verl-style OPD journey,
Expand Down
16 changes: 7 additions & 9 deletions PYPI.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,9 @@ python -m pip install "miniverl[train]"
miniverl data sample --format verl-parquet --out prompts.parquet
miniverl plan --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]'
--set 'data.train_files=["prompts.parquet"]' --out plan.json
miniverl run --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]' --dry-run
--plan plan.json --dry-run
```

These commands need no Git checkout. `data sample` creates a real structured
Expand Down Expand Up @@ -199,12 +198,11 @@ successful bridge check never means that a distributed verl job ran. Review
the [bridge contract](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-bridge.md) and [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md).

The intended operating loop is **plan → inspect → run → inspect → export**.
Direct execution remains convenient for experiments, but the JSON plan and
compatibility report are the durable review surfaces: they expose model pins,
data paths, loss details, physical batches, unsupported fields, estimated
memory and every local reinterpretation before weights are allocated. Run
artifacts then add measurements and hashes without rewriting the original
source intent.
`plan --out` byte-binds the YAML, ordered overrides and scanned Parquet inputs
to the exact native config; `run --plan` rejects drift before loading weights.
Its digest follows the run manifest, teacher cache and checkpoints. Direct
`run --config` remains available for experiments. See [immutable execution
plans](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/immutable-plans.md).

## Research and validation

Expand Down
16 changes: 7 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,9 @@ python -m pip install "miniverl[train]"
miniverl data sample --format verl-parquet --out prompts.parquet
miniverl plan --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]'
--set 'data.train_files=["prompts.parquet"]' --out plan.json
miniverl run --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]' --dry-run
--plan plan.json --dry-run
```

These commands need no Git checkout. `data sample` creates a real structured
Expand Down Expand Up @@ -199,12 +198,11 @@ successful bridge check never means that a distributed verl job ran. Review
the [bridge contract](docs/verl-bridge.md) and [compatibility policy](docs/compatibility.md).

The intended operating loop is **plan → inspect → run → inspect → export**.
Direct execution remains convenient for experiments, but the JSON plan and
compatibility report are the durable review surfaces: they expose model pins,
data paths, loss details, physical batches, unsupported fields, estimated
memory and every local reinterpretation before weights are allocated. Run
artifacts then add measurements and hashes without rewriting the original
source intent.
`plan --out` byte-binds the YAML, ordered overrides and scanned Parquet inputs
to the exact native config; `run --plan` rejects drift before loading weights.
Its digest follows the run manifest, teacher cache and checkpoints. Direct
`run --config` remains available for experiments. See [immutable execution
plans](docs/immutable-plans.md).

## Research and validation

Expand Down
12 changes: 6 additions & 6 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,9 @@ python -m pip install "miniverl[train]"
miniverl data sample --format verl-parquet --out prompts.parquet
miniverl plan --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]'
--set 'data.train_files=["prompts.parquet"]' --out plan.json
miniverl run --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]' --dry-run
--plan plan.json --dry-run
```

这些命令不需要 Git checkout。`data sample` 生成真实的结构化 Parquet;`plan` 在不加载
Expand Down Expand Up @@ -165,9 +164,10 @@ snapshot materialize 之前仍报告 `launchable: false`。产物完整性、上
launchability、算法语义与 distributed execution 分开报告。bridge doctor 通过并不表示运行过
分布式 verl。详见[桥接契约](docs/verl-bridge.md)与[兼容性政策](docs/compatibility.md)。

建议操作闭环是 **plan → inspect → run → inspect → export**。JSON plan 与兼容报告在分配
权重前明确模型 pin、数据路径、loss、physical batch、不支持字段、显存 estimate 与所有本地
重解释;run artifact 只在其上补充 measurement 与 hash,不会改写源意图。
建议操作闭环是 **plan → inspect → run → inspect → export**。`plan --out` 把 YAML、按顺序
应用的 override、扫描后的 Parquet 与精确 native config 绑定;`run --plan` 在加载权重前拒绝
任何漂移。plan digest 同时进入 run manifest、teacher cache 和 checkpoint;直接
`run --config` 仍适合快速实验。详见[不可变执行计划](docs/immutable-plans.md)。

## 研究与验证

Expand Down
4 changes: 4 additions & 0 deletions docs/config-overrides.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ This flag accepts only the high-risk mappings printed in that compiled report.
It never enables an unsupported algorithm, multi-GPU field, FSDP behavior or
policy-gradient objective.

For a reviewed run, bind the acceptance and exact inputs once with
`miniverl plan --out plan.json`, then use `miniverl run --plan plan.json`; see
[Immutable execution plans](immutable-plans.md).

## Official-example field coverage

The generated [coverage report](generated/verl-opd-v08-official-fields.json)
Expand Down
5 changes: 5 additions & 0 deletions docs/for-verl-users.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ matrix. The development CLI records repeated `--set`, override files and
trailing tokens with deterministic precedence; it does not execute `${...}`
interpolations or shell text. See [Config overrides](config-overrides.md).

For a serious run, add `--accept-local-reinterpretations --out plan.json`,
inspect the immutable artifact, then execute `miniverl run --plan plan.json`.
This binds the source YAML, overrides and scanned Parquet bytes to the exact
native config; see [Immutable execution plans](immutable-plans.md).

## Same fields, different placement

An upstream-shaped fragment can stay recognizable:
Expand Down
41 changes: 41 additions & 0 deletions docs/immutable-plans.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Immutable execution plans

For a serious run, freeze the resolved config and Parquet inputs before loading
model weights:

```bash
miniverl plan --config verl-opd.yaml \
--accept-local-reinterpretations \
--out plan.json --offline
miniverl run --plan plan.json --offline
```

`plan --out` is still weight-free. It scans the actual Parquet source and
atomically writes a deterministic JSON artifact containing:

- the source YAML byte digest and every ordered override;
- the full compatibility report and explicit reinterpretation acceptance;
- immutable student and teacher revisions;
- every Parquet file hash plus schema, content and row-count identities;
- the exact validated native `RunConfig`, loss semantics and physical
recommendations;
- declared tokenizer revisions. Structural tokenizer identities remain
`declared_not_loaded` until the bounded hardware probe loads them.

The plan's canonical digest excludes only its own digest fields. `run --plan`
validates the schema and miniVERL minor-version boundary, recomputes that digest,
rechecks the YAML and every Parquet byte, rescans the data manifest, and consumes
the sealed native config. It never recompiles with new defaults. Model
construction starts only after all checks pass.

The same plan digest is recorded in the run manifest, teacher-cache index,
checkpoint state and checkpoint identity. A cache or checkpoint from another
plan is refused. Direct `run --config` remains available for quick experiments,
but `--plan` is mutually exclusive with config and override options.

## Deliberate limits

An immutable plan does not download or hash model snapshots, allocate CUDA, or
claim measured memory. Those identities become measured only through the
explicit probe/materialization stages. Moving model revisions, unsupported verl
semantics and unaccepted local reinterpretations prevent plan publication.
9 changes: 8 additions & 1 deletion docs/release-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ verl check before squash merge `8d3ebb2`.

- [x] Add upstream-shaped override UX with complete source and precedence
provenance while preserving the repeatable `--set` interface.
- [ ] Bind execution to an immutable plan artifact and fail closed when its
- [x] Bind execution to an immutable plan artifact and fail closed when its
config, data, model, tokenizer or compatibility acceptance has drifted.
- [ ] Add a bounded hardware probe, transactional model materialization and a
realistic one-GPU quickstart without widening the documented algorithm.
Expand All @@ -42,6 +42,13 @@ non-GPU/non-network tests at 85.03% branch coverage, 8 RTX 4080 GPU tests,
rendered SVG instances across four Playwright viewports, package/Twine checks
and the pinned-verl compatibility tests available in the training environment.

The immutable-plan candidate passed 2,196 local non-GPU/non-network tests at
85.01% branch coverage, 8 RTX 4080 GPU tests, and 13 network tests with 2
environment-dependent skips. Determinism and tamper tests cover plan bytes,
source YAML, Parquet files, acceptance, native config, cache identity and
checkpoint identity. Ruff, mypy, actionlint, strict MkDocs, package/Twine,
generated-description, link/text and all four Playwright viewports also pass.

## v0.8.0 single-GPU verl OPD pivot

- [x] Implement and validate the documented `verl-opd-v0.8-single-gpu-v1`
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ nav:
- Home: index.md
- For verl users: for-verl-users.md
- Config overrides: config-overrides.md
- Immutable plans: immutable-plans.md
- Run verl-style OPD: opd-quickstart.md
- Start on one GPU: single-gpu-guide.md
- Align:
Expand Down
1 change: 1 addition & 0 deletions scripts/check_docs_visual.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
"/",
"/for-verl-users/",
"/config-overrides/",
"/immutable-plans/",
"/alignment-lab/alignment-lab-v1/",
"/alignment-external/alignment-external-v1/",
"/consumer-runtime/",
Expand Down
Loading
0