Add explicit rewarded OPD profile by DaoyuanLi2816 · Pull Request #94 · DaoyuanLi2816/mini-verl

DaoyuanLi2816 · 2026-08-29T13:09:21Z

Summary

add a separate conformance-only rewarded sampled-k1 OPD profile without changing the four existing profile identities
add typed reward requests/results/providers, deterministic exact-answer scoring, injectable Python API providers, and versioned group advantage composition
bind reward/composer identity through Parquet rows, reward and advantage logs, caches, checkpoints, native plans, and exports
preserve the direct-GKD and reward-free PG objectives; the new path has no critic, value model, global advantage normalization, GRPO, or distributed-execution claim

ruff check .; ruff format --check .; mypy src/miniverl; actionlint 1.7.12
full non-GPU/non-network suite with coverage: 2456 passed, 9 skipped, 24 deselected; 83% coverage
GPU: 10 passed; network: 16 passed
extracted final sdist: 2443 passed, 23 skipped, 18 deselected
strict MkDocs build and browser checks across 1440, 1024, 820, and 390 px, including the profile table
package build, Twine, clean core wheel import, clean training-extra doctor, generated-artifact, Markdown/link, text-integrity, and privacy regression checks
frozen calculator benchmark SHA-256 remains 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc
existing profile identity digests were compared directly with current main and are unchanged

No benchmark or task-level result was changed, and this PR makes no task-quality claim for the new profile.

Add explicit rewarded OPD profile

ccdf0a7

DaoyuanLi2816 merged commit b7a81b4 into main Aug 29, 2026
16 checks passed

DaoyuanLi2816 deleted the v0.11.0-rewarded-opd branch August 29, 2026 13:15