DaoyuanLi2816 · Aug 29, 2026 · Aug 29, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -36,6 +36,21 @@ All notable changes to miniVERL are recorded here. The format follows
   prompt/group cursors, trajectory counts, backend synchronization, cache
   identity and export provenance to exact interruption/resume state.
 
+### Rewarded OPD profile
+
+- Added the separate conformance-only
+  `verl-opd-v0.8-single-gpu-pg-k1-rewarded-v1` profile. It combines an
+  explicit deterministic task-reward advantage with the existing sampled-k1
+  distillation advantage; the direct-GKD and reward-free PG profiles retain
+  their original contracts and identities.
+- Added torch-free reward request/result/provider types, a closed exact-answer
+  provider, versioned group transformations, fail-closed Parquet validation,
+  and provider/composer identity binding across reward logs, teacher caches,
+  checkpoints and exports.
+- Kept the objective local and critic-free: there is no value model, global
+  normalization, arbitrary reward code, GRPO claim or distributed execution
+  claim. No task-quality benchmark was run for this profile.
+
 ### Documentation narrative
 
 - Rebuilt the English, Chinese and PyPI landing narratives around the local

diff --git a/PYPI.md b/PYPI.md
@@ -135,6 +135,8 @@ the complete mapping. [Compatibility profiles](https://github.com/DaoyuanLi2816/
 how profile identity follows plans, caches, checkpoints and exports. Separate
 conformance-only grouped profiles add transactional Parquet `n>1` samples
 without changing either measured `n=1` profile or introducing GRPO semantics.
+A separate conformance-only rewarded profile adds deterministic exact-answer
+rewards and explicit group advantage composition; it has no task-quality claim.
 
 ## Measured systems evidence
 

diff --git a/README.md b/README.md
@@ -135,6 +135,8 @@ the complete mapping. [Compatibility profiles](docs/profiles/index.md) explains
 how profile identity follows plans, caches, checkpoints and exports. Separate
 conformance-only grouped profiles add transactional Parquet `n>1` samples
 without changing either measured `n=1` profile or introducing GRPO semantics.
+A separate conformance-only rewarded profile adds deterministic exact-answer
+rewards and explicit group advantage composition; it has no task-quality claim.
 
 ## Measured systems evidence
 

diff --git a/README.zh-CN.md b/README.zh-CN.md
@@ -124,6 +124,8 @@ distillation:
 [兼容 profile](docs/profiles/index.md)说明 profile identity 如何跟随 plan、cache、checkpoint
 与 export。另有两个仅完成一致性验证的 grouped profile，为 Parquet prompt 提供事务化
 `n>1` 独立样本；它们不改变两个实测 `n=1` profile，也不引入 GRPO 语义。
+另有一个仅完成一致性验证的 rewarded profile，加入确定性的 exact-answer reward 与显式
+group advantage 组合；目前不对任务质量作结论。
 
 ## 实测系统证据
 <
8BD0
br>

diff --git a/docs/compatibility.md b/docs/compatibility.md
@@ -22,13 +22,16 @@ migrations, and readers retain the schema-specific behavior needed to load them.
 
 ## Pinned OPD profiles
 
-v0.8 adds the separately versioned `verl-opd-v0.8-single-gpu-v1` executable
-profile: one local actor, one teacher, one generation, pure GKD
-`forward_kl_topk`, token-mean aggregation, LoRA/QLoRA and Parquet prompts.
+v0.8 introduced `verl-opd-v0.8-single-gpu-v1`: one local actor, one teacher,
+pure GKD `forward_kl_topk`, token-mean aggregation, LoRA/QLoRA and Parquet
+prompts. The registry now also contains sampled-k1, grouped-independent and a
+separate rewarded sampled-k1 contract.
 `import-verl --config` publishes a canonical profile plus field report;
-`export-verl` recognizes a completed compatible run and emits reward-free OPD
-overrides. PG OPD, task-reward mixtures, multiple teachers, multimodal and all
-distributed semantics fail closed. Local physical phase scheduling is always
+`export-verl` recognizes a completed compatible run and emits its profile-bound
+OPD overrides. The rewarded profile accepts only a closed deterministic
+exact-answer provider and remains critic/value/GRPO-free. Multiple teachers,
+arbitrary reward code, multimodal and all distributed semantics fail closed.
+Local physical phase scheduling is always
 labelled as a reinterpretation of upstream resource fields.
 
 The [machine-readable field matrix](generated/verl-opd-v0.8-compatibility.json)
@@ -54,8 +57,8 @@ Optimizer state, distributed RNG, FSDP/Megatron native checkpoints, Ray runtime
 state and teacher-cache conversion sit beyond that exchange contract. The
 release smoke records artifact/config validation and distributed execution as
 separate evidence fields. Legacy reward-scaffold bundles use
-`launch.template.sh`; current pure-OPD bundles become launchable through the
-documented materialization checks.
+`launch.template.sh`; current profile exports remain explicit about their
+individual launch blockers and semantic-parity status.
 See the [current local runtime](verl-opd-runtime.md), [current scale-out
 contract](verl-opd-scaleout.md), and [legacy bridge](legacy-verl-bridge.md).
 

diff --git a/docs/for-verl-users.md b/docs/for-verl-users.md
@@ -1,7 +1,8 @@
 # For verl users
 
-miniVERL brings two documented verl v0.8 OPD profiles into a local, inspectable
-workflow. Familiar field names and Parquet data flow through a typed compiler,
+miniVERL brings five closed verl v0.8 OPD profiles into a local, inspectable
+workflow; two have measured systems records and three are conformance-only.
+Familiar field names and Parquet data flow through a typed compiler,
 which turns resource intent into sequential phases on one CUDA GPU and records
 every local mapping in the resulting plan.
 
@@ -18,7 +19,8 @@ by plans, caches, checkpoints and exports.
 ## Supported input shape
 
 - A resolved YAML using the `verl-opd-v0.8-single-gpu-v1` field subset.
-- Reward-free verl-style Parquet prompts with structured chat messages.
+- Verl-style Parquet prompts with structured chat messages; the rewarded
+  profile additionally requires bound deterministic exact-answer metadata.
 - One actor, one teacher, token-mean aggregation, and either direct
   forward-top-k GKD or sampled-k1 vanilla policy loss. The measured profiles
   remain `n=1`; their grouped counterparts accept independent `n>1` samples.
@@ -100,8 +102,9 @@ produces top-k token IDs and log-probabilities on actor-generated tokens;
 top-k mass and overlap are diagnostics, not an explicit tail bucket. In the PG
 k1 profile the target is only the sampled token's teacher log-probability,
 bound to the rollout actor log-probability and policy version; current actor
-log-probability is recomputed at update time. Both profiles use reward-free
-distillation with one actor and one teacher. Local runtime strategies
+log-probability is recomputed at update time. The rewarded PG profile adds a
+separately logged task advantage from the closed exact-answer provider; it has
+no critic, value model or GRPO semantics. Local runtime strategies
 may keep quantized roles resident, swap movable unquantized roles, or share a
 compatible backbone while preserving distinct role identities in provenance.
 
@@ -127,8 +130,8 @@ has progressed.
   unsupported values.
 - **Algorithm field rejected:** select the explicit PG-k1 profile for its
   narrow sampled policy-loss path, or a grouped profile for Parquet `n>1`.
-  Rewards, critics, external advantages, KL penalties, group-relative
-  baselines, multiple teachers and distributed counts remain unsupported.
+  Arbitrary reward code, critics, value models, KL penalties, multiple teachers
+  and distributed counts remain unsupported.
 - **Interpolation rejected:** resolve Hydra/OmegaConf in your trusted verl
   environment first. miniVERL will not execute `${...}`.
 - **High-risk reinterpretation not accepted:** inspect `miniverl plan`, then

diff --git a/docs/generated/upstream-support-matrix.json b/docs/generated/upstream-support-matrix.json
@@ -34,6 +34,14 @@
       "upstream_commit": "7aed6b230776f963fa09509c10d9c3a767d1102c",
       "objective": "sampled_k1_grouped_independent_samples",
       "distributed_execution_tested": false
+    },
+    {
+      "name": "verl-opd-v0.8-single-gpu-pg-k1-rewarded-v1",
+      "status": "active",
+      "upstream_tag": "v0.8.0",
+      "upstream_commit": "7aed6b230776f963fa09509c10d9c3a767d1102c",
+      "objective": "task_reward_plus_sampled_k1_vanilla_policy_loss",
+      "distributed_execution_tested": false
     }
   ],
   "unprofiled_versions": "unsupported"

diff --git a/docs/profiles/index.md b/docs/profiles/index.md
@@ -45,13 +45,19 @@ name.
 | `verl-opd-v0.8-single-gpu-pg-k1-v1` | sampled `k1` + vanilla policy loss | sampled-token teacher log-probability | sampled-token signal; smaller target artifact | measured |
 | `verl-opd-v0.8-single-gpu-grouped-v1` | direct GKD over independent grouped samples | top-k token IDs and log-probabilities | configurable `n>1`; no group baseline | conformance only |
 | `verl-opd-v0.8-single-gpu-pg-k1-grouped-v1` | sampled `k1` over independent grouped samples | sampled-token teacher log-probability | configurable `n>1`; no GRPO estimator | conformance only |
-
-All four are reward-free, strict current-policy, one-actor/one-teacher profiles
-on one CUDA GPU. The PG profiles use a detached distillation-derived advantage
-with the pinned vanilla policy-loss form. The measured records compare runtime
-and semantic conformance; task-quality questions belong to a declared benchmark.
-The two grouped profiles add transactional Parquet prompt groups while leaving
-the published `n=1` identities unchanged; each sample is optimized independently.
+| `verl-opd-v0.8-single-gpu-pg-k1-rewarded-v1` | task reward + sampled `k1` through vanilla policy loss | sampled-token teacher log-probability + deterministic exact-answer reward | explicit group transform; no critic or value model | conformance only |
+
+The first four profiles are reward-free. The rewarded profile is a separate,
+strict current-policy contract with a closed deterministic provider and
+versioned task/distillation advantage composition. All five use one actor and
+one teacher on one CUDA GPU. The PG profiles retain the pinned vanilla
+policy-loss form. The measured records compare runtime and semantic
+conformance; the rewarded profile has no task-quality result yet.
+Library callers may inject an object implementing `RewardProvider` with
+`reward.provider: python_api`; YAML and run artifacts never name or load Python
+modules. The built-in verl-shaped profile remains fixed to `exact_answer`.
+The group-capable profiles use transactional Parquet prompt groups while
+leaving the published `n=1` identities unchanged.
 See [For verl users](../for-verl-users.md), the
 [PG-k1 ADR](../adr/0010-verl-v0.8-pg-k1-contract.md), and the runtime evidence
 for the measured boundary.
diff --git a/docs/verl-opd-runtime.md b/docs/verl-opd-runtime.md
@@ -1,8 +1,9 @@
 # Current verl-style OPD runtime
 
-This is miniVERL's current executable path: two documented, resolved profiles
-from official verl `v0.8.0`, pinned at `7aed6b23`, compiled into local phases
-on one NVIDIA CUDA GPU.
+This is miniVERL's current executable path: five closed profiles from official
+verl `v0.8.0`, pinned at `7aed6b23`, compiled into local phases on one NVIDIA
+CUDA GPU. Two profiles have measured systems records; the grouped and rewarded
+extensions are conformance-only.
 
 ## Start from the pinned profile
 
@@ -44,6 +45,12 @@ policy-loss semantics. Its export carries sampled-token signals in place of a
 top-k target requirement. See [Which profile should I use?](profiles/index.md)
 for the choice guide.
 
+`verl-opd-v0.8-single-gpu-pg-k1-rewarded-v1` keeps that sampled-k1 path and
+adds an explicit deterministic task-reward advantage. The provider, group
+transform and both coefficients are versioned and carried by cache, checkpoint
+and export identity. This is a local task-plus-distillation objective, not a
+GRPO or distributed verl result.
+
 ## Placement strategy
 
 - NF4/int8 roles use resident local phases.

diff --git a/scripts/check_docs_visual.py b/scripts/check_docs_visual.py
@@ -34,6 +34,7 @@
     "/for-verl-users/",
     "/config-overrides/",
     "/immutable-plans/",
+    "/profiles/",
     "/hardware-planning/",
     "/verl-opd-runtime/",
     "/verl-opd-reference-workload/",

diff --git a/src/miniverl/bridge/export.py b/src/miniverl/bridge/export.py
@@ -25,7 +25,10 @@
     required_verl_text,
     validate_target_verl,
 )
-from miniverl.bridge.opd_pg_v08 import VERL_OPD_PG_K1_V08_PROFILE
+from miniverl.bridge.opd_pg_v08 import (
+    VERL_OPD_PG_K1_REWARDED_V08_PROFILE,
+    VERL_OPD_PG_K1_V08_PROFILE,
+)
 from miniverl.bridge.opd_v08 import VERL_OPD_V08_PROFILE
 from miniverl.bridge.profiles import (
     VERL_OPD_GROUPED_V08_PROFILE,
@@ -432,13 +435,20 @@ def _write_hashes(root: Path) -> None:
     write_text(checksum, "\n".join(lines) + "\n")
 
 
-_PG_PROFILES = frozenset({VERL_OPD_PG_K1_V08_PROFILE, VERL_OPD_PG_K1_GROUPED_V08_PROFILE})
+_PG_PROFILES = frozenset(
+    {
+        VERL_OPD_PG_K1_V08_PROFILE,
+        VERL_OPD_PG_K1_GROUPED_V08_PROFILE,
+        VERL_OPD_PG_K1_REWARDED_V08_PROFILE,
+    }
+)
 _OPD_PROFILES = frozenset(
     {
         VERL_OPD_V08_PROFILE,
         VERL_OPD_PG_K1_V08_PROFILE,
         VERL_OPD_GROUPED_V08_PROFILE,
         VERL_OPD_PG_K1_GROUPED_V08_PROFILE,
+        VERL_OPD_PG_K1_REWARDED_V08_PROFILE,
     }
 )
 
@@ -460,7 +470,9 @@ def _opd_run_contract(run: Path) -> tuple[dict[str, Any], dict[str, Any]] | None
     if report.get("executable") is not True or not isinstance(source, dict):
         raise ConfigError("source run does not carry an executable verl OPD compatibility plan")
     required: dict[str, Any] = {
-        "distillation.distillation_loss.use_task_rewards": False,
+        "distillation.distillation_loss.use_task_rewards": (
+            profile == VERL_OPD_PG_K1_REWARDED_V08_PROFILE
+        ),
         "actor_rollout_ref.actor.use_kl_loss": False,
         "algorithm.use_kl_in_reward": False,
     }
@@ -591,8 +603,11 @@ def _opd_launch_script(adapter: dict[str, Any], teacher: dict[str, Any]) -> str:
 
 
 def _opd_bundle_readme(profile: str) -> str:
+    rewarded = profile == VERL_OPD_PG_K1_REWARDED_V08_PROFILE
     semantics = (
-        "sampled k1 teacher log-probabilities and pinned vanilla policy-gradient loss"
+        "explicit task and sampled-k1 distillation advantages through the pinned vanilla policy loss"
+        if rewarded
+        else "sampled k1 teacher log-probabilities and pinned vanilla policy-gradient loss"
         if profile in _PG_PROFILES
         else "pure GKD `forward_kl_topk` targets"
     )
@@ -601,8 +616,7 @@ def _opd_bundle_readme(profile: str) -> str:
 This checksummed bundle preserves a local `{profile}` run as standard
 PEFT, tokenizer and Parquet artifacts plus {semantics}
 overrides for [`verl {VERL_TAG}`]({VERL_REPOSITORY}/tree/{VERL_TAG}) at
-`{VERL_COMMIT}`. It contains no reward scaffold because task rewards are
-disabled by contract.
+`{VERL_COMMIT}`. {"The rewarded profile uses a closed deterministic provider and no critic, value model or GRPO normalization." if rewarded else "Task rewards are disabled by contract."}
 
 `recipe/launch.template.sh` remains fail-closed until the exact student and
 teacher snapshots are materialized. The bundle has not run distributed verl;
@@ -657,6 +671,11 @@ def _export_opd_bundle(
         "teacher base snapshot is not bundled",
         "distributed verl execution was not tested",
     ]
+    rewarded = profile == VERL_OPD_PG_K1_REWARDED_V08_PROFILE
+    if rewarded:
+        blockers.append(
+            "rewarded task-plus-distillation composition has no claimed distributed verl parity"
+        )
     if teacher_adapter_path is not None:
         blockers.append(
             "teacher adapter requires an explicit merge/materialization step for upstream verl"
@@ -711,7 +730,9 @@ def _export_opd_bundle(
             },
             "miniverl_version": __version__,
             "target_semantics": (
-                "pure OPD sampled k1 policy gradient"
+                "rewarded OPD task plus sampled-k1 distillation policy gradient"
+                if rewarded
+                else "pure OPD sampled k1 policy gradient"
                 if profile in _PG_PROFILES
                 else "pure GKD forward_kl_topk OPD"
             ),
@@ -722,8 +743,13 @@ def _export_opd_bundle(
                 "samples_per_prompt": samples_per_prompt,
                 "trajectory_schema_version": 3,
                 "sample_semantics": "independent_current_policy_trajectory",
-                "group_baseline": False,
+                "group_baseline": (
+                    _get(source, "distillation.distillation_loss.task_advantage_mode")
+                    if rewarded
+                    else False
+                ),
                 "grpo": False,
+                **({"critic": False, "value_model": False} if rewarded else {}),
             },
             "artifact_complete": False,
             "artifact_bundle_complete": True,
@@ -738,7 +764,13 @@ def _export_opd_bundle(
             "launchable": False,
             "distributed_execution_tested": False,
             "algorithm_semantic_parity": False,
-            "reward_required": False,
+            "reward_required": rewarded,
+            "reward_provider": (
+                _get(source, "distillation.distillation_loss.reward_provider") if rewarded else None
+            ),
+            "advantage_composer_version": (
+                "miniverl-task-distill-advantage-v1" if rewarded else None
+            ),
             "launch_blockers": blockers,
             "data_round_trip": data_evidence,
             "unsupported_semantics": [