8000 Bump trl from 1.8.0 to 1.12.0 by dependabot[bot] · Pull Request #109 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]
Skip to content

Bump trl from 1.8.0 to 1.12.0 - #109

Closed
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/pip/trl-1.12.0
Closed

Bump trl from 1.8.0 to 1.12.0#109
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/pip/trl-1.12.0

Conversation

@dependabot
@dependabot dependabot Bot commented on behalf of github Sep 1, 2026
Copy link
Copy Markdown

Bumps trl from 1.8.0 to 1.12.0.

Release notes

Sourced from trl's releases.

v1.12.0

⚠️ v1.12.0 is an accidental duplicate of v1.11.0

During the v1.11.0 release, VERSION was bumped to 1.12.0 instead of 1.12.0.dev0. Our publish workflow fires on any push to main that touches VERSION and only skips the upload when the version contains "dev", so trl 1.12.0 was auto-published to PyPI at 19:46 on 2026-08-26, 90 seconds after 1.11.0, with identical code.

trl==1.12.0 on PyPI is bit-identical to trl==1.11.0. There are no features, fixes, or behavior changes in this version.

The version number is burned (PyPI does not allow yanking + re-uploading the same version), so we skip 1.12 entirely for real development. The next feature release is v1.13.0.

Full Changelog: huggingface/trl@v1.11.0...v1.12.0 (empty)

v1.11.0

Features

Use vLLM's own server instead of TRL's

trl vllm-serve used to run a custom FastAPI app around vLLM's LLM, with its own data-parallel fan-out and a worker extension for weight sync. vLLM's own server covers all of that. The custom server is gone: trl/scripts/vllm_serve.py drops from 1218 to ~130 lines and now translates its flags into vllm serve, prints the equivalent command, forwards any extra argument, and sets the three settings TRL needs (--weight-transfer-config, --logprobs-mode processed_logprobs, --max-logprobs -1). fastapi, uvicorn and pydantic leave the vllm extra.

Weight sync now uses vLLM's NCCL weight-transfer engine — tensors are announced once and streamed in a single packed broadca 7438 st, instead of one HTTP request + broadcast per tensor. Distillation teacher logprobs come from prompt_logprobs.

Benchmark (GRPO server-mode on H100, Qwen2.5-1.5B / Qwen2.5-VL-3B, same seed, same data):

Setup before after speedup
Text, TP=1 1.08, 0.91 s/step 0.63, 0.62 s/step 1.59×
Text, TP=2 0.96, 0.96 s/step 0.62, 0.60 s/step 1.57×
VLM, TP=1 2.14, 2.13 s/step 1.48, 1.48 s/step 1.44×

Bit-identical completions vs the old server (16/16 over 4 prompts × n=4, logprobs identical to 0.0 over 499 tokens). trl vllm-serve still accepts the same flags — it just forwards to vllm serve now.

by @​qgallouedec in huggingface/trl#6765

New experimental AsyncDistillationTrainer (+ multi-teacher MOPD)

Async on-policy distillation, architected like AsyncGRPOTrainer: a background rollout worker generates the student's own completions and scores them against a teacher served over HTTP, so generation and training overlap instead of alternating. The teacher is never loaded locally — just a vLLM server URL.

Also supports MOPD (multi-teacher on-policy distillation): pass more than one entry in teacher_server_urls and route each sample to a teacher via a teacher_id column (e.g. a math teacher and a code teacher, each served independently).

from trl.experimental.async_distillation import AsyncDistillationConfig, AsyncDistillationTrainer
trainer = AsyncDistillationTrainer(
model="Qwen/Qwen3-4B",
args=AsyncDistillationConfig(
teacher_server_urls=["http://teacher-math:8000", "http://teacher-code:8000"],
teacher_top_k=16,
),
train_dataset=dataset,  # rows carry a teacher_id column for MOPD
)
trainer.train()
</tr></table>

... (truncated)

Commits
  • 59c4a8e ⬆️ Bump dev version (#6938)
  • d0a2cd5 Release: v1.11 (#6937)
  • 8f7a634 Remove one host sync from SFT chunked_nll loss (#6842)
  • 671cc0b Suggest migrating to DistillationTrainer when the GKD config is fully covered...
  • f10615d Raise when packing is combined with context parallelism (#6843)
  • 429d2e1 Rewrite the TRL skill around the Python API (#6794)
  • 060c8c9 Fix the Slack report posting twice (#6936)
  • 3699748 Make the Slack CI report readable at a glance (#6934)
  • 7cc415a Drop trl skills install and move the skill source out of the package (#6793)
  • 67290a0 Use vLLM's own server instead of TRL's (#6765)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot
Bump trl from 1.8.0 to 1.12.0
1d61742 
Bumps [trl](https://github.com/huggingface/trl) from 1.8.0 to 1.12.0.
- [Release notes](https://github.com/huggingface/trl/releases)
- [Changelog](https://github.com/huggingface/trl/blob/main/RELEASE.md)
- [Commits](huggingface/trl@v1.8.0...v1.12.0)

---
updated-dependencies:
- dependency-name: trl
  dependency-version: 1.12.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot added dependencies Pull requests that update a dependency file python Pull requests that update python code labels Sep 1, 2026
@DaoyuanLi2816
DaoyuanLi2816 commented Sep 8, 2026
Copy link
Copy Markdown
Owner

Closing this automated update without merge. Dependency changes are maintained on first-party branches so repository attribution stays with the project maintainer.

@dependabot @github
dependabot Bot commented on behalf of github Sep 8, 2026
Copy link
Copy Markdown
Author

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.

@dependabot
dependabot Bot deleted the dependabot/pip/trl-1.12.0 branch September 8, 2026 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

dependencies Pull requests that update a dependency file python Pull requests that update python code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@DaoyuanLi2816
0