Alibaba’s QwenGyre Tackles The Million-Token Problem With Elastic GPU Scheduling For XLong RL
In Brief
Alibaba’s QwenGyre brings elastic GPU scheduling and trajectory deduplication to extreme-long-horizon RL, boosting NL2RepoBench scores by 6% and cutting training time up to 44%.
Alibaba has introduced QwenGyre, a reinforcement learning framework designed for what the research team calls “XLong-horizon” agent training — tasks in which a single rollout can run for hours, involve hundreds of model–environment interactions, and consume close to one million tokens. As LLM agents move from resolving individual repository issues to building entire codebases, conventional online RL pipelines struggle with two problems: severe variance in rollout duration leaves GPUs idle while a few straggler executions finish, and branching execution paths generate massive redundant training data.
The first component, the Elastic Scheduler, addresses GPU underutilization. Traditional frameworks either alternate an entire GPU pool between rollout and training (colocate), leaving hardware idle during long rollout tails, or statically partition GPUs between the two stages (async), preventing either side from borrowing the other’s spare capacity. QwenGyre instead organizes hardware into independently switchable “cells” that reallocate between inference and training as the volume of unfinished rollout work changes. Live harness executions are never interrupted: model requests are rerouted to remaining rollout engines mid-flight, with KV caches migrated over RDMA to avoid recomputation. Newly freed cells join an ongoing training step as additional data-parallel replicas, pulling the authoritative weights from a core cell, while streaming training distributes micro-batches according to each group’s progress so cells finish in step.
Role transitions are fast — averaging 8.5 seconds from rollout to training and 3.5 seconds back — negligible against rollout executions lasting hours.
Taming Branching Trajectories
The second component, the Trajectory Processor, handles the training data itself. At XLong horizons, agent harnesses compact saturated contexts, delegate subtasks to sub-agents, and retry failed paths, so one execution yields a branching tree of divergent context paths rather than a linear trajectory. Naively expanding every path duplicates shared prefixes and inflates training cost. QwenGyre reconstructs the execution as a prefix-shared trajectory tree, preserving each model output’s exact original conditioning context. Only tokens actually generated by the policy are trained on, and shared targets are counted once through on-tree masking.
The framework also scores partial progress: executions that hit their timeout are still evaluated on their preserved workspace state, so partial attempts carry meaningful rewards rather than being discarded. At training admission, a bounded number of trajectories — capped by role priority (main agent first, then sub-agents, then summary paths) — is selected per execution, with token losses averaged within each execution to prevent path count from skewing weighting.
Scaled to Alibaba’s flagship Qwen 3.8-2.4T model at roughly 700K tokens per rollout, QwenGyre improved the NL2RepoBench pass rate from 52.5% to 58.5% in 48 training steps, completing them in 75.4 hours versus 134.6 for async and 91.5 for colocate baselines — speedups of up to 1.78× and 1.85× respectively, achieved while matching baseline training scores. Across the three evaluated workloads — NL2RepoBench, DeepSWE, and TerminalBench — gains held consistently under equal GPU budgets.
Disclaimer
In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.
About The Author
Alisa, a dedicated journalist at the MPost, specializes in crypto, AI, investments, and the expansive realm of Web3. With a keen eye for emerging trends and technologies, she delivers comprehensive coverage to inform and engage readers in the ever-evolving landscape of digital finance.
More articles
Alisa, a dedicated journalist at the MPost, specializes in crypto, AI, investments, and the expansive realm of Web3. With a keen eye for emerging trends and technologies, she delivers comprehensive coverage to inform and engage readers in the ever-evolving landscape of digital finance.
