Conductor: Dynamically Orchestrating Pipeline Parallelism with Multi-Granularity Control

Abstract

Pipeline parallelism is a fundamental pillar of large scale model training yet its efficiency is frequently constrained by straggler induced pipeline bubbles. This issue is exacerbated by static scheduling approaches including handcrafted heuristics and Integer Linear Programming which are inherently brittle when facing real world execution time variance. In this work we introduce Conductor which is a dynamic and two tiered scheduling framework designed to virtually eliminate straggler induced bubbles under realistic stochastic conditions. The key technical insight is to decouple global and long horizon scheduling from local and instantaneous load balancing. At a coarse grain a reinforcement learning agent leverages millisecond scale inference to generate robust global schedules and adapts to runtime dynamics in scenarios where traditional static solvers are computationally intractable. At a fine grain we introduce a dynamic computation migration mechanism that resolves residual micro bubbles by offloading sub computations such as attention heads from transiently slower workers to faster ones within a single timestep. Evaluated on large scale language model training configurations our framework outperforms state of the art static scheduling baselines by 5% to 14% in throughput and demonstrates superior resilience to injected system noise and execution variance.

Highlights

Introduce a dynamic two-tier framework that separates global scheduling from local load balancing.
Use reinforcement learning for fast, adaptive global scheduling under execution-time variation.
Mitigate residual micro-bubbles through fine-grained computation migration.

Method

Conductor separates long-horizon operation ordering from short-horizon workload balancing, so each control layer addresses the source of pipeline idle time at its own timescale.

Conductor's two-tier control design.
Control layer	Mechanism	Objective
Global scheduling	Heuristic-guided PPO policy	Adapt the feasible operation order to runtime variation.
Local balancing	Fine-grained computation migration	Offload eligible sub-computations from transiently slower workers.

Pipeline slots illustrating stragglers across four devices

A straggler sets the pace of a pipeline slot and leaves faster devices idle.

Comparison between ZBPP-based and reinforcement-learning based scheduling — Pass-time imbalance

Scheduling under an unexpected transmission congestion event — Bursty network congestion

Representative runtime perturbations: the learned policy reorders feasible operations to reduce idle time and adapt to transmission congestion.

Fine-grained Computation Migration

Fine-grained migration of MLP computation between devices

Fine-grained migration balances a temporarily slower worker by redistributing an eligible MLP sub-computation.

Results

Across the evaluated large-scale language model training configurations, Conductor improves throughput by 5% to 14% over the reported static scheduling baselines and remains resilient to injected system noise and execution variance.

Ablation study under 10% runtime noise.
Algorithm	Completion time (ms)	Throughput improvement	Bubble ratio
ZB-2p	342.2315	0.00%	25.15%
ZB-2p + RL	321.2232	6.54%	20.26%
ZB-2p + Migration	258.8029	32.24%	1.02%
Conductor	242.7314	40.99%	0.79%

Pipeline schedules comparing pure ZBPP ordering and RL-driven dynamic scheduling

Illustrative schedules for pure ZBPP ordering and RL-driven dynamic scheduling under the same pipeline configuration.

Megatron-LM throughput for computation balancing (samples/GPU/s).
Schedule	GPT-3 1.5B	GPT-3 6.2B	GPT-3 14.6B
ZB-2p / Conductor migration	14.5 / 15.1	4.32 / 4.51	1.81 / 1.90
ZB-1p / Conductor migration	12.9 / 13.4	3.88 / 4.02	1.61 / 1.66
1F1B-I / Conductor migration	13.1 / 13.6	4.01 / 4.16	1.54 / 1.62
1F1B / Conductor migration	11.8 / 13.0	3.50 / 3.78	1.40 / 1.57

Citation

@article{dong2027conductor,
  title={Conductor: Dynamically Orchestrating Pipeline Parallelism with Multi-Granularity Control},
  author={Dong, Xingbo and Liu, Ziyuan and Yang, Yuezhe and Lai, Yen-Lung and Jin, Zhe},
  journal={Future Generation Computer Systems},
  year={2027},
  doi={10.1016/j.future.2026.108762}
}