GraphFuse-R1: A Compiler-Side Bound and a Learned Graph-Level Policy for Operator Fusion

Authors: Xueyan Zhang, Jiakang Huang, YiTian Ding, Shuhao Guan, Yining Wang, Yuchen Li, Jinman Zhao, Gerald Penn

Status: Under review

Summary: Operator fusion in PyTorch 2.0 is still decided by TorchInductor’s local, pairwise, rule-based heuristic. We ask two questions: how much fusion decision is actually left to make, and can a language model make that decision over the whole graph? A policy-independent probe shows that fusible work is only 2.9% of wall-clock time at batch 1 and 24.3% at batch 256, which sets a hard ceiling on any fusion policy. Within that ceiling, GraphFuse-R1, an 8B open-weight LLM (Qwen3-8B) trained with SFT and GRPO, serializes the scheduler graph as text and proposes whole-graph fusion groups that TorchInductor validates before applying. It compiles 28.5% fewer kernels than TorchInductor and fuses 39% more nodes per graph.

My role: I adapted TorchInductor’s fusion pass to accept the model’s proposals, and owned the SFT/RL data: a deduplicated 6.5K-graph corpus from seven sources, plus SFT traces kept only when they matched the compiler.