TARE: Timescale-Aware Row-wise Energy Redistribution for Muon Optimizers

Authors: Jinman Zhao, Xueyan Zhang, Jiakang Huang, Xutao Mao, Jingcheng Niu, Xi Chen, Yining Wang, Changyi Yang, YiTian Ding, Jiahe Liu, Gerald Penn, Jiaru Li

Status: Under review

Summary: TARE is a Muon-family optimizer that shifts update energy away from weight rows whose recent gradients conflict with the longer-term trend. It reaches NorMuon’s loss with 1.07–1.10x higher token efficiency from 130M to 1.1B parameters, at under 5% extra wall-clock time.

My role: I ran the 1.1B scale-up, integrated TARE into the distributed training code, and measured its overhead.