Jiakang Huang

Jiakang Huang

CS student at UBC; visiting researcher at UCLA BigML. Data selection, model merging, and ML compilers.

HR-Merge: Model Merging with Headroom-Weighted Residual Refinement

Published in NeurIPS 2026 Workshop on Neural Artifacts (NeuralArtifacts), Poster, 2026

Paper

Authors: Jiakang Huang, Xueyan Zhang, Renyi Cai, Jingcheng Niu, Jiaru Li, Yining Wang, Josh Zhang, Jiahe Jiang, Jinman Zhao, Gerald Penn, Baharan Mirzasoleiman

Venue: NeurIPS 2026 NeuralArtifacts Workshop (Poster)

Summary: Model-merging methods apply one rule to every task, but the accuracy they leave behind is concentrated in a few tasks. HR-Merge is a training-free, merge-operator-agnostic wrapper: it measures each task’s validation headroom (how far the merge falls below that task’s expert), re-merges the leftover per-task residuals weighted toward the tasks that need it, and adds them back at a fail-closed strength, so it provably never underperforms the base merge on validation. It gives the best result in every CLIP ViT and LoRA setting we tested, reaching 94.29 on the LLaMA-3 8B six-NLI benchmark, above the prior best of 94.16.