Decentralized gradient tracking with local steps
Decentralized gradient tracking with local steps
Gradient tracking (GT) is an algorithm designed for solving decentralized optimization problems over a network (such as training a machine learning model). A key feature of GT is a tracking mechanism that allows us to overcome data heterogeneity between nodes. We develop a novel decentralized tracking mechanism, K-GT, which enables …