Cross-feature Contrastive Loss for Decentralized Deep Learning on Heterogeneous Data
Cross-feature Contrastive Loss for Decentralized Deep Learning on Heterogeneous Data
The current state-of-the-art decentralized learning algorithms mostly assume the data distribution to be Independent and Identically Distributed (IID). However, in practical scenarios, the distributed datasets can have significantly heterogeneous data distributions across the agents. In this work, we present a novel approach for decentralized learning on heterogeneous data, where data-free …