Speeding up distributed machine learning using codes

Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos, Kannan Ramchandran

Type: Article

Publication Date: 2016-07-01

Citations: 147

DOI: https://doi.org/10.1109/isit.2016.7541478

View Publication

Locations

arXiv (Cornell University) - View
2022 IEEE International Symposium on Information Theory (ISIT) - View

Similar Works

Action	Title	Year	Authors
+	Speeding Up Distributed Machine Learning Using Codes	2017	Kangwook Lee Maximilian Lam Ramtin Pedarsani Dimitris Papailiopoulos Kannan Ramchandran
+	On the Fundamental Limits of Coded Data Shuffling for Distributed Machine Learning	2020	Adel Elmahdy Soheil Mohajer
+	Fundamental Limits of Distributed Data Shuffling	2018	Kai Wan Daniela Tuninetti Mingyue Ji Pablo Piantanida
+	Information Theoretic Limits of Data Shuffling for Distributed Learning	2016	Mohamed Adel Attia Ravi Tandon
+	Information Theoretic Limits of Data Shuffling for Distributed Learning	2016	Mohamed Attia Ravi Tandon
+ PDF Chat	Information Theoretic Limits of Data Shuffling for Distributed Learning	2016	Mohamed Adel Attia Ravi Tandon
+	Near Optimal Coded Data Shuffling for Distributed Learning	2019	Mohamed Adel Attia Ravi Tandon
+	On the Fundamental Limits of Coded Data Shuffling for Distributed Learning Systems.	2018	Adel Elmahdy Soheil Mohajer
+ PDF Chat	Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication	2020	Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi
+	Near Optimal Coded Data Shuffling for Distributed Learning	2018	Mohamed Adel Attia Ravi Tandon
+	Near Optimal Coded Data Shuffling for Distributed Learning	2018	Mohamed Adel Attia Ravi Tandon
+	On the Worst-case Communication Overhead for Distributed Data Shuffling	2016	Mohamed Attia Ravi Tandon
+	On the Worst-case Communication Overhead for Distributed Data Shuffling	2016	Mohamed Adel Attia Ravi Tandon
+ PDF Chat	On the worst-case communication overhead for distributed data shuffling	2016	Mohamed Adel Attia Ravi Tandon
+ PDF Chat	Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication	2020	Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi
+	Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication	2018	Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi
+ PDF Chat	Rateless codes for near-perfect load balancing in distributed matrix-vector multiplication	2022	Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi
+ PDF Chat	Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication	2019	Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi
+	Straggler-resistant distributed matrix computation via coding theory	2020	Aditya Ramamoorthy Anindya Bijoy Das Li Tang
+	Straggler Mitigation through Unequal Error Protection for Distributed Matrix Multiplication	2020	Büşra Tegin Eduin E. Hernandez Stefano Rini Tolga M. Duman

Works That Cite This (88)

Action	Title	Year	Authors
+	Computing Linear Transformations With Unreliable Components	2017	Yaoqing Yang Pulkit Grover Soummya Kar
+	Minimizing Latency for Secure Coded Computing Using Secret Sharing via Staircase Codes	2020	Rawad Bitar Parimal Parag Salim El Rouayheb
+	On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems	2021	Baoqian Wang Junfei Xie Kejie Lu Yan Wan Shengli Fu
+	Coded Computation Against Processing Delays for Virtualized Cloud-Based Channel Decoding	2018	Malihe Aliasgari Jörg Kliewer Osvaldo Simeone
+	A Pliable Index Coding Approach to Data Shuffling	2019	Linqi Song Christina Fragouli Tianchu Zhao
+ PDF Chat	Collaborative Decoding of Polynomial Codes for Distributed Computation	2019	Adarsh M. Subramaniam Anoosheh Heidarzadeh Krishna R. Narayanan
+ PDF Chat	Coded Network Function Virtualization: Fault Tolerance via In-Network Coding	2016	Ali Al-Shuwaili Osvaldo Simeone Jörg Kliewer Petar Popovski
+ PDF Chat	Coded Computation Across Shared Heterogeneous Workers With Communication Delay	2022	Yuxuan Sun Fan Zhang Junlin Zhao Sheng Zhou Zhisheng Niu Deniz Gündüz
+ PDF Chat	Fundamental resource trade-offs for encoded distributed optimization	2020	A. Salman Avestimehr Seyed Mohammadreza Mousavi Kalan Mahdi Soltanolkotabi
+	A Fundamental Tradeoff Between Computation and Communication in Distributed Computing	2017	Songze Li Mohammad Ali Maddah-Ali Qian Yu A. Salman Avestimehr

Works Cited by This (11)

Action	Title	Year	Authors
+ PDF Chat	Efficient task replication for fast response times in parallel computation	2014	Da Wang Gauri Joshi Gregory W. Wornell
+ PDF Chat	Network Coding for Distributed Storage Systems	2010	Alexandros G. Dimakis P. Brighten Godfrey Yunnan Wu Martin J. Wainwright Kannan Ramchandran
+ PDF Chat	Parallel stochastic gradient algorithms for large-scale matrix completion	2013	Benjamin Recht Christopher Ré
+ PDF Chat	Fundamental Limits of Caching	2014	Mohammad Ali Maddah-Ali Urs Niesen
+	Parallelized Stochastic Gradient Descent	2010	Martin Zinkevich Markus Weimer Lihong Li Alex Smola
+	Speeding Up Distributed Machine Learning Using Codes	2017	Kangwook Lee Maximilian Lam Ramtin Pedarsani Dimitris Papailiopoulos Kannan Ramchandran
+ PDF Chat	When do redundant requests reduce latency ?	2013	Nihar B. Shah Kangwook Lee Kannan Ramchandran
+	HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent	2011	Feng Niu Benjamin Recht Christopher Ré Stephen J. Wright
+	Speeding up distributed machine learning using codes	2016	Kangwook Lee Maximilian Lam Ramtin Pedarsani Dimitris Papailiopoulos Kannan Ramchandran
+ PDF Chat	TOFEC: Achieving optimal throughput-delay trade-off of cloud storage using erasure codes	2014	Guanfeng Liang Ulaş C. Kozat