Channel-Directed Gradients for Optimization of Convolutional Neural Networks

Dong Lao, Peihao Zhu, Peter Wonka, Ganesh Sundaramoorthi

Type: Preprint

Publication Date: 2020-01-01

Citations: 2

DOI: https://doi.org/10.48550/arxiv.2008.10766

View Publication

Locations

arXiv (Cornell University) - View
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant	2023	Xianbiao Qi Jianan Wang Lei Zhang
+ PDF Chat	Adaptive Gradient Regularization: A Faster and Generalizable Optimization Technique for Deep Neural Networks	2024	Huixiu Jiang Ling Yang Yu Bao Rutong Si Sikun Yang
+	Optimization Methods in Deep Learning: A Comprehensive Overview	2023	David Shulman
+	Gradient Descent: The Ultimate Optimizer	2019	Kartik Chandra Erik Meijer Samantha Andow Emilio Arroyo-Fang Irene Dea Johann George Melissa Grueter Basil Hosmer Steffi Stumpos Alanna Tempest
+	Training Neural Networks with Stochastic Hessian-Free Optimization	2013	Ryan Kiros
+	Continuous vs. Discrete Optimization of Deep Neural Networks	2021	Omer Elkabetz Nadav Cohen
+	Training Neural Networks with Stochastic Hessian-Free Optimization	2013	Ryan Kiros
+	Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks	2018	Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu
+	Block-Normalized Gradient Method: An Empirical Study for Training Deep Neural Network	2017	Adams Wei Yu Lei Huang Qihang Lin Ruslan Salakhutdinov Jaime Carbonell
+ PDF Chat	Gradient Networks	2024	Shreyas Chaudhari Srinivasa Pranav José M. F. Moura
+	How to guess a gradient	2023	Utkarsh Singhal Brian Cheung Kartik Chandra Jonathan Ragan‐Kelley Joshua B. Tenenbaum Tomaso Poggio Stella X. Yu
+ PDF Chat	LPGD: A General Framework for Backpropagation through Embedded Optimization Layers	2024	Anselm Paulus Georg Martius Vít Musil
+	Continuous vs. Discrete Optimization of Deep Neural Networks	2021	Omer Elkabetz Nadav Cohen
+ PDF Chat	ChannelDropBack: Forward-Consistent Stochastic Regularization for Deep Networks	2024	Evgeny Hershkovitch Neiterman Gil Ben-Artzi
+	The Break-Even Point on Optimization Trajectories of Deep Neural Networks	2020	Stanisław Jastrzȩbski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras
+	Regularization for Deep Learning: A Taxonomy	2017	Jan Kukačka Vladimir Golkov Daniel Cremers
+	Regularization for Deep Learning: A Taxonomy	2017	Jan Kukačka Vladimir Golkov Daniel Cremers
+	On the Overlooked Structure of Stochastic Gradients	2022	Zeke Xie Qian-Yuan Tang Zheng He Mingming Sun Ping Li
+	The Break-Even Point on Optimization Trajectories of Deep Neural Networks	2020	Stanisław Jastrzȩbski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras
+	Channel Normalization in Convolutional Neural Network avoids Vanishing Gradients	2019	Zhenwei Dai Reinhard Heckel

Works That Cite This (2)

Action	Title	Year	Authors
+ PDF Chat	CONetV2: Efficient Auto-Channel Size Optimization for CNNs	2021	Yi Ru Wang Samir Khaki Weihang Zheng Mahdi S. Hosseini Konstantinos N. Plataniotis
+	CONetV2: Efficient Auto-Channel Size Optimization for CNNs.	2021	Yi Ru Wang Samir Khaki Weihang Zheng Mahdi S. Hosseini Konstantinos N. Plataniotis

Works Cited by This (15)

Action	Title	Year	Authors
+	Stochastic variational inference	2013	Matthew D. Hoffman David M. Blei Chong Wang John Paisley
+	Mirrorless Mirror Descent: A More Natural Discretization of Riemannian Gradient Flow.	2020	Suriya Gunasekar Blake Woodworth Nathan Srebro
+	Very Deep Convolutional Networks for Large-Scale Image Recognition	2014	Karen Simonyan Andrew Zisserman
+ PDF Chat	Deep Residual Learning for Image Recognition	2016	Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
+	Practical Riemannian Neural Networks	2016	Gaétan Marceau-Caron Yann Ollivier
+ PDF Chat	The Cityscapes Dataset for Semantic Urban Scene Understanding	2016	Marius Cordts Mohamed Omran Sebastian Ramos Timo Rehfeld Markus Enzweiler Rodrigo Benenson Uwe Franke Stefan Roth Bernt Schiele
+	Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms	2017	Xiao Han Kashif Rasul Roland Vollgraf
+	Laplacian Smoothing Gradient Descent	2018	Stanley Osher Bao Wang Penghang Yin Xiyang Luo Minh Phạm A. T. Lin
+	Adaptive Gradient Methods with Dynamic Bound of Learning Rate	2019	Liangchen Luo Yuanhao Xiong Yan Liu Xu Sun
+	Decoupled Weight Decay Regularization	2017	Ilya Loshchilov Frank Hutter