BigNAS: Scaling up Neural Architecture Search with Big Single-Stage Models

Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, Quoc V. Le

Type: Book-Chapter

Publication Date: 2020-01-01

Citations: 198

DOI: https://doi.org/10.1007/978-3-030-58571-6_41

View Chat PDF

Locations

Lecture notes in computer science - View
arXiv (Cornell University) - View - PDF

Similar Works

Action	Title	Year	Authors
+	Efficient Neural Architecture Search: A Broad Version.	2020	Zixiang Ding Yaran Chen Nannan Li Dongbin Zhao C. L. Philip Chen
+	BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models	2020	Jiahui Yu Pengchong Jin Hanxiao Liu Gabriel Bender Pieter-Jan Kindermans Mingxing Tan Thomas S. Huang Xiaodan Song Ruoming Pang Quoc V. Le
+	Neural Architecture Search: Insights from 1000 Papers	2023	Colin White Mahmoud Safari Rhea Sanjay Sukthanker Binxin Ru Thomas Elsken Arber Zela Debadeepta Dey Frank Hutter
+ PDF Chat	EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search	2021	Vasco Lopes Saeid Alirezazadeh Luı́s A. Alexandre
+ PDF Chat	Local Search is a Remarkably Strong Baseline for Neural Architecture Search	2021	T. Den Ottelander Arkadiy Dushatskiy Marco Virgolin Peter A. N. Bosman
+ PDF Chat	EAT-NAS: elastic architecture transfer for accelerating large-scale neural architecture search	2021	Jiemin Fang Yukang Chen Xinbang Zhang Qian Zhang Chang Huang Gaofeng Meng Wenyu Liu Xinggang Wang
+ PDF Chat	NAS-Count: Counting-by-Density with Neural Architecture Search	2020	Yutao Hu Xiaolong Jiang Xuhui Liu Baochang Zhang Jungong Han Xianbin Cao David Doermann
+	Neural Architecture Transfer 2: A Paradigm for Improving Efficiency in Multi-Objective Neural Architecture Search	2023	Simone Sarti Eugenio Lomurno Matteo Matteucci
+ PDF Chat	DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search	2020	Xiyang Dai Dongdong Chen Mengchen Liu Yinpeng Chen Lu Yuan
+	Towards Less Constrained Macro-Neural Architecture Search	2022	Vasco Lopes Luı́s A. Alexandre
+	NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search	2020	Xuanyi Dong Yi Yang
+ PDF Chat	NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search	2020	Xuanyi Dong Yi Yang
+	NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search	2020	Xuanyi Dong Yi Yang
+	NAS-Bench-102: Extending the Scope of Reproducible Neural Architecture Search	2020	Xuanyi Dong Yi Yang
+ PDF Chat	Evolving Search Space for Neural Architecture Search	2021	Yuanzheng Ci Lin Chen Ming Sun Boyu Chen Hongwen Zhang Wanli Ouyang
+	Evolving Search Space for Neural Architecture Search	2020	Yuanzheng Ci Lin Chen Ming Sun Boyu Chen Hongwen Zhang Wanli Ouyang
+	Evolving Search Space for Neural Architecture Search.	2020	Yuanzheng Ci Chen Lin Ming Sun Boyu Chen Hongwen Zhang Wanli Ouyang
+	A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions	2020	Pengzhen Ren Yun Xiao Xiaojun Chang Po-Yao Huang Zhihui Li Xiaojiang Chen Xin Wang
+ PDF Chat	A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions	2020	Pengzhen Ren Yun Xiao Xiaojun Chang Po-Yao Huang Zhihui Li Xiaojiang Chen Xin Wang
+ PDF Chat	Redefining Neural Architecture Search of Heterogeneous Multinetwork Models by Characterizing Variation Operators and Model Components	2023	Unai Garciarena Roberto Santana Alexander Mendiburu

Cited by (129)

Action	Title	Year	Authors
+ PDF Chat	Omni-Sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR Via Supernet	2022	Haichuan Yang Yuan Shangguan Dilin Wang Meng Li Pierce Chuang Zhang Xiao-hui Ganesh Venkatesh Ozlem Kalinli Vikas Chandra
+ PDF Chat	AutoML: A survey of the state-of-the-art	2020	Xin He Kaiyong Zhao Xiaowen Chu
+ PDF Chat	Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search	2021	Kaicheng Yu René Ranftl Mathieu Salzmann
+ PDF Chat	GAIA: A Transfer Learning System of Object Detection that Fits Your Needs	2021	Xingyuan Bu Junran Peng Junjie Yan Tieniu Tan Zhaoxiang Zhang
+ PDF Chat	Transfer-Once-For-All: AI Model Optimization for Edge	2023	Achintya Kundu Laura Wynter Rhui Dih Lee Luis Angel D. Bathen
+	Reconsidering CO2 emissions from Computer Vision	2021	Andre Fu Mahdi S. Hosseini Konstantinos N. Plataniotis
+ PDF Chat	ScaleNet: Searching for the Model to Scale	2022	Jiyang Xie Xiu Su Shan You Zhanyu Ma Fei Wang Chen Qian
+ PDF Chat	ShiftNAS: Improving One-shot NAS via Probability Shift	2023	Mingyang Zhang Xinyi Yu Haodong Zhao Linlin Ou
+ PDF Chat	AdaBits: Neural Network Quantization With Adaptive Bit-Widths	2020	Qing Jin Linjie Yang Zhenyu Liao
+ PDF Chat	Dynamic Slimmable Network	2021	Changlin Li Guangrun Wang Bing Wang Xiaodan Liang Zhihui Li Xiaojun Chang
+ PDF Chat	Network pruning via resource reallocation	2023	Yuenan Hou Zheng Ma Chunxiao Liu Zhe Wang Chen Change Loy
+	RobustART: Benchmarking Robustness on Architecture Design and Training Techniques	2021	Shiyu Tang Ruihao Gong Yan Wang Aishan Liu Jiakai Wang Xinyun Chen Fengwei Yu Xianglong Liu Dawn Song Alan Yuille
+ PDF Chat	Teachers Do More Than Teach: Compressing Image-to-Image Models	2021	Qing Jin Jian Ren Oliver J. Woodford Jiazhuo Wang Geng Yuan Yanzhi Wang Sergey Tulyakov
+ PDF Chat	ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search	2021	Lumin Xu Yingda Guan Sheng Jin Wentao Liu Chen Qian Ping Luo Wanli Ouyang Xiaogang Wang
+ PDF Chat	FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining	2021	Xiaoliang Dai Alvin Wan Peizhao Zhang Bichen Wu Zijian He Zhen Wei Kan Chen Yuandong Tian Matthew Yu Péter Vajda
+ PDF Chat	Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap	2021	Lingxi Xie Xin Chen Kaifeng Bi Longhui Wei Yuhui Xu Lanfei Wang Zhengsu Chen An Xiao Jianlong Chang Xiaopeng Zhang
+ PDF Chat	AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling	2021	Dilin Wang Meng Li Chengyue Gong Vikas Chandra
+ PDF Chat	ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild	2022	Lumin Xu Sheng Jin Wentao Liu Chen Qian Wanli Ouyang Ping Luo Xiaogang Wang
+ PDF Chat	Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search	2021	Mingzhu Shen Feng Liang Ruihao Gong Yuhang Li Chuming Li Lin Chen Fengwei Yu Junjie Yan Wanli Ouyang
+ PDF Chat	Cyclic Differentiable Architecture Search	2022	Hongyuan Yu Houwen Peng Yan Huang Jianlong Fu Hao Du Liang Wang Haibin Ling
+ PDF Chat	NASRec: Weight Sharing Neural Architecture Search for Recommender Systems	2023	Tunhou Zhang Dehua Cheng Yuchen He Zhengxing Chen Xiaoliang Dai Liang Xiong Feng Yan Hai Li Yiran Chen Wei Wen
+ PDF Chat	AutoFormer: Searching Transformers for Visual Recognition	2021	Minghao Chen Houwen Peng Jianlong Fu Haibin Ling
+ PDF Chat	ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding	2022	Dailan He Ziming Yang Weikun Peng Rui Ma Hongwei Qin Yan Wang
+ PDF Chat	Towards Automated Model Design on Recommender Systems	2024	Tunhou Zhang Dehua Cheng Yuchen He Zhengxing Chen Xiaoliang Dai Liang Xiong Yudong Liu Feng Cheng Yufan Cao Feng Yan
+ PDF Chat	Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search	2024	Alexandre Heuillet Ahmad Nasser Hichem Arioui Hedi Tabia
+	AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System	2021	Pengyu Zhao Kecheng Xiao Yuanxing Zhang Kaigui Bian Wei Yan
+ PDF Chat	Repnas: Searching for Efficient Re-Parameterizing Blocks	2023	Mingyang Zhang Xinyi Yu Jingtao Rong Linlin Ou
+ PDF Chat	UGC: Unified GAN Compression for Efficient Image-to-Image Translation	2023	Yuxi Ren Jie Wu Peng Zhang Manlin Zhang Xuefeng Xiao Qian He Rui Wang Min Zheng Xin Pan
+	RepNAS: Searching for Efficient Re-parameterizing Blocks	2021	Mingyang Zhang Xinyi Yu Jingtao Rong Linlin Ou Feng Gao
+	Integrating Large Circular Kernels into CNNs through Neural Architecture Search	2021	Kun He Chao Li Yixiao Yang Gao Huang John E. Hopcroft
+ PDF Chat	MutualNet: Adaptive ConvNet via Mutual Learning From Different Model Configurations	2021	Taojiannan Yang Sijie Zhu Matías Mendieta Pu Wang Ravikumar Balakrishnan Minwoo Lee Tao Han Mubarak Shah Chen Chen
+	Wisdom of Committees: An Overlooked Approach To Faster and More Accurate Models	2020	Xiaofang Wang Dan Kondratyuk Eric Christiansen Kris Kitani Yair Alon Elad Eban
+ PDF Chat	SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems	2022	Xin Dong B. De Salvo Meng Li Chiao Liu Zhongnan Qu H. T. Kung Ziyun Li
+ PDF Chat	FlexiViT: One Model for All Patch Sizes	2023	Lucas Beyer Pavel Izmailov А. И. Колесников Mathilde Caron Simon Kornblith Xiaohua Zhai Matthias Minderer Michael Tschannen Ibrahim Alabdulmohsin Filip Pavetic
+ PDF Chat	An Analysis of Super-Net Heuristics in Weight-Sharing NAS	2021	Kaicheng Yu René Ranftl Mathieu Salzmann
+ PDF Chat	DATA: Domain-Aware and Task-Aware Self-supervised Learning	2022	Qing Chang Junran Peng Lingxi Xie Jiajun Sun Haoran Yin Qi Tian Zhaoxiang Zhang
+ PDF Chat	SuperTickets: Drawing Task-Agnostic Lottery Tickets from Supernets via Jointly Architecture Searching and Parameter Pruning	2022	Haoran You Baopu Li Zhanyi Sun Xu Ouyang Yingyan Lin
+ PDF Chat	Joslim: Joint Widths and Weights Optimization for Slimmable Neural Networks	2021	Ting-Wu Chin Ari S. Morcos Diana Marculescu
+ PDF Chat	MRF-UNets: Searching UNet with Markov Random Fields	2023	Zifu Wang Matthew B. Blaschko
+ PDF Chat	Towards Regression-Free Neural Networks for Diverse Compute Platforms	2022	Rahul Duggal Hao Zhou Shuo Yang Jun Fang Yuanjun Xiong Wei Xia

Citing (30)

Action	Title	Year	Authors
+ PDF Chat	Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification	2015	Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
+ PDF Chat	Deep Residual Learning for Image Recognition	2016	Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
+ PDF Chat	Identity Mappings in Deep Residual Networks	2016	Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
+	Neural Architecture Search with Reinforcement Learning	2016	Barret Zoph Quoc V. Le
+	MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications	2017	Andrew Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand Marco Andreetto Hartwig Adam
+ PDF Chat	Squeeze-and-Excitation Networks	2018	Jie Hu Li Shen Gang Sun
+	Searching for Activation Functions	2017	Prajit Ramachandran Barret Zoph Quoc V. Le
+	Efficient Neural Architecture Search via Parameter Sharing	2018	Hieu Pham Melody Y. Guan Barret Zoph Quoc V. Le Jeff Dean
+ PDF Chat	ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design	2018	Ningning Ma Xiangyu Zhang Hai-Tao Zheng Jian Sun
+	ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware	2018	Han Cai Ligeng Zhu Song Han
+	Searching for MobileNetV3.	2019	Andrew Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen Mingxing Tan Weijun Wang Yukun Zhu Ruoming Pang Vijay Vasudevan
+	Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift	2015	Sergey Ioffe Christian Szegedy
+	EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks	2019	Mingxing Tan Quoc V. Le
+ PDF Chat	MobileNetV2: Inverted Residuals and Linear Bottlenecks	2018	Mark Sandler Andrew Howard Menglong Zhu Andrey Zhmoginov Liang-Chieh Chen
+ PDF Chat	Squeeze-and-Excitation Networks	2019	Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu
+	Slimmable Neural Networks	2018	Jiahui Yu Linjie Yang Ning Xu Shuicheng Yan Thomas S. Huang
+	Hierarchical Representations for Efficient Architecture Search	2017	Hanxiao Liu Karen Simonyan Oriol Vinyals Chrisantha Fernando Koray Kavukcuoglu
+ PDF Chat	Progressive Neural Architecture Search	2018	Chenxi Liu Barret Zoph Maxim Neumann Jonathon Shlens Hua Wei Li-Jia Li Li Fei-Fei Alan Yuille Jonathan Huang Kevin Murphy
+ PDF Chat	Bag of Tricks for Image Classification with Convolutional Neural Networks	2019	Tong He Zhi Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li
+ PDF Chat	MnasNet: Platform-Aware Neural Architecture Search for Mobile	2019	Mingxing Tan Bo Chen Ruoming Pang Vijay Vasudevan Mark Sandler Andrew Howard Quoc V. Le
+ PDF Chat	Learning Transferable Architectures for Scalable Image Recognition	2018	Barret Zoph Vijay Vasudevan Jonathon Shlens Quoc V. Le
+	Regularized Evolution for Image Classifier Architecture Search	2019	Esteban Real Alok Aggarwal Yanping Huang Quoc V. Le
+ PDF Chat	FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search	2019	Bichen Wu Kurt Keutzer Xiaoliang Dai Peizhao Zhang Yanghan Wang Fei Sun Yiming Wu Yuandong Tian Péter Vajda Yangqing Jia
+ PDF Chat	Universally Slimmable Networks and Improved Training Techniques	2019	Jiahui Yu Thomas S. Huang
+	Once-for-All: Train One Network and Specialize it for Efficient Deployment	2019	Han Cai Chuang Gan Tianzhe Wang Zhekai Zhang Song Han
+ PDF Chat	Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours	2020	Dimitrios Stamoulis Ruizhou Ding Di Wang Dimitrios Lymberopoulos Bodhi Priyantha Jie Liu Diana Marculescu
+ PDF Chat	DSNAS: Direct Neural Architecture Search Without Parameter Retraining	2020	Shoukang Hu Sirui Xie Hehui Zheng Chunxiao Liu Jianping Shi Xunying Liu Dahua Lin
+ PDF Chat	CARS: Continuous Evolution for Efficient Neural Architecture Search	2020	Zhaohui Yang Yunhe Wang Xinghao Chen Boxin Shi Chao Xu Chunjing Xu Qi Tian Chang Xu
+ PDF Chat	AOWS: Adaptive and Optimal Network Width Search With Latency Constraints	2020	Maxim Berman Leonid Pishchulin Ning Xu Matthew B. Blaschko Gérard Medioni
+ PDF Chat	Single Path One-Shot Neural Architecture Search with Uniform Sampling	2020	Zichao Guo Xiangyu Zhang Haoyuan Mu Wen Heng Zechun Liu Yichen Wei Jian Sun