A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking

Lorenzo Papa, Paolo Russo, Irene Amerini, Luping Zhou

Type: Article

Publication Date: 2024-04-24

Citations: 8

DOI: https://doi.org/10.1109/tpami.2024.3392941

View Publication

Locations

IEEE Transactions on Pattern Analysis and Machine Intelligence - View
arXiv (Cornell University) - View - PDF
PubMed - View
IRIS Research product catalog (Sapienza University of Rome) - View - PDF

Similar Works

Action	Title	Year	Authors
+	A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking	2023	Lorenzo Papa Paolo Russo Irene Amerini Luping Zhou
+	Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work	2022	Khawar Islam
+	Fast Vision Transformers with HiLo Attention	2022	Zizheng Pan Jianfei Cai Bohan Zhuang
+	A survey of the Vision Transformers and its CNN-Transformer based Variants	2023	Asifullah Khan Zunaira Rauf Anabia Sohail Abdul Rehman Hifsa Asif Aqsa Asif Umair Farooq
+ PDF Chat	A Comprehensive Survey of Transformers for Computer Vision	2023	Sonain Jamil Md. Jalil Piran Oh -Jin Kwon
+ PDF Chat	ELFATT: Efficient Linear Fast Attention for Vision Transformers	2025	Chong Wu Maolin Che Renjie Xu Zhuoheng Ran Hong Yan
+	A Comprehensive Survey of Transformers for Computer Vision	2022	Sonain Jamil Md. Jalil Piran Oh‐Jin Kwon
+	Post-Training Quantization for Vision Transformer	2021	Zhenhua Liu Yunhe Wang Kai Han Wei Zhang Siwei Ma Wen Gao
+ PDF Chat	Mixed Non-linear Quantization for Vision Transformers	2024	Gihwan Kim Jemin Lee Sihyeong Park Yongin Kwon Hyungsin Kim
+ PDF Chat	A Survey on Vision Transformer	2022	Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo Zhenhua Liu Yehui Tang An Xiao Chunjing Xu Yixing Xu
+	A Survey on Vision Transformer	2020	Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo Zhenhua Liu Yehui Tang An Xiao Chunjing Xu Yixing Xu
+ PDF Chat	A Comprehensive Survey of Transformers for Computer Vision	2023	Sonain Jamil Md. Jalil Piran Oh‐Jin Kwon
+ PDF Chat	AutoFormer: Searching Transformers for Visual Recognition	2021	Minghao Chen Houwen Peng Jianlong Fu Haibin Ling
+ PDF Chat	LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition	2024	Youbing Hu Yun Cheng Anqi Lu Zhiqiang Cao Dawei Wei Jie Liu Zhijun Li
+	Multi-Tailed Vision Transformer for Efficient Inference	2022	Yunke Wang Boxue Du Chang Xu
+ PDF Chat	Visformer: The Vision-friendly Transformer	2021	Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian
+	Visformer: The Vision-friendly Transformer	2021	Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian
+ PDF Chat	FMViT: A Multiple-Frequency Mixing Vision Transformer	2024	Weijie Tan Yifeng Geng Xuansong Xie
+	Convolutional Xformers for Vision	2022	Pranav Jeevan Amit sethi
+	Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer	2022	Yanjing Li Sheng Xu Baochang Zhang Xianbin Cao Peng Gao Guodong Guo

Works That Cite This (1)

Action	Title	Year	Authors
+ PDF Chat	Adfq-Vit: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers	2024	Yanfeng Jiang Ning Sun Xueshuo Xie Fei Yang Tao Li

Works Cited by This (57)

Action	Title	Year	Authors
+	Distilling the Knowledge in a Neural Network	2015	Geoffrey E. Hinton Oriol Vinyals Jay B. Dean
+ PDF Chat	ImageNet Large Scale Visual Recognition Challenge	2015	Olga Russakovsky Jia Deng Hao Su Jonathan Krause Sanjeev Satheesh Sean Ma Zhiheng Huang Andrej Karpathy Aditya Khosla Michael S. Bernstein
+	Neural Machine Translation by Jointly Learning to Align and Translate	2014	Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio
+ PDF Chat	Deformable Convolutional Networks	2017	Jifeng Dai Haozhi Qi Yuwen Xiong Yi Li Guodong Zhang Han Hu Yichen Wei
+ PDF Chat	Unified Perceptual Parsing for Scene Understanding	2018	Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun
+	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	2018	Jacob Devlin Ming‐Wei Chang Kenton Lee Kristina Toutanova
+ PDF Chat	Panoptic Feature Pyramid Networks	2019	Alexander Kirillov Ross Girshick Kaiming He Piotr Dollár
+ PDF Chat	Learning Efficient Convolutional Networks through Network Slimming	2017	Zhuang Liu Jianguo Li Zhiqiang Shen Gao Huang Shoumeng Yan Changshui Zhang
+ PDF Chat	Non-local Neural Networks	2018	Xiaolong Wang Ross Girshick Abhinav Gupta Kaiming He
+ PDF Chat	Focal Loss for Dense Object Detection	2017	Tsung-Yi Lin Priya Goyal Ross Girshick Kaiming He Piotr Dollár