Demystify Transformers & Convolutions in Modern Image Deep Networks

Jifeng Dai, Min Shi, Wei‐Yun Wang, Sitong Wu, Linjie Xing, Wenhai Wang, Xizhou Zhu, Lewei Lu, Jie Zhou, Xiaogang Wang

Type: Preprint

Publication Date: 2022-01-01

Citations: 1

DOI: https://doi.org/10.48550/arxiv.2211.05781

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Blending Anti-Aliasing into Vision Transformer	2021	Shengju Qian Hao Shao Yi Zhu Mu Li Jiaya Jia
+	CoAtNet: Marrying Convolution and Attention for All Data Sizes	2021	Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan
+	TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition	2023	Meng Lou Hong-Yu Zhou Sibei Yang Yizhou Yu
+	CvT: Introducing Convolutions to Vision Transformers	2021	Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Yuan Lü Lei Zhang
+ PDF Chat	CvT: Introducing Convolutions to Vision Transformers	2021	Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang
+	Do Vision Transformers See Like Convolutional Neural Networks?	2021	Maithra Raghu Thomas Unterthiner Simon Kornblith Chiyuan Zhang Alexey Dosovitskiy
+	Convolutional Xformers for Vision	2022	Pranav Jeevan Amit sethi
+	A survey of the Vision Transformers and its CNN-Transformer based Variants	2023	Asifullah Khan Zunaira Rauf Anabia Sohail Abdul Rehman Hifsa Asif Aqsa Asif Umair Farooq
+	UniNeXt: Exploring A Unified Architecture for Vision Recognition	2023	Fangjian Lin Jianlong Yuan Sitong Wu Fan Wang Zhibin Wang
+	CoAtNet: Marrying Convolution and Attention for All Data Sizes	2021	Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan
+ PDF Chat	ACC-ViT : Atrous Convolution's Comeback in Vision Transformers	2024	Nabil Ibtehaz Ning Yan Masood Mortazavi Daisuke Kihara
+ PDF Chat	Rethinking Spatial Dimensions of Vision Transformers	2021	Byeongho Heo Sangdoo Yun Dongyoon Han Sanghyuk Chun Junsuk Choe Seong Joon Oh
+	Interpret Vision Transformers as ConvNets with Dynamic Convolutions	2023	Chong Zhou Chen Change Loy Bo Dai
+	ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond	2022	Qiming Zhang Yufei Xu Jing Zhang Dacheng Tao
+	DMFormer: Closing the Gap Between CNN and Vision Transformers	2022	Zimian Wei Hengyue Pan Xin Niu Dongsheng Li
+	DMFormer: Closing the gap Between CNN and Vision Transformers	2023	Zimian Wei Hengyue Pan Lujun Li Menglong Lu Xin Niu Peijie Dong Dongsheng Li
+	TiC: Exploring Vision Transformer in Convolution	2023	Song Zhang Qingzhong Wang Jiang Bian Haoyi Xiong
+	Fcaformer: Forward Cross Attention in Hybrid Vision Transformer	2022	Haokui Zhang Wenze Hu Xiaoyu Wang
+	LightViT: Towards Light-Weight Convolution-Free Vision Transformers	2022	Tao Huang Lang Huang Shan You Fei Wang Qian Chen Chang Xu
+	Transformer in Convolutional Neural Networks	2021	Yun Liu Guolei Sun Yu Qiu Le Zhang Ajad Chhatkuli Luc Van Gool

Works That Cite This (1)

Action	Title	Year	Authors
+	UniNeXt: Exploring A Unified Architecture for Vision Recognition	2023	Fangjian Lin Jianlong Yuan Sitong Wu Fan Wang Zhibin Wang

Works Cited by This (0)

Action	Title	Year	Authors

Demystify Transformers &amp; Convolutions in Modern Image Deep Networks

Locations

Similar Works

Works That Cite This (1)

Works Cited by This (0)

Demystify Transformers & Convolutions in Modern Image Deep Networks