When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism

Guangting Wang, Yucheng Zhao, Chuanxin Tang, Chong Luo, Wenjun Zeng

Type: Preprint

Publication Date: 2022-01-01

Citations: 2

DOI: https://doi.org/10.48550/arxiv.2201.10801

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+ PDF Chat	When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism	2022	Guangting Wang Yucheng Zhao Chuanxin Tang Chong Luo Wenjun Zeng
+	Twins: Revisiting the Design of Spatial Attention in Vision Transformers	2021	Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin Wei Huaxia Xia Chunhua Shen
+	Twins: Revisiting the Design of Spatial Attention in Vision Transformers	2021	Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin Wei Huaxia Xia Chunhua Shen
+	Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work	2022	Khawar Islam
+	Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention	2023	Xuran Pan Tianzhu Ye Zhuofan Xia Shiji Song Gao Huang
+	DAT++: Spatially Dynamic Vision Transformer with Deformable Attention	2023	Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang
+	CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows	2021	Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen Baining Guo
+ PDF Chat	PSLT: A Light-Weight Vision Transformer With Ladder Self-Attention and Progressive Shift	2023	Gaojie Wu Wei‐Shi Zheng Yutong Lu Qi Tian
+	Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer	2021	Zilong Huang Youcheng Ben Guozhong Luo Pei Cheng Gang Yu Bin Fu
+	Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention	2021	Sitong Wu Tianyi Wu Haoru Tan Guodong Guo
+	TiC: Exploring Vision Transformer in Convolution	2023	Song Zhang Qingzhong Wang Jiang Bian Haoyi Xiong
+	BViT: Broad Attention based Vision Transformer	2022	Nannan Li Yaran Chen Weifan Li Zixiang Ding Dongbin Zhao
+ PDF Chat	BViT: Broad Attention-Based Vision Transformer	2023	Nannan Li Yaran Chen Weifan Li Zixiang Ding Dongbin Zhao Shuai Nie
+ PDF Chat	CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows	2022	Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen Baining Guo
+	Fast Vision Transformers with HiLo Attention	2022	Zizheng Pan Jianfei Cai Bohan Zhuang
+	Demystifying Local Vision Transformer: Sparse Connectivity, Weight Sharing, and Dynamic Weight	2021	Qi Han Zejia Fan Qi Dai Lei Sun Ming‐Ming Cheng Jiaying Liu Jingdong Wang
+	LKCA: Large Kernel Convolutional Attention	2024	Chenghao Li Boheng Zeng Yi Lu Pengbo Shi Qingzi Chen Jirui Liu Lingyun Zhu
+ PDF Chat	Swin Transformer: Hierarchical Vision Transformer using Shifted Windows	2021	Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin Baining Guo
+	Three things everyone should know about Vision Transformers	2022	Hugo Touvron Matthieu Cord Alaaeldin El-Nouby Jakob Verbeek Hervé Jeǵou
+	A survey of the Vision Transformers and its CNN-Transformer based Variants	2023	Asifullah Khan Zunaira Rauf Anabia Sohail Abdul Rehman Hifsa Asif Aqsa Asif Umair Farooq

Works That Cite This (1)

Action	Title	Year	Authors
+ PDF Chat	Towards an Effective and Efficient Transformer for Rain-by-Snow Weather Removal	2023	Tao Gao Yuanbo Wen Kaihao Zhang Peng Cheng Ting Chen

Works Cited by This (0)

Action	Title	Year	Authors