Kernel Deformed Exponential Families for Sparse Continuous Attention.

Alexander Moreno, Supriya Nagesh, Zhenke Wu, Walter Dempsey, James M. Rehg

Type: Preprint

Publication Date: 2021-11-01

Citations: 0

Locations

arXiv (Cornell University) - View

Similar Works

Action	Title	Year	Authors
+	Kernel Deformed Exponential Families for Sparse Continuous Attention	2021	Alexander Moreno Supriya Nagesh Zhenke Wu Walter Dempsey James M. Rehg
+	Sparse and Continuous Attention Mechanisms	2020	André F. T. Martins António Farinhas Marcos V. Treviso Vlad Niculae Pedro M. Q. Aguiar Mário A. T. Figueiredo
+	Designing Robust Transformers using Robust Kernel Density Estimation	2022	Han Xing Tongzheng Ren Tan M. Nguyen Khai Nguyen Joydeep Ghosh Nhat Ho
+	Sparse Continuous Distributions and Fenchel-Young Losses	2021	André F. T. Martins Marcos Treviso António Farinhas Pedro M. Q. Aguiar Mário A. T. Figueiredo Mathieu Blondel Vlad Niculae
+	Squared Neural Families: A New Class of Tractable Density Models	2023	Russell Tsuchida Cheng Soon Ong Dino Sejdinović
+	Kernel Conditional Exponential Family	2017	Michael Arbel Arthur Gretton
+	Kernel Conditional Exponential Family.	2017	Michael Arbel Arthur Gretton
+ PDF Chat	Quadratic Gating Functions in Mixture of Experts: A Statistical Insight	2024	Pedram Akbarian Huy Nguyen Xing Han Nhat Ho
+	Hybrid Random Features	2021	Krzysztof Choromański Haoxian Chen Han Lin Yuanzhe Ma Arijit Sehanobish Deepali Jain Michael S. Ryoo Jake Varley Andy Zeng Valerii Likhosherstov
+	Hybrid Random Features	2021	Krzysztof Choromański Haoxian Chen Han Lin Yuanzhe Ma Arijit Sehanobish Deepali Jain Michael S. Ryoo Jake Varley Andy Zeng Valerii Likhosherstov
+	Convolutional Deep Exponential Families	2021	Chengkuan Hong Christian R. Shelton
+	Learning deep kernels for exponential family densities	2018	Li Kevin Wenliang Danica J. Sutherland Heiko Strathmann Arthur Gretton
+	Learning deep kernels for exponential family densities	2018	Li Kevin Wenliang Danica J. Sutherland Heiko Strathmann Arthur Gretton
+ PDF Chat	A Simple Exponential Family Framework for Zero-Shot Learning	2017	Vinay Kumar Verma Piyush Rai
+ PDF Chat	Deep learning for presumed probability density function models	2019	Marc Henry de Frahan Shashank Yellapantula Ryan King Marc Day Ray Grout
+	Function-Space Distributions over Kernels	2019	Gregory W. Benton Wesley J. Maddox Jayson P. Salkey Julio Albinati Andrew Gordon Wilson
+	Function-Space Distributions over Kernels	2019	Gregory W. Benton Wesley J. Maddox Jayson P. Salkey Julio Albinati Andrew Gordon Wilson
+ PDF Chat	Large Separable Kernel Attention: Rethinking the Large Kernel Attention design in CNN	2023	Kin Wai Lau Lai-Man Po Yasar Abbas Ur Rehman
+ PDF Chat	Rethinking Softmax: Self-Attention with Polynomial Activations	2024	Hemanth Saratchandran Jianqiao Zheng Yiping Ji Wenbo Zhang Simon Lucey
+	Deep Kernel Learning	2015	Andrew Gordon Wilson Zhiting Hu Ruslan Salakhutdinov Eric P. Xing

Works That Cite This (0)

Action	Title	Year	Authors

Works Cited by This (13)

Action	Title	Year	Authors
+	Estimators, escort probabilities, and phi-exponential families in statistical physics	2004	Jan Naudts
+	Kernel exponential family estimation via doubly dual embedding	2019	Bo Dai Hanjun Dai Arthur Gretton Le Song Dale Schuurmans Niao He
+	Attention is All you Need	2017	Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser Illia Polosukhin
+ PDF Chat	Attend and Diagnose: Clinical Time Series Analysis Using Attention Models	2018	Huan Song Deepta Rajan Jayaraman J. Thiagarajan Andreas Spanias
+	Learning deep kernels for exponential family densities	2018	Li Kevin Wenliang Danica J. Sutherland Heiko Strathmann Arthur Gretton
+	Efficient and principled score estimation with Nyström kernel exponential families	2018	Danica J. Sutherland Heiko Strathmann Michael Arbel Arthur Gretton
+	Density Estimation in Infinite Dimensional Exponential Families	2017	Bharath K. Sriperumbudur Kenji Fukumizu Arthur Gretton Aapo Hyvärinen Revant Kumar
+	Kernel Conditional Exponential Family	2017	Michael Arbel Arthur Gretton
+	Neural Machine Translation by Jointly Learning to Align and Translate	2015	Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio
+	Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel.	2019	Yao-Hung Hubert Tsai Shaojie Bai Makoto Yamada Louis‐Philippe Morency Ruslan Salakhutdinov