Estimation of KL Divergence: Optimal Minimax Rate

Yuheng Bu, Shaofeng Zou, Yingbin Liang, Venugopal V. Veeravalli

Type: Article

Publication Date: 2018-02-13

Citations: 59

DOI: https://doi.org/10.1109/tit.2018.2805844

View Publication

Locations

IDEALS (University of Illinois Urbana-Champaign) - View - PDF
IEEE Transactions on Information Theory - View

Similar Works

Action	Title	Year	Authors
+	Estimation of KL Divergence: Optimal Minimax Rate	2016	Yuheng Bu Shaofeng Zou Yingbin Liang Venugopal V. Veeravalli
+	Estimation of KL divergence between large-alphabet distributions	2016	Yuheng Bu Shaofeng Zou Yingbin Liang Venugopal V. Veeravalli
+	Minimax Optimal Estimation of KL Divergence for Continuous Distributions	2020	Puning Zhao Lifeng Lai
+	Minimax Optimal Estimation of KL Divergence for Continuous Distributions	2020	Puning Zhao Lifeng Lai
+	Minimax Estimation of KL Divergence between Discrete Distributions.	2016	Yanjun Han Jiantao Jiao Tsachy Weissman
+	Minimax Rate-optimal Estimation of KL Divergence between Discrete Distributions.	2016	Yanjun Han Jiantao Jiao Tsachy Weissman
+	Local likelihood methods : estimation of Kullback-Leibler divergence and extension to random fields	2004	이영경
+	Minimax Estimation of the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math> </inline-formula> Distance	2018	Jiantao Jiao Yanjun Han Tsachy Weissman
+	Exact Expressions for Kullback–Leibler Divergence for Univariate Distributions	2024	Victor Mooto Nawa Saralees Nadarajah
+	Analysis of K Nearest Neighbor KL Divergence Estimation for Continuous Distributions	2020	Puning Zhao Lifeng Lai
+ PDF Chat	Bounds on $L_p$ Errors in Density Ratio Estimation via $f$-Divergence Loss Functions	2024	Yoshiaki Kitazawa
+ PDF Chat	Minimax Optimal Additive Functional Estimation with Discrete Distribution: Slow Divergence Speed Case	2018	Kazuto Fukuchi Jun Sakuma
+	Estimation of Kullback–Leibler Divergence by Local Likelihood	2006	Young Lee Byeong U. Park
+ PDF Chat	On Semi-supervised Estimation of Discrete Distributions under f-divergences	2024	Hasan Sabri Melihcan Erol Lizhong Zheng
+	Minimax estimation with divergence loss function	1974	R.L. Kashyap
+	Kullback–Leibler divergence: A quantile approach	2016	P. G. Sankaran S. M. Sunoj N. Unnikrishnan Nair
+	Minimax Optimal Additive Functional Estimation with Discrete Distribution	2018	Kazuto Fukuchi Jun Sakuma
+ PDF Chat	Minimax Estimation of Discrete Distributions Under <inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> Loss	2015	Yanjun Han Jiantao Jiao Tsachy Weissman
+	Divergence estimation for machine learning and signal processing	2013	Masashi Sugiyama
+	Minimax Estimation of Divergences Between Discrete Distributions	2020	Yanjun Han Jiantao Jiao Tsachy Weissman

Works That Cite This (24)

Action	Title	Year	Authors
+	Minimax Optimal Estimation of KL Divergence for Continuous Distributions	2020	Puning Zhao Lifeng Lai
+ PDF Chat	Optimal rates of entropy estimation over Lipschitz balls	2020	Yanjun Han Jiantao Jiao Tsachy Weissman Yihong Wu
+	On the Competitive Analysis and High Accuracy Optimality of Profile Maximum Likelihood	2020	Yanjun Han Kirankumar Shiragur
+	Optimal Estimation of Wasserstein Distance on a Tree With an Application to Microbiome Studies	2019	Shulei Wang Tommaso Cai Hongzhe Li
+	Minimax Optimal Estimation of KL Divergence for Continuous Distributions	2020	Puning Zhao Lifeng Lai
+	Analysis of K Nearest Neighbor KL Divergence Estimation for Continuous Distributions	2020	Puning Zhao Lifeng Lai
+ PDF Chat	Comparing and Weighting Imperfect Models Using D-Probabilities	2019	Meng Li David B. Dunson
+	Optimal rates of entropy estimation over Lipschitz balls	2017	Yanjun Han Jiantao Jiao Tsachy Weissman Yihong Wu
+	Minimax Estimation of the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math> </inline-formula> Distance	2018	Jiantao Jiao Yanjun Han Tsachy Weissman
+ PDF Chat	Minimax Rates of ℓp-Losses for High-Dimensional Linear Errors-in-Variables Models over ℓq-Balls	2021	Xin Li Dongya Wu

Works Cited by This (21)

Action	Title	Year	Authors
+	Introduction to Nonparametric Estimation	2008	Alexandre B. Tsybakov
+	Theory of Approximation of Functions of a Real Variable	1963	A.F. TIMAN
+	Probability and Computing: Randomized Algorithms and Probabilistic Analysis	2005	Michael Mitzenmacher Eli Upfal
+	The Rational Approximation of Real Functions	1979	Daniel Wulbert
+ PDF Chat	Minimax Estimation of Functionals of Discrete Distributions	2015	Jiantao Jiao Kartik Venkat Yanjun Han Tsachy Weissman
+ PDF Chat	Ensemble estimation of multivariate f-divergence	2014	Kevin R. Moon Alfred O. Hero
+	Nonparametric Estimation of Küllback-Leibler Divergence	2014	Zhiyi Zhang Michael Grabchak
+	Polynomial Approximation on Polytopes	2014	Вилмос Тотик
+ PDF Chat	Rényi Divergence and Kullback-Leibler Divergence	2014	Tim van Erven Peter Harremoës
+	Two-Sample Test Statistics for Measuring Discrepancies Between Two Multivariate Probability Density Functions Using Kernel-Based Density Estimates	1994	Niall Anderson Peter A. Hall D. M. Titterington