Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization
We develop and analyze <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</i> -estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a nonasymptotic variational characterization of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> -divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The …