Minimax entropy estimation beyond two derivatives
Fix $d \ge 3$, $s > 2$, $2 \le p < \infty$ and a sufficiently large $L$. Put $r = \lceil s \rceil$ and, for $f$ on $\mathbb{R}^d$ and $h \in \mathbb{R}^d$, $$\Delta_h^r f(x) = \sum_{j=0}^{r} (-1)^{r-j} \binom{r}{j} f\bigl(x + (j - r/2)h\bigr), \qquad \omega_r(f,t)_p = \sup_{|h| \le t} \|\Delta_h^r f\|_{L^p(\mathbb{R}^d)}.$$ Let $\mathcal{F}_{s,p,d}(L)$ be the set of probability densities $f$ supported in $[0,1]^d$, extended by zero to $\mathbb{R}^d$, with $$\|f\|_{L^p(\mathbb{R}^d)} + \sup_{t > 0} t^{-s} \omega_r(f,t)_p \le L;$$ no positive lower bound on $f$ is imposed. From $n$ independent observations with density $f$, the target is the differential entropy $H(f) = -\int_{[0,1]^d} f \log f$, with $0 \log 0 = 0$, and $$R_n = \inf_{\widehat{H}} \sup_{f \in \mathcal{F}_{s,p,d}(L)} \bigl(\mathbb{E}_f |\widehat{H} - H(f)|^2\bigr)^{1/2},$$ the infimum over all estimators. Han, Jiao, Weissman and Wu proved that for $0 < s \le 2$, every $d \ge 1$ and $2 \le p < \infty$, the minimax risk over this class is of order $(n \log n)^{-s/(s+d)} + n^{-1/2}$, and that the lower bound $$R_n \gtrsim (n \log n)^{-s/(s+d)} + n^{-1/2}$$ holds for every $s > 0$.
Problem. Determine the order of $R_n$ as $n \to \infty$, up to constants depending on $s$, $p$, $d$ and $L$. In particular, is the lower bound above sharp for all such $d$, $s$, $p$ and $L$?
Source: Han, Jiao, Weissman and Wu (2020), §4.2 (the case s > 2); stated in this form by AIM — Open Applied Problems.
Imported from github.com/MColbrook/AIM.
Classification
- Primary
- ST — Statistics Theory
- Secondary
- IT
- Keywords
- AIM Open Applied Problems, differential entropy, functional estimation, minimax rates
References
@article{han2020,
author = {Han, Yanjun and Jiao, Jiantao and Weissman, Tsachy and Wu, Yihong},
title = {Optimal rates of entropy estimation over {L}ipschitz balls},
journal = {Ann. Statist.},
volume = {48},
number = {6},
year = {2020},
pages = {3228--3250},
doi = {10.1214/19-AOS1927},
eprint = {1711.02141},
archivePrefix = {arXiv},
}
@misc{aim-high-smoothness-entropy-estimation,
author = {Brunton, Steve and Colbrook, Matthew J. and de Hoop, Maarten V. and Stepaniants, George and Townsend, Alex and Ward, Rachel},
title = {{Minimax entropy estimation beyond two derivatives}},
howpublished = {AIM --- Open Applied Problems, GitHub repository},
year = {2026},
url = {https://github.com/MColbrook/AIM/blob/a153f469ee0ea64e876b2ad97c79d252c19a3e22/problems/493-high-smoothness-entropy-estimation.md},
note = {Entry 493 at commit a153f46, 2026-10-13},
}
Citing this problem
This identifier is permanent. It will not change if the problem is solved, reclassified or withdrawn.
OP-00289, "Minimax entropy estimation beyond two derivatives", clopen.solutions.