Preventing False Discovery in Interactive Data Analysis Is Hard
Preventing False Discovery in Interactive Data Analysis Is Hard
We show that, under a standard hardness assumption, there is no computationally efficient algorithm that given n samples from an unknown distribution can give valid answers to n3+o(1) adaptively chosen statistical queries. A statistical query asks for the expectation of a predicate over the underlying distribution, and an answer to …