Aaditya Ramdas – Multiple hypothesis testing

For the last 5 years, I have been interested in a variety of topics in multiple hypothesis testing, and related topics in selective inference (post-selection inference), simultaneous inference, etc.

Online multiple hypothesis testing (package) (vignette) (shiny app, FDR) (shiny app, FWER)

Online multiple testing considers the setting in which a sequence of decisions has to be made one at a time in an online fashion. Usually, these decisions are tests of different hypotheses, to either test a global null hypothesis or control the familywise error rate or false discovery rate. However, one may instead be interested in constructing a series of confidence intervals, or just making sign decisions on parameters, and controlling the false coverage rate or false sign rate.

• ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls
  J. Tian, A. Ramdas       NeurIPS, 2019   arxiv   code

• Online control of the false coverage rate and false sign rate
  A. Weinstein*, A. Ramdas*       ICML, 2020   arxiv   proc

• Online control of the familywise error rate
  J. Tian, A. Ramdas       Statistical Methods in Medical Research, 2021   arxiv   proc

• Asynchronous online testing of multiple hypotheses
  T. Zrnic, A. Ramdas, M. Jordan       JMLR, 2021   arxiv   code   blog   proc

• The power of batching in multiple hypothesis testing
  T. Zrnic, D. Jiang, A. Ramdas, M. Jordan       AISTATS, 2020   arxiv   talk   proc

• SAFFRON: an adaptive algorithm for online FDR control
  A. Ramdas, T. Zrnic, M. Wainwright, M. Jordan       ICML, 2018   arxiv   proc   github   (full oral talk)

• MAB-FDR: Multi (A)rmed/(B)andit testing with online FDR control
  F. Yang, A. Ramdas, K. Jamieson, M. Wainwright       NeurIPS, 2017   arxiv   code   30-min talk   proc   (spotlight talk)

• Online control of the false discovery rate with decaying memory
  A. Ramdas, F. Yang, M. Wainwright, M. Jordan       NeurIPS, 2017   arxiv   summary   proc   15-min talk from 44:00   (full oral)

• Dynamic Algorithms for Online Multiple Testing
  Z. Xu, A. Ramdas       Mathematical and Scientific ML, 2021   arxiv

Interactive/dynamic multiple testing

I'm interested in the question of how to involve a human-in-the-loop while testing one or multiple hypotheses. These new types of procedures are “interactive”, and involve the human making a sequence of (arbitrary, but hopefully smart) data-dependent decisions. The aim is two-fold: (a) no matter what the human does, type-1 error should be controlled, (b) if the human made smart decisions (based on the data, and any heuristic model), the power should be higher than what one can achieve with a noninteractive test.

• Interactive identification of individuals with positive treatment effect while controlling false discoveries
  B. Duan, L. Wasserman, A. Ramdas       arxiv   talk

• STAR: A general interactive framework for FDR control under structural constraints
  L. Lei, A. Ramdas, W. Fithian       Biometrika, 2020   arxiv   proc   code   talk

• Familywise error rate control by interactive unmasking
  B. Duan, A. Ramdas, L. Wasserman       ICML, 2020   arxiv   code

• Interactive martingale tests for the global null
  B. Duan, A. Ramdas, S. Balakrishnan, L. Wasserman       Electronic J. of Stat., 2020   arxiv   code   proc

• Interactive rank tests
  B. Duan, A. Ramdas, L. Wasserman       arxiv

• Simultaneous high-probability bounds on the FDP in structured, regression and online settings
  E. Katsevich, A. Ramdas       Annals of Stat., 2020   arxiv   proc   code

• Large-scale simultaneous inference under dependence
  J. Tian, X. Chen, E. Katsevich, J. Goeman, A. Ramdas       arxiv  

Structured multiple testing

There is much interest in how to incorporate various notions of structure into multiple testing to improve both power and precision of off-the-shelf procedures. I have considered various different types of “structure” in my past work, but it seems like there is much that remains to be explored.

• False discovery rate control with e-values
  R. Wang, A. Ramdas       (JRSSB, minor revision)   arxiv

• A unified framework for bandit multiple testing
  Z. Xu, R. Wang, A. Ramdas       NeurIPS, 2021   arxiv  

• A unified treatment of multiple testing with prior knowledge using the p-filter
  A. Ramdas, R. F. Barber, M. Wainwright, M. Jordan       Annals of Stat., 2019   arxiv   code   proc

• DAGGER: A sequential algorithm for FDR control on DAGs
  A. Ramdas, J. Chen, M. Wainwright, M. Jordan       Biometrika, 2019   arxiv   code   BM

• p-filter: multi-layer FDR control for grouped hypotheses
  R. F. Barber*, A. Ramdas*       J of Royal Stat. Soc., Series B, 2016   arxiv   code   JRSSB

• QuTE: decentralized FDR control on sensor networks
  A. Ramdas, J. Chen, M. Wainwright, M. Jordan       IEEE CDC, 2017   code   CDC

Miscellaneous

• Fast and powerful conditional randomization testing via distillation
  M. Liu, E. Katsevich, L. Janson, A. Ramdas       Biometrika, 2021   arxiv   code

• Optimal rates and tradeoffs for multiple testing
  M. Rabinovich, A. Ramdas, M. Wainwright, M. Jordan       Stat. Sinica, 2019   arxiv   proc