Aaditya Ramdas
 |
Aaditya Ramdas
132H Baker Hall |
How do I spend my research time?
I am motivated by broad practical issues in the application of statistics, data science, ML and AI to solve problems in science and technology, that I attempt to address using a theory-backed methodology lens. Most of my published work appears in the top proceedings in ML/AI (NeurIPS, ICML, COLT, ALT, AISTATS, UAI, AAAI, IJCAI, etc) and the top journals in statistics (AoS, AoAS, Biometrika, JRSSB, EJS, JCGS, PNAS, etc), with several one-off papers in the top IEEE, SIAM and probability journals.
How should we quantify prediction uncertainty for black box ML algorithms? We have developed several “distribution-free” methods that achieve clean, nonasymptotic guarantees without making any distributional or algorithmic assumptions. Our state-of-the-art QOOB algorithm s the most computationally and statistically efficient out-of-the-box regression UQ method thus far — it combines nested conformal prediction, quantile random forests, out-of-bag residual scores, and the jackknife+ (AoS’21). Our NeurIPS’19 paper showed how to handle covariate shift using weighted exchangeability, and our NeurIPS’20 paper studied distribution-free calibration in binary classification, which we are currently extending to multiclass settings. (supported by American Institute of Math SQUARE grant)
How should we quantify estimation uncertainty in sequential settings? Our Probability Surveys paper developed a wide variety of sophisticated martingale techniques that unify and generalize a wide-variety of classical and modern self-normalized concentration inequalities for scalars, vectors and matrices in discerete and continuous time. These can be used to sequentially test hypotheses or estimate functionals using time-uniform, nonasymptotic confidence sequences (AoS’21) that are valid at arbitrary stopping times under nonparametric conditions. We have examined their admissibility, and have developed state-of-the-art confidence sequences for a wide variety of fundamental problems, like means of bounded random variables, both with and without replacement, as well as quantiles and CDFs of arbitrary distributions. (supported by an NSF DMS Small grant)
How can we avoid making false discoveries while testing tens of thousands of hypotheses? We have designed novel multiple hypothesis testing algorithms in a wide variety of structured settings. ADDIS (NeurIPS’19), SAFFRON (ICML’18) and LORD++ (NeurIPS’17) are the state-of-the-art FDR algorithms in the online setting where one encounters an infinite sequence of hypotheses, while DAGGER (Biometrika’19) handles directed acyclic graphs of hypotheses. The p-filter framework (JRSSB’16, AoS’19) can enforce FDR control simultaneously for different partitions of hypotheses into groups, while handling various types of weights and dependence. We now know how to provide post-hoc bounds on the false discovery proportion for knockoffs and other problems (AoS’21), online control of the false coverage rate, false sign rate (ICML’20) and the familywise error rate (SMMR’21). Our e-BH procedure generalizes the Benjamini-Hochberg procedure to work with e-values instead of p-values, and the distilled conditional randomization test is a powerful, computationally efficient method for testing conditional independence in the model-X setting. (supported by an NSF CAREER award)
How can we involve a human-in-the-loop during scientific data collection and analysis? In order to avoid selection bias and p-hacking, we have developed several techniques for “masking” the data and progressively unmasking it, in an interactive loop between human and algorithm. These frameworks allow for utilizing covariates, learning/modeling algorithms, a scientist's prior information, and enforcing arbitrary structural constraints. For example, STAR (Biometrika’21) is an interactive algorithm for FDR control, but we also have interactive methods for familywise error rate (ICML’20) and for global null testing (EJS’21), as well as interactive Wilcoxon rank test for causal inference. We have also qualitatively and quantitatively characterized the selection bias due to adaptive sampling and stopping in multi-armed bandits (NeurIPS’19, ICML’20). (supported by an Adobe Faculty Research award)
High-dimensional and dimension-agnostic inference via sample splitting and machine learning. My earlier work debunked the myth (AAAI’15) that kernel/distance-based tests do not suffer from a curse of dimensionality. In fact, the null distribution of many tests differs in low- and high-dimensional settings. This motivated the development of dimension-agnostic inference, which is the goal of designing inferential methods that are valid (and powerful) in low- and high-dimensional settings. Sample splitting and machine learning algorithms can be together used effectively for high-dimensional inference, for example classification accuracy for two-sample testing (AoS’21) or regression accuracy for conditional independence testing. Our universal inference paper (PNAS’20), in addition to being dimension-agnostic, also provides finite sample guarantees without any regularity assumptions, leading to the first tests and confidence sets for a wide variety of problems involving dependence, mixtures or shape constraints.
I have various works in optimization, active learning, information theory, kernel methods, and several scientific collaborations (see the last few sections of my publication page). More details in my Curriculum Vitae.
How do I lead my life outside research?
I lead a very high energy and intense lifestyle, and I like the fact that an academic life provides a large amount of freedom and space to take the lead with new initiatives. I believe that it is the responsibility of the young faculty members (the “new guard”, who will be tomorrow's academic leaders), to innovate not just in research, but push the boundaries of what we want tomorrow's universities and academia to look like.
Teaching. I am a passionate teacher and enjoy developing novel classes. I have designed and taught new PhD courses on martingales, sequential analysis and concentration, statistical methods for reproducibility, and the ABCDE of statistical methods in ML. In Fall 2020, I designed and co-taught a new tri-department multidisciplinary undergraduate freshman seminar class on Voting, coinciding with the US elections. In Spring 2021, I am teaching a new tri-university class on game-theoretic statistical inference. I spend a lot of time designing the syllabi, homeworks, projects and course policies/structure, and one of my major inspirations is anonymous student feedback and the resulting faculty course evaluations. As a PhD student, I completed the Future Faculty Program offered by the Eberley Center for Teaching Excellence, and received a Graduate Teaching Award from the ML department (2014) and then the Alan J. Perlis Graduate Student Teaching Award (2015, one student in the School of CS). I have always involved myself in STEM outreach activities in various countries (USA, Oman, India) and to various audiences (primary, middle and high schools), including historically underrepresented communities (all-girls programs, lower income neighborhoods).
Communication. I enjoy communicating my research broadly and work hard at creating engaging talks and tutorials, many of which are linked here. I have given over 100 invited seminar talks in a variety of university departments (Stat, CS, EE, Math, OR) and industry research labs. It moves me to get positive feedback from people I have never met. I enjoy organizing cool workshops on diverse topics with fun colleagues, such as optimization, active learning, adaptive data analysis, nonparametrics, and learning theory. In summer 2021 (modulo Covid), Peter Grunwald and I will be co-organizing a week-long workshop on safe, anytime-valid inference. I was invited to be the primary instructor for a two-week Bocconi-Oxford-Imperial summer school in advanced statistics+probability at Lake Como, for which I chose the topic to be “Statistical inference using betting scores, e-values and martingales”, and Glenn Shafer has graciously accepted to be my co-instructor.
Service. I enjoy helping conference or journals experiment with new modern reviewing protocols. In the last few years, I have served as a meta-reviewer (or area chair or senior program committee member) for most major ML conferences, such as NeurIPS, ICML, COLT, ALT, AISTATS and UAI. These have each succesfully upgraded the peer-review experience and conference model in different ways to meet the increasing challenges of volume of papers and breadth of topics (OpenReview, GatherTown, randomized experiments, matching and bidding, etc). For 2021-22, I will serve as an associate editor for an exciting new journal with an interesting review process, the New England Journal of Statistics in Data Science. I also review dozens of papers per year for the top statistics journals (AoS, JRSSB, JASA, Biometrika, Stat. Sci., EJS, Bernoulli, etc). I am also one of the two arXiv stat.ML moderators, the highest load category within statistics.
Mentorship. I take my mentorship role very seriously, and I closely advise a talented group of postdocs, senior and junior PhD students, masters students and undergraduates in Stats and ML. I work with them not only on their research, but on a wide variety of other complementary skills such as writing and speaking. I have developed a variety of checklists for Stat-ML PhD students, which include topics like poster presentations, writing effective rebuttals, work-life balance, and so on, which seem important but are hard to get advice or mentorship on. I will continue to populate these over the next few years into a handbook involving other topics like navigating the academic job market.
Outside of work. I enjoy bursting my bubble via immersive travel; I have traveled to over 55 countries on all 6 habitable continents mostly via long, low-grade backpacking trips involving lots of walking, hostels and trains. I love the outdoors and wilderness; I am an advanced open-water certified scuba diver, and have completed multiple month-long courses in mountaineering and climbing, as well as first aid certifications. I try my best to lead a trash-free life; since Jan 1, 2016, I rely primarily on reusable, compostable and recyclable materials, and personally allowed myself a budget of about one big garbage bag of landfill trash per year. In 2013, I completed an Ironman triathlon in Louisville, but had a bad accident in 2014, and after recovery in 2015, I used multiple long-distance bicycle rides in California and Zambia as a way to fight back. Yoga is my moving-meditation, with running, swimming and biking being my other weekly mind-cleansers.
Given the huge freedom awarded to academics and the scope for creativity in all spheres of an academic life, the success of the scientific enterprise depends on the deep involvement of its members outside of pure research, leading me to strongly believe in the holistic development and enthusiastic contributions of faculty members in all aspects of helping departments, universities and disciplines flourish.
Selected recent papers
Estimating means of bounded random variables by betting
I. Waudby-Smith, A. Ramdas arxiv
Dimension-agnostic inference
I. Kim, A. Ramdas arxiv
Admissible anytime-valid sequential inference must rely on nonnegative martingales
A. Ramdas, J. Ruf, M. Larsson, W. Koolen arxiv
False discovery rate control with e-values
R. Wang, A. Ramdas (JRSSB, major revision) arxiv
Testing exchangeability: fork-convexity, supermartingales, and e-processes
A. Ramdas, J. Ruf, M. Larsson, W. Koolen Intl J of Approximatte Reasoning, 2021 arxiv proc
On the power of conditional independence testing under model-X
E. Katsevich, A. Ramdas arxiv
Fast and powerful conditional randomization testing via distillation
M. Liu, E. Katsevich, L. Janson, A. Ramdas Biometrika, 2021 arxiv code
Time-uniform, nonparametric, nonasymptotic confidence sequences
S. Howard, A. Ramdas, J. Sekhon, J. McAuliffe Annals of Stat., 2021 arxiv proc code talk1 tutorial
Time-uniform Chernoff bounds via nonnegative supermartingales
S. Howard, A. Ramdas, J. Sekhon, J. McAuliffe Prob. Surveys, 2020 arxiv proc
Are sample means in multi-armed bandits positively or negatively biased?
J. Shin, A. Ramdas, A. Rinaldo NeurIPS, 2019 arxiv poster proc
A unified treatment of multiple testing with prior knowledge using the p-filter
A. Ramdas, R. F. Barber, M. Wainwright, M. Jordan Annals of Stat., 2019 arxiv code proc
Classification accuracy as a proxy for two sample testing
I. Kim*, A. Ramdas*, A. Singh, L. Wasserman Annals of Stat., 2021 arxiv proc
Predictive inference with the jackknife+
R. Barber, E. Candes, A. Ramdas, R. Tibshirani Annals of Stat., 2020 arxiv code
Simultaneous high-probability bounds on the FDP in structured, regression and online settings
E. Katsevich, A. Ramdas Annals of Stat., 2020 arxiv code proc
Biography
Aaditya Ramdas is an assistant professor at Carnegie Mellon University, in the Departments of Statistics and Machine Learning. He was one of the inaugural inductees of the COPSS Leadership Academy, and a recipient of the Bernoulli New Researcher Award. His work is supported by an NSF CAREER Award, an Adobe Faculty Research Award, an ARL Grant on Safe Reinforcement Learning, the Block Center Grant for Technology and Society, amongst several others.
Aaditya's main theoretical and methodological research interests include selective and simultaneous inference (interactive, structured, post-hoc control of false decision rates, etc), sequential uncertainty quantification (confidence sequences, always-valid p-values, bias in bandits, etc), and distribution-free black-box predictive inference (conformal prediction, calibration, etc). His areas of applied interest include neuroscience, genetics and voting.
He is one of the organizers of the amazing and diverse StatML Group at CMU.
Outside of work, here are some easy topics for conversation: travel/outdoors (hiking, scuba, etc.), trash-free living, completing the Ironman triathlon and long-distance bicycle rides.