STAMPS@CMU and the NSF AI Planning Institute for Data-Driven Discovery in Physics jointly present:

Benefits of saying “I Don’t Know” when analyzing and modeling the climate system with ML

by Elizabeth Barnes (Department of Atmospheric Science, Colorado State University)

Online webinar December 3, 2021 at 1:30-2:30 PM ET.
For connection information, join our mailing list here.

The atmosphere is chaotic. This fundamental property of the climate system makes forecasting weather incredibly challenging: it’s impossible to expect weather models to ever provide perfect predictions of the Earth system beyond timescales of approximately 2 weeks. Instead, atmospheric scientists look for specific states of the climate system that lead to more predictable behaviour than others. Here, we demonstrate how neural networks can be used, not only to leverage these states to make skillful predictions, but moreover to identify the climatic conditions that lead to enhanced predictability. We introduce a novel loss function, termed “abstention loss”, that allows neural networks to identify forecasts of opportunity for regression and classification tasks. The abstention loss works by incorporating uncertainty in the network’s prediction to identify the more confident samples and abstain (say “I don’t know”) on the less confident samples. Once the more confident samples are identified, explainable AI (XAI) methods are then applied to explore the climate states that exhibit more predictable behavior.


Dr. Elizabeth (Libby) Barnes is an associate professor of Atmospheric Science at Colorado State University. She joined the CSU faculty in 2013 after obtaining dual B.S. degrees (Honors) in Physics and Mathematics from the University of Minnesota, obtaining her Ph.D. in Atmospheric Science from the University of Washington, and spending a year as a NOAA Climate & Global Change Fellow at the Lamont-Doherty Earth Observatory. Professor Barnes' research is largely focused on climate variability and change and the data analysis tools used to understand it. Topics of interest include earth system predictability, jet-stream dynamics, Arctic-midlatitude connections, subseasonal-to-decadal (S2D) prediction, and data science methods for earth system research (e.g. machine learning, causal discovery).

She teaches graduate courses on fundamental atmospheric dynamics and data science and statistical analysis methods. Professor Barnes is involved in a number of research community activities. In addition to being a lead of the US CLIVAR Working Group: Emerging Data Science Tools for Climate Variability and Predictability, a member of the National Academies’s Committee on Earth Science and Applications from Space, a funded member of the NSF AI Institute for Research on Trustworthy AI in Weather, Climate and Coastal Oceanography (AI2ES), and on the Steering Committee of the CSU Data Science Research Institute, she recently finished being the lead of the NOAA MAPP S2S Prediction Task Force (2016-2020).