Abstract:
Key global sustainability challenges, ranging from ensuring food and energy security to managing disaster response, critically depend on our ability to accurately forecast weather and project climate. While current approaches are limited by our physical understanding of the atmosphere, improvements in sensory capabilities and large-scale machine learning (ML) present immense opportunities for designing alternative solutions. In this talk, I will discuss key design principles for developing foundation models for atmospheric sciences. Unlike language and vision, scientific domains present unique challenges due to the significant heterogeneity of available datasets. Moreover, downstream tasks in atmospheric sciences require generalizing across a broad range of variables and spatiotemporal resolutions. To address these challenges, I will demonstrate novel strategies for data engineering, model design, spatiotemporal optimization, and cross-modal finetuning aimed at scalable learning of atmospheric foundation models. Our resulting models exhibit exceptional skill, speed, and adaptability across a range of predictive tasks in weather and climate science. Finally, I will summarize the broader implications of this research for scientific ML and growing external efforts to adapt our models for diverse sustainable development goals.