The Cloud Hunter's Problem: An Automated Decision Algorithm to Improve the Productivity of Scientific Data Collection in Stochastic Environments

Abstract

A decision algorithm is presented that improves the productivity of data collection activities in stochastic environments. The algorithm was developed in the context of an aircraft field campaign organized to collect in situ data from boundary layer clouds. Required lead-times implied that aircraft deployments had to be scheduled in advance, based on imperfect forecasts regarding the presence of conditions meeting specified requirements. Given an overall cap on the number of flights, daily Fly/No-fly decisions were taken traditionally using a discussion-intensive process involving heuristic analysis of weather forecasts by a group of skilled human investigators. An alternative automated decision process uses self-organizing maps to convert weather forecasts into quantified probabilities of suitable conditions, together with a dynamic programming procedure to compute the opportunity costs of using up scarce flights from the limited budget. Applied to conditions prevailing during the 2009 RACORO campaign of the U.S. Department of Energy’s Atmospheric Radiation Measurement Program, the algorithm shows a 21% increase in data yield and a 66% improvement in skill over the heuristic decision process used traditionally. The algorithmic approach promises to free up investigators’ cognitive resources, reduce stress on flight crews, and increase productivity in a range of data collection applications.

Keywords: weather, risk management, dynamic programming, weather forecasting, decision analysis, decision making under uncertainty

JEL Classification: C61, D81, D83