Benefits assessment. Red dart arrow hitting in the target center of dartboard Target hit in the center.

Benefits Assessment

Science Serving Society requires in-depth evaluation of the weather impact guidance accuracy as well as the actions undertaken based on utilizing that guidance

Assessing the benefits of investments in improved decision making to minimize weather impacts on flight operations are key. The return on investment will have to be assessed against a baseline of current operational decisions. Benefits can be achieved in various forms, such as improved confidence in or timing of a decision, reduced delays, flight time or fuel burn, smoother rides, less airspace congestion, reduced airborne holding time or fewer diversions, less injuries of passengers and crew, etc. Each stakeholder may have a different expectation on what they would like to achieve based on their investment. Moreover, the evaluation should look at the weather prediction accuracy, the accuracy of the translation of weather into operational impacts, and ultimately also the effectiveness of the decision making.

Research and Development Areas

Weather Prediction Accuracy

Evaluating the quality and accuracy of a weather prediction is fundamental to, yet only part of a comprehensive benefits assessment. The evaluation needs to be cognizant of what aspects of weather yield operational impacts and pay particular attention to those weather aspects, including the timing, duration, location, intensity, and organization, etc. thereof. NCAR has developed novel assessment techniques that can provide specific insights to weather prediction accuracy.

Impact Prediction Accuracy

The weather impact prediction methodology and decision support display systems have to be tuned to the needs of different users—for example, a line of severe convective storms predicted in the wrong place may be perceived as a bad forecast from a water resources manager (e.g., dam operator), yet the same forecast might be quite good for an en-route air traffic control manager. Moreover, an improved weather forecast may not necessarily yield a better impact prediction. The evaluation needs to carefully examine the translation of weather into operational impacts based on a stakeholder’s use case and ideally capture the associate uncertainties. Understanding the baseline for this translation is essential for identifying improvements.

  • Novel impact translation methodologies

Decision Making Effectiveness

Today, most decisions are still made by humans. Thus, the evaluation of effectiveness of the decision-making process needs to capture the human behavior on both the decision making as well as the implementation side. This assessment is rather involved and has to be carefully planned in order to yield insights. Again, establishing a baseline of the decision-making process prior to the changes is essential.

Resources

Contact

Matthias Steiner

Director, Aviation Applications Program

email

Arnaud Dumont

Deputy Director Engineering, Aviation Applications Program

email

James Pinto

Deputy Director Science, Aviation Applications Program

email