Model Evaluation Tools Version 5.1 Released

Moving Closer to Matching Forecasts to Observations
Monday, March 7, 2016 - 10am
MODE Time Domain (MTD) tool applied to hourly snowfall accumulations from the High-Resolution Rapid Refresh Time-Lagged Ensemble (HRRR-TLE).

In today’s highly technological world, quality control is everything. Whether it’s the latest smart phone, driverless car, or even weather forecast model, performance testing is the unsung hero in building a high-quality product.

The Air Force 557th Weather Wing has been using MET to evaluate cloud and precipitation deterministic  and stochastic model forecasts.  

With the delivery of MET version 5.1, the process got much easier with the new regridding options  to place data on the same grids, eliminating lots of regridding code.

We are very pleased with the tool and NCAR's support of the tool.

-Bob Craig
557th Model Analysis Team

NCAR’s state-of-the-art verification system, METv5.1 (Model Evaluation Tools Version 5.1), was released in October, and it offers a multitude of enhancements to its already extensive capability for research and operational meteorologists and climatologists. The additions can be grouped into new tools, enhanced controls over preexisting capabilities, and new statistics.  It may be fair to say there is something new for everyone.

Statistical verification of forecasts is a critical component to forecast development and improvement. Verification also benefits forecasters and end users by supplying them with objective data about the quality or accuracy of the forecasts, which can feed into decision processes. NCAR is in a unique position to serve two very different science communities in verification. Its Developmental Testbed Center represents a collaboration between research (NCAR and universities) and operational (National Oceanic and Atmospheric Administration, [NOAA], and Air Force) Numerical Weather Prediction (NWP) communities. There are nearly 3,000 registered users of the MET package from over 125 countries.

The newest version is chock full of improvements. “We worked very hard and are very excited about the new features in v5.1,” said Tara Jensen, head of the MET team. In direct response to user requests and needs, and guidance from NOAA’s Science Advisory Board, this latest version includes:

New Tools

MODE Time Domain (MTD) tool applied to hourly snowfall accumulations from the High-Resolution Rapid Refresh Time-Lagged Ensemble (HRRR-TLE).
MODE Time Domain (MTD) tool applied to hourly snowfall accumulations from the High-Resolution Rapid Refresh Time-Lagged Ensemble (HRRR-TLE).
  • Automated regridding;
  • Method for Object Based Evaluation (MODE) Time Domain (to show the evolution of storms systems, for example);
  • Tools to read the Gridpoint Statistical Interpolation binary diagnostic files.

Enhanced Controls

  • Combining Gen_Poly_Mask and Gen_Circle_Mask into a more generalized tool Gen_Vx_Mask;
  • Computing conditional continuous statistics;
  • Ability to define a rapid intensification/rapid weakening event for a tropical cyclone in a more generic way with TC-Stat;
  • Finer controls of thresholding.

New Statistics

  • A “climo mean” specification to the Grid-Stat, Point-Stat, and Ensemble-Stat configuration files for computation of climatology-based statistics.

NCAR’s Model Evaluation Team has a proven track record of providing the broad Numerical Weather Prediction community with state-of-the-art verification package. It will continue to incorporate newly developed and advanced verification methodologies, including new methods for diagnostic and spatial verification, as well as existing systems for verification of NWP forecasts. 

What’s Next?

While MET v5.1 represents a wealth of improvements, the MET team is not resting on its laurels; a refined version is in the works and will be released later this year.

With continued funding, the team will focus on verification unification with NOAA packages (all using standard measures), tool and database development, user interface and visualization, cloud verification, and training and user support.