Multiscale and Probabilistic Prediction

Assessing merits of innovative NWP techniques for multiscale deterministic and probabilistic model guidance
Community Leveraged Unified Ensemble (CLUE) dataset: Two 9-member ensemble subsets from the CLUE 2016 data were examined to investigate whether there was an advantage to using multiple microphysics/planetary boundary layer (PBL) parameterizations (multip) compared to one common physics suite (singlp), both with perturbed initial/boundary conditions. Fractions Skill Score (FSS) was one verification metrics used to assess how forecast skill varies with spatial scale.
Community Leveraged Unified Ensemble (CLUE) dataset: Two 9-member ensemble subsets from the CLUE 2016 data were examined to investigate whether there was an advantage to using multiple microphysics/planetary boundary layer (PBL) parameterizations (multip) compared to one common physics suite (singlp), both with perturbed initial/boundary conditions. Fractions Skill Score (FSS) was one verification metrics used to assess how forecast skill varies with spatial scale.

Multiscale and probabilistic prediction activities in the Joint Numerical Testbed program are focused primarily within the Developmental Testbed Center (DTC). The DTC is a distributed facility with components in the JNT at NCAR's Research Applications Laboratory (RAL), and the Global Systems Division (GSD) of NOAA's Earth System Research Laboratory (ESRL). It facilitates the transfer of research results into operations and provides the research community with an easily accessible state-of-the-art Numerical Weather Prediction (NWP) system for research. One of the DTC's focal points is regional forecasting systems, with a goal of accelerating the rate at which new technology is infused into operational weather forecasting.

NWP systems can generally be configured to suit a broad spectrum of weather regimes. The JNT conducts a variety of testing and evaluation (T&E) activities ranging from diagnostic case studies to extensive, seasonal or year-long experiments. While efforts continue related to evaluating deterministic model output, as operational centers move towards ensemble-based forecasting the JNT is becoming more involved in the T&E of probabilistic information. Emerging multi-scale global models are also an area of recent expansion and will continue to be a strong focus topic into the future.

The design for each T&E activity is created in consultation with the model developers, relevant area experts, and verification experts. Evaluation of these retrospective cases includes traditional verification approaches, as well as advanced verification techniques using the Model Evaluation Tools (MET). All verification statistics include a statistical significance (SS) and practical significance (PS) assessment when appropriate.

By conducting carefully controlled testing, including the generation of objective verification statistics, the JNT is able to provide the operational community with guidance for selecting new NWP technologies with potential value for operational implementation.  In addition, the extensive testing provides the research community with baselines against which the impacts of new techniques can be evaluated.  The statistical results also provide guidance to researchers in selecting model configurations to use for their projects.

In addition to extensive T&E through the DTC, the JNT is assisting the user community (especially students) with efficiently running NWP components and making connections with future collaborators through the establishment of software containers. Many times, the biggest hurdle when running a new software system is getting it set up and compiled on the intended computer platform. Building complex systems that require a number of external libraries can be a prohibitive hurdle for users to overcome. In order to reduce some of this difficulty, software containers are being exploited to ship complete software systems to users. The containers have everything that is needed to run a software application, including the necessary operating system components (tools and libraries) and compiled executable (or code and compiler), thus, allowing for the user to quickly produce output without being delayed by technical issues. DTC staff members created UPP and MET containers to supplement those containers that had already been established by others in the community (including, WPS, WRF, and NCL) so that an end-to-end NWP system can be fully employed through containers.

Representative Projects

  • DTC Global Model Test Bed (GMTB) Providing tools to facilitate the new physical parameterization testing for global NWP Models.
  • DTC Regional Ensemble Assessing the merits of new and innovative ensemble modeling techniques.
  • Numerical Weather Prediction (NWP) Containers These software containers have been established for community use to quickly spin up an NWP forecast system that can then be post-processed and verified.
  • NOAA Testbeds A collaborative partnership with an overarching goal to accelerate improvement of high-impact weather forecasting capability that significantly affect the lives and property of U.S. inhabitants.

Search through all publications in NCAR's OpenSky Library.

Harrold, M., H. Jiang and J. Wolff, 2013: Investigating Seasonal and Regional Differences of the WRF-ARW HIRES Window Physics Suite. 13th WRF Users Workshop, Boulder, CO, June 24-27, 2013.

Jiang, H., M. Harrold and J. Wolff, 2013: Investigating the impact of surface drag parameterization schemes available in WRF on surface winds. 13th WRF Users Workshop, Boulder, CO, June 24-27, 2013.

Wolff, J.K., P. Jimenez, J. Dudhia, G. Lackmann, K. Mahoney and M. Harrold, 2013: Demonstrating the utility of the Mesoscale Model Evaluation Testbed (MMET) in a research environment. 13th WRF Users Workshop, Boulder, CO, June 24-27, 2013.

Wolff, J. K. and M. Harrold, 2013: Tracking WRF performance: How do the three most recent versions compare? 13th WRF Users Workshop, Boulder, CO, June 24-27, 2013.

Xu, Mei, J. Wolff and M. Harrold, 2013: Inter-Comparison of AFWA Operational Configurations Using WRFv3.3.1 and WRFv3.4. 13th WRF Users Workshop, Boulder, CO, June 24-27, 2013.

Harrold, M., J. Wolff, T. Jensen, H. Jiang, C. Phillips. T. Slovacek, G. Lackmann, J. Dudhia, P. Jimenez, 2013. The Mesoscale Model Evaluation Testbed (MMET): Assisting with the transition of promising NWP techniques from research to operations. NOAA Testbeds Workshop, College Park, MD, April 2-3, 2013.

Du, J., G. DiMego, B. Zhou, D. Jovic, B. Ferrier, M. Pyle, B. Yang, G. Manikin, J. Zhu, J. Wolff and B. J. Etherton, 2013: Newly Upgraded 16km Short-Range Ensemble Forecast (SREF) System at NCEP. Special Symposium on Advancing Weather and Climate Forecasts: Innovative Techniques and Applications, Austin, TX, January 7-10, 2013.

Fowler, T. L., B. G. Brown, J. K. Wolff and L. B. Nance, 2013: Meaningful evaluation strategies for numerical model forecasts. Third Conference on Transition of Research to Operations, Austin, TX, January 7-10, 2013.

Wolff, J. K., M. Harrold, T. Jensen, H. Jiang, T. Slovacek, G. M. Lackmann, J. Dudhia, P. Jimenez, B. A. Colle and C. F. Mass, 2013: Utilizing the Mesoscale Model Evaluation Testbed (MMET) to Transition Promising New Research Techniques from Research to Operations. Special Symposium on Advancing Weather and Climate Forecasts: Innovative Techniques and Applications, Austin, TX, January 7-10, 2013.

Wolff, Jamie K., Brad S. Ferrier, Clifford F. Mass, 2012: Establishing Closer Collaboration to Improve Model Physics for Short-Range Forecasts. Bull. Amer. Meteor. Soc., 93, ES51-ES53. doi: http://dx.doi.org/10.1175/BAMS-D-11-00248.1

Harrold, M., J. Wolff, J. Halley Gotway, P. Oldenburg, Z. Trabold, 2012: Inter-Comparison of the WRFv3.3.1 AFWA Operational and RRTMG-Replacement Configurations. 13th WRF Users Workshop, Boulder, CO, June 26-29, 2012.

Wolff, J. K., L. Nance, B. S. Ferrier, C. F. Mass, B. G. Brown, Y-H Kuo, 2012: Transitioning Promising New Mesoscale Innovations from Research to Operations: Defining a Process to Bridge the "Valley of Death." 13th WRF Users Workshop, Boulder, CO, June 26-29, 2012.

Wolff, J. K., L. Nance, B. S. Ferrier, C. F. Mass, B. G. Brown, Y-H Kuo, 2012: Bridging the valley of death: Defining a process for transitioning promising new mesoscale innovations from research to operations. 25th Conference on Weather Analysis and Forecasting/21st Conference on Numerical Weather Prediction, Montreal, Canada, May 29 - June 1, 2012.

Harrold, M., J. Wolff, J. Halley Gotway, P. Oldenburg, Z. Trabold, 2012: Inter-Comparison of the AFWA Operational and RRTMG-Replacement Configurations using WRFv3.3.1. 25th Conference on Weather Analysis and Forecasting/21st Conference on Numerical Weather Prediction, Montreal, Canada, May 29 - June 1, 2012.

Wolff, J. K., 2011: DTC: Serving to bridge the research and operational NWP communities. NWP Workshop on Model Physics with and Emphasis on Short-Range Prediction, Camp Springs, MD, July 26-28, 2011.

Harrold, M., J. Wolff, Z. Trabold and L. Nance, 2011: Testing and Evaluation of WRF Reference Configurations. 12th WRF Users Workshop, 20-24 June 2011, Boulder, CO.

Wolff, J. K., B. G. Brown, J. Halley Gotway, M. Harrold, Z. Trabold, L. Nance and P. Oldenburg, 2011: Quantitative Precipitation Forecast (QPF) Verification Comparison Between the Global Forecast System (GFS) and North American Mesoscale (NAM) Operational Models. 12th WRF Users Workshop, 20-24 June 2011, Boulder, CO.

Wolff, J. K., B. Brown, J. Halley Gotway, M. Harrold, Z. Trabold, L. Nance and P. Oldenburg, 2011: Quantitative precipitation forecast verification comparison between the Global Forecast System and North American Mesoscale operational models. Preprints, 20th Conference on Weather and Forecasting/20th Conference on Numerical Weather Prediction, Seattle, WA, January 24-27, 2011.

Wolff, J. K., L. Nance, J. Halley Gotway, P. Oldenburg, M Harrold and Z. Trabold, 2011: WRF QNSE Test and Evaluation. 20th Conference on Weather and Forecasting/20th Conference on Numerical Weather Prediction, Seattle, WA, January 24-27, 2011.

Wolff, J. K., B. Brown, J. Halley Gotway, M. Harrold, Z. Trabold, L. Nance and P. Oldenburg, 2010: QPF verification comparison between the GFS and NAM operational models. 3rd WMO International Conference on Quantitative Precipitation Estimation and Quantitative Precipitation Forecasting and Hydrology. 18-22 October 2010, Nanjing, China.

Wolff, J. K., L. Nance, J. Halley Gotway and P. Oldenburg, 2010: WRFv3.1.1+ QNSE Test and Evaluation. 11th WRF Users Workshop, 21-25 June 2010, Boulder, CO.

Wolff, J. K., L. Nance, L. R. Bernardet and B. G. Brown, 2010: Implementation of WRF Reference Configurations. 11th WRF Users Workshop, Boulder, CO, June 21-25, 2010.

Wolff, J. K., L. Nance, L. R. Bernardet and B. G. Brown, 2009: WRF Reference Configurations - Update. 10th WRF Users Workshop, Boulder, CO, June 23-26, 2009.

Bernardet, L. R., J. Wolff, L. Nance, A. Loughe, B. Weatherhead, E. Gilleland and B. G. Brown, 2009: Comparison between ARW and NMM objective forecast verification scores. 23rd Conference on Weather Analysis and Forecasting/19th Conference on Numerical Weather Prediction. 1-5 June 2009, Omaha, NE, American Meteorological Society.

Wolff, J. K., L. Nance, L. R. Bernardet and B. G. Brown, 2009: WRF Reference Configurations. 23rd Conference on Weather Analysis and Forecasting/19th Conference on Numerical Weather Prediction. 1-5 June 2009, Omaha, NE, American Meteorological Society.

Bernardet, L., J. Wolff, L. Nance and E. Gilleland, 2008: Objective verification results from forecasts generated with the ARW and NMM dynamic cores of the WRF model. 9th WRF Users Workshop, 23-27 June 2008, Boulder, CO.

Wolff, J. K., B. Weekley, L. Nance, L. Bernardet and B. Brown, 2008: WRF Reference Configurations - Concept and Plans.9th WRF Users Workshop, 23-27 June 2008, Boulder, CO.

Bernardet, L., L. Nance, M. Demitras, S. Koch, T. Fowler, A. Loughe, J. L. Mahoney, H. Chuang, M. Pyle and R. Gall, 2008: The Developmental Testbed Center and its winter forecasting experiment. Bull. Amer. Meteor. Soc. 89, 611-627.

Brown, J. M., S. Benjamin, T. G. Smirnova, G. A. Grell, L. R. Bernardet, L. B. Nance, R. S. Collander and C. W. Harrop, 2007: Rapid Refresh Core Test: aspects of WRF-ARW and WRF-NMM performance relevant to the Rapid Refresh application. 22nd Weather Analysis and Forecasting Conf., Park City, Utah, AMS.

Bernardet, L.R., L. Nance, H.-Y. Chuang, A. Loughe, M. Demirtas, S. Koch and R. Gall, 2005: The Developmental Testbed Center Winter Forecasting Experiment. 21st Conf. on Weather Analysis and Forecasting, Washington, D.C., Amer. Meteor. Soc., CD-ROM, 7.1.

Demirtas, M., L.B. Nance, L.R. Bernardet, Y. Lin, A. Loughe, R. Gall and S. E. Koch, 2005: Quantitative precipitation forecasts (QPF) verification of DWFE. 17th Conf. on Numerical Weather Prediction, Washington, D.C., Amer. Meteor. Soc., CD-ROM, 7.2.

Koch, S.E., R. Gall, G. DiMego, E. Szoke, J. Waldstreicher, P. Manousos, B. Meisner, N. Seaman, M. Jackson, R. Graham, A. Edman and D. Nietfeld, 2005: Lessons learned from the DTC Winter Forecast Experiment. 17th Conf. on Numerical Weather Prediction, Washington, D.C., Amer. Meteor. Soc., CD-ROM, 7.6.

Szoke, E., S. Koch and D. Novak, 2005: An examination of the performance of two high-resolution numerical models for forecasting extended snow bands during the DTC Winter Forecast Experiment. 21st Conf. on Weather Analysis and Forecasting, Washington, D.C., Amer. Meteor. Soc., CD-ROM, 7.3.

Bernardet, L., P. Bogenschutz, J. Snook, A. Loughe, 2005. WRF Forecasts over the Southeast United States: Does a larger domain lead to better results? WRF/MMM5 Users Workshop, Boulder, CO.

Nance, L., L. Bernardet, H. Chuang, G. DiMego, M. Demirtas, R. Gall, S. Koch, Y. Lin, A. Loughe, J. Mahoney and M. Pyle, 2005: The WRF Developmental Testbed Center: A status report. WRF/MM5 Users Workshop, Boulder, CO, Amer. Meteor. Soc., CD-ROM, 2.8.

Bernardet, L.R., L. Nance, H.-Y. Chuang, A. Loughe and S.E. Koch, 2004: Verification statistics for the NCEP WRF pre-implementation test. Part 1: Deterministic verification of ensemble members. First Joint WRF/MM5 Users Workshop, Boulder, CO, 229-232.

Davis, C., L. Nance, L. Bernardet, M. Pyle and H. Chuang, 2004: WRF forecasts of recent significant weather events: A comparison of ARW and NMM cores. First Joint WRF/MM5 Users Workshop, Boulder, CO, 243-246.

Seaman, N., R. Gall, L. Nance, S. Koch, L. Bernardet, G. DiMego, J. Powers and F. Olson, 2004: The WRF process: Streamlining the transition of new science from research into operations. First Joint WRF/MM5 Users Workshop, Boulder, CO, 173-17.

Multiscale and Probabilistic Prediction