Figure 1. Fractions Skill Score (FSS) of hourly-accumulated rainfall for the thresholds of 1mm (left) and 2.5mm (right) over the WRF 3km domain.  The radius of influence used in the FSS computation is 10km. Skills from 3 experiments (explained in the text) are compared.

The model-based QPF effort had three focus areas in FY16. First was the evaluation and verification of the real-time results from the 2015 STEP Hydromet Experiment; second was the continued development of convective-scale data assimilation using WRFDA; and third was the real-time demonstration of data assimilation and QPF systems along with other STEP-Hydromet components during the summer of 2016.

Figure 2. Scattered plot of HRRR vs RADAR FSSs for the hourly precipitation threshold of 1mm (left) and 5mm (right).

The evaluation and verification of QPF was also performed over the entire 3km model domain (see Figure 3 for the domain size) and for the nowcast domain (Figure 3). The verification for the whole 3km domain was done for eight convective days in 2015 summer by comparing three WRF 0-12h QPF systems/configurations against MRMS gauge-corrected precipitation analysis.

Figure 3. WRF hourly precipitation forecasts (t=5h) from the experiments HRRR, CYCLE, and RADAR, valid at 2015081211 UTC and initialized at 2015081206 UTC. MRMS precipitation analysis is shown in the upper-left panel for verification.

The Fraction Skill Scores for 1mm and 2.5mm are shown in Figure 1, each compares the skills for the following QPF runs:

• CYCLE: initialized by WRFDA 3DVar analysis with continuous 3-hourly update cycles,
• assimilating only conventional observations;
• RADAR: partial cycled hourly radar data assimilation with CYCLE as the first guess at the multiples of 3 hour;
• HRRR: operational HRRR mapped to the same domain as the other runs; 

The results in Figure 1 indicate that the runs with radar data assimilation (HRRR and RADAR) improve the skill over CYCLE for the entire 12 hour forecast period. WRFDA 3DVar-based radar data assimilation run RADAR has higher skill than HRRR at the most forecast hours. Figure 2 compares the skills between HRRR and RADAR for each of the eight cases. HRRR has superior skill only for two of the eight cases. These two cases were chosen for in-depth study to find the issues in the WRFDA data assimilation scheme for future improvement. Figure 3 compares the three 5h forecasts from HRRR, CYCLE, and RADAR against the MRMS analysis for one of the failure cases. It is shown that the precipitation forecasts both by CYCLE and RADAR is contaminated by the false prediction near the corner of Colorado, New Mexico, and Oklahoma, resulting in lower skill score than that of HRRR. Several research efforts are being made to tackle the issues discovered from our evaluation of the 2015 QPF.

Figure 4. Comparison of the 3DVAR, 3DVAR_DA, and GFS models in 2014. 

The evaluation and verification of the performance of WRF 3DVAR forecasts with and without data assimilation (DA) and the HRRR model was also conducted for a set of days from June, July and August days in 2015 over the smaller Front Range nowcast domain. These results have been compared to 2014 QPF verification statistics for the same models.  Two verification regions were specified in this study: one domain within 60km of the KFTG Denver radar, the second centered over the eastern Denver region, where a high density of surface rain gauges were used as truth.  The verification over the KFTG region used the MRMS gauge-corrected precipitation analysis as the truth dataset. The 3DVar with DA has shown some apparent improvement over 3DVAR without DA from 2014 to 2015. This is cautiously suggested by comparing of the performance in Figure 4 with Figure 5.  Note that in these Figures, the experiment 3DVAR are the same as CYCLE in Figure 3 and DA the same as RADAR. 

Figure 5. Comparison of 3DVAR, 3DVAR_DA, and HRRR for 2015.

Figure 5 shows the performance of the models versus forecast lead time for each model. After a 2 hr lead time, the HRRR performance was significantly inferior to the 3DVAR methods. 

The effort to improve convective-scale data assimilation in FY15 focused on the improvement of water vapor analysis via a more objectively based cloud analysis, the assimilation of “no rain” observations from radar, and surface data assimilation. The WRFDA 4DVar radar data assimilation system was also actively studied and developed with leveraged external fund from the Central Weather Bureau of Taiwan.  

Figure 6. Correlation as a function of forecast lead time for 3DVAR_DA with rain gauge-measured rainfall for the average hourly rainfall accumulation over Denver urban area. Performance for 2014 and 2015 are shown. 

During the STEP Hydromet Experiment conducted in the summer of 2016, the QPF systems were run on the same 3km domain (Figure 3) as in 2015. Instead of running one deterministic forecast with radar data assimilation, 15 ensemble runs were conducted in real time by varying physics and large-scale analysis background (for cold start initial conditions and boundary conditions). Results of the ensemble forecasts are being evaluated.