The primary role of the nowcast environment is to collect weather data, execute algorithms for producing a combined thunderstorm forecast, and provide a graphical display tool for viewing the various datasets.
Data incorporated into the nowcast environment includes WSR-88D (NEXRAD) radar, satellite, sounding, and surface data.
The software applications in the nowcast environment include algorithms for identifying and tracking thunderstorm movement, identifying boundaries, wind retrieval from radar, as well as a fuzzy-logic engine which allows the user to combine the weighted outputs from the various algorithms to produce a single, combined forcast. Subsequent verification of generated forcasts are available both visually and statistically.
In realtime operations the nowcast environment initiates and maintains process control through an auto-restart mechanism. The auto-restarter provides the user with an automated, hands-free forecasting environment.
Many components of the nowcast environment also can be operated in archive mode. Archive mode allows the user to re-run algorithms with newly specified parameters and to view historical datasets.
The key hardware components of the Auto-Nowcast Environment are:
The key software components of the Auto-Nowcast Environment are:
The key data components of the Auto-Nowcast Environment are:
Other features include:
Project | Year | Sponsor | Operational Site Location |
RAPS | 1995 | Federal Aviation Administration | Northeastern Colorado |
Memphis | 1996 | Federal Aviation Administration | Memphis, Tennessee |
SCAN | 1997, 1998, 1999, 2000 | National Weather Service | Sterling, Virginia |
ATEC/WSMR | 1997, 1998, 1999, 2000, 2001, 2002, 2003 | U.S. Army | White Sands Missile Range, New Mexico |
ATEC/Aberdeen | 1998 | U.S. Army | Aberdeen Proving Ground, Virginia |
ATEC/RTTC | 1999, 2000 | U.S. Army | Redstone Technical Test Center, Alabama |
Summer Olympics 2000 | 1999, 2000 | World Weather Research Program | Sydney, Australia |
2002, 2003 | Federal Aviation Administration | Lincoln Labs at Massachusetts Institute of Technology |
Because the Auto–Nowcast Environment involves many complex algorithms, the power of many machines or hosts are combined to make up the entire environment. All machines in the Auto–Nowcast Environment are set up to operate under the "nowcast" user login. See your site administrator for the password for this login. Some of the Auto–Nowcast Environment machines are configured with auto–login at boot–up some are not, depending of the needs of the particular machine and the local area network configuration at the site. In an auto–login setup, booting the machine will automatically result in a login session as user "nowcast" and an will initiate an X–windows session. On machines which are not configured for auto–login at boot–up, the user must manually login as user "nowcast" and start X–windows using the following commands:
| login: | nowcast |
| password: | (see site administrator) |
| startx |
Once the X–windows session is started (either manually or via auto–login) the user will see a console window for system messages, an xterm for running UNIX commands, and a window manager tool bar. (look on reflect...)
The Auto–Nowcast Environment can be managed by any host through command line program execution; however, an X interface with pull–down menus is available on the control host. The control host can be determined by typing the command:
echo $CONTROL_HOST
The realtime Auto–Nowcast Environment is started from the control host either by typing the command:
niwot_startup
or via a pulldown menu on the $CONTROL_HOST:
It only takes a few minutes for all of the Auto–Nowcast Environment process to start up. Several graphical windows will appear on the control host at startup. Each of these windows and their role in the Auto–Nowcast Environment will be discussed in detail below. Because each of the algorithms within the Auto–Nowcast Environment requires different amounts of input data for their processing, the results of the algorithms will not begin to arrive until the Auto–Nowcast Environment has been up and running for a while. Some algorithms will produce data within six minutes, others will not produce data until there is significant weather.
The realtime Auto–Nowcast Environment is shutdown from any host by typing the command:
niwot_shutdown
or via a window pulldown menu on the $CONTROL_HOST:
Operations Guide (for downloadable pdf, click here)
Q. Will the radar data degrade the network performance of my LAN?
A. No. The Auto–Nowcast Environment exists on two separate local area networks. The radar data is broadcast on a dedicated LAN which does not interfere with the LAN on which your other machines reside.
Q. What should I do when I cannot login to one of the Auto–Nowcast Environemnt host machines?
A. If the host which contains the shared project space is down then it will not be possible to login to any other Nowcast hosts. This is because the .cshrc for all hosts are linked to the .cshrc in <shared project space host>:$PROJ_DIR/.cshrc.If you can't login to any nowcast host, try rebooting the host of the shared project space.
If the problem is isolated to one host, try rebooting that host.
Q. What should I do when a dataset is late or missing?
Q.How can I be sure that radar data is coming into a Auto-Nowcast host?
A.First, check the raw radar data directories directories to see when the last data files arrived. A second check can be run on the hosts which have the dual ethernet cards installed for receiving the radar data. This check requires root login privileges.
To find out whether a host has 2 ethernet connections (expected: eth0 and eth1), run the command:
ifconfig -a
If 2 ethernet connections are not listed, this host is not directly receiving the radar data feed so this check is not valid. If both eth0 and eth1 are listed, look for which one has "Bcast" set to xxx.xxx.199.xxx or xxx.xxx.197.xxx. This is the port which would be receiving radar data. To check that data is actually arriving on that port, run the command (for eth0, replace eth1 below with eth0):
tcpdump -i eth1 udp port 3280 If radar data is arriving, you will see a data stream similar to the following:
16:35:06.810338 128.117.199.255.3280 > 128.117.199.255.3280: udp 1450
16:35:06.810338 128.117.199.255.3280 > 128.117.199.255.3280: udp 998
16:35:06.820338 128.117.199.255.3280 > 128.117.199.255.3280: udp 1450
16:35:06.820338 128.117.199.255.3280 > 128.117.199.255.3280: udp 998
If radar data are not being broadcast on the ethernet port, you will need to check on communications upstream of the Auto-Nowcast Environment.
If you are receiving radar data via an LDM then you can type ldmadmin watch to see if bundles of radar data that are being received for a particular radar ID. The output of ldmadmin watch looks like this:
Oct 13 20:34:50 pqutil: 11144 20031013203446.553 NEXRD2 125000 L2-BZIP2/KIND/20031013203501/125/0
Oct 13 20:34:52 pqutil: 12277 20031013203448.979 NEXRD2 181020 L2-BZIP2/KLOT/20031013203139/181/20
Oct 13 20:35:01 pqutil: 12226 20031013203457.569 NEXRD2 181021 L2-BZIP2/KLOT/20031013203139/181/21
Oct 13 20:35:09 pqutil: 11593 20031013203506.282 NEXRD2 181022 L2-BZIP2/KLOT/20031013203139/181/22
Oct 13 20:35:10 pqutil: 11322 20031013203506.248 NEXRD2 125001 L2-BZIP2/KIND/20031013203501/125/1
Oct 13 20:35:14 pqutil: 8206 20031013203511.688 NEXRD2 181023 L2-BZIP2/KLOT/20031013203139/181/23
Q. What happens to the Auto-Nowcast Environment when a data feed goes down?
A. Because so many processes in the Auto–Nowcast Environment rely on the radar data feed, very few processes will continue to generate results if the radar feed goes down.
If other data feeds go down, some algorithms which rely on those data may output "missing data" datasets, while others will wait for the input data to become available before producing any output.
In general, the Auto–Nowcast Environment will continue to run, and when data become available again the algorithms will resume outputing meaningful results.
How much data is generated each day?
This will vary depending on the type and number of datasets at your installation.
Q. What should I do when one of the data partitions becomes full?
A. If the "/home" or cross–mounted shared data partition disk is 100% full, you should check that the events list is not saving so many cases that the data scrubber cannot free up sufficient disk space. If this is the case, the older cases should be archived and removed from $DATA_HOME/params/events.list.
If the "/" (a.k.a. "root") partition is 100% full, you must reboot the machine.
Q. What should I do when a process is restarting or missing?
A. Following are some actions to take for diagnosing restarting or missing processes:
To figure out the correct kill and start scripts for the process, examine the Host and Script specifications listed in the sections on Analysis Algorithms and The Project Directory.
NOTE: Be sure to kill and re–start the process on the correct host. The logical host is specified either in the start script or in the corresponding parameter file. The physical host is then determined using the UNIX echo command. For example: echo $TITAN_HOST
Q. What happens to the Auto–Nowcast Environment when a process in the procmap is restarting continuously or is missing?
A. The process will not be able to generate its expected output. This may also affect other "downstream" processes which need that process's output.
In this situation, it is important to diagnose the source of the problem. For information on trouble shooting a process which is failing, see the discussion above.
Q. What is the difference between Mode 1 and Mode 2 process restarts?
A. Mode 1 process restarts are due to a process missing from the procmap. This occurs if a process exits prematurely.
Mode 2 process restarts are due to a process not registering frequently enough with the procmap (check the Heartbeat time in the procmap window). In this case, the process is believed to be "hung" and is stopped and restarted by the auto–restarter.
Q. What does it mean when I get the message "No more processes" or "Cannot exec" or "Cannot fork" when I try to run a command?
A. This usually indicates that the host has run out of processes, i.e., the process table is full. In order to run the command you need to free up some process space. If you are running extra processes, which are not part of the Auto–Nowcast Environment, try to exit them. If that does not free up sufficient process space, try running the command exec top.You can start terminating processes until you can begin executing commands in the xterm. By examining the output of top you may also be able to determine which processes are filling up the process table and diagnose the problem causing the table to fill up.
Q. The Auto–restart log for the previous few days is empty. Is this a problem?
A. The auto–restarter only outputs to the log file when a process is restarted. An empty log file indicates that no processes were restarted, the system is up and fine.
Q. Can I rearrange the location of datasets or add new machines to the Auto-Nowcast Environment?
A. Yes. In order to relocate processes and datasets to a new machine the following must be edited:
Q. Can I get software upgrades?
A. When the Auto–Nowcast Environment is installed at a field site, a snapshot of the application software is stored on disk so that any software problems that arise can be debugged with a version of the software which is running in the field. In the meantime, these same applications undergo changes at NCAR as a part of the ongoing effort to improve the software.
Depending on the magnitude of the changes that have taken place in the software at NCAR, an upgrade for any one application may be a simple matter of taking a new snapshot and copying the executable to a field machine. However, upgrading an entire Auto–Nowcast Environment may require reconfiguring the Environment to incorporate enhancements to the operating system, networking, interprocess communication, and process control.
In order to maximize stability in the Auto–Nowcast Environment, we do not install full software upgrades during an operational season. A full upgrade and reconfiguration typically takes four weeks of NCAR staff time.
Thunderstorm Auto Nowcasting Training
Laboratory Exercise
