On Tuesday, Patrick Marsh wanted a distraction from his dissertation and embarked on an idle investigation of tornado warnings and impacted areas (my thoughts on what “impact” means are below). Using some very rough approximations, he calculated the percentage of warned persons who are impacted by a tornado. Even under the most generous set of assumptions, the verification by population is generally below 20%. It’s worth noting that 2011 (the most recent year that official tornado data is available) was the best year in the analysis, but there is no indication of a general improvement trend.
Despite some of the problems I’ve previously noted in the polygon warning system, it’s still better than warning entire counties. Still there’s a lot of room to improve the false alarm rate. Much of the population-based false alarm comes from warnings that have no tornado at all. The rest either comes from too-large warnings or not-small-enough warnings (“not-small-enough” warnings are small enough to be justifiable, but still larger than absolutely necessary).
It’s not always easy to shrink warnings. Only the supercell storms relatively close to a radar site seem suitable. In those cases, it’s possible to make the warning only a few miles wide, or the width of the mesocyclone with uncertainty added as you go downstream. This would minimize the area under the warning, but it got me wondering: would that be too small?
At the scale of a mile or two, how do you explain the warned area to the public? Storm-based warnings are already difficult to communicate quickly, and microwarnings would only compound the problem. Even in Lafayette, the 10th largest city in Indiana, the covered area might look something like:
...TEAL ROAD BETWEEN 4TH STREET AND 26TH ST... ...KOSSUTH ST BETWEEN 9TH STREET AND SAGAMORE PARKWAY... ...SOUTH ST BETWEEN FIVE POINTS AND FARABEE DR... ...18TH ST BETWEEN BECK LN AND FERRY ST...
And so on. Or maybe it would use neighborhoods and landmarks instead:
...LAFAYETTE COUNTRY CLUB... ...HIGHLAND PARK... ...JEFFERSON HIGH SCHOOL... ...FIVE POINTS... ...WALLACE TRIANGLE... ...COLUMBIAN PARK...
Either way, it’s much more complicated than a simple “LAFAYETTE”. Yes, it’s more detailed, does that help? First, it takes much longer to read the text. Secondly, can you count on people, especially those who are new to the area, to know the streets, neighborhoods, and landmarks well enough to quickly figure out if they’re affected or not? I suspect the answer is “no”. Perhaps some day someone with the time, energy, and funding can look at this.
Sidebar: What does it mean to be “affected” by a tornado?
When Patrick commented on Twitter about his post from Tuesday, I remarked that the results depend on how “affected” is defined. His analysis was based on population, but that doesn’t necessarily convey all impacts. If my office is wiped out by a tornado but my house is untouched, I am still affected. You can expand this out even further and incorporate businesses that saw decreased revenue as a result of a tornado, even if they were not directly hit. Businesses that see increased revenue (e.g. home improvement stores) might also be included, even though the effect is a positive one. The broader (and, I would argue, more accurately) we define being affected, the more difficult it becomes to get accurate data.
I agree with your discussion on “affected”. It’s definitely a tricky subject and one that I hope to explore. One possibility is to try and use some measure of the number of structures in the path. Now…to find such a dataset… =)
The smaller the warning area, the shorter the valid life of the warning.
Even in a tiny area, tornado damage is never sure, only probable, before it occurs.
So it may be a hopeless cause.
An actual improvement has been the availability of real-time radar, especially on portable devices.
Looking out the window is good, too.
Wally, you raise a very good point. Reducing the false alarm rate to zero is a bad goal, simply because it comes at the cost of reducing the probability of detection and the lead time. False alarm rate hovers around 75%, not including people in verified warnings who are not impacted. Clearly there is room for improvement.
I’m not sure the availability of radar to mobile device users is relevant to the issue of polygon size, although with location awareness, it does make it easier for the public to see if they are inside a warning.
I use a portable device to watch radar if I head to the basement, but maybe it’s just me who does that…
Wally, you’re certainly not the only one. I doubt, though, that a high percentage of the public 1) looks at radar while taking shelter and 2) knows enough to interpret it correctly.
The NWS I think had a lower false alarm rate in the past since they tended to only warn from ground observations.
Aren’t most of the false alarms these days based on radar-indicated rotation?
The policy must be, better to have false alarms than fail to give one.
When I can see a radar image, polygon size becomes less relevant, but maybe I have more confidence from knowing how tornadoes relate to rain patterns.
By the way, the depiction of wind direction on Doppler images has never been good. That could be a user interface improvement that needs to happen.
“The NWS I think had a lower false alarm rate in the past since they tended to only warn from ground observations.”
The stats I could find quickly show a stable false alarm rate from 1986 through 2006. This includes several years before and after Doppler radars were installed at NWS sites.
“Aren’t most of the false alarms these days based on radar-indicated rotation?
The policy must be, better to have false alarms than fail to give one.”
That’s fair to say. There are three key statistics in warnings: false alarm rate (FAR), probability of detection (POD), and lead time. These are all interrelated. In order to reduce the FAR, the POD and lead time would also reduce without a significant improvement in science or technology. The public, and thus the NWS, favors a high POD and a high lead time, so the FAR suffers by policy.
“By the way, the depiction of wind direction on Doppler images has never been good. That could be a user interface improvement that needs to happen.”
Agreed. That’s more of a technology limitation than a presentation limitation. Radars can only detect motion along the radar beam. Forecasters have to set the mean storm motion in the radar software, from which velocity images can be generated. It would take several overlapping radars to build better wind field images.
Thanks for the stats. I don’t think the Doppler-indicated warnings ramped up until some years after the Dopplers were installed.
Lafayette has 3 or 4 radars overlapping, so wind direction determination might be easy here, but not closer to the radar sites.
The elevation of the wind readings far from the radars might hurt, though.
Wally, you’re right that Lafayette has four radars that can cover the area. Theoretically, it would be possible to take the data from the four sites and combine them into a better wind field. However, they don’t all scan the same area at the same time, and they all scan Lafayette at different elevations above ground, so even then, the resulting image would often be meaningless.
It would be interesting to try to assemble such an image, though, just to see what it looks like. I’ll add it to my list of projects that I’ll probably never get to. :/
Two things would dramatically improve the quality of the velocity images. The first is smaller “bins” (i.e. improved resolution) and the second is using frequency shift instead of phase shift to measure velocity. Neither of those are cheap.
Perhaps one radar site could do it, using a 3-D simulation technique like one used in some animated GIFs
(http://www.bing.com/images/search?q=animated+GIF+3-d&id=DA2A82AFD85D9DE40C39E4924371684869EF8DD3&FORM=IQFRBA#view=detail&id=B9D7AA247A847B3C4263CD1ECDC05A813E0819BE&selectedIndex=245).
Say a storm is moving, and within it a possible vortex.
The wind direction is going to be more steady outside the vortex than within.
Lay a point grid over the radar coverage area.
Track changes in wind speed and direction over time intervals at each point.
The group of points with the highest deltas might relate somehow to the position and intensity of a vortex.
Perhaps a tornado could interpolated?
Nice post. I learn something new and challenging on sites I stumbleupon everyday.
It’s always exciting to read through content from other writers
and practice something from other websites.