Talking Michigan Transportation

New research yields surprises on traffic safety messages

April 28, 2022 Michigan Department of Transportation Season 4 Episode 102
Talking Michigan Transportation
New research yields surprises on traffic safety messages
Show Notes Transcript

On this week’s Talking Michigan Transportation podcast, conversations with two academics studying the effect of safety messages on driver behavior. 

First, Joshua Madsen, a professor of accounting and behavioral economics at the University of Minnesota, talks about a research report he co-authored — and highlighted in the Journal Science — that examined whether highway signs displaying traffic deaths reduce crashes. 

In the second segment, Jerry Ullman, a senior research engineer at the Texas A&M Transportation Institute, talks about a perspective he wrote to accompany the story in Science. 

Madsen explains how he first encountered the messages while driving in Illinois and was struck by the starkness of the numbers. He wondered about context and how the numbers were derived. 

As outlined in the story: 

"Researchers focused on Texas, which consistently displayed the messages for one week every   month on 880 signs across the state’s highways. Researchers gathered data on all traffic crashes that happened on affected roads between 2010 and 2017. They compared crashes that occurred in weeks when fatality stats were displayed with those that happened during the rest of the month, taking care to compare only the accidents that happened at the same hour and on the same day of the week. They also controlled for weather and for holidays, which can independently affect the number of crashes. "

While conceding the difficulty of researching these topics, determining cause and affect and discerning what and when messages can influence behavior, Madsen cites one initiative with resonance: placing the wreckage of vehicles, which had been driven by a teenager, at rest areas. 

During his segment, Ullman talks about whether the effect of higher fatality numbers is plausible and questioned whether drivers are really processing larger and smaller death rates differently. He says he would like to see more research on the cause of the increase.  

Ullman also talks about the importance of message design and other research on how optimism bias informs our judgment.

Podcast player photo: MDOT Dynamic Message Sign board displaying a safety message.

[Music] 

Jeff Cranson: Welcome to the Talking Michigan Transportation podcast. I'm your host, Jeff Cranson. 

[Music] 

Cranson: You may have seen messages on freeway signs, the electronic ones known as dynamic message signs, or DMS, displaying the toll of crash deaths on Michigan roads. Since MDOT began posting on one Wednesday a month some years ago, the practice has provoked varied reactions. New research, highlighted in the journal Science, takes a look at the practice and whether the messages have had an effect on driver behavior. Today, I’ll be speaking first with Josh Madsen, a professor of accounting and behavioral economics at the University of Minnesota, who co-authored the study. And later, I’ll speak with Jerry Ullman, a senior research engineer at the Texas A&M Transportation Institute, who authored a column that accompanied the story in the journal Science, offering his own perspective. The study was conducted on Texas roads. So, again, first Professor Madsen, thank you for taking time to be here. Could you talk a little bit about what motivated you to look into this? 

Joshua Madsen: Sure, thank you so much again, Jeff, for letting me join your show. I came across my first fatality message in Illinois while I was doing my PhD in accounting. As you mentioned, I’m an accounting professor, and accountants, they don’t study debits and credits; we study information, and specifically how information such as financial press releases of 10ks affects market participants. So, I was knee-deep in research about information disclosure and how that can have effects on different market participants. And I came across a fatality message while I was driving on the freeway, and just kind of did a double take. I had a small freakout moment where I was trying to digest the information and figure out what was the context; was this only on highway crashes or crashes across all road networks, what was the exact period? I found myself being distracted, but also interested in understanding how effective these things could be, right? So, I felt that I had the tools and the interest and started to explore. A couple months later I found myself in Tennessee and saw another one in that state. And at that point, realized that this was not just some isolated occurrence on one specific highway in Illinois. And so I started to dig deeper, reached out to a fellow PhD student at Chicago, Jonathan Hall, who was in traffic safety. He was working on a dissertation focused in that direction. So, he had a lot of the missing tool sets and domain knowledge that I didn't have. And so our skills complemented each other well, then we just dived into the study.

 Cranson: Were you alarmed? I mean, did you find it disturbing just to be hit with that number right between the eyes? 

Madsen: I certainly did. I will say, I didn't start out expecting or even hoping that it was increasing crashes. My best hypothesis or hope for a finding was that it would have some sort of transitory effect, that maybe crashes would go down immediately, but then that wouldn't last very long. Sort of like the rubber neck type effect; that we can have maybe a transitory effect on drivers’ behavior. But yeah, I certainly was affected. As I’ve talked with other people, my favorite was talking with this elderly lady at church. She's 80 years old, and I was describing what I was doing and these messages, and she's never seen one before. But her face just visibly went crazy, she was shaking her head, and she's like, “They disclose things like that? They show that?” You could tell that she also had a strong reaction, which is, in part, what it was intended from these signs, right? They wanted to get a sobering message. They wanted to hopefully shock drivers into driving safer. Recognition that standard approaches, click it or ticket, reduce your speed; those things don't seem to have a benefit. So, let's try something else, maybe we need to get a little bit more extreme to get drivers to finally take driving risks seriously, slow down, save lives.  

Cranson: That's the thing. You and I talked a little bit about this before we started recording, but traffic safety engineers and people that are advocates for traffic safety are constantly trying to think of creative ways to get the message across. They're frustrated. I mean, with everything we've done to make roads and cars safer, we keep developing new technologies: auto braking, and lane assist, and vehicle to infrastructure communications, and all those things we're doing. Yet, 40,000 people are still dying on our nation's roads. And we wouldn't tolerate even the smallest plane crash with one or two fatalities, gets wall-to-wall news coverage. So, and I know you've said that that's why obviously the goal is laudable, but if it's counterintuitive, if it's going the wrong direction, then we got to rethink it. I think maybe one of the other professors quoted in that story in the Science journal said, what it really illustrates is that people just have limited cognitive capacity. And I guess I kind of knew that, but this underscores that.  

Madsen: Right, we're all humans. And drivers are no different than the rest of us; many of us drive frequently, and we can only handle so much. And so, if you're involved in an already dangerous, complicated, complex task of driving down a freeway at 70-80 miles an hour, dealing with lane changes, maybe you've got a screaming baby in the back seat, you're worried about the meeting that you're running late to—just throwing one more thing to think about to distract you to take your eyes off of the road with something that is arguably not informative. So, giving very specific feedback about, there's a crash ahead, or you need to change lanes; that's specific to where you need to go and how you need to get there. Obviously, more benefits to that, but one of these more generic messages that are just trying to nudge behavior…again, I'm a behavioral economist. I'm all about nudges, like they can do fantastic things, they just need to be done—what my research is showing, you got to watch out for when you do it and worry about how much cognitive bandwidth. How much attention can be sacrificed to this nudge, to this piece of information that you're trying to shove into a driver's space?  

Cranson: One argument against signs and their effectiveness would be that we already have signs telling you what the speed limit is, and not a lot of people seem to pay attention to those anymore. And you wouldn't put a sign on a bank that says no bank robbers allowed. Do you think your study, I mean, thinking in terms of the pushback, was broad enough? Is your sample large enough to draw that conclusion? Is there somebody out there that might say, look, the numbers would have been even worse if not for this? I mean, it's the proving the negative thing, right. You can't say how many more people would have died if not for these messages. 

Madsen: Yeah, you're talking about the missing counterfactual. Like what really would happen in the absence of this. Many places, many states that show this, unfortunately, there's just no way to rigorously test that. Texas gave us a research boon in the sense that they decided, for whatever reasons, to only show this one week each month. So, you have an entire week where the intensity of treatment goes up. And then I can think about, as a fairly reliable counter factual, what is happening that specific growth segment, that day of the week, that hour, on the other weeks in that month. And so to that extent, I feel like that counter factual really, that’s the reason this got published where it did in Science is the rigor to be able to say, it's not experimental evidence. I didn't get the perfect randomization if I were to design this in a lab, but it is pretty darned good and pretty high and able to enable us to, with a fairly high amount of confidence, say this seems to be a causal effect, not just some random association.

 Cranson: Yeah, and we found, as the pandemic started, speeds increased everywhere. I know that was the case in Michigan, I think it was the case in Minnesota, too. And state police in many states were finding people on radar going 120 miles an hour faster. And then as people returned to the roads and traffic increased again, some of those habits seemed to just stick with people, and some of those high speeds. And so the crash rates were up. Actually, in some cases, crash rates overall were flat or went down a little bit, but crash deaths went up. So the crashes that did occur had a lot more severe consequences.

 Madsen: …were very egregious. Yeah, you empty the roads out, you feel like you can drive faster for sure. And I think you're just saying that sometimes that behavior sticks, and we continue to do what might be safe in one scenario, but as the environment changes, our behavior is slow to adapt.

 Cranson: Yeah, so I guess, coming out of this—and I remember being interviewed by a TV reporter in the Detroit market early on when MDOT started posting these messages, just in a very limited way like you talked about Texas. And she interviewed the mother of someone who had died at the hands of a drunk driver, and the mother was very alarmed at first when she saw the numbers going up, but then she decided that was humanizing it for people. These crash stats really do involve human beings, and she decided she was comfortable with it. I think it'd be interesting now to go back and cite that research and see. I haven't seen very many news stories, I mean, much exploration of this. Obviously, your study has provoked a lot of stories, and a lot of news, and a lot of discussion, which is good. But I don't know, what do you think; this wasn't part of your study, but what do your instincts tell you from studying behavior? What would resonate? What does make somebody start to think driving really is a privilege, it's not a right?

 Madsen: Sure, I'll tell you what's resonated with me the most, is at least two times now at various rest stops, I've seen a totaled vehicle on display with the biography of the teen that was in that car and was killed. And that was just horrifying, shocking, but I was parked, I was out of my car. I was able to digest that information, talk about it with my kids. I mean, hats off to the family member that's willing to do that, to be willing to share that tragic story, but with the motivation of really humanizing, as you said, what really can happen. And that people's lives really do end in tragic and unnecessary ways. So, again, that humanizes it, but it's delivered in a time like, again—I love the rest stop idea. I'm out of my car, I'm able to think about it, I'm able to digest it. If I'm having a really hard time working with that, I don't have to get in my car right away. I can just take a few more minutes, and then when I enter back onto the freeway, hopefully my behavior is a little bit better. I've got kids in the back seat, I'm going to have a teenage driver soon, what am I going to teach him? How am I going to prepare him? So fine, yeah, be shocking. Just get the message across, but don't distract the drivers while they are in the act of operating. You wouldn't go onto some factory floor with someone that's operating a very expensive and dangerous piece of equipment and suddenly flash some lights in their eyes and give them a message that suddenly, hey, this is a dangerous machine. Well I'm operating the machine, I need to focus on operating the sheet machine right now, alright. You need to warn me as I am walking out onto the factory floor, not while I'm in the process of operating the machine.

 Cranson: Yeah, that makes a lot of sense. I will have to say that both my kids are out of the house now, and I had always hoped that by the time they started driving, we would have fully autonomous vehicles that just couldn't crash. But that was a pipe dream on my part. You would think that’s what should get everybody's attention; if it's not thinking about yourself and the potential for catastrophe, it's thinking about your kids being out there. On a lighter note, because you've studied behavior, I found some of your other research kind of interesting. Just talk a little bit about this—because I think it relates, again, to why being an economist or an accounting professor is about a lot more than just numbers—what you found about how weather affects people's processing of things like earnings news and investing.

Madsen: Yeah, sure. That was a really fun study. We're looking at basically investigating what we all kind of know to be true, but there's not a lot of evidence on it of the rainy day blues. The rain comes down, the sky is overcast. I've got a little bit lower energy. It's maybe harder to get up in the morning, harder to get motivated, harder to give it my all. And so we wanted to test that, but we wanted to focus not just on normal people, but maybe on people that have really strong incentives to not be affected. People that, I've got a deadline, I’ve got to get it done, so I'll get my act together, and I'll make sure that happens. And so, again, I am a county professor interested in financial markets, so we went to financial analysts. Individuals that it's their job to process earnings information and put out forecasts in terms of, okay, what is the implication? Google just announced earnings, how do we think this is going to affect their future profitability, should my investors buy, or sell, or hold, etcetera. And so, research design the ability to try and get a causal statement here is extremely important. And so, what we do in that study is we take advantage of the fact that firms are followed by multiple analysts. So Google has 40 plus analysts following it, but each of those analysts is located in a different city, experiencing arguably different, somewhat random weather. And so when Google announces its earnings on March 15th, you've got 40 analysts all needing to respond to it, all recognizing or experiencing their own weather conditions. And so, what we find there is that those analysts that happen to be in a situation that that day the weather is more overcast, rainy, windy; they're just slower to get it out. Their forecast, their update doesn't come out nearly as quickly as analysts that are experiencing better weather conditions. And then when their forecast does come out, it's overall more pessimistic about the firm's prospects. And again, I find it beautiful, I'm grateful that you brought it up, and you seem to enjoy it as well, just looking at how the behavior, the environment around us can actually affect us. We're all humans, be it you, me, or a fortune 500 CEO; we all are susceptible to our environment, and it has the ability to influence us.

Cranson: Yeah, I'd start to wonder why anybody that lives in Western Michigan, at least Southwestern Michigan, where we have more than our share of cloudy days, would not be terminally affected by seasonal affected disorder and probably never want to invest.

Madsen: Yeah no, there certainly are regional effects. I will say that in that study, we wanted to make sure that it wasn't just your Southwestern Michigan analysts. And so, we needed that analyst to also be following or responding to an earnings announcement for another firm. So we want to take out the individual. It's not just the individual, but it's you, Jeff, on a rainy day versus a sunny day, relative to someone else that is also experiencing rainy and sunny days as the benchmark. That's getting a little bit more into the weeds, but yeah, the general idea; let's see how people respond differently based off of the weather that they're experiencing.

Cranson: Yeah no, it's really interesting, and I think that it's a good thing that you question everything and that you're willing to look at things that other people might just take for granted and ask those kinds of questions. And that's what led you to do this research on traffic safety, which is really important and really noble. I appreciate it, and I very much appreciate you taking time to talk about it.

Madsen: Thank you so much. It's been a joy to be with you, and good luck with your podcast.

Cranson: Thanks. Stay with us, we'll have more on the other side of this important message.

Narrator: The Michigan Department of Transportation reminds you that when a vehicle collides with another vehicle, person, or other object, it is a crash, not an accident. By reducing human error, we can prevent crashes, and rebuild Michigan roads safely.

Cranson: And for our second segment, I'll be speaking with Jerry Ullman, who is a senior research engineer at the Texas A&M Transportation Institute. And he has done some of his own research related to safety messages and the effect it has, kind of its own look at that behavior and driver behavior. And he wrote what was dubbed a perspective that went in the journal Science that accompanied the story about the study, and it wasn't so much a counter point or a descent, but it brought a different context to it or, I guess, a different perspective as they called it. So, thank you, first, for taking time to be here. I appreciate it. 

Jerry Ullman: Oh no problem, thanks for having me. 

Cranson: So, tell me what you took away from all this discussion, states including Texas, where you are doing these messages about fatal crashes, and the number of fatalities on a weekly basis. In the case of Michigan, one day per month, not a weekly basis, one day per month in Michigan, but I guess for the full week, once a month in Texas. And they've kind of curtailed that some but give me your broad perspective on this whole thing. 

Ullman: Well, I think certainly the intent and the goal of what was being done with these kinds of messages were certainly in the right frame. It's just that the messages themselves, traffic safety messages, have had pretty much a lack of direction and guidance on what to put on them, how much to put on them, and those kind of things. Unlike messages we use for traffic management purposes, when we're talking about incidents or providing notices about road work ahead and that kind of thing, we have a very solid way of quantifying, or how much information are we going to put on those messages. We were very concerned about driver information overload; any traffic engineer, that's drilled into them from a really early age. I think one of the things that we took away from the review of the Batson paper is that even in these traffic safety messages, you've got to be concerned about the type of information, how you're laying it out, and how much information you're putting on those messages. So I think that's the big takeaway. 

Cranson: Yeah, I think information overload is a problem for all of us even when we're not driving. But what would you say to the proponents of this kind of thing, the advocates always say, look, if we just get people talking about it, then we've done our job? And I don't know how you measure that, and I think that's one of the problems is studying what's effective, and how do you adequately study what works, and figure out if anything works: if telling teens to not text and drive and telling people in a certain age demographic not to speed, or that they have to wear their seatbelt, all those things. Because I certainly agree with you that putting things on the message boards, that's news you can use. There's a safety component to telling people that there's a crash ahead, and you should seek a detour and prevent the kind of congestion that causes other crashes, or construction, or something like that. So what do you think about that? 

Ullman: Well yeah, I think it is important that safety campaigns are important. I think the real question becomes, to what extent do these dynamic message signs support versus try to become the prime outreach mechanism? I think they can certainly supplement, but they should be used as a supplement to the broad campaign itself. One of the nice, I think one of the interesting things about this study is really the outcome is what we're most interested in. Does it improve safety and result in fewer crashes? And this is the first time I’ve ever seen an effort to focus specifically on the use of safety messages on dynamic message signs, and part of it was because of the way that the process which was performed in Texas for those few years that allowed them to do this analysis and come to the conclusions that they did. So I don't think you say, “Safety messages should never be posted on a sign”, as much as it's, “Hey, we've gotta make sure we don't overload the driver with messages”. Fatality numbers, I’m not so much whether it's a fatality number, whether it's a number of total cashes, if it's a percentage of tickets written, or any of those kinds of things. The numbers themselves are just, we know they're a bit challenging to process from an information standpoint. We're, as humans, we're really good at understanding small numbers, but you start throwing in larger and larger numbers; it's just something we as humans don't do a really good job of. So throwing out 1,237 fatalities this year or whatever, that's quite a bit of information and contributes to information overload when you add it to a lot of other words, and maybe another phrase with another slogan, and that kind of a thing. 

Cranson: Yeah, it's all about context, and when you've only got so many characters, you're greatly restricted on the DMS board. So it's hard to provide a lot of that context, and that's one of the things I hear from people a lot; compared to what?  

Ullman: Yeah, the other thing is you have to think that you've got a limited amount of space on the board, but you've only got a limited amount of time to view it, too, because you're coming at it at whatever highway speed. So, those signs are designed really to only give you about eight seconds or less of a visible readable view, and time during which you're supposed to be kind of glancing at the sign, back at the road, back at your instruments, back up to the sign. So, the amount of time you can actually put your eyes on that sign is really limited, and so message design is so important in that. 

Cranson: Yeah, well, you know the story and Science quote, she was saying it's counter-intuitive, but the analysis is solid. But it also later says that you're not totally convinced. So could you explain that? 

Ullman: Yeah, there is other literature that suggests that the use of statistics, fatality numbers, and stuff really doesn't move the needle when it comes to driver sort of attitudes about driving safer or those kind of things. We all have what's often referred to as optimism biases; we're all above average in our driving abilities. So, when he looked at that literature that really suggests that use of statistics and that kind of thing doesn't affect their attitudes and their stated behaviors, and then you look at this data, you've got to go back. And that's where I thought, those two don't line up. But if you boil it back to the amount of information in that limited viewing time that's being presented, larger numbers that are a little more complicated for people to even interpret what that number is, and what it means, then you've got that potential issue. I guess the other thing we have to also keep in mind is, this is one study using one state, and the available data in that state. It's a statistical analysis, and I think the authors did a good job of looking at all the potential alternative hypothesis and that kind of thing, but at the end of the day, there still is a statistical chance that this result was the result of something extraneous that wasn't measured. And/or it just was randomness that lined up in a way that looks like it's an effect, when in fact it really comes from the same population—if you don't use statistical terminology. So, what would be better is a follow-up study, maybe another state, or a similar kind of analysis to validate what's done here, to know that this really wasn't just a neat confluence of unmeasured variables, but truly an effect that's due to the use of the safety messages that include fatality numbers, particularly large fatality numbers.  

Cranson: Well, I think that you nailed it, and I’ve never seen this play out, but I’ve always believed that most people think that “I’m a good driver, but these other people aren't”. 

Ullman: Yep, that's exactly it.  

Cranson: That's the bias. Is there anything else you want to say about this, or you'd want to get across for better understanding? 

Ullman: No, I think like I said, I think what you summarize, what it was our interpretation. I think again, just encouraging agencies to take a look at what they're doing. I believe there's better ways to use these messages for safety related messages that don't have to rely on fatality numbers, or really any type of statistics or numbers in them. And I think it's important to be having this kind of conversation about what do we really want to put on these signs.  

Cranson: Yeah, and keep trying to figure out what might work, and what might resonate. I think especially with young drivers, who are still learning as they go and naturally feel invincible, as we all did at that age.  

Ullman: Exactly, I totally agree with you. 

Cranson: Well, thank you very much. I really appreciate it, and I think this helps add to the conversation 

Ullman: You're quite welcome, thanks for the call. 

[Music] 

Cranson: Thank you again for listening to this week's edition of the Talking Michigan Transportation podcast. I would like to thank Randy Debler and Corey Petee for engineering this week’s podcast. To subscribe, to show notes, and more, go to Apple podcasts and search for Talking Michigan Transportation. 

[Music]