surprising findings about randomized government inspections

Research & Ideas

OSHA Inspections: Protecting Employees or Killing Jobs?

Published:	May 21, 2012
Author:	Michael Blanding

Executive Summary:

As the federal agency responsible for enforcing workplace safety, the Occupational Safety and Health Administration is often at the center of controversy. Associate Professor Michael W. Toffel and colleague David I. Levine report surprising findings about randomized government inspections. Key concepts include:

• In a natural field experiment, researchers found that companies subject to random OSHA inspections showed a 9.4 percent decrease in injury rates compared with uninspected firms.
• The researchers found no evidence of any cost to inspected companies complying with regulations. Rather, the decrease in injuries led to a 26 percent reduction in costs from medical expenses and lost wages—translating to an average of $350,000 per company.
• The findings strongly indicate that OSHA regulations actually save businesses money.

About Faculty in this Article:

Michael Toffel is an associate professor in the Technology and Operations Management unit at Harvard Business School.

• More Working Knowledge from Michael W. Toffel
• Michael W. Toffel - Faculty
  Research
• E-mail Michael W. Toffel: mtoffel@hbs.edu

With an election looming and the economy continuing to struggle, the effectiveness of government regulation has become a political football. While advocates hold regulations up as necessary to protect public health and safety, critics see them as arbitrary and costly to business, reducing wages and killing jobs at a time when the United States can ill afford to lose them. Few regulatory agencies have a more direct effect on businesses than the Occupational Safety and Health Administration (OSHA), the federal agency responsible for enforcing regulations to keep workplaces healthy and accident-free.

"If people are skeptical of OSHA, it's likely based on anecdotes. The difference in our study is we are looking at hundreds of companies over a long period of time, and we find that those anecdotes are not typical."
—Michael Toffel

"Needless rules and onerous regulations are often roadblocks to economic growth and job creation," groused one congressman during a House subcommittee hearing on OSHA last year, sharing the private frustrations of many employers. The debate over the agency has gone on for years, in part because there has been little evidence to prove the case for either side. In order to get the biggest bang for taxpayers' bucks, OSHA typically inspects companies most likely to have problems, often following accidents and complaints, stacking the deck with companies that are worse than average.
"Where there is smoke, there is usually fire, so the fact that government inspectors find safety problems is not surprising," says Michael W. Toffel, an associate professor and the Marvin Bower Fellow at Harvard Business School.
At the same time, when problems are resolved, there's no way of telling whether the inspections themselves helped fix them, because the law of averages implies that a company with a bad year would usually improve the following year even without an inspection. "When they do find a problem, it's not entirely obvious that it wouldn't resolve itself anyway," he says.
Toffel has long studied the effectiveness of regulations, focusing on voluntary measures. When he and colleague David I. Levine heard of a program at California OSHA to conduct randomized inspections of workplaces, they realized they had the perfect real-world experiment to settle the debate over workplace inspections.
"When I learned that Cal-OSHA was conducting some of its inspections at random, I felt almost an obligation to analyze that experiment," says Levine, a professor at UC Berkeley's Haas School of Business. "To let these debates on OSHA and other regulatory agencies continue for another generation when this experiment had already been done seemed inexcusable."
Little did the researchers know at the time how difficult it would be to extract the data for analysis. That task fell to Toffel, who was still completing his doctoral studies at Haas when the research began in 2005. "We received all this data from Cal-OSHA on cassettes, the likes of which I have never seen," Toffel laughs. "They were Memorex tapes about three-quarters of an inch tall and six inches square. We had to hire someone to make them machine-readable."
The team brought on Matthew Johnson, then an HBS research associate who is now an economics doctoral student at Boston University, to help with the data analysis.
Thus began the long saga of extracting statistics on inspections and sorting out which were randomly assigned, which followed accidents, and which followed complaints. In addition, Toffel and Levine essentially became deputized by another California regulatory agency to gain access to workers' compensation data. Because the data were collected at the company level, the researchers limited their analysis to firms with only one plant, where the effects of an inspection on injury rates and costs could be clearly identified. Finally, they matched companies by size, industry, and other characteristics to end up with some 800 companies. Half of the companies had been subject to random inspections; half of them were eligible for inspections but not chosen.

Surprising findings

The results of their analysis, published in Science last week, are definitive: inspections worked. Compared with uninspected firms, the companies subject to random inspections showed a 9.4 percent decrease in injury rates. What's more, the findings were consistent for both large and small accidents. "We thought our results might have been driven by fewer big problems, like preventing storage racks from collapsing and other major accidents; or perhaps by a particularly dramatic decline in smaller injuries prevented by workers more regularly wearing personal protective gear," says Toffel, who worked as an environment, health, and safety manager in the private sector before pursuing his doctorate. "But we found it to be an across-the-board effect."
Just as important are the findings about the costs to companies of complying with regulations. Testing every measure they could find—jobs, wages, sales, and credit ratings among them—the researchers found no evidence (within the margin of error) of any cost to businesses that had been inspected. In fact, quite the contrary: the decrease in injuries led to a 26 percent reduction in costs from medical expenses and lost wages, translating to an average of $350,000 per company. While those costs would be felt most immediately by the firms' workers' comp insurance companies, over time that would translate to lower insurance premiums for the employers.
In other words, those who charge that OSHA regulations cost business money have it completely wrong. In fact, the regulations save money. The magnitude of the results surprised even Toffel and Levine, who expected perhaps a small savings if any. But the strength of the findings, they say, should persuade even skeptical antiregulatory critics.
"If people are skeptical of OSHA, it's likely based on anecdotes," Toffel says. "The difference in our study is we are looking at hundreds of companies over a long period of time, and we find that those anecdotes are not typical. If they know of a company that has been shut down, that is not typical." In fact, the inspected firms were just as likely to be in business at the end of the study period as the control firms.
Of course, a single study cannot settle the debate over regulation, or even the debate over OSHA. This study is limited to one regulatory agency in one state; other states and other agencies could show different results. One thing the research does show, though, is the value of randomized inspections as a way to help gauge regulations' effectiveness. What's more, say Toffel and Levine, the potential benefits from randomizing isn't limited to government inspections, but can also extend to the private sector in the form of randomizing the deployment of new safety, environmental, and quality programs.
But randomizing, of necessity, involves a trade-off. Focusing on short-term cost-effectiveness often leads managers to implement new programs at sites or at times when they think they will do the most good. But such targeted interventions make it impossible to evaluate the program's overall effectiveness, as one could with randomized deployment.
"There is a perceived cost in doing a portion of the work in a randomized fashion," Toffel says. "But if you want to learn, it requires an investment and some patience. You have to be willing to delay or forgo deploying a new program at some of the neediest sites in order to evaluate whether the program is effective." Or as Levine puts it: "It's costly to learn, but it's more costly to be ignorant."

About the author

Boston-based writer Michael Blanding is a fellow at the Edmond J. Safra Center for Ethics at Harvard University and author of The Coke Machine: The Dirty Truth Behind the World's Favorite Soft Drink.