Regulatory Analysis and Regulatory Reform: An Update

Congress and the executive branch have attempted to improve the quality of regulatory decisions by adopting laws and executive orders that require agencies to identify the problem they are trying to address and assess its significance, examine a wide range of alternatives to solve the problem, assess the costs and benefits of the alternatives, and choose to regulate only when the benefits justify the costs. A research team from the Mercatus Center at George Mason University has assessed the quality and use of regulatory analysis accompanying every economically significant, prescriptive regulation proposed by executive branch regulatory agencies between 2008 and 2012. 

The team found that, while it varied widely, the quality of regulatory analysis was generally low and did not alter much with the change of administrations. For 60 percent of the regulations, agencies failed to provide any significant evidence that any part of the regulatory analysis helped inform their decisions. Improving the quality and use of regulatory analysis will require institutional reforms to ensure that regulatory impact analysis is required, objective, and used to inform decisions about whether and how to regulate. 

What Is Regulatory Impact Analysis?

Somewhere along the line, most people learn a few basic steps to take before making a major decision. These steps include:  

1. Understand the root causes of the problem,
2. Define the goal to achieve, 
3. Develop a list of alternative ways to solve the problem, and 
4. Assess the pros and cons of each alternative. 

Call these steps “Decision-making 101.” 

For nearly four decades, presidential administrations have required executive branch agencies to follow these steps when they conduct Regulatory Impact Analyses (RIAs) that accompany major regulations. In 1993, President Clinton’s Executive Order 12866 laid out the fundamental requirements that have governed regulatory analysis and review ever since. In January 2011, President Obama’s Executive Order 13563 reaffirmed the principles and processes in the Clinton executive order:

Our regulatory system must protect public health, welfare, safety, and our environment while promoting economic growth, innovation, competitiveness, and job creation. It must be based on the best available science. It must allow for public participation and an open exchange of ideas. It must promote predictability and reduce uncertainty. It must identify and use the best, most innovative, and least burdensome tools for achieving regulatory ends. It must take into account benefits and costs, both quantitative and qualitative. It must ensure that regulations are accessible, consistent, written in plain language, and easy to understand. It must measure, and seek to improve, the actual results of regulatory requirements.

Analytical requirements are especially rigorous for economically significant regulations, defined as regulations that have a material adverse effect on the economy or have an annual effect on the economy of $100 million or more. 

Assessing the Quality and Use of Regulatory Analysis

The Mercatus Center at George Mason University has developed a qualitative framework to assess both the quality and use of regulatory analysis in federal agencies. The scoring process evaluates the quality of regulatory analysis using twelve criteria grouped into three categories:

1. Openness: how easily can a reasonably informed, interested citizen find the analysis, understand it, and verify its underlying assumptions and data?
2. Analysis: how well does the analysis define and measure the outcomes or benefits the regulation seeks to provide, define the systemic problem the regulation seeks to solve, identify and assess alternatives, and evaluate costs and benefits?
3. Use: how much did the analysis affect decisions in the proposed rule, and what provisions did the agency make for tracking the rule’s effectiveness in the future?

A research team evaluated each economically significant, prescriptive rule between 2008 and 2012—a total of 108 regulations. For each criterion, the evaluators assigned a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points. 

Quality of Analysis Is Low

The average total score was just 31.2 out of 60 possible points—barely 50 percent—the equivalent of a grade of “F.” Figure 1, below, shows that the majority of regulations, slightly over 60 percent, scored below 36 points—the equivalent of a “D.” No RIA did an excellent job on all aspects of regulatory analysis. The highest total score ever achieved was 48 out of 60 possible points (80 percent), equivalent to a “B−.” This was the joint Environmental Protection Agency-National Highway Traffic Safety Administration regulation proposed in 2009 that revised Corporate Average Fuel Economy standards. 

Greatest Weaknesses: Retrospective Analysis 

Table 1, below, shows the average scores on each of the 12 criteria. The two final criteria relating to retrospective analysis score particularly low. Few regulations or analyses set goals, establish measures, or establish protocols to gather data so that the effects of the regulation may be evaluated after it is implemented.  

For each Report Card criterion, there are a few examples of reasonably good quality or use of analysis. But best practices are not widespread.

Analysis Rarely Used to Inform Decisions

Table 1 shows that most of the lowest scores are for criteria measuring the use of analysis. The broadest Report Card criterion measuring use of analysis (criterion 9) asks whether the agency claimed or appeared to use any part of the analysis to guide any decisions. As figure 2 demonstrates below, agencies often fail to provide any significant evidence that any part of the RIA helped inform their decisions. Perhaps the analysis affects decisions more frequently than these statistics suggest, but agencies fail to document this in the Notice of Proposed Rulemaking or the RIA. If so, then at a minimum there is a significant transparency problem. 

Improving the Quality and Use of Regulatory Analysis

Average scores for prescriptive regulations are relatively low, earning slightly more than 50 percent of the total possible points. Clearly, agency regulatory analysis is often incomplete and seldom used in decisions. This pattern persists across administrations, indicating that the source of the problem is institutional, not political. Fundamental institutional reforms are necessary to ensure that agencies conduct high-quality regulatory impact analysis and use it in decisions. In short, regulatory impact analysis should be: 

• Required: Congress should require federal agencies to conduct thorough regulatory impact analysis before they write and propose significant regulations. Agencies tend to pay attention to what the law says they should do, because otherwise a court might vacate the regulation. The congressional requirement should include independent agencies that are not currently subject to the executive orders on regulatory analysis and review. Scholarly research has found that many independent agencies conduct even less thorough economic analysis than executive branch agencies. Requiring independent agencies to conduct regulatory impact analysis and explain how they used it in decisions would likely improve their quality and use of analysis.
• Objective: All too often, regulatory analyses read as an afterthought. Agencies should be required to publish analysis of the systemic problem they seek to solve and alternative solutions (along with all underlying data and research) for public comment before making decisions about proposed regulations. Agency economists should have the independence to conduct objective analysis instead of simply justifying decisions that have already been made. Public hearings on regulations after they are proposed would give agencies an incentive to produce better analysis because they would have to defend it publicly. Expanding the resources of the Office of Information and Regulatory Affairs, which reviews all major regulations, can also help to improve the quality and use of analysis. 
• Used: Congress should require all agencies to explain, when proposing regulations, how the major elements of regulatory analysis affected decisions about the regulation. Consistent with the Government Performance and Results Modernization Act of 2010, agencies should also be required to explain how major regulations advance their high-priority goals and establish measures to track the regulation’s actual results.

Conclusion

Regulatory impact analysis assesses the need for, alternatives to, and benefits and costs of proposed regulations. Although administrations of both political parties have required regulatory impact analysis, the quality of that analysis has generally been low. To make matters worse, agencies often fail to provide any evidence that regulatory analysis, however imperfect, informed their decisions. In addition, agencies rarely establish plans for retrospective analysis to evaluate the effectiveness of their regulations. To be genuinely effective, regulatory impact analysis should be required by statute, objective, and used by federal agencies.