Score: 26 / 60
RULE SUMMARY
FRA proposes regulations implementing a requirement of the Rail Safety Improvement Act of 2008 that certain passenger and freight railroads install positive train control systems. The proposal includes required functionalities of the technology and the means by which it would be certified. The proposal also describes the contents of the positive train control implementation plans required by the statute and contains the proposed process for submission of those plans for review and approval by FRA. These proposed regulations could also be voluntarily complied with by entities not mandated to install positive train control systems.
METHODOLOGY
There are twelve criteria within our evaluation within three broad categories: Openness, Analysis and Use. For each criterion, the evaluators assign a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points.
| Criterion | Score |
|---|---|
Openness | |
| 1. How easily were the RIA , the proposed rule, and any supplementary materials found online? RIN 2130-AC03 can be found on regulations.gov using the RIN and a keyword search. The Department of Transportation's website has links to regulations.gov and additional information regarding regulations. A link to the proposed regulation and the RIA is available six clicks from the DOT home page, if one knows to look under "Federal Railroad Administration" and then follow a series of regulatory links. | 5/5 |
| 2. How verifiable are the data used in the analysis? Cost data are simply attributed to discussions with a working group composed of people in industry who have experience with these systems. Locomotive counts come from some industry source that is not well-referenced. Benefit data are based on several studies of costs avoided as a result of PTC; full names and links for studies are not given. Citations and links would be helpful. | 1/5 |
| 3. How verifiable are the models and assumptions used in the analysis? It is not clear what justifies many of the assumptions—eg, "high cost," "low cost," percentage effectiveness of the regulation, phase-in schedule, etc. Citations and links would be helpful. | 1/5 |
| 4. Was the analysis comprehensible to an informed layperson? The RIA begins with a well-written explanation of the issue and history. The actual analysis of benefits and costs is a bit turgid and has some repetition. The results are usually clear, but it probably requires a background in economics or other analytical field to infer how DOT arrived at the results. The conclusion is understandable but the analysis is filled with acronyms, jargon, and industry-specific references. | 3/5 |
Analysis | |
| 5. How well does the analysis identify the desired outcomes and demonstrate that the regulation will achieve them? | 3/5 |
| Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life? The principal benefits are those attributable to reduced accidents: reduced casualties, property damage, environmental damage, track/road closures, evacuations, and reduced risk of hazmat incidents. The analysis notes there may also be cost savings or efficiency increases for railroads but does not include these in the main benefit calculation. | 4/5 |
| Does the analysis identify how these outcomes are to be measured? Benefits for most of the factors mentioned above were measured in a 2005 Volpe Center study that examined 14 years' worth of accidents that would have been prevented by positive train control. Benefits were monetized, and the current analysis updates the benefits for inflation and for DOT's increase in its standard willingness to pay to avoid a fatality. The analysis adds some figures from "headline" accidents not in the database used in the Volpe study. Unfortunately, the analysis does not always measure the benefits in their natural units, but simply adjusts dollar figures from the Volpe study. | 4/5 |
| Does the analysis provide a coherent and testable theory showing how the regulation will produce the desired outcomes? The analysis does not state how it determined which accidents would have been prevented by positive train control, so there is not really a clear theory explaining how requiring PTC would prevent some or all accidents. | 1/5 |
| Does the analysis present credible empirical support for the theory? The analysis cites a couple past studies that estimate benefits from PTC, but these appear to be projections based on past accidents someone judged could have been prevented, not studies of PTC's effects in practice. | 2/5 |
| Does the analysis adequately assess uncertainty about the outcomes? Analysis acknowledges that some benefits are highly uncertain and declines to include these in the main benefit-cost comparison. An stimation of benefits acknowledges that PTC will not be 100 percent effective. | 2/5 |
| 6. How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve? | 0/5 |
| Does the analysis identify a market failure or other systemic problem? There is no explanation of a market failure related to safety. Since the analysis never explains how to determine what accidents PTC would have prevented, there isn't even a theory of a systemic problem. The closest thing to an explanation is that PTC is expensive and FRA developed the rule in response to a congressional mandate: "FRA began the process of developing a proposed rule after RSIA08 was signed into law on October 16, 2008 . . . Despite these successes, the widespread deployment of these various train control systems, particularly on the general freight system, remained very much constrained by prohibitive capital costs." | 0/5 |
| Does the analysis outline a coherent and testable theory that explains why the problem (associated with the outcome above) is systemic rather than anecdotal? At one point the analysis does just the opposite—asserting that some railroads' voluntary adoption of PTC shows that they expect substantial private business benefits. But if this is true, then what is the underlying need for regulation? Elsewhere, the analysis claims that private benefits plus safety benefits could exceed costs. There could be a germ of a theory here if the analysis could explain why railroads do not capture the safety benefits privately, but no such discussion occurs. Contradictory claims make this confusing. | 0/5 |
| Does the analysis present credible empirical support for the theory? The small benefits, based on the relatively small number of casualties or other problems preventable by PTC, suggests that there may not be a significant systemic problem. | 1/5 |
| Does the analysis adequately assess uncertainty about the existence or size of the problem? No relevant discussion. | 0/5 |
| 7. How well does the analysis assess the effectiveness of alternative approaches? | 2/5 |
| Does the analysis enumerate other alternatives to address the problem? The analysis does not consider any substantial alternatives. In the preamble, FRA explains several instances where discretion was exercised to contain costs, but there is no real regulatory analysis of these. A working group was used and may have considered options, but RIA goes no further in explaining what they were. The regulatory flexibility section notes, "FRA is unaware of any significant alternatives that would meet the intent of RSIA08 and that would minimize the economic impact on small entities." | 2/5 |
| Is the range of alternatives considered narrow (e.g., some exemptions to a regulation) or broad (e.g., performance-based regulation vs. command and control, market mechanisms, nonbinding guidance, information disclosure, addressing any government failures that caused the original problem)? Alternative implementation dates are occasionally discussed, but not in great detail. | 2/5 |
| Does the analysis evaluate how alternative approaches would affect the amount of the outcome achieved? There is some reference to alternative implementation dates' effect on costs. | 2/5 |
| Does the analysis adequately address the baseline? That is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future? The baseline is assumed to be accidents that occurred in a 14-year period in the past. At one point, the analysis mentions that if rail volumes increase, the potential for the regulation to create benefits could also increase, but this is not incorporated into the calculations. | 2/5 |
| 8. How well does the analysis assess costs and benefits? | 2/5 |
| Does the analysis identify and quantify incremental costs of all alternatives considered? Costs are calculated for the only alternative chosen and for different implementaiton dates. | 4/5 |
| Does the analysis identify all expenditures likely to arise as a result of the regulation? The analysis seems pretty thorough, though not well-documented. "The costs anticipated to accrue from adopting this proposed rule would include: (1) Costs associated with developing implementation plans and administrative functions related to the implementation and operation of PTC systems, including the information technology and communication systems that make up the central office; (2) hardware costs for onboard locomotive system components, including installation; (3) hardware costs for wayside system components, including installation; and (4) maintenance costs for all system components." | 3/5 |
| Does the analysis identify how the regulation would likely affect the prices of goods and services? No relevant discussion. | 0/5 |
| Does the analysis examine costs that stem from changes in human behavior as consumers and producers respond to the regulation? No relevant discussion. | 0/5 |
| If costs are uncertain, does the analysis present a range of estimates and/or perform a sensitivity analysis? The analysis presents a range of estimates but does not do much to justify the range. | 2/5 |
| Does the analysis identify the alternative that maximizes net benefits? Only one alternative is analyzed, and net benefits are very negative. "FRA recognizes that the likelihood of business benefits is uncertain and that the cost-to-benefit comparison of this rule, excluding any business benefits, is not favorable. However, FRA has taken measures to minimize the rule’s adverse impacts and to provide as much flexibility as FRA is authorized to grant under RSIA08." | 2/5 |
| Does the analysis identify the cost-effectiveness of each alternative considered? There is no relevant discussion, though cost-effectiveness could perhaps be calculated from the information in the analysis. "FRA is unable to estimate the potential savings if Congress provided a longer implementation schedule or provided incentives, rather than mandates, for PTC system installation." | 1/5 |
| Does the analysis identify all parties who would bear costs and assess the incidence of costs? Railroads are usually assumed to bear the costs. There is some discussion in passing of whether railroads could pass the costs, or cost savings, on to their customers, and how customers might respond, but this is not incorporated into the calculations. | 2/5 |
| Does the analysis identify all parties who would receive benefits and assess the incidence of benefits? Railroads may receive some benefits; recipients of other benefits are not explicitly discussed. | 1/5 |
Use | |
| 9. Does the proposed rule or the RIA present evidence that the agency used the analysis? The analysis indicates that FRA is sensitive to the costs and made several decisions intended to keep costs down. This suggests the RIA affected some decisions by making FRA sensitive to costs, but it does not appear that any choice flowed directly from the RIA's findings. | 3/5 |
| 10. Did the agency maximize net benefits or explain why it chose another alternative? The RIA estimates that net benefits of the chosen option would be negative; no substantially different options are analyzed. FRA explains why it made the choices it made: "Although costs associated with implementation of the proposed rule are significant and such costs would far exceed the benefits, FRA is constrained by the requirements of RSIA08, which do not provide latitude to for implementing PTC differently. Nevertheless, FRA has taken several steps to avoid triggering unnecessary costs in the proposed rule." | 3/5 |
| 11. Does the proposed rule establish measures and goals that can be used to track the regulation's results in the future? FRA establishes no measures or goals. The analysis is based on estimates of various benefits related to avoided costs of accidents. Accidents could be tracked in the future to see if this regulation reduces them. | 1/5 |
| 12. Did the agency indicate what data it will use to assess the regulation's performance in the future and establish provisions for doing so? FRA makes no explicit commitment, but accident data could be used to track outcomes. | 2/5 |
| Total | 26 / 60 |
Additional details
- Agency
- Department of Transportation
- Regulatory Identification Number
- 2130-AC03
- Agency Name
- Department of Transportation
- Rule Publication Date
- 07/21/2009
- Comment Closing Date
- 08/20/2009