- | Regulation Regulation
- | Policy Briefs Policy Briefs
- |
Motor Vehicle Safety Standards, Ejection Mitigation
Proposed Rule
Score: 31 / 60
RULE SUMMARY
This notice of proposed rulemaking would establish a new Federal Motor Vehicle Safety Standard (FMVSS) No. 226, to reduce the partial and complete ejection of vehicle occupants through side windows in crashes, particularly rollover crashes. The standard would apply to the side windows next to the first three rows of seats in motor vehicles with a gross vehicle weight rating (GVWR) of 4,536 kilogram (kg) or less (10,000 pounds (lb) or less). To assess compliance, the agency is proposing a test in which an impactor would be propelled from inside a test vehicle toward the windows. The ejection mitigation safety system would be required to prevent the impactor from moving more than a specified distance beyond the plane of a window. To ensure that the systems cover the entire opening of each window for the duration of a rollover, each side window would be impacted at up to four locations around its perimeter at two time intervals following deployment.
METHODOLOGY
There are twelve criteria within our evaluation within three broad categories: Openness, Analysis and Use. For each criterion, the evaluators assign a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points.
| Criterion | Score |
|---|---|
Openness | |
| 1. How easily were the RIA , the proposed rule, and any supplementary materials found online? The Federal Register notice is available on regulations.gov via a keyword or RIN search. It is mis-filed under "rules" rather than "proposed rules." The RIA is available by opening the docket folder. Neither can be found easily via links or the search function on the DOT web site. | 3/5 |
| 2. How verifiable are the data used in the analysis? Data on fatalities, injuries, etc. appear to be high-quality data from government databases. It is not obvious how a non-specialist could access these data, but specialists knowledgeable about this kind of regulation may know. Appendices contain some of the data without much explanation or citation to sources. | 2/5 |
| 3. How verifiable are the models and assumptions used in the analysis? Most studies used are cited in footnotes but not linked. They are usually by NHTSA, DOT, or contractors. Most assumptions underlying benefit estimates are based on studies or analysis of either test data or accident data; some others seem logical but are not explicitly documented or justified. Costs for redesigning models are not estimated because the agency believes it has allowed sufficient lead time for redesign; no documentation supports this assumption. | 4/5 |
| 4. Was the analysis comprehensible to an informed layperson? Text is quite turgid and technical -- mostly because of technological and engineering jargon rather than economics discussion. Lengthy descriptions of tests were difficult to follow, and would ahve been even more difficult to follow were it not for the window diagrams. Not always clear how the RIA got from assumptions to results. | 3/5 |
Analysis | |
| 5. How well does the analysis identify the desired outcomes and demonstrate that the regulation will achieve them? | 4/5 |
| Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life? It's clear the ultimate human outcomes the agency cares about are reduced fatalities and severe injuries caused by ejection during vehicle crashes. | 5/5 |
| Does the analysis identify how these outcomes are to be measured? The analysis measures fatalities, measures injuries, and monetizes them. | 5/5 |
| Does the analysis provide a coherent and testable theory showing how the regulation will produce the desired outcomes? Compliance with the proposed standards will physically prevent many, but not all, ejections during accidents by keeping occupants inside the windows. No discussion of behavioral changes (less seatbelt use, less careful driving) that might negate some of these benefits. | 3/5 |
| Does the analysis present credible empirical support for the theory? Evidence is based on tests and calculations; tests confirm that the side air bags NHTSA believe manufactures would adopt would prevent a lot of ejections. NHTSA acknowledges that it lacks actual crash data for some parts of the analysis. The baseline discussion notes that some manufacturers make vehicles that already comply with the standard, but the fatality and accident data predate most of these models, so the agency cannot see if models that comply with the standard have a better safety record. | 3/5 |
| Does the analysis adequately assess uncertainty about the outcomes? NHTSA is pretty upfront when it is unsure about something in this analysis. Alternate calculations are done assuming people have equal or weighted risks of being ejected through different parts of the window opening. Sensitivity analysis performed for several different effectiveness assumptions. | 4/5 |
| 6. How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve? | 1/5 |
| Does the analysis identify a market failure or other systemic problem? NHTSA goes to great lengths to describe the status quo of car crashes - that passengers are more likely to die in a rollover accident and that in a rollover accident passengers are more likely to die if they get ejected from the vehicle. But the analysis does not identify a systemic explanation of why a seemingly cost-effective safety measure is not adopted. Why wouldn't manufacturers voluntarily supply these or other safety measures if they thought consumers were willing to pay for them? To the contrary, the analysis lays out loads of things that car manufacturers are doing voluntarily like investing in glazing technologies to make windows stronger and installing rollover sensors. A majority of cars even have the type of airbags NHTSA says are good. Yet the agency jumps straight to regulating airbag safety tests without ever making a case for why such regulation is necessary. | 0/5 |
| Does the analysis outline a coherent and testable theory that explains why the problem (associated with the outcome above) is systemic rather than anecdotal? No theory. | 0/5 |
| Does the analysis present credible empirical support for the theory? The closest it comes is presentation of statistics showing that the risk of death from total ejection is much higher than the risk from partial or no ejection. This shows the total ejection problem is bigger than others, but not necessarily systemic. | 1/5 |
| Does the analysis adequately assess uncertainty about the existence or size of the problem? Analysis includes some cursory mentions of uncertainty in the beginning as it describes the problem, then addresses some concerns in more detail in the sensitivity analysis later in the RIA. | 1/5 |
| 7. How well does the analysis assess the effectiveness of alternative approaches? | 3/5 |
| Does the analysis enumerate other alternatives to address the problem? The RIA lays out two alternative options. | 3/5 |
| Is the range of alternatives considered narrow (e.g., some exemptions to a regulation) or broad (e.g., performance-based regulation vs. command and control, market mechanisms, nonbinding guidance, information disclosure, addressing any government failures that caused the original problem)? The alternatives are all pretty similar. The chosen alternative is a performance standard that does not mandate particular technologies. However, the current technology most likely to meet the standard is side curtain air bags, and this is assumed for assessing costs. The analysis calculates benefits for several different ways of meeting the standards. Two regulatory alternatives were considered: a standard based on tests at a lower speed for one quadrant of the window, and one that tests only one impact point on the window instead of four. These are narrow tweaks on the performance standard. It seems strange that the RIA did not consider an alternative focused on increasing seat belt use, since most ejections occur to occupants not wearing seat belts. | 2/5 |
| Does the analysis evaluate how alternative approaches would affect the amount of the outcome achieved? Benefits calculated and compared for the two alternatives and the chosen alternative. | 4/5 |
| Does the analysis adequately address the baseline? That is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future? Baseline uses manufacturers' projected model year 2011 sales. RIA acknowledges that projections for 2013, when the regulation goes into effect, would be better, but these do not exist. Baseline adjusts for effects of other rulemakings and some sub-categories the standards might not affect, such as complete ejections of people wearing safety belts. | 3/5 |
| 8. How well does the analysis assess costs and benefits? | 3/5 |
| Does the analysis identify and quantify incremental costs of all alternatives considered? Analysis focuses mainly on material costs and compliance testing costs. NHTSA doesn't think that redesign costs or manufacturing costs will change because the regulation gives the industry such a big lag time to implement the changes, but it's unclear to the reader that such an assumption is believable. It seems a bit arbitrary. | 3/5 |
| Does the analysis identify all expenditures likely to arise as a result of the regulation? Some other potential costs are mentioned but not analyzed. For example, the analysis mentions the possibility of separate air bags for each window. | 3/5 |
| Does the analysis identify how the regulation would likely affect the prices of goods and services? The analysis breaks the cost estimates down to the per-vehicle level and it's something really small, so it's implied that it won't have a big impact on the ultimate price of vehicles. No discussion of how costs would be passed through to consumers, except that the regulatory flexibility analysis asserts that small manufacturers could pass costs through because they produce unique automobiles. | 2/5 |
| Does the analysis examine costs that stem from changes in human behavior as consumers and producers respond to the regulation? Analyis considers changes in producer behavior but not consumer behavior. | 2/5 |
| If costs are uncertain, does the analysis present a range of estimates and/or perform a sensitivity analysis? Sensitivity analysis performed for several cost assumptions. | 4/5 |
| Does the analysis identify the alternative that maximizes net benefits? Net benefits calculated for 3 alternatives, and the option that maximizes net benefits is identified. | 4/5 |
| Does the analysis identify the cost-effectiveness of each alternative considered? Cost per life saved calculated for 3 alternatives. | 4/5 |
| Does the analysis identify all parties who would bear costs and assess the incidence of costs? Analysis discusses some differences based on vehicle or manufacturer -- i.e. how the material costs will be different for cars with two rows vs. three rows -- and reference some small manufacturers. No substantial discussion of cost incidence for consumers vs. automakers. | 2/5 |
| Does the analysis identify all parties who would receive benefits and assess the incidence of benefits? Analysis is specific about what types of crashes they are talking about, the age of the person, whether the person is wearing a seat belt, etc. such that it's reasonably thorough as to who is likely to benefit from the rule. RIA implies that most of the benefits are for individuals who do not wear safety belts, because they are much more likely to be ejected in accidents. But it does not follow up on this obvious indication that different parties may bear the benefits and costs. | 3/5 |
Use | |
| 9. Does the proposed rule or the RIA present evidence that the agency used the analysis? A congressional mandate required this type of regulation. The law did not require the specific aspects of the test. NHTSA appears to have used the anaysis of benefits to find out which alternative would be most effective, based on lab/engineering tests. The range of alternatives is very limited, and some alternatives mentioned in the preamble weren't evaluated in the RIA. | 2/5 |
| 10. Did the agency maximize net benefits or explain why it chose another alternative? NHTSA appears to have chosen the alternative that maximizes net benefits and offers lowest cost per life saved. The range of alternatives was very limited, apparently because the law specified that NHTSA must issue a performance standard, but NHTSA never explicitly explained why it considered such narrow alternatives. | 3/5 |
| 11. Does the proposed rule establish measures and goals that can be used to track the regulation's results in the future? No explicit measures established. Since the RIA projects the number of fatalities and severe injuries with and without the regulation, these could be regarded as targets. | 1/5 |
| 12. Did the agency indicate what data it will use to assess the regulation's performance in the future and establish provisions for doing so? Since the regulation is phased in, it should be possible to track safety record of vehicles that comply and do not comply each year. It is clear that the agency could track the % of cars that meet the standards, crash data, and fatalities over time. | 2/5 |
| Total | 31 / 60 |
Additional details
- Agency
- Department of Transportation
- Regulatory Identification Number
- 2127-AK23
- Agency Name
- Department of Transportation
- Rule Publication Date
- 12/02/2009
- Comment Closing Date
- 02/01/2010