- | Regulation Regulation
- | Policy Briefs Policy Briefs
- |
Real-Time System Management Information Program
Proposed Rule
Score: 23 / 60
RULE SUMMARY
Section 1201 of the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (SAFETEA–LU) requires the Secretary of Transportation (Secretary) to establish a Real-time System Management Information Program that provides, in all states, the capability to monitor, in real-time, the traffic and travel conditions of the major highways of the United States and to share that data with state and local governments and with the traveling public. This proposed rule would establish minimum parameters and requirements for states to make available and share traffic and travel conditions information via real-time information programs.
METHODOLOGY
There are twelve criteria within our evaluation within three broad categories: Openness, Analysis and Use. For each criterion, the evaluators assign a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points.
| Criterion | Score |
|---|---|
Openness | |
| 1. How easily were the RIA , the proposed rule, and any supplementary materials found online? The NPRM can be found via a keyword or RIN search in regulations.gov. It is mis-filed as a "rule" rather than a "proposed rule." The NPRM says there is a "cost analysis." A benefit-cost analysis is filed in the docket folder as "other." So the NPRM and RIA are in regulations.gov but very difficult to find. The NPRM but not the RIA can be found via a keyword search on the DOT web page. We could not find the RIA until 2011, two-and-a-half years after the NPRM. | 2/5 |
| 2. How verifiable are the data used in the analysis? Some cost data are sourced and linked; some are not. Some numbers appear to be plucked from the air. A table shows sources of data used for benefits; these are sourced and usually linked. An appendix lists miles of road to be covered in each city. | 2/5 |
| 3. How verifiable are the models and assumptions used in the analysis? Not a very complicated "model"; calculations assume that costs and benefits identified in case studies can be extrapolated across all cities and states. Assumptions used to classify cities as low, medium, and high cost are clear but not explicitly justified. Others -- such as operation and maintenance costs -- are justified based on "case studies" and "engineering principles" without further citation. Benefits are extrapolated from a Georgia study that sounds like a plausible starting point. Some benefit assumptions, such as use of national average data rather than city-specific data, seem arbitrary. | 3/5 |
| 4. Was the analysis comprehensible to an informed layperson? Analysis was quite readable with few acronyms. It is clear how the analysis estimated costs based on prior experience. Benefits are extrapolated from the Atlanta study, but one would have to read that study to fully understand how the calculations were performed. | 4/5 |
Analysis | |
| 5. How well does the analysis identify the desired outcomes and demonstrate that the regulation will achieve them? | 3/5 |
| Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life? RIA mentions reduced delays, improved safety, emissions reductions, and fuel savings as the primary benefits of the rule. These are all outcomes linked to quality of life. Other outcomes are suggested (such as measuring the benefits from HOV or setting HOT pricing) but not clearly identified. | 5/5 |
| Does the analysis identify how these outcomes are to be measured? Based on the Atlanta study, the RIA estimates hours of reduced delay, reduced emissions, gasoline savings, and reduced crashes. All of these are monetized. Analysis notes that other benefits travelers might derive from the system are not measured. Estimates are for regulatory requirements that must be satisfied within 2 years, but not for other requirements that must be satisfied within 4 years. | 3/5 |
| Does the analysis provide a coherent and testable theory showing how the regulation will produce the desired outcomes? The RIA asserts the systems will produce benefits but does not offer an explicit theory that explains how mandated implementation of a real-time information system leads to the projected benefits. The NPRM states that the regulation allows drivers to better plan to avoid congestion by giving them advance notice of incidents and road conditions. Aside from a couple references to the 511 system, it is unclear how once the information is available it will be transmitted to drivers or how they will use it. If data measurement is a benefit, then clearly measuring data will produce that outcome but the benefits of that are not made clear. | 2/5 |
| Does the analysis present credible empirical support for the theory? The principal empirical evidence is a study of Atlanta's real-time information system. Benefit estimates are calculated using the results of the Atlanta study. There is no explicit justification for why the Atlanta results can be generalized or whether those results reflect factors unique to Atlanta. | 2/5 |
| Does the analysis adequately assess uncertainty about the outcomes? RIA acknowledges an important confounding factor in the Georgia program was the presence of rapid response emergency vehicles. It performs a sensitivity analysis assuming the information program is responsible for only 1/3 and 1/10 of the benefits. A statement at the end lists numerous uncertainties that would affect the size of benefits, but no further analysis attempts to assess the size of these uncertainties or their effects. No attempt to measure uncertainty arising from different outcomes in varying cities. | 2/5 |
| 6. How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve? | 1/5 |
| Does the analysis identify a market failure or other systemic problem? The NPRM contains a succinct explanation of traffic congestion: demand outstrips supply because payment is rarely matched with use. While this identifies the institutional problem that leads to congestion, it does not explain why real-time information is under-provided. No discussion in the RIA of a market or government failure that creates the need for this particular regulation, or why a federal mandate is required. | 2/5 |
| Does the analysis outline a coherent and testable theory that explains why the problem (associated with the outcome above) is systemic rather than anecdotal? The only theory of a systemic problem is the pricing discussion in the NPRM. But there is no theory explaining why real-time information is under-provided. The NPRM explains why real-time information is valuable to travelers but does not explain why a federal regulation is necessary, rather than state/local information initiatives or private solutions (eg, traffic updates via GPS). | 2/5 |
| Does the analysis present credible empirical support for the theory? NPRM cites a university study and DOT project that demonstrate congestion is increasing. There is evidence that the systems produce benefits (described above), but no evidence cited supporting a theory of market or government failure that would motivate the regulation. | 1/5 |
| Does the analysis adequately assess uncertainty about the existence or size of the problem? No relevant discussion. | 0/5 |
| 7. How well does the analysis assess the effectiveness of alternative approaches? | 1/5 |
| Does the analysis enumerate other alternatives to address the problem? The NPRM mentions that DOT considered alternative approaches related to phasing in of the requirements and the information content required. There is no further elaboration, and these are not analyzed at all in the RIA. | 1/5 |
| Is the range of alternatives considered narrow (e.g., some exemptions to a regulation) or broad (e.g., performance-based regulation vs. command and control, market mechanisms, nonbinding guidance, information disclosure, addressing any government failures that caused the original problem)? Alternatives such as requiring information be available for public transport are somewhat broad (although not really alternatives to the rule proposed) but this is never considered in the RIA and only mentioned in the NPRM. | 1/5 |
| Does the analysis evaluate how alternative approaches would affect the amount of the outcome achieved? No alternatives analyzed in the RIA. | 0/5 |
| Does the analysis adequately address the baseline? That is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future? RIA uses data from a tracking survey to determine which areas already have some or almost all of the required tracking capability in place. No discussion of whether states or localities would continue to deploy or expand deployment of this technology in the absence of the mandate. The analysis attributes all benefits of such a system to the rule but not all costs of implementing the system because some states have already partially done so. It seems reasonable to say that if some states have already partially implemented a system then they would have received the benefits from doing so regardless of the rule. | 1/5 |
| 8. How well does the analysis assess costs and benefits? | 1/5 |
| Does the analysis identify and quantify incremental costs of all alternatives considered? Analysis calculates capital and operating costs for cities and states for the chosen alternative only, for the first 2 years' requirements only. | 3/5 |
| Does the analysis identify all expenditures likely to arise as a result of the regulation? Expenditures appear to be fairly comprehensive for the costs the RIA estimates. However, the analysis considers only the requirements for the first two years, not additional measures in years 3 and 4. It was also unclear to what extent the cost already incurred by states may have been a response to anticipated future regulation. | 3/5 |
| Does the analysis identify how the regulation would likely affect the prices of goods and services? Taxes will presumably fund these expenditures. The NPRM notes states and localities can use federal highway funds but offers no further analysis. | 1/5 |
| Does the analysis examine costs that stem from changes in human behavior as consumers and producers respond to the regulation? No discussion of these costs. In some cases these might be significant if the expenditures are state/local funds that would otherwise be used for important health/safety programs. The reduction in total cost of driving would presumably lead people to rely more on cars, live further from cities, perhaps even own less fuel efficient cars. | 0/5 |
| If costs are uncertain, does the analysis present a range of estimates and/or perform a sensitivity analysis? Analysis acknowledges substantial uncertainty because no technology is mandated. Cost calculations assume the most expensive technology to be conservative. | 2/5 |
| Does the analysis identify the alternative that maximizes net benefits? RIA calculates net benefits of the alternative chosen under three different assumptions about the size of total benefits; net benefits are always positive. No discussion of alternatives. | 1/5 |
| Does the analysis identify the cost-effectiveness of each alternative considered? No explicit discussion of cost-effectiveness defined as outcomes/cost. There are multiple outcomes, and the RIA does calculate a benefit/cost ratio for the chosen alternative. | 1/5 |
| Does the analysis identify all parties who would bear costs and assess the incidence of costs? The NPRM mentions several categories of federal funds that states could use to implement these systems. It is not clear if this means the initiative will be wholly funded by the federal government. | 1/5 |
| Does the analysis identify all parties who would receive benefits and assess the incidence of benefits? Implicitly, motorists appear to receive most of the benefits, and the analysis implies that differences in circumstances may lead to higher or lower benefits in different locations. No further discussion of the incidence of benefits. | 1/5 |
Use | |
| 9. Does the proposed rule or the RIA present evidence that the agency used the analysis? The NPRM states several times that the regulation is required by legislation, so the analysis did not motivate the decision to issue the regulation. The calculation of benefits and costs appears to have been done after the decisions about the regulation were made, more to document the consequences of the decision than to inform the decision. | 1/5 |
| 10. Did the agency maximize net benefits or explain why it chose another alternative? Net benefits of the chosen alternative were calculated, but no alternatives were considered, so DOT did not have information on the net benefits of alternatives. DOT clearly chose to issue the regulation because the law directed it to, not because the analysis showed net benefits. | 2/5 |
| 11. Does the proposed rule establish measures and goals that can be used to track the regulation's results in the future? No commitment to goals or measures, but estimates of benefits and costs in the RIA could be used as goals. | 1/5 |
| 12. Did the agency indicate what data it will use to assess the regulation's performance in the future and establish provisions for doing so? The RIA conveys the impression that DOT has access to data that could be used to monitor the regulation's results and compare them to the RIA's predictions. | 2/5 |
| Total | 23 / 60 |
Additional details
- Agency
- Department of Transportation
- Regulatory Identification Number
- 2125-AF19
- Agency Name
- Department of Transportation
- Rule Publication Date
- 01/14/2009
- Comment Closing Date
- 04/14/2009