Donate Today

Large Aircraft Security Program, Other Aircraft Operator Security Program, and Airport Operator Security Program

Proposed Rule

Score: 38 / 60

RULE SUMMARY

The Transportation Security Administration (TSA) proposes to amend current aviation transportation security regulations to enhance the security of general aviation by expanding the scope of current requirements and by adding new requirements for certain large aircraft operators and airports. The main objectives of the proposed rule are to merge the Partial, Private Charter, and Twelve-Five Standard Security Programs into a Large Aircraft Standard Security Program (LASP); to apply this LASP to general aviation operators using aircraft with a maximum certificated takeoff weight (MTOW) over 12,500 pounds (“large aircraft”); and to enhance the security of these operations. The LASP would have a core component based largely on the current Twelve-Five Security Program which would apply to all operators using large aircraft and two additional components which would incorporate requirements specific to the Private Charter and Twelve-Five All-Cargo Security Programs. So the main objectives of the proposed rule are: (1) To merge the partial, private charter and twelve-five programs into a large aircraft security program and to expand its scope to include general aviation operators using aircraft with a MTOW of over 12,500 pounds and (2) to enhance the security of these operations.

METHODOLOGY

There are twelve criteria within our evaluation within three broad categories: Openness, Analysis and Use. For each criterion, the evaluators assign a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points.

CriterionScore

Openness

1. How easily were the RIA , the proposed rule, and any supplementary materials found online?
The "Regulatory and Economic Analysis" for "Notice of Proposed Rulemaking: Large Aircraft Security Program" is easily accessible on regulations.gov using either the RIN or a keyword search (four clicks to download). Additionally, tsa.gov provides direct links to the proposed rule via its Security Regulations section, where rules are listed by RIN, title, and docket number.
5/5
2. How verifiable are the data used in the analysis?
The data relies on some TSA expert estimations for data that is not verifiable. Other calculations rely on data/estimates from DOT publications rather than peer-reviewed publications. The estimate costs for a 9/11 type come from a non-published report (or at least the RIA doesn't list where it's published). Data for operations costs are better documented.
2/5
3. How verifiable are the models and assumptions used in the analysis?
Assumptions are often based on published reports by DOT and other agencies, rather than peer-reviewed publications (although DOT may have based its studies on peer-reviewed pubications). Others came from "experts" with no way of verification. Cost assumptions and models are better done than benefits, and that may reflect the difficult nature of estimating benefits of averting terrorism. While most documentation is evident, however, accessibility to literature could be improved by providing more direct links.
3/5
4. Was the analysis comprehensible to an informed layperson?
The RIA is clear and understandable. To avoid confusion, the RIA even states that "no attempt is made to precisely replicate the regulatory language" of the proposed rule (1). While some technical jargon is used, all of it is explained with efforts like a detailed abbreviations sheet (6). Although not cited as well as they could be, the various charts and graphs on total annualized costs are also very accessible. Further, especially helpful in illuminating the changes of the proposed rule to the layperson is the chart entitled "Proposed Changes to the Existing Regulatory Framework" (29–30).
5/5

Analysis

5. How well does the analysis identify the desired outcomes and demonstrate that the regulation will achieve them?
3/5
Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life?
Some outcomes are clear, such as avoiding terrorists using planes as weapons preserves life, etc. Other benefits are not linked to outcomes; e.g. it's not clear how putting all programs for large aircraft under one regulatory framework necessarily leads to "better governance of and for stakeholders."
3/5
Does the analysis identify how these outcomes are to be measured?
No, but it points out that the benefits from averting terrorism are likely un-measureable. Instead, the RIA relies on break-even analysis, which is about as good as it can do. Other benefits, such as "better governance," could be measureable, but the RIA does not say how specifically.
3/5
Does the analysis provide a coherent and testable theory showing how the regulation will produce the desired outcomes?
The benefits section outlines the main regulatory changes/additions that LASP will provide in detail. By highlighting elements of LASP like Watch List Matching, Expanding Security, and Operations and Security Program Oversight describing how each will enhance security in detail, the RIA provides a theory showing how the proposed rule will produce the desired outcomes. Testability is difficult/impossible because terrorist attacks averted are not measureable.
3/5
Does the analysis present credible empirical support for the theory?
Previous aircraft regulations are discussed in the history, but no empirical support of their success in affecting outcomes is given.
0/5
Does the analysis adequately assess uncertainty about the outcomes?
Since key data are not available to calculate the risks associated with a potential terrorist attack and the willingness of individuals to pay for this incremental risk reduction (see comment 2A-2), the RIA notes that TSA conducts break-even analyses to supply probable estimates for four different "attack scenarios."
4/5
6. How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve?
2/5
Does the analysis identify a market failure or other systemic problem?
The Introduction and Background section of the RIA asserts the importance of the proposed rule and the TSA more generally on the grounds that "On September 11, 2001, several terrorist attacks were made against the United States that resulted in catastrophic human casualties and property damage." Since national defense measures necessitate federal response, Congress established the TSA and charged it with the responsibility of securing all of the nation's transportation modes. The RIA therefore implies that a market failure exists that prevents individuals and states from assessing threats to transportation and developing policies for national security (18). The RIA could be improved if the market failure/systemic problem was explained more thoroughly.
2/5
Does the analysis outline a coherent and testable theory that explains why the problem (associated with the outcome above) is systemic rather than anecdotal?
Because market failure is not clearly addressed, the RIA provides no direct theory explicitly linking how said federal regulation will lead to improvements in national security above and beyond listing the benefits of LASP.
1/5
Does the analysis present credible empirical support for the theory?
No, the analysis only provides anecdotal evidence of terrorism events.
0/5
Does the analysis adequately assess uncertainty about the existence or size of the problem?
The break-even analysis lists different "attack scenarios" with different economic/life impacts. The likelihoods are not knowable a priori, but the analysis does give a good picture of how much impact aviation terrorism may have in different scenarios. The break-even curves for three scenarios shown in figure 6 could be more useful if axes were labeled; still, it is useful in showing what baseline probability of each attack scenario must exist for LASP to at least break even. The use of these break-even curves is a potential best practice.
4/5
7. How well does the analysis assess the effectiveness of alternative approaches?
4/5
Does the analysis enumerate other alternatives to address the problem?
TSA considers four other alternatives, including maintaining status quo (watch list matching), using TSA inspectors instead of TSA-approved auditors, implementing a Web-based passenger information system without LASP, and implementing a Web-based system in conjunction with LASP. Different aircraft weight thresholds are also considered.
5/5
Is the range of alternatives considered narrow (e.g., some exemptions to a regulation) or broad (e.g., performance-based regulation vs. command and control, market mechanisms, nonbinding guidance, information disclosure, addressing any government failures that caused the original problem)?
Although thoroughly documented, the range of the four alternatives considered is narrow because they all are variations of elements within the proposed or existing rules. Rather than proposing alternatives to the type of regulation or regulation itself, the RIA focuses on alternatives based on changes to particular elements of the process underlying the proposed rule, like watch list matching or aircraft weight thresholds (125). Although two of the four alternatives do posit different processes, they still fall within the regulatory reach of the TSA (129). Such alternative descriptions are only useful when the RIA has established that the proposed rule is actually the most effective means to the end of increased transportation security and the RIA has not made it obvious that this is the case.
2/5
Does the analysis evaluate how alternative approaches would affect the amount of the outcome achieved?
The alternatives and their intermediate outcomes are well-assessed, and their relation to final outcomes is discussed but not really assessed (this may not be possible). For instance, for the "Watch List Matching" alternative, TSA argues that expanding access to the watch list increases the opportunity for the list to be compromised and prevents TSA from actually verifying that watch list vetting is properly conducted (127). For the alternatives proposing to use TSA inspectors instead of approved third-party auditors, the RIA notes that the alternative would decrease effective TSA oversight (134). When considering using a Web-based application for transmission of passenger information instead of the current program, costs and uncertainties of this alternative are mostly addressed (129–133). Lastly though, in terms of Evaluating Different Aircraft Weight Thresholds, the RIA argues that lower weight requirements are less inclusive and the 12,5000 pound MTOW in proposed rule mitigates potential security risk and damage of more large aircraft (135–138).
3/5
Does the analysis adequately address the baseline? That is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future?
The RIA defines the baseline in terms of the “break-even” analysis which indicates the tradeoffs between program cost and program benefits in the form of impact on baseline risk of a significant aviation-related terror attack (12). Since the exact risk of attack we face at the present moment and our exact willingness to pay for a program that decreases said risk is highly uncertain, this method seems like an adequate way to establish the baseline.
4/5
8. How well does the analysis assess costs and benefits?
4/5
Does the analysis identify and quantify incremental costs of all alternatives considered?
The RIA presents very explicit 10-year annualized cost charts (discounted at 3% and/or 7%) for how each aspect of the LASP will affect both current and newly regulated aircraft operators (52–96). Annualized costs include opportunity costs to passengers.
5/5
Does the analysis identify all expenditures likely to arise as a result of the regulation?
All possible expenditures are explained and monetized. The RIA gives a detailed quantification of the impact of LASP requirements on existing and new operators, both aircraft and airport, and expenditures that will be incurred by TSA itself. For instance, newly regulated aircraft operators incur "expenses associated with designating security coordinators and ensuring they meet training requirements, subjecting flight crews to STAs, and controlling access to weapons" (52–3). TSA estimates covered aircraft operators would expend $1.1 billion over 10 years to comply with LASP, discounted at 7% (48). In addition, TSA would incur costs to reach out to covered operators during the implementation stage, "to process security programs and profiles submitted by ... operators," and to conduct inspections to ensure compliance, all of which amounts to expenditures of $834,000 over a ten year period (97). TSA calculates that the 10-year cost of the program to be $1.4 billion, discounted at 7% (47).
5/5
Does the analysis identify how the regulation would likely affect the prices of goods and services?
It's not analyzed, but comment is invited on some specific prices that may change. This could have been done based on previous regulations of airline security. Further, the costs of each aircraft operator contracting with an auditor approved by TSA are based on an assumption that TSA-approved auditors "would charge prices comparable to those currently charged by FAA safety inspectors." TSA requests comments on this assumption, however, including comments on how this requirement may affect the prevailing industry structure and prices (87). A wider discussion of LASP's effect on prices of goods and services would improve the RIA.
2/5
Does the analysis examine costs that stem from changes in human behavior as consumers and producers respond to the regulation?
It is impressive that the RIA considers passenger opportunity costs. According to TSA, passenger opportunity costs derive from two distinct events. The first arises from the "requirement for passengers to provide personal information to the aircraft operator preliminary to the watch list matching process." The second results from those instances when the aircraft operator receives a No-Fly message from the watch list service provider and the flight is delayed while the aircraft operator attempts to resolve the No-Fly status (115). While TSA's quantification of the effects of delays is outstanding, it does not anticipate how passengers may react to increased waiting times. Explanation of possible producer responses is also lacking.
3/5
If costs are uncertain, does the analysis present a range of estimates and/or perform a sensitivity analysis?
A Sensitivity Analysis, provided in appendix A, gives several areas of uncertainty in the cost estimates. For instance, the "most significant source of uncertainty in the cost estimates is the actual number of annual operations conducted by aircraft subject to the proposed rule" (185–193). To capture this uncertainty, a random variable for annual flights was created and TSA ran a Monte Carlo simulation. TSA also simulated the "variability in three major cost elements (which contribute to the total program costs discussed above)." In the remainder of this appendix, the resulting distributions for each cost element are described in turn (190). Some areas of uncertainty aren't recognized, however (such as some assumptions made in the analysis).
4/5
Does the analysis identify the alternative that maximizes net benefits?
In some scenarios, yes. Given the approaches considered and the potential costs of choosing alternative approaches, it seems reasonable to think that the RIA has identified an approach that may (in some scenarios) maximize net benefits.
4/5
Does the analysis identify the cost-effectiveness of each alternative considered?
Yes, in break-even analysis for the main regulation and in the appendix for the alternatives.
5/5
Does the analysis identify all parties who would bear costs and assess the incidence of costs?
The direct costs are well assessed and incidence described in multiple charts and graphs, but some possible indirect costs (due to changes in prices or substitution across modes) are not considered. The exception is opportunity cost to passengers, which is one indirect cost considered.
4/5
Does the analysis identify all parties who would receive benefits and assess the incidence of benefits?
The RIA's Benefits section gives a breakdown of how each aspect of the LASP will increase aviation security and consolidate the regulatory regimen (34–46). Specific assessment of the parties who will receive these benefits is not given, although it is assumed that the general welfare of the United States and government officials will share in these benefits, respectively.
2/5

Use

9. Does the proposed rule or the RIA present evidence that the agency used the analysis?
The Introduction section of the proposed rule implies that it used the RIA to establish the higher weight thresholds of LASP because "incidents involving heavier aircrafts have the potential to lead to greater damages and loss of life under one of the scenarios studied in [their] regulatory impact analysis" (64–92). Also, given the extensive calculations of marginal cost compliance for each participant and carefully calculated benefits of the break even analysis that TSA found in this RIA, it would be surprising if the TSA did not use the results to make major decisions.
4/5
10. Did the agency maximize net benefits or explain why it chose another alternative?
TSA calculates the costs of multiple alternatives and claims that the options it chose are efficient and effective when compared to the other options considered in the RIA. No chart or table explicitly sums up the costs/benefits of options considered.
4/5
11. Does the proposed rule establish measures and goals that can be used to track the regulation's results in the future?
The analysis does not address this topic.
0/5
12. Did the agency indicate what data it will use to assess the regulation's performance in the future and establish provisions for doing so?
The various 10-year annual estimated compliance cost charts can be thought of as data to gauge actual against estimated costs, though the RIA does not explicitly state that TSA will do so. Also, break-even analysis using cost data could be performed with these data.
2/5
 
Total38 / 60

Additional details

Agency
Department of Homeland Security
Regulatory Identification Number
1652-AA53
Agency Name
Department of Homeland Security
Rule Publication Date
10/30/2008