Donate Today

State Fiscal Stabilization Fund Program -- Notice of Proposed Requirements, Definitions, and Approval Criteria

Proposed Rule

Score: 23 / 60

RULE SUMMARY

The Secretary of Education proposes requirements, definitions, and approval criteria for the State Fiscal Stabilization Fund (Stabilization) program. One or more of these requirements, definitions, and approval criteria may be used in awarding funds under this program in fiscal year (FY) 2010. The requirements, definitions, and approval criteria proposed in this notice are based on the assurances regarding education reform that grantees are required to provide in exchange for receiving funds under the Stabilization program. We take this action to specify the data and information that grantees must collect and report with respect to those assurances and to help ensure grantees’ ability to collect and report the required data and information.

METHODOLOGY

There are twelve criteria within our evaluation within three broad categories: Openness, Analysis and Use. For each criterion, the evaluators assign a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points.

CriterionScore

Openness

1. How easily were the RIA , the proposed rule, and any supplementary materials found online?
1810-AB04 can be found on regulations.gov as well as on the Department of Education website. On the Department of Education website, the regulation can be found by entering either a keyword or the RIN into the search box on the top right section of their home page. Also, the regulation can be found in just two clicks: first by clicking on "Ed Recovery Act: Read about stimulus spending," and then by clicking on "State Stabilization Notice" about half way down the page. 1810-AB04 immediately appears.
5/5
2. How verifiable are the data used in the analysis?
Estimates of hours needed to comply with various data requirements are simply stated, not sourced. Other numbers, such as numbers of students or schools, are from identified and linked data sources.
3/5
3. How verifiable are the models and assumptions used in the analysis?
Assumptions about which data states already have and which data they would need to make extra effort to gather are just stated, not sourced. Several studies showing that data-driven decisions can improve student achievement are cited and linked.
2/5
4. Was the analysis comprehensible to an informed layperson?
The language is clear but the analysis isn't complete and is therefore, not quite understandable from step to step. It could have been more thorough, as it is very cursory except for compliance costs. Those calculations and results are pretty straightforward. Existence and size of benefits is nebulous, but this is due to lack of analysis rather than hard-to-understand analysis.
3/5

Analysis

5. How well does the analysis identify the desired outcomes and demonstrate that the regulation will achieve them?
2/5
Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life?
Motivate and improve education reforms by providing better data for assessment, stabilize state finances, avoid reductions in education. The ultimate outcome mentioned is student achievement.
4/5
Does the analysis identify how these outcomes are to be measured?
Student achievement would be the ultimate measure, but it is not calculated or estimated in the RIA. The principal measure in the RIA is whether states furnish data and plans that satisfy criteria the regulation lays out.
2/5
Does the analysis provide a coherent and testable theory showing how the regulation will produce the desired outcomes?
The underlying theory, which is not elaborated in great detail, is that requiring states to furnish data will induce them to gather data; these data will be used to make decisions about reforms, and these decisions will lead to improved student achievement.
1/5
Does the analysis present credible empirical support for the theory?
Several academic studies are cited which show that data-driven reform decisions can improve student acheivement.
2/5
Does the analysis adequately assess uncertainty about the outcomes?
There is no discussion of uncertainty about outcomes.
0/5
6. How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve?
0/5
Does the analysis identify a market failure or other systemic problem?
There is no explanation of why states do not or would not furnish these or other useful data voluntarily. A rationale for requiring the data is especially necessary because states can use federal funding to pay the costs of gathering and processing the data.
0/5
Does the analysis outline a coherent and testable theory that explains why the problem (associated with the outcome above) is systemic rather than anecdotal?
No relevant discussion.
0/5
Does the analysis present credible empirical support for the theory?
No relevant discussion.
0/5
Does the analysis adequately assess uncertainty about the existence or size of the problem?
No relevant discussion.
0/5
7. How well does the analysis assess the effectiveness of alternative approaches?
2/5
Does the analysis enumerate other alternatives to address the problem?
The only option considered is to give funding to states, require them to improve education in the enumerated ways, but don't require them to report data.
2/5
Is the range of alternatives considered narrow (e.g., some exemptions to a regulation) or broad (e.g., performance-based regulation vs. command and control, market mechanisms, nonbinding guidance, information disclosure, addressing any government failures that caused the original problem)?
The only alternative considered is analogous to "no action."
2/5
Does the analysis evaluate how alternative approaches would affect the amount of the outcome achieved?
The analysis asserts that providing the funds without requiring data would be "a missed opportunity for bringing about needed educational reforms at a critical time." No real analysis, such as evaluation of states' incentives to pursue or report on reforms, is provided to back up this claim.
1/5
Does the analysis adequately address the baseline? That is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future?
The analysis acknowledges that some states have better data than others, but it does not really develop a baseline and just asserts that the requirement will improve data availability and (hopefully) student achievement by some unspecified amount.
1/5
8. How well does the analysis assess costs and benefits?
1/5
Does the analysis identify and quantify incremental costs of all alternatives considered?
State costs of gathering/processing/providing data are estimated for each requirement, but only for the option chosen.
3/5
Does the analysis identify all expenditures likely to arise as a result of the regulation?
The regulatory analysis calculates $61.7 million in compliance costs for states, but then it lists only the annual $12.6 billion in transfers to the states in the accounting statement. Since the analysis focuses on the data requirements, it seems more accurate to list the costs as $61.7 million. If the regulation is intended to change behavior beyond just requiring the states to report data, then the costs of those behavior changes (eg, different school reform decisions) should also be included.
3/5
Does the analysis identify how the regulation would likely affect the prices of goods and services?
No relevant discussion.
0/5
Does the analysis examine costs that stem from changes in human behavior as consumers and producers respond to the regulation?
The description of benefits anticipates better results from school reform as a result of better data, but there is no discussion of costs that might arise as people make use of the data to make different decisions. This allows the department to count better school reforms as a benefit with zero additional cost.
0/5
If costs are uncertain, does the analysis present a range of estimates and/or perform a sensitivity analysis?
No relevant discussion.
0/5
Does the analysis identify the alternative that maximizes net benefits?
Benefits and costs of multiple alternatives are not calculated, so this could not be done. The department asserts that benefits exceed costs: "Although these benefits are not easily quantified, the Department believes they will exceed the projected costs."
1/5
Does the analysis identify the cost-effectiveness of each alternative considered?
Benefits and costs of multiple alternatives are not calculated, so this could not be done.
0/5
Does the analysis identify all parties who would bear costs and assess the incidence of costs?
The analysis notes that states can cover the costs using federal funds. It also calculates costs for different types of entities.
3/5
Does the analysis identify all parties who would receive benefits and assess the incidence of benefits?
There is no explicit discussion of incidence. A discussion of equalizing quality of teachers across schools implies that students in high-poverty areas may benefit more, since these are the schools that have fewer highly qualified teachers.
1/5

Use

9. Does the proposed rule or the RIA present evidence that the agency used the analysis?
The decision seems to be driven by research suggesting that data-driven school reforms can improve student achievement. So there is some effect of analysis. But it appears the comparison of benefits with costs played no role in the decision, and the only alternative considered is not analyzed in any depth.
2/5
10. Did the agency maximize net benefits or explain why it chose another alternative?
Net benefits are not calculated. Nothing in the rule or RIA suggests that the agency compared costs with benefits when making its decision, though the department says it believes benefits will exceed costs. Decisions seem driven by a desire for more and better school reforms, but the analysis does not present enough of a basis to claim that the alternative chosen is even the most effective way of achieving the benefits.
1/5
11. Does the proposed rule establish measures and goals that can be used to track the regulation's results in the future?
There are no provisions in the regulation to assess how effective the data requirements are in spurring school reform or increasing student achievement. In some cases, the regulation says the state data must demonstrate state progress to the department's satisfaction, so this might provide some basis for goals for the regulation. The data gathered could be used to establish goals and measures for the regulation.
1/5
12. Did the agency indicate what data it will use to assess the regulation's performance in the future and establish provisions for doing so?
The proposed rule suggests a number of ways the required data might be used for retrospective analysis or otherwise affect decisions in the future: "The provision of better information (on teacher qualifications, teacher and principal evaluation systems, State student longitudinal data systems, State standards and assessment systems, student success in high school and postsecondary education, efforts to turn around low-performing schools, and charter school reforms) to policymakers, educators, parents, and other stakeholders will assist in their efforts to further the reforms. In addition, State reporting of these data will help the Department determine the impact of the unprecedented level of funding made available by the ARRA. Further, the data and plans that States submit will inform Federal education policy, including the upcoming reauthorization of the ESEA." These all involve evaluating effects of school reforms or federal expenditures, not effects of this regulation requiring states to furnish data.
1/5
 
Total23 / 60

Additional details

Agency
Department of Education
Regulatory Identification Number
1810-AB04
Agency Name
Department of Education
Rule Publication Date
07/29/2009
Comment Closing Date
08/28/2009