Score: 33 / 60
RULE SUMMARY
This proposed rule would modify two of the medical data code set standards adopted in the Transactions and Code Sets final rule published in the Federal Register. It would also implement certain provisions of the Administrative Simplification subtitle of the Health Insurance Portability and Accountability Act (HIPAA) of 1996.
METHODOLOGY
There are twelve criteria within our evaluation within three broad categories: Openness, Analysis and Use. For each criterion, the evaluators assign a score ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices). Thus, each analysis has the opportunity to earn between 0 and 60 points.
Criterion | Score |
---|---|
Openness | |
1. How easily were the RIA , the proposed rule, and any supplementary materials found online? The RIA could not be found on the HHS website via links or search. It only appears on regulations.gov using an RIN search, not a keyword search. | 2/5 |
2. How verifiable are the data used in the analysis? Raw data and findings from studies are cited, linked, and from credible sources such as the US Census Bureau. The data from the working group are discussed and summarized in tables but not linked via url. Overall, the analysis does a good job of linking and using real studies. | 4/5 |
3. How verifiable are the models and assumptions used in the analysis? The analysis is primarily based on studies by RAND and Robert Nolan, which appear to be credible and carefully done. Links to both are provided. The analysis gives sensible reasons for relying less heavily on a report estimating costs commissioned by health insurers, though it reaches similar results in some cases. | 5/5 |
4. Was the analysis comprehensible to an informed layperson? The analysis is explained well. It is clear how they reached the results. The principal detractor from the score is that there are many acronyms and a lot of specialized language. | 4/5 |
Analysis | |
5. How well does the analysis identify the desired outcomes and demonstrate that the regulation will achieve them? | 4/5 |
Does the analysis clearly identify ultimate outcomes that affect citizens’ quality of life? It identifies six outcomes (benefits) from the RAND study: more accurate payments for new procedures, fewer rejected claims, fewer improper claims, better understanding of new procedures, improved disease management, better understanding of health conditions and health care outcomes, and harmonization of disease monitoring and reporting worldwide. | 5/5 |
Does the analysis identify how these outcomes are to be measured? The first four are measured and monetized. The fifth is measured but not monetized. The last is not measured or monetized. | 4/5 |
Does the analysis provide a coherent and testable theory showing how the regulation will produce the desired outcomes? Mostly the effects are just asserted with references to sources. | 3/5 |
Does the analysis present credible empirical support for the theory? The analysis uses two prominent studies with url links, uses other studies when needed, and established a cross-agency working group. The working group seems to be an unusual but clever way to go about benefit-cost analysis and does a much better job than other RIAs. | 3/5 |
Does the analysis adequately assess uncertainty about the outcomes? A range of benefit estimates is provided, based on a review of the two primary studies and supplemental information from others. | 3/5 |
6. How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve? | 1/5 |
Does the analysis identify a market failure or other systemic problem? The current system is running out of room for new codes and probably leads to inaccuracies. Since HHS designed the previous system, it's up to HHS to change it. | 3/5 |
Does the analysis outline a coherent and testable theory that explains why the problem (associated with the outcome above) is systemic rather than anecdotal? Though implicit in the above explanation, it is not spelled out in any detail. | 1/5 |
Does the analysis present credible empirical support for the theory? Mostly just assumed/asserted, but plausible given the above entry. | 1/5 |
Does the analysis adequately assess uncertainty about the existence or size of the problem? Uncertainty is indirectly acknowledged in the calculation of ranges for benefits, but the existence of problem(s) is taken as certain. | 0/5 |
7. How well does the analysis assess the effectiveness of alternative approaches? | 3/5 |
Does the analysis enumerate other alternatives to address the problem? The analysis lists the alternatives considered, gives more detail to them in the regulatory flexibility analysis, and briefly discusses their costs or inefficiencies. Examples include: continue to use current system, delay introduction of new system, use other coding systems, and wait until next generation system is developed. | 4/5 |
Is the range of alternatives considered narrow (e.g., some exemptions to a regulation) or broad (e.g., performance-based regulation vs. command and control, market mechanisms, nonbinding guidance, information disclosure, addressing any government failures that caused the original problem)? The analysis discusses options that are moderately different. The options are alternative federally required coding sets, but not alternatives to federally required coding sets. | 3/5 |
Does the analysis evaluate how alternative approaches would affect the amount of the outcome achieved? A brief discussion rules out alternatives based on effectiveness, but there is not much in-depth analysis or calculation of the amount of benefit the alternatives would produce. The discussion does note a GAO study that was critical of one of the options. | 2/5 |
Does the analysis adequately address the baseline? That is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future? The analysis never really establishes a baseline, but one can gather the assumption that it is the current coding system. It assumes no one will come up with a solution, which might be reasonable, but not much effort is spent justifying it. | 1/5 |
8. How well does the analysis assess costs and benefits? | 2/5 |
Does the analysis identify and quantify incremental costs of all alternatives considered? The analysis quantifies implementation/compliance costs for federal government and private entities. It does not examine costs of alternatives. There's a reasonably thorough analysis of costs of the rule, but not of other options. | 3/5 |
Does the analysis identify all expenditures likely to arise as a result of the regulation? Seems pretty comprehensive; based on several previous analyses of costs undertaken by outside parties. | 3/5 |
Does the analysis identify how the regulation would likely affect the prices of goods and services? Costs are only extended to the providers and payers, but not to any effects to the actual patients themselves. That would have been very informative. | 1/5 |
Does the analysis examine costs that stem from changes in human behavior as consumers and producers respond to the regulation? There are a few limited assumptions about how the change in coding categories might affect the size of costs or benefits. But there is no analysis of how the increased costs might filter through and affect behavior, or of any perverse behavioral consequences created by the adoption of the new classification system. All health care entities will be affected and they have to figure out a way to deal with the new codes; it doesn't talk about whether some will switch to other ways of using the codes and billing systems. In some areas the RIA briefly talks about productivity minutes lost, number of procedures affected, and person-year training time, but it is inconsistent and mildly informative. | 2/5 |
If costs are uncertain, does the analysis present a range of estimates and/or perform a sensitivity analysis? Ranges are presented but not extensively justified. Brief reasons for some judgment calls are given. | 3/5 |
Does the analysis identify the alternative that maximizes net benefits? The analysis dismisses alternatives with short arguments that they will not achieve benefits as large, or as soon, or may have higher costs. It only gives net benefits for the rule. | 2/5 |
Does the analysis identify the cost-effectiveness of each alternative considered? It asserts that it adopted the most cost-effective alternative, but does not calculate the monetized net benefits of the other options. | 1/5 |
Does the analysis identify all parties who would bear costs and assess the incidence of costs? It estimates costs for different types of providers/suppliers, but not for patients. | 3/5 |
Does the analysis identify all parties who would receive benefits and assess the incidence of benefits? The benefits presented are society-wide; the analysis sometimes identifies the particular part of industry where a benefit may occur. | 1/5 |
Use | |
9. Does the proposed rule or the RIA present evidence that the agency used the analysis? It sounds like the pre-existing studies of costs and benefits may have prompted HHS to pursue the chosen alternative, and the benefit-cost analysis confirmed this decision. The chosen alternative seems to have already been chosen when the RIA was written. When a non-monetized benefit of international cooperation is used, that is the option that will be picked. Additionally, benefit-cost analysis was not done fully on any of the alternatives. | 3/5 |
10. Did the agency maximize net benefits or explain why it chose another alternative? The agency claims to have rejected alternatives because they would not produce benefits as large or as soon, but the analysis did not provide calculations to back this up. HHS explicitly asserts it chose the most cost-effective alternative. | 2/5 |
11. Does the proposed rule establish measures and goals that can be used to track the regulation's results in the future? No measures or goals are established, but the RIA's list of benefits could be used to develop measures and goals. | 1/5 |
12. Did the agency indicate what data it will use to assess the regulation's performance in the future and establish provisions for doing so? HHS mentions that new, more accurate codes could be useful for research. However, this is not the same thing as research evaluating the effects of the change in codes. It appears that the effects of this regulation on many of the benefits could be tracked by analyzing claims data CMS has access to. | 2/5 |
Total | 33 / 60 |
Additional details
- Agency
- Department of Health and Human Services
- Regulatory Identification Number
- 0958-AN25
- Agency Name
- Department of Health and Human Services
- Rule Publication Date
- 08/22/2008