Technical Expert Panel on Electronic Health Record Data Quality Best Practices for Increased Scientific Acceptability

We are pleased to be able to comment on the National Quality Forum’s draft report titled “Technical Expert Panel on Electronic Health Record Data Quality Best Practices for Increased Scientific Acceptability.”

The Mercatus Center at George Mason University is dedicated to advancing knowledge relevant to current policy debates. Toward this end, its scholars conduct independent, nonpartisan analyses of agencies’ rules and proposals. With that in mind, this comment does not represent the views of any particular affected party or special interest group.

Electronic health records (EHRs) will be an essential component of healthcare in the future—a means of extending the capacity of the healthcare system and increasing the positive effect of that system on the health of the American population. To fulfill that promise, however, EHRs will have to evolve far beyond their current state. Today’s EHRs focus mainly on facilitating billing, not on enabling clinical applications of use to patients and providers. One can argue that contemporary EHRs are more of a hindrance than a help to care—an often-frustrating bureaucratic artifact that encumbers a healthcare provider’s time and distracts from the task of care.

If they are to fulfill their promise, future EHRs will need to incorporate broader data sources, including patient input. Entries should include more natural language, in contrast to today’s highly structured drop-down menus and checkboxes. The data will have to be more portable across platforms and applications—the much-vaunted, but elusive, interoperability. In addition, EHRs will have to meet two market tests absent in today’s environment: do patients and providers find the EHRs useful and use them voluntarily?

The National Quality Forum’s draft report makes a positive contribution in this direction and deserves a great deal of praise concerning its insights and thoroughness. Many of the draft report’s recommendations deserve a place in any future manifestation of EHRs. However, the draft report does not fully escape the environment that is the downfall of today’s EHRs. In our view, a more useful regime of EHRs would rely less on mandates and top-down specification of data and more on bottom-up, organically designed data configurations that constantly adapt and evolve in accordance with input by patients and healthcare providers. This approach will require greater reliance on natural language inputs, broad interoperability, and adaptive structure of data and user interfaces.

High Points of the Draft Report

We share the task force’s overall goal: to improve the quality, availability, and utility of healthcare data contained in the nation’s EHRs, and there is much to like in the draft report. The first paragraph makes excellent points, which are echoed throughout the report. In this comment, we address the first four points made in that initial paragraph. The first of those reads, “One of the promises of electronic health records (EHRs) is that they enable automated clinical quality measure reporting.” We view vastly improved, restructured EHRs as essential to the task of improving the efficacy and efficiency of healthcare, improving health and doing so in more cost-effective ways than the status quo.

The second point states, “EHR systems are primarily designed to support patient care and billing, not necessarily capture additional data . . . to support quality measurement.” We would amend this statement by stating that today’s EHRs are primarily designed to support billing and record keeping and only secondarily designed for patient care. In general, patients have little interaction with their EHRs, and to a large extent healthcare providers detest them. EHRs have been cited as a significant motivation for healthcare provider retirements. We stress that this is a function of how EHRs are structured today. Designed differently, EHRs could serve patients and healthcare providers alike and still perform their role with respect to billing and record keeping.

The third point states, “However, since EHR data routinely collected for patient care can be used for clinical quality measures, they can be reused to reduce provider burden associated with public reporting and value-based purchasing programs.” We agree that this reuse is a desirable goal, but one not well served by current EHRs. At this point, rather than reducing healthcare provider burdens, EHRs often are a provider burden—and a significant one at that.

The fourth point states: “Despite high adoption rates in multiple care settings, the promises of EHRs haven’t yet been fully realized because of considerable variation in data quality, due to a number of factors . . . .” Data quality is certainly one problem with present-day EHRs, but they have other serious problems. EHRs tend to create fragmented data lakes; that is, a given patient is associated with numerous disconnected EHR programs in the possession of different healthcare organizations who all may document information differently. The user interfaces for healthcare providers are often cumbersome, with rigidly defined data fields or simple free text. The ability of current EHRs to interface with nontraditional data sources, such as fitness watches and other remote telemetry, is limited or nonexistent. Patients have virtually no way to input data into their EHRs, resulting in EHRs that include data points from only the brief periods when the patient is actually in the presence of a healthcare provider. Nor can patients easily dispute the data entered by the healthcare provider, leaving them further disenfranchised from their own record.

The much-vaunted goal of interoperability has fallen far short; moving data among a patient’s different healthcare providers can be difficult or impossible. In March 2020, Katie Keith posted on the Health Affairs blog, “Despite gains in many providers moving from paper records to electronic health records (EHRs) over the past decade, interoperability has been an ongoing source of frustration for providers and patients alike.” That post was in part a response to the release of two official documents addressing these problems: a 474-page final rule from the Centers for Medicare and Medicaid Services (CMS) and a 1,244-page final rule from the Office of the National Coordinator for Health Information Technology within the Department of Health and Human Services.

The most vivid and iconic commentary on the failure of EHRs and interoperability is the fact that the atavistic fax machine remains a critical tool in medicine.

The draft report is heavily concerned with the utility of EHRs in producing aggregate data across the patient population. All the shortcomings we have described affect the capacity to accumulate useful population data. We suggest as a general principle that if patients do not provide regular input to EHRs, if patients rarely see or use output from EHRs, and if healthcare providers find EHRs to be a stumbling block rather than a tool, then the usefulness and quality of the data will never remotely approach the potential of the technology.

The draft report makes some valuable contributions. It discusses the potential value of unstructured natural language inputs into EHRs. In addition, as it should, it discusses the challenges involved in converting such inputs into coherent, accessible information for a given patient or for a population. By contrast, quite a bit of the draft report is devoted to further refining structured data. Both the use of unstructured natural language and the refinement of structured data are worthy goals, but there is some danger of overemphasizing the second at the expense of the first. Unstructured natural language is innately more valuable in supporting positive provider–patient interactions.

The draft report includes ample discussion of incentives designed to encourage implementation and innovation. We have some concerns that many of these ideas require someone in central authority to prospectively judge which actors will best advance the development and implementation of EHRs. For an analogy, imagine a program in the mid-1970s, pouring development funding onto Data General and Honeywell rather than Microsoft or Apple. Current experts on and users of clinical data may not be the ideal agents for envisioning the future. As information technology has showed over the past 50 years, the greatest innovations tend to come from unexpected innovators in unexpected places, with advancements growing rapidly and organically from constant interaction among users and innovators. On January 9, 2007, Steve Jobs, introducing the first iPhone, said, “Every once in a while, a revolutionary product comes along that changes everything,” and then pronounced his new device such a product. He was correct, and no one could have imagined that panels of government officials, academics, and established corporate leaders would ever have envisioned anything remotely resembling the iPhone—or identified in advance that Jobs would be the one to produce this world-changing product.

Our Recommendations

In 2016, two of us (Graboyes and Bryan) proposed a tentative set of principles for reimagining EHRs. We went so far as to term our vision digital health biographies (DHBs) because of the significant revulsion by healthcare providers at the very notion of EHRs. We thought this rebranding could help avoid many of the preconceived notions about current and future EHRs. In a larger work in 2018, we tweak and expand upon those principles, based on our conversations and research in the interim. We plan to revisit these principles again soon in an upcoming research paper. No doubt, our thinking on some will have changed. Nevertheless, we think the list presented in the 2018 paper still holds up rather well, at least as a conversation starter. We present those principles here verbatim from the 2018 paper:

1. As a default, patients, not doctors, should own the DHBs and the data contained within them.
2. Each patient should have precisely one DHB.
3. A patient’s DHB should incorporate data from multiple providers—primary care physicians, specialists, hospitals, nurse practitioners, emergency rooms, pharmacists, therapists, and so on.
4. The DHB should also incorporate data from wearable telemetry such as Fitbits, insulin pumps, and heart monitors.
5. The DHBs should incorporate subjective data entered by patients, including family history, childhood illness recollections, fears, and feelings.
6. To the greatest extent possible, data entry should use natural language (ordinary spoken or written sentences) rather than structured queries (such as drop-down menus).
7. Machine learning capabilities should extract and organize output for specific users, limiting the output as much as possible to the needs of each specific provider or of the patient.
8. Input and output should be recognized as very different functions that require different software, allowing vendors at both ends of the system to compete on the basis of functionality and aesthetics.
9. A common protocol or protocols should be set up to minimize the cost and difficulty of shifting from one input or output vendor to another.
10. To maximize competition among vendors, the government should not mandate or subsidize any particular vendors or data requirements.
11. DHB usage should be voluntary on the part of healthcare providers so that the systems must continually prove their worth.
12. The prime motivation of DHBs should be improved patient health and provider efficiency.

In a key takeaway from that paper, we cite the federal government’s role in creating the internet as a potential model to emulate: “The model for this principle lies in the shift in the early 1990s from the federal government’s tightly controlled ARPANET [Advanced Research Projects Agency Network] to the wide-open internet. The government made a set of interoperability protocols available to the public but did not mandate or heavily tilt the market toward the adoption of those protocols. The code was merely available for web developers and users. Adoption was widespread because the code worked, it was relatively unobtrusive, and these protocols passed the test of the market. The question of whether clinical-quality EHRs can emerge from decentralized processes rather than from a centralized command may be the most contentious point in determining the success or failure of EHRs or DHBs.”

Our paper also includes a number of caveats to these principles. When writing our DHB paper, the two of us (Graboyes and Bryan) were unaware of the work of our coauthor on this comment (Berkowitz). In a 2010 presentation at the Mayo Clinic, Lyle Berkowitz shared ideas that anticipated some of Graboyes and Bryan’s ideas on DHBs. In the Mayo Clinic speech, Berkowitz stated that EHRs were “dead.” (He actually used the related term “electronic medical record,” or EMR, though we will continue to use EHR here.) Berkowitz attributed the failure, in part, to EHR designers doing what doctors had asked them to do, which was to make the EHRs “look like paper.” Speaking to information technology specialists behind the EHRs, he said, “You created something that looked like Word, that looked like Excel, tried to make it look like paper. Not much innovation there.” These stylistic features, the three of us argue, are as important as the structure of the data. On the basis of Berkowitz’s work, some of our additional thoughts include the following:

• Use of the Health Information Technology for Economic and Clinical Health (HITECH) Act to focus on rewarding only features, functions, and outcomes was a mistake. The process should have started with insistence on a common data model so that (a) all the systems could communicate with each other and (b) the focus of EHR vendors could be on creating better top-level (user interface) systems versus creating and maintaining their own proprietary data models.
• Because a common data model does not exist (yet), the initial focus should be on how to harmonize all data into a single, unified national database that can then be used for measurement, reporting, and so on. This approach is technically possible, and various groups are doing it to some extent, but a true national plan is lacking.
• There is a tradeoff between trying to capture data in a structured way up front and using simple free text; it is more difficult and takes more time and thus is costlier. Meanwhile, natural language processing and artificial intelligence are enabling an easier conversion of free text into structured data elements.
• For the topic at hand, one must clarify what problem is being solved (i.e., this more about sharing data, or reporting out data, and so on?). Then, one must develop the proper incentives around that approach, and let the technology experts help make it happen.
• The structure of the data must be flexible and evolutionary. Data should not be pigeonholed into an immutable structure.

For financing innovation, we would urge that consideration be given to retrospective prizes, as opposed to prospective grants that often dictate to the innovator. Perhaps the best-known of such prizes today are the XPRIZEs, awarded by the XPRIZE Foundation. (Currently, the foundation is offering a number of prizes to be given to those who meet certain research and development goals with respect to the COVID-19 pandemic.) With such programs, a goal is stated up front, and money is awarded retroactively to whomever achieves the goal first. Thus, central authorities do not need to guess ahead of time the identity of the best innovator, as they must with prospective grants. Such prizes have been around since at least Great Britain’s Longitude Prize of 1714. In this vein, Congress created the Eureka Prizes as part of the 21st Century Cures Act.

Summary

We appreciate the goals laid out in the National Quality Forum draft report and the thought and detail that went into its production. However, we also advise CMS to focus on ways to enable decentralized innovators to autonomously develop the means for improving the efficacy, efficiency, and proliferation of EHRs. This approach requires flexibility, interoperability, and a highly competitive environment, with both traditional and nontraditional agents offering innovations from the bottom up. Such an environment is well served by ensuring that patients and providers have the capacity to accept or reject products. This organic, evolutionary method contrasts with previous top-down approaches involving encyclopedic specifications and heavy-handed mandates