Certification Retrospective Part 2 – Method to our Madness?

Note: this blog post will continue discussing my experiences in CCHIT. The opinions are mine, and do not necessarily reflect the views of CCHIT or Siemens.

In Part 1, I talked about CCHIT giving me the privilege to work with great people over the years. But we didn’t volunteer for CCHIT just to meet people, but to do important work to advance the HIT industry. So what did we accomplish, and what were we thinking as we evolved?

One thing I learned is that no group can please all the people all the time. Sometimes, people wondered why we proposed some criteria and not others, or proposed the timeframes that we did. So we tried to explain these “whys” in a white paper Interoperability, Supplying the Building Blocks for a Patient-centered EHR in mid-2009 which still exists on the website of writer John Morrissey. It described the context for what we were doing and the “methodical march” to increasing levels of information interoperability that we proposed. I don’t think this was well understood by the public, who might have thought some of what we did was arbitrary. But here’s what was in our mind (well, I can only speak for myself – mine at least).

Key principles:

• We aren’t starting with a blank sheet of paper. The existing state of adoption (immature as it might be) must be considered, so we started each year with an “environmental scan” on the state of the HIT industry.
• Adequate time must be provided for EHR products to make the transition to a more desirable state. We aimed for 18 months lead time from our first “signal to the market” (new draft criteria) until certification testing of those criteria.

Given these principles, our methodical march followed the path below.

• Data Capture precedes interoperability. You can’t share what you don’t have! When we started CCHIT, not all EHRs even captured all the data that other systems would want, so functionality criteria (ambulatory, inpatient, ED) needed to capture the data elements that could then be shared. Thus in the first certification year, 2006, there were no interoperability criteria required (though several criteria, such as Lab Results and e-prescribing, were roadmapped for later years).
• Information Liquidity (a term we didn’t use, but which implicit in our criteria) is beneficial even prior to semantic interoperability. Thus, the first clinical summaries that CCHIT certified (Continuity of Care Documents, CCDs) in 2008 required human-readable sections for meds, allergies, problems, etc., but not structured data. We were later criticized for this, as if we didn’t care about structured data, but this was always intended as a stepping stone to later stages (as explained in the white paper). Even today (3 years later than the 2008 criteria), many voices espouse liquidity as a more important and realistic for the near-term than semantic interoperability (“don’t let the perfect be the enemy of the good”).  
• Standardized Content (formats and vocabulary). Human-readability is fine for humans, but doesn’t help most EHR systems to “understand” (semantics) and process the information they receive from others. Because of the existing Tower of Babel syndrome most EHRs have been “confounded” in their ability to share information with other EHRs, and to “consume” (not just display) it. So CCHIT supported the goals of ONC and HITSP to progress toward standardization of vocabularies. But it realized that changing an EHR’s vocabulary overnight wasn’t realistic, and that mapping from one vocabulary to another is not simple or foolproof and must be done carefully for patient safety and other reasons. Thus a “glide path” from narrative to standard vocabularies was roadmapped.
• Discrete Data Import (Consumption). When liquidity and standardized content are established, the EHRs that receive or retrieve clinical data can “do something” with the data, more than just file and display it. As early as 2007 criteria, we published certification criteria for ambulatory EHR discrete import of Laboratory Results that referenced the HITSP-endorsed HL7 2.5.1 Lab Results Reporting Implementation Guide, including a common subset of LOINC observation codes. Thus, many of us were disappointed when the ONC Final Rule for certification, published in 2010, lacked a specific HL7 standard and LOINC requirement, not going as far as what CCHIT had been certifying since 2007. There are 127 CCHIT Certified® ambulatory EHRs that already meet this requirement. For CCDs, the challenge of discrete data import is  greater because of the wider variety of data, and multiple applicable vocabularies. Rather than declaring the problem too big to tackle in one shot, we proposed chipping away a few sections at a time, starting in 2010 with discrete import of medications (using RxNorm as specified by HITSP), allergies (using RxNorm and UNII), Immunizations (using CVX) and demographics, then moving on to problems (using SNOMED-CT). These criteria are a matter of public record, and can be found on the CCHIT website: 2008 criteria. See 2008 Criterion IO-AM 11.06 for example, which originally roadmapped discrete data import for 2009, though the date changed to 2010 in the 2009 criteria (published but then withdrawn)***. You can decide based on the facts whether we were too aggressive, or too laggardly, or about right.

So what’s the point of this retrospective? I’ll sum up lessons learned in the next post. I’ll also speak more about how CCHIT’s interoperability criteria compared to ONC’s criteria.

I’ll also discuss some of the agonizing controversies that we faced, particularly in 2008 and 2009 over issues like standardized transport, discrete data import, data reconciliation turf (is it interoperability or functionality or both?) and the feasibility of requiring SNOMED-CT. We wrestled with each of these, and did our best. We might not have succeeded perfectly in all of them, and our criteria development work was superseded by ONC before we could resolve them all, but we learned much in the methodical march!

David

*** (Long Footnote)

The 2009 Ambulatory Certification criteria, along with the CCHIT 2009 certification program, were not launched, since they were in limbo awaiting decisions on ONC-ATCB certification. But eventually most of these criteria (without the roadmap) were relabeled and launched as the 2011 CCHIT Certified® ambulatory criteria. If the 2009 program had proceeded, Criterion IO-AM 11.06 would have required the following in 2010, adding much specificity beyond what had been proposed in the same criterion in 2008:

The system shall provide the ability to display CCD documents, using a subset of the HITSP C32 specification for the Medication and Immunization History module, file them as intact documents in the EHR, and import the discrete data from one or more of the entries in a structured form into the patient record. The system shall provide the ability to import as discrete data the following data elements from the C32 Medications Module: Free text SIG, free text product name, free text brand name, coded product name, coded brand name, product concentration, status of medication, quantity ordered, order date/time, order expiration date/time, prescription number, quantity dispensed, and fill number. (Note: fill number means the number of fills dispensed, not the number of fills prescribed). The “required if known” (R2) data elements will not always be known, and therefore will not be present in every instance of a C32 document, but if present, the system shall be able to import them

More was explained in the Comments column:

Medication and Immunization History-Coded Subset includes the following content modules of the HITSP C32: Person Information, Healthcare Provider, Medications-Prescriptions and Non-Prescription, Information Source, Comments

The intent for 10 Certification is not to require specific capability to use the imported data but only to show the system is capable of processing discrete data elements from a CCD. In future certification years, more functionality criteria to “use” the data may be added. Fulfillment of the 10 Certification criteria may be demonstrated in a variety of ways, including but not limited to any one of the following:

- Reconciliation functionality that allows CCD data to be compared with existing data, and selectively imported into a list

- A display of data captured from external sources, including the imported discrete data. The physical storage location (database table, file, etc.) is at the vendor’s discretion and does not need to be the same location as the active allergies.

- Creation of an CCD that includes the elements previously imported as discrete data

Vendors may innovate and differentiate their products by how well such data are used, but 10 Certification will not limit vendors to any particular approach

Certification Retrospective -- Part 1

Note: this blog post will discuss my experiences in CCHIT. The opinions are mine, and do not necessarily reflect the views of CCHIT or Siemens.

It’s been a long time since I blogged, even though there have been many HIT topics to blog about, such as Stage 2 meaningful use, the ONC Standards and Interoperability Framework, the ONC Strategic Plan, the NPRM on Medicare Shared Savings (Accountable Care Organizations), and more. I’m getting back on the wagon. One topic I’ve wanted to blog about for a while, even though it’s not a current “hot topic” in the news, is my experience in the Certification Commission for Health Information Technology (CCHIT) prior to the establishment of the ONC certification program. This bubbled to the surface when I read a HISTalk “Time Capsule” posting. So I’ll write another mini-series (probably two or three parts). This first post will simply introduce the “cast of characters” and how they interacted. The next posts will talk about what the Interoperability Workgroup of CCHIT did, and how its work through 2009 related to the ONC-ATCB certification program as we know it today. There are useful lessons learned.

As a matter of disclosure, I’ve been a CCHIT volunteer for more than five years, and am still officially a co-chair of CCHIT’s Interoperability Workgroup, but this group has been inactive for about a year though it still nominally exists. The reason for its inactivity is that it was chartered as a group to propose certification criteria EHRs, a role that was displaced by ONC and the HIT Standards Committee. CCHIT still offers its branded certification program called “CCHIT Certified®” that is completely distinct from the ONC certification for which CCHIT is an authorized testing and certification body (ATCB). The branded program includes criteria that I and many others worked on until 2009. In interoperability, CCHIT is currently performing testing based on the NIST test scripts and some optional tools, but it is not developing any new interoperability criteria at this time.

Industry collaboration – whether in standards organizations or groups like CCHIT – has been a rewarding and mind-expanding aspect of my career. It’s about the people who work together to benefit many more people (e.g., patients and providers), trying to leave their agendas and “organization hats” at the door. So this first post focuses on some facts about the volunteers of CCHIT interoperability.

I found it fascinating to compile a list of the people I’ve served with on Interoperability workgroups at CCHIT. See this link for a table of volunteer names. It’s a pretty diverse and impressive list, IMHO. The list of CCHIT workgroup members has always been publically posted, though I don’t know if it’s preserved historically, so I had to dig through my archives. My apologies if there are some typos in my list. The organizations listed were as of the time the people were on the workgroup, not necessarily where they are now. And occasionally there were people who had to resign for lack of time and a few were replaced but that may not be reflected in my table.

Each year, the Interop Workgroup had between 13 and 24 members (the 24 divided into two groups). Over 60 members served on the workgroup at some time. Of these, 9 were from EHR vendors (only counting vendors of products subject to certification, not vendors of different product types like e-prescribing, lab systems, or HIE tools). The EHR vendor reps were generally active participants, but they didn’t dominate the workgroup. Of course, there are many other interoperability experts who could have been on CCHIT but weren’t, since there was limited space. In that sense, CCHIT was not open like a Standards Development Organization, but it tried to establish a small committed group of volunteers, selected by an application process according to guidelines for stakeholder balance. Sometimes, we knew what we didn’t know, and invited guests to educate us at our meetings, on topics like e-prescribing, vocabularies, and clinical documents.

Sometimes the group was split into an Inpatient Interop WG and and Ambulatory Interop WG (in which case I was on the Inpatient side). In the last year, it divided into an Advanced Interoperability WG and a “regular” Interoperability WG (I co-chaired the latter). Usually a small subset of volunteers would work on specific subsets of criteria (e.g., Lab, eRx, clinical documents) and the entire WG would review and refine.

I have very positive feelings towards these people. They’re some of the finest I’ve worked with, and I am grateful to them. There was a strong sense of collegiality and teamwork, and the sense that we were doing something important. While there were downsides to the group being selected rather than open, that also helped motivate most members since they couldn’t passively leave the work to “someone else” as tends to happen in larger groups. Collaboration was real across different stakeholders and even among vendor competitors. Sure we wouldn’t always agree. Some members thought criteria should be much more aggressive. Others couldn’t understand why features they regarded as only “nice-to-haves” should be mandatory for certification. Sometimes (as co-chair) I’d feel frustrated if I thought someone wasn’t pulling their weight (e.g., not showing up for meetings). It’s inevitable that in volunteer organizations not everyone contributes equally, but everyone had the opportunity.

In acknowledgments, there’s always a risk of omitting someone important, but I especially want to thank Amit Trivedi and Anita Samarth who alternated as CCHIT staff leads for interoperability. Their support was tireless and outstanding: I still remember 4am e-mails from both of them as they worked towards tight deadlines! And I really appreciated the commitment, passion, and the desire to do the right thing, from my fellow-co-chairs such as Dr. Alan Zuckerman, Dr. Pat Hale, and George Robinson.

The Board of Commissioners was mostly “out of sight” for the workgroup. They respected and endorsed the expertise of the volunteers, but would attend our meetings if we needed them (Dr. Leavitt did this several times), offering guidance but not dictating what we should do. I only remember being “overridden” by the Board on one substantive issue.

In the subsequent post(s), I’ll talk more about the interoperability work products that we developed, how it related to standards organizations, ONC, and what I would have done differently if I could start over.

David