Patient-Centeredness: will "Direct-only" get us there?

I read that the HIT Standards Committee recommended that Direct (SMTP, S/MIME secure email) be the only transport protocol for certification. While on the one hand this sounds like a good step toward parsimony (similar to having only one vocabulary for problems or one Consolidated CDA standard for summary of care records), I think it’s different and has some unintended consequences against the patient-centered view that can be advanced with HIEs.

Those of us who worked on Direct recognized its strength for particular “push” use cases. Many also felt strongly that Direct needed a bridge to the HIE world in terms of protocols (hence the XDR and XDM for Direct Messaging specification). The Direct Project Overview made it very clear that Direct “push” is not intended to solve every use case but to coexist with other forms of exchange including “pull” as well explained in this EHRA whitepaper.

This is easy to understand if we look at how businesses are run, and how people manage their personal data. Email push is highly beneficial and essential in my business and personal life. But it’s not all there is. At work, when people collaborate on projects (as providers ought to  collaborate in patient care), they store information in shared resources. They don’t rely upon the project documents existing in every individual’s email or personal folders! That would lead to waste and confusion. Instead, there are repositories that handle version-controlled documents, source code, etc. Wiki pages rather than emails can be used to record the shared project experience. Emails are great for initial notification, but not for the ongoing management of the information. And even individuals don’t email their photos to everyone in their address book: they share them in a single place (like Facebook) from which trusted friends can pull (view and download) them. Information exchange  via Direct should not “push out of the picture” information sharing that benefits the patient and provider community.

HIT SC’s recommendation of a single protocol for certification is fine as a “minimum requirement.” But I’m very concerned that certifying only Direct, if accepted by ONC, could bias providers toward Direct to the detriment of other types of exchange. CMS’ Incentive Program proposes that meaningful use measures only count transmissions to providers using the standards included in certification (page 13708 of the NPRM). This would have the unintended consequence of discouraging sharing through HIEs (at any level), that use IHE profiles such as XDS, XDR, XCA, etc., as most use these instead of Direct. Those profiles require sufficient metadata to support queries against a patient-centered document registry, so that providers can find what they’re looking for. Direct by itself does not provide or require those metadata, so even if Direct were used to push documents to an HIE, the HIE would be hard pressed to correctly index and file the documents.

I’m not criticizing Direct for what it is. I am saying that it was not designed to handle all exchanges, and that overemphasizing it as the only “meaningful use” method that “counts” will have negative impacts on HIEs that provide a patient-centered aggregate view. Point-to-point transmissions don’t provide such a view since each provider’s view from “push alone” is only a “silo” unless everyone copies everyone else on every document (the dreaded “reply to all” email) – not a good idea! While HIEs aren’t required for MU, they should not be discriminated against either. CMS regulations incenting Direct only (implicitly disincenting HIE) could harm progress toward patient-centricity. That would be like a business rewarding people for sending their documents via email but not rewarding them for sharing controlled versions of those documents in a repository. Between ONC and CMS, I hope this problem can be avoided by focusing on the real objective – providers and patients sharing data electronically, without CMS measures counting only Direct SMTP exchanges as qualifying for meaningful use. “Pull” exchanges from an HIE or “push” exchanges using other protocols (e.g., XDR SOAP) should also count toward MU measures. And I hope that HHS goes beyond “permitting” HIEs for MU, but that they will continue to invest in and work with others (e.g., states) on a national healthcare infrastructure (NwHIN Exchange as it evolves) to enable a patient-centered, cross-provider view of data as a public good.

Meaningful Use and Consolidated CDA

I invite anyone who is interested in commenting on the ONC Stage 2 Standards and Certification Rule, and/or the HL7 Consolidated CDA standard, to visit this page within the ONC S&I Framework.

http://wiki.siframework.org/ToC+MU+Analysis

As you know Meaningful Use 2 has specified Consolidated CDA and the data to be exchanged with providers (upon transitions of care) and with patients (view, download, and transmit; clinical summaries after visits). But the mapping of the MU2 data to consolidated CDA has been fuzzy, so the work on that page is an attempt to clarify and help "harmonize" ONC's rule with the standard it references. I have a spreadsheet posted that explains this, and Keith Boone does too (and Keith has written extensively about this on his Healthcare IT Standards blog).

The time is short, as comments to ONC are due by early May. We are conducting this analysis via an open process facilitated by ONC S&I, in cooperation with HL7 Structured Documents workgroup.

David

Meaningful Use Seeking Standards!

Update: 10 minutes after I posted below, the ONC NPRM was issued!
================================================

Hello! I, like many of you, are in the midst of reviewing the Medicare and Medicaid Programs; EHR Incentive Program - Stage 2 Proposed Rule (NPRM) on Meaningful Use Stage 2 (MU2), which was published late yesterday afternoon but was probably not seen by most people until today. The reason I titled this post as I did was because MU2 needs to be “married” to the Standards, but we’re all limited by the fact that the ONC NPRM on Standards and Certification Criteria was not published on the same day as the CMS NPRM. We’re still waiting... So for MU 2, we don’t have the actual standards in our hands yet, which we need in order to understand the criteria and assess their effects on providers, vendors, and others. Since I’ve been following the HIT Standards Committee for months, and there were sneak previews given at HIMSS, I know about many standards that ONC supposedly included such as Consolidated CDA, NwHIN Direct, and SNOMED-CT. But I’d sure like to see the details ASAP.

From CMS’ NPRM, many data elements in the “summary of care record,” “clinical summaries,” and “view and download” overlap but aren’t quite the same. As worded they don’t exactly match terms used in standards. Hopefully someone in ONC did the mapping to translate from the HIT Policy Committee’s and CMS’s phrases into the right “data buckets” in Consolidated CDA, for example. If not, I’m available to help through efforts like ONC’s S&I Framework Transitions of Care workgroups.

Margalit Gur-Arie wrote a helpful MU2 summary that helps distinguish what’s the same, what’s slightly new, what’s really new, etc.

As an interoperability champion, I see that ONC really means it when it talks about a big push for interoperability in MU2. MU2 has no more of those “check the box” tests without real exchange that characterized much of MU1! But stage 1 still has a few more years of life in it, for at least some providers, even if it is just a stage setter, Farzad Mostashari spoke of the “massive river flowing” of advances HIT, and I’ll be glad when real information exchanges flow abundantly among providers and patients, as I wrote about in some of my very first blogs.

Unfinished HIT Business

Happy New Year! Typically it’s a time for reflection and looking ahead. But since Keith Boone already blogged about the Top 10 HIT Standards efforts of 2011 and John Halamka predicted the Top 16 HIT Standards Committee Topics for 2012, where does that leave me? I’ll try blending 2011 and 2012, since many things started in 2011 but haven’t yet come to conclusion.

In the hilarious comedy Throw Momma from the Train, Billy Crystal plays a writer who is so obsessed with his ex-wife, that he has serious writer’s block and can’t get past the first sentence of his new novel (“The night was humid…”). He eventually gets his writing groove back (“A writer writes, always”), and doesn’t leave unfinished business. Sometimes I feel like I can’t get a blog started either, so I’m going to just start writing and close out the year!

Here are 11 items of HIT standards unfinished business that I hope reach some closure in ‘12.

1. Meaningful Use Stage 2. More than a hope, it’s a safe bet that this will reach closure in 2012. I predict that items #3, 5, 6, and #8 below will be partially resolved by MU2, but most won’t because they’re complex and/or outside the scope of MU2.
2. Sustainability of federal standards initiatives. HITSP was the previous ONC’s sanctioned standards harmonization effort, but it is gone. Some wheels were reinvented. Now the ONC S&I Framework is here, but an election year and budget cuts loom. What are the chances for national continuity of direction beyond 2012?
3. Sensible certification rules. The HIT SC’s Implementation Workgroup made recommendations based on testimony from the difficult Stage 1 experiences. What practical improvements will be made to certification? 
4. CDA. Just when we thought there was convergence on Consolidated CDA and achievement of a critical mass of CDA-capable products (driven largely by Stage 1 certification), significant CDA changes are being discussed: Detailed Clinical Models (CIMI) and Green CDA standards. What is the path forward that achieves simplification and yet doesn’t send everyone back to the drawing board?
5. Transitions of care. ONC put a lot of effort into the ToC initiative, and some fine clinical recommendations were issued, but they haven’t been fully connected to the Consolidated CDA work yet. How will the MU standards be adjusted so that they truly help clinicians in care coordination?
6. Vocabularies. What appear to be strong recommendations for singular standard vocabularies were made, but seemingly watered down by statements that they are “for quality measures only.” Will converged vocabularies be a reality for interoperability as well as quality measures?
7. Patient-sourced information. Let’s face it, much data in EHRs comes straight from the patient already, yet patient-sourced information seems to be controversial. What HIT standards, will enable patient information (from devices, interviews, messages, PHRs) to have their proper place?
8. Lab reporting and ordering. I hope that the ONC Laboratory Reporting Initiative has achieved widespread consensus.  Will these standards finally result in standardized interfaces in the real world, vs. just on paper?
9. Holistic view of exchange patterns. ONC has focused heavily on the specifications that it commissioned. But will ONC recognize the role of community-based HIEs using standards such as IHE XDS, and not leave a “hole in the middle” by considering only Direct and cross-community NwHIN Exchange?
10. Certificate and/or Provider Directory. Much effort has been poured into supporting only the Direct Project. Will the limited use case of certificate discovery given a Direct address be enough to push Direct into mainstream use? Will there be robust universal certificate directories and provider directories?
11. Query Health. This is not likely to be in MU2, and with so many other priorities, there may not be many organizations geared up to be query responders. What critical mass of industry adoption is necessary in order for population health queries to be statistically significant?

"Six Months of Strategery?"

I read a story in Modern Healthcare about a discussion with HHS CTO Todd Park at a recent conference in Nashville. He talked about unleashing innovation. I applaud Todd for his bravery in giving out his email address publicly seeking input on innovations, and for his goal of “unlocking the mojo” of HHS staff, rather than having good ideas doomed by “six months of strategery.” I liked that!

There has been plenty of recent discussion, and guidance from NIST about improving EHR Usability (for patient safety, efficiency, and many other reasons). There has also been similar discussion in the HIT Standards Committee implementation workgroup about improving the usability of specifications and standards to remove barriers to adoption so the “little guy” isn’t left behind, and so that everyone doesn’t waste time navigating specs that point to other specs that point to other specs (called “the indirection problem”). Some recent standards efforts, such as the HL7 Consolidated CDA implementation guide, are trying to improve usability of specs in this way.

I plan to write Todd speaking as an individual citizen (remember this blog doesn’t speak for my company though my writing is influenced by working in the Health IT industry). Usability is important for both a positive and a negative reason. Lack of usability (of specs, or software, or anything) tends to waste people’s time and “raise their blood pressure” due to frustration. Wasted a few hours per person multiplied by thousands or millions of people, and you have some serious non-productivity! Worse, it can lead to mistakes (like clinical errors jeopardizing patients) or people throwing up their hands and just not doing things that are too complicated (like refusing to use an EHR). On the positive side, when something is usable, like I consider my search engine, Word processing, email, and blog editing software to be, then people can get amazingly more productive, and can do great things.

What about the usability of the specifications that hover over all of us in the EHR industry, Meaningful Use and Certification regulations? Or for that matter, all Federal regulations published in the Federal Register with its three columns and small print, no index and poor searchability? It’s so “1950’s” and paper-oriented. You can’t just scroll down the page like any normal document or web page, because you have to keep backing up for the next column. Try fitting a page on a screen and you can’t read the tiny font. You can’t search it to find a referenced section number, because the numbers are not a sensible scheme like 178.25.3.2.7, where you always know your context: instead section numbers aren’t even contiguous and searchable. For example, the ONC Standards and Certification Final Rule has text like “The standard specified in § 170.207(a)(2)”  OK, if I want to find that paragraph in the document, it’s easily done in the Acrobat search box, right? Nope, can’t be done! It only finds is the references to the paragraph, but not the paragraph itself. The only way to get to § 170.207(b)(2) is by scrolling around and finally locating 170.207 near the end, then finding subparagraph (b) dangling by itself, and finally finding subsubparagraph (2), hopefully still inside of (b) unless I missed a changed letter in between. And when I finally get there, I still can’t see what the standard is, because it says “the code set specified in 45 CFR 162.1002(a)(2)” which isn’t defined in the document. I have to go out to Google. Those like me who have been doing this for a while know some tricks and short cuts and contacts to ask, but a lot of people don’t have that luxury.

So my suggestion to Todd is very low-tech and has nothing to do with HIT, but a lot to do with usability, productivity, and helping folks actually do what the regulations want them to do! First of all, they need to be able to understand the regs in a reasonable amount of time. I’ve spent hundreds of hours reading regulations affecting HIT. I estimate that just their formatting and numbering alone causes me to spend 50-100% more time to digest them than it should. And I probably also miss important information in the process. Multiply that by thousands of others who have the same problem. While it helps that others have created “Readers Digest versions” to simplify and explain the regulations, but why did they have to create them in the first place?

Could this problem be solved by reformatting new regulations for usability, using a few smart people including human factors engineers, armed with common sense and putting themselves in the “consumer’s” shoes? Could this be done without “six months of strategery?” Hopefully yes, if Todd’s right and “mojo gets unlocked” in this area. I admit that I and probably most people and companies have similar entrenched traditions and rules that get in the way of us doing what we ought to do. I just blogged about this today because I read the interview with Todd and connected his ideas with my reading of the regulations. But the Federal Government’s regulations affect thousands of providers, vendors, and others. There’s a real opportunity for a simple low-tech innovation in the government regulation writing and formatting to help many people avoid wasting time, so they can direct more of their energies to creating innovative solutions that will really help people!

ONC S&I Framework Face-to-Face "Aha!" Moments

I attended the ONC Standards and Interoperability Framework meeting last week, and found it very helpful, though what I came out with was not what I expected going in. It shows that I need to keep and open and flexible mind! I’m not going to recap the entire meeting, but will focus on three “Aha!” moments that I had. While I had intended to focus on Transitions of Care (ToC), I ended up splitting my time between ToC, Query Health (QH), and Data Segmentation (DS).

The S&I Framework is the place to be to understand where the industry (including Meaningful Use) is headed. It has some of the strengths (open participation) and challenges (difficulty integrating the workgroups) of HITSP, but has more direct buy-in from ONC, and also has the admirable goal of pilot testing standards to prove their viability before incorporating them into regulations. However, not everything done in S&I Framework is connected to Meaningful Use stage 2; some is more futuristic.

So here are my three “Aha!” moments.

• Transitions of Care. The dots still need to be connected across the many pieces in this initiative. While there’s convergence upon a single standard (CDA Consolidation Guide, which is close to passing its HL7 ballot), that in itself is both too much and yet not enough for ToC. Too much? Many parts of the guide aren’t necessary for ToC. Not enough? The sections that are necessary don’t have the core data element constraints of the Clinical Information Model (CIM) specified clearly enough to know what must be implemented and piloted. I was also surprised to learn that the current reference implementation work in ToC is based on HITSP C32/C83/C80, not yet based on the CIM and CDA Consolidation work because they’re waiting finalization. So they’ll have to flip over quickly to address the emerging standard in order to pilot it. Piloting tools based on C32 would only be nice to have for developers who haven’t already done it, and it wouldn’t make sense to have to pilot them prior to Stage 2 final rulemaking.
• Query Health. I had thought a “query” was something like an SQL query or an IHE document requestry or patient identity query, and assumed it would be synchronous. As Gershwin would say, “it ain’t necessarily so!” While those are indeed queries, QH has a broader definition, and seems to be leaning strongly to asynchronous messages. For example, a query might be composed of two separate (but correlated) push transactions, like a Direct message conveying the “query” (question to be asked, such as “How many diabetic patients with elevated A1C were on X therapy?”). Then the EHR or HIE would run the query and send back the results (again possibly via a push transaction). So it could be like a secure email and a reply to that email. Or it could be like an HL7 Lab Order and a subsequent Result observation message. The QH Technical Workgroup will evaluate the various technical standards to satisfy the use cases. But beyond whatever technical standards are endorsed, I’ll be interested to see how QH defines a mix of predefined query parameters vs totally ad hoc queries, and how the systems are expected to understand what is being requested.
• Data Segmentation (for Privacy?). I was only at about an hour of these discussions, which were moving slowly. But what struck me was the debate about whether this initiative should focus solely on privacy. Clearly, data can be “segmented” for multiple purposes, but given the difficulty up till now of getting a standardized privacy solution deployed even for simpler use cases, I was surprised that broadening the scope was being advocated by some. Other efforts have struggled between “narrow vs broad” use cases. In this particular case, I think the challenge is big enough on privacy that it makes sense to have a narrow focus rather than to “boil the ocean” with other uses of data segmentation. Perhaps a clear definition of phases and scope would help: Phase 1 is solely focused on standards and pilots for privacy data segmentation; later phases can expand the scope. I view this as primarily a problem (granular privacy consent) in search of a solution, rather than a solution (segmentation) in search of many problems. 

So all in all, it was a productive meeting. Much work still remains to be done, and there’s way too much going on for most organizations to stay on top of. Still, I appreciate ONC’s leadership in convening and “uniting the tribes” as Dr. Mostashari said at the Care Transitions meeting.

Forecast Cloudy but Clearing: a Crystal Ball for Meaningful Use

Now that the HIT Standards Committee met on September 28^th to wrap up their “Standards Summer Camp” and are well along in several of the ONC Standards and Interoperability Framework initiatives, how do we stand in getting a clear weather forecast for Meaningful Use Stage 2? Activity was fluid during the hot summer, but by the chill of winter, much will have “frozen” into the form of ONC’s Notice of Proposed Rulemaking (NPRM) for Stage 2 Standards and Certification Criteria.

I was pleased to see an initial draft of certification criteria produced by the Implementation Workgroup. While it’s very preliminary, I understood it to be the first cut at the criteria that ONC will refine over the next few months as they write the NRPM, expected to be released for public comment perhaps in January. Here are a few highlights of what I found new or surprising. While my main focus is generally on information exchange criteria, there is some noteworthy new information on other objectives too. Numbers refer to the rows in the document.

• 10 Clinical Decision Support. There’s more specificity around the ability to configure the CDS rules based on user role, patient setting, and points in workflow; also display of the evidence/source of each notification or care suggestion
• 15 Lab Results. Lots of work has been done in the ONC S&I Framework and HL7 on the new Lab Results Interface (LRI) implementation guide, in an effort to bridge the gap between competing standards. I applaud efforts to work together to reach consensus, even if it means no one gets everything they want, so I hope that this standards effort is leveraged.
• 18 Electronic Notes. They provide clarity that it must not be a scanned document, but that pretty much every other kind of electronic note is OK. While functionally that is achievable, hopefully there will be careful consideration in how to measure this objective to avoid burdening providers with workflow issues.
• 23 Patient View and Download. A definite shift to patient-initiation via electronic access (a secure web portal for example) to more information than was required in Stage 1, rather than have to request a copy through provider personnel. There’s still much debate about alternative ways to satisfy this (e.g., is an export to a patient-controlled PHR acceptable?), so it will be important for guidance to be clear here. 
• 29 Exchange Clinical Information. In this objective, as well as Patient View and Download, clarity is needed on how many types of clinical documents are required (one-size-fits-all vs. multiple different documents?). For example, the ONC S&I Framework Transitions of Care initiative has recommended specific documents -- Discharge Summary and Discharge Instructions from a hospital and Consultation Request and a Consultation Summary from an EP, but the MU objective is written as if it is just a single clinical summary.
• 33 Syndromic Surveillance. As expected, there is a new implementation guide, though it is only applicable to hospitals. But there is still uncertainty over considering submission of Reportable Cancer Conditions by EPs. If that is to be included, what is the recommended standard?
• 52 Amendments to information. This is a requirement that had not been included in previous presentations of Stage 2 MU. Will it be in the NPRM? If so, it would include replacing a data element while preserving a record of the previous value, and appending patient-supplied information as free text or scanned image/document, for patient engagement (e.g., if the patient disputes something in the record). Note, they use the word "user" but are not clear whether they mean "patient" for the requirement to "Make electronically available, upon a user's request, a historical account of amendment(s).”
• 53 Patient Matching. This is also something not mentioned in the Policy Committee's MU objectives. It is fuzzy at this point. If included in certification, it would impose more validation upon some demographic data.

I encourage you to read these thoroughly and work through the channels available to you (e.g., a provider association, a trade association, an industry consortium, or comments to ONC’s blog) to submit suggestions for clarification. The people on policy and standards committees work very hard, but it’s impossible for them to anticipate every case where someone might have trouble interpreting. The implementation workgroup already has flagged areas that they know need more work. If you can give examples of how interpretations may vary, and offer constructive suggestions for how it could be worded unambiguously and made testable, I think it will help ONC to write the clearest possible regulatory language.

Certification Retrospective Part 4 – Lessons Learned

Note: this blog post concludes my discussion of experiences in CCHIT. The opinions are mine, and do not necessarily reflect the views of CCHIT or Siemens.

In life in general, and in development of software, standards, and certification criteria, we learn and grow more by trial and error than we do when everything goes smoothly. Frederic Brooks’ classic software engineering book The Mythical Man-Month said “Plan to throw one away; you will anyhow.“ Hence we need prototyping and iterative refinement. The CDA Consolidation Project is fresh on my mind. I voted negative in its HL7 ballot, but that doesn’t mean it won’t succeed, only that it has issues to fix. It would have been a miracle to get that complex task right the first time.

So it is with certification. The CCHIT Interoperability Workgroup (IOWG) experienced growing pains in its early years. 2006 produced only a roadmap, 2007 produced the first (modest) interoperability criteria. 2008 had stronger criteria and roadmap, and 2009 was poised to make big strides toward semantic interoperability. What were my main lessons learned through these years?

What We Got Right (IMHO):

• The “methodical march” sequence of building blocks. Putting the horse before the cart sure helps! As described in Method to our Madness?, we proposed a logical sequence for interoperability:

1. Data Capture
2. Liquidity
3. Standardized Content
4. Data Consumption

Our views crystallized especially in 2008-2009. CCHIT’s criteria included many of the standards that were eventually endorsed by ONC. For example, Lab Results, CCD, e-prescribing, vocabularies (LOINC, RxNorm, SNOMED CT), and leveraging HITSP.

• The Roadmap (“Glide Path”) Concept. I love maps (online, GPS, paper) and want to know ahead of time where I’m going and the milestones along the way. CCHIT published roadmaps for the next two years beyond each certification year. Roadmaps aimed to provide adequate lead time and to encourage developers to plan and head in the right direction. While the HIT Standards Committee did a great job proposing a “Glide Path” to more robust standards in their proposals in late 2009 (see Jamie Ferguson’s August 20th 2009 Clinical Operations Workgroup presentation), that concept was mostly lost when the ONC regulations were published (see comments on John Halamka’s Feb. 10, 2010 blog). For interoperability, developers need to know the target standards, more than a high-level meaningful use matrix, to move forward.
• Staying a Course. Once we defined the roadmap, we might adjust timing and provide more detailed criteria in our next development cycle, but overall there was continuity and stability with a minimum of “surprises.”
• Transparency. CCHIT posted responses to every public comment on its website. Of course not everyone would agree, but we thought everyone deserved to know their comments were considered and our rationale for decisions.
• Teamwork. The IOWG built relationships and trust among multiple stakeholders within CCHIT, as mentioned in Part 1. That enabled us to work through differences and get things done, despite conflicting opinions. In contrast, it’s harder to capture the “team” element in groups where 50-100 people are nominally on a project but attendance is sporadic, yet I understand the need for openness and transparency. Ultimately, a small group forms a cohesive team within the larger group. For example, the Documentation and Testing workgroup within the Direct Project gradually developed a “team” spirit similar to what the IOWG had.

Regrets:

• Lack of Opportunity to Finish What We Started. Last week I used football, now here’s my first baseball analogy. CCHIT, as the “starting pitcher” for EHR certification was relieved by ONC in the 6^th inning – enough to get a quality start, but not enough to win the game. Our methodical march didn’t reach its goal, but I hope that ONC will be a good “closer” to save the win for interoperable EHRs, coordination of care, and the good of the patient.
• Lack of End-to-End Coverage. Interoperability involves senders, receivers/requesters, and perhaps intermediaries. To succeed, it must encompass all participants (e.g., EHRs, HIEs, PHRs, labs, pharmacies, public agencies), because so much exchange goes beyond EHRs. We didn’t achieve this in CCHIT, but not for lack of trying! We envisioned covering some end-to-end interoperability through certification programs for HIEs and PHRs in addition to EHRs. Excellent people were in the HIE and PHR workgroups, and the IOWG had good dialogues with both. The HIE program was launched with poor uptake followed by suspension in light of impending ONC program changes. The PHR program was not launched.
• All-or-Nothing Approach. CCHIT pondered a modular certification approach early on, especially for inpatient EHRs, but didn’t implement it until ONC mandated it in the Certification Final Rule. In hindsight, I wish we had done that sooner. I don’t think we needed an “all or nothing” approach. Truth in labeling would have disclosed each product’s gaps, to help people make informed decisions. Vendors with complete EHR functionality could still differentiate themselves as a “one-stop shop” but others (“little guys”) would not be excluded from certification just because they didn’t provide 100% of all capabilities.
• A year in limbo. From mid-2009 through mid-2010, the industry was paralyzed by uncertainty and lost momentum on interoperability. It was clear that CCHIT criteria wouldn’t be adopted as is by ONC, but no one knew what would replace them. There was no assurance of staying a course or a clear glide path, and people who had done their best to follow the CCHIT roadmap were concerned and confused. Interoperability features (such as standardized coding and discrete data consumption) that the IOWG expected to be in products in 2010 have been delayed, possibly to MU Stages 2 or 3.

Whether right or regret, there are lessons learned that can be applied going forward.

• Follow the methodical march – a logical sequence of capture, liquidity, standardization, and consumption.
• Define a roadmap (glide path) with ample lead time.
• Stay the course.
• Take an end-to-end perspective.
• Be transparent.
• Be flexible, not “all or nothing.”

That’s it! This has been a long blog series, and if you made it to the end, thanks!

Certification Retrospective Part 3 – Controversies Along the Way

Note: this blog post continues my discussion of experiences in CCHIT. The opinions are mine only and do not necessarily reflect the views of CCHIT or Siemens. Please bear this in mind especially considering that the topic is “Controversies”

From my last Certification Retrospective post, I promised to share lessons learned and some “agonizing controversies” we faced in the CCHIT Interoperability workgroup (IOWG). I thought I could do both in one post, but now realize that I’ll need a “Fourth Movement” after this. Each controversy could easily take up a post of its own, but I’ll try to squeeze them into this post, deferring Lessons Learned to Part 4.

1. When should HITSP specs be incorporated into certification? Once HITSP was formed, it (not CCHIT) was designated to select standards. CCHIT, however, could decide when/if those standards would be certified, striking a difficult balance between supporting the federal strategy (HITSP was HHS-funded, CCHIT was partially HHS-funded) and yet being pragmatic. It would have been difficult to require EHRs to develop too much at once when certification was “all or nothing” and not modular. From working in both CCHIT and HITSP, CCHIT tended to give more weight to considerations such as market adoption and lead times, not just whether a suitable standard existed. I along with Jamie Ferguson were appointed to co -chair the 10-member HITSP-CCHIT joint working group, whose goal was to coordinate the work of HITSP and CCHIT and address how/when to include HITSP specs in certification. But some HITSP-selected standards were still in “trial implementation” with little adoption, and the approval of a standard (HL7, HITSP, or any other) didn’t mean it should be mandatory within a specific timeframe. And there were just so many of them to digest, let alone certify! Some people weren’t pleased, thinking that CCHIT should be a more aggressive “enforcement arm” of HITSP. In the end, we included more of HITSP in 2008 and 2009 certification criteria than ONC did in its 2011 certification criteria, most notably the HL7 2.5.1 Laboratory results with LOINC vocabulary starting in 2007, C32 CCD starting in 2008 (adding RxNorm, UNII, SNOMED CT vocabularies in 2009) and HITSP TP13 (IHE XDS.b – see point #2 below). But we also roadmapped (2-3 years out) most other HITSP specs when we published our 2009 criteria. I believe this was a sensible middle of the road approach. IMHO, if we had mandated many more HITSP specs in certification, there would be few or zero certified products today!
2. Should certification require transport standards? Fourth & Ten – we’d better punt! Seriously, we debated this during 2008 and 2009. We recognized this as a glaring gap in interoperability, and wanted to make headway, realizing that lack of secure transport standards would hinder interoperability even if content were standardized. The Direct Project didn’t exist back then. We were aware of diverse ways that HL7 messages were transported, that IHE had defined SOAP-based web services for document transport, and that HITSP had selected many IHE specifications. Many vendors had already tested IHE Cross-Enterprise Document Sharing (XDS.b) at Connectathon. So the IOWG approved XDS.b as 2009 certification criteria (along with PIX/PDQ to support it). But the Board of Commissioners decided to designate these as optional interoperability criteria, not required for certification, because despite the IOWG’s recommendation and strong EHRA support, some commented against it. This was the one case I can recall where my workgroup’s recommendation was overridden by the Board, which I know was a tough decision since the Board usually didn’t do that. This was also a case where we were less aggressive in CCHIT than the EHR Association had proposed. EHRA had sometimes criticized CCHIT for not providing enough lead time, but in this case they were very disappointed that CCHIT had not required this in certification. This one still bothers me as a missed opportunity.
3. Should we require discrete data import, and who owns that decision anyway? We felt that the main point of requiring standardized content formats and vocabularies, not just human readability/liquidity, was for the discrete data to be consumable. Following our “methodical march” mentioned in part 2, we thought in 2008 that the time had come to propose discrete data import (e.g., update your medication, allergy, or problem lists using data from other providers). But now we were into a “boundary condition.” Where did interoperability’s “turf” end and functionality begin? We had a series of “discrete data import” meetings between the IOWG and the Ambulatory and Inpatient Functionality workgroups. We recognized that data must not only be exchanged, it needs to be reconciled (medication reconciliation being an example), and that you can’t just “slam in” all meds, allergies, problems from many external sources to automatically update the active med/allergy/problem list. The functionality workgroups didn’t think the time was right to require such capability in certification for 2009 or 2010. But how were we going to keep making progress toward semantic interoperability? In the end, we proposed criteria to import very specific discrete data elements for medications, allergies, demographics, and immunizations (with problems to come later), as described in my last post. But we didn’t define the functionality, as long as the discrete data were stored somewhere. And in any case, reconciliation criteria would belong to the functionality workgroups, not the IOWG. I think this would have been a reasonable “baby step” but because of the change in certification that started in 2009, those roadmap criteria never had the chance to be certified. I hope that similar careful thought will go into decisions in ONC certification, whenever it gets to that point. End-to-end interoperability involves content creation, secure standardized transport, and content consumption, all of which should be considered holistically.
4. Should we push the industry toward a single standard for clinical problems? This was really part of the broader topic of converging on a singular standard vocabulary for each type of data, but SNOMED CT for problems was the lightning rod topic for several reasons: a) the difficulties of getting physicians to create medical problem lists at all; b) ICD-9 diagnosis codes were already required for reimbursement and some physicians systems were built around ICD-9; c) ICD-10 was already complicating things by looming on the horizon; d) anything that imposed more work on physicians could hinder usability and adoption. So though SNOMED CT seemed a clear winner for an interoperable clinical vocabulary and was HITSP-endorsed, we received considerable pushback from some members of the other workgroups as to how fast SNOMED CT could be required. I was pleased that many in the IOWG stepped up to provide statistics and research facts such as analysis of mapping of ICD-9 to SNOMED CT and clarification that standard codes don’t have to be visible in the clinician’s UI. We knew this was a contentious issue, but staked out our positions in the 2009 roadmap and a Q&A document.

The above four were prototypical of the controversies we faced in content, vocabulary, and transport. I could go on with more examples. That’s part of the challenge of interoperability work, as evidenced by the fact that ONC and the HIT Standards Committee still have all of the above issues to ponder for Stages 2 and beyond. I should mention that one more big controversy arose in the very first year, when CCHIT was newly formed and struggling to figure out how to set the interoperability bar. Eyes were upon those first members to resolve the CDA vs. CCR debate. Unlike the four issues above, where we took a stand, we weren’t ready to force a decision in 2006, and then HITSP was formed by ONC to deal with such “standards harmonization” issues. So at least our punt had a receiver (and football wasn’t locked out)!

I promise that Part 4 will really conclude this series with my thoughts on Lessons Learned, proudest accomplishments, regrets, and hopes for the future, stemming from my CCHIT experience. Thanks for listening to my sharing from “inside the trenches” of discussions of which the public was not aware.

Certification Retrospective Part 2 – Method to our Madness?

Note: this blog post will continue discussing my experiences in CCHIT. The opinions are mine, and do not necessarily reflect the views of CCHIT or Siemens.

In Part 1, I talked about CCHIT giving me the privilege to work with great people over the years. But we didn’t volunteer for CCHIT just to meet people, but to do important work to advance the HIT industry. So what did we accomplish, and what were we thinking as we evolved?

One thing I learned is that no group can please all the people all the time. Sometimes, people wondered why we proposed some criteria and not others, or proposed the timeframes that we did. So we tried to explain these “whys” in a white paper Interoperability, Supplying the Building Blocks for a Patient-centered EHR in mid-2009 which still exists on the website of writer John Morrissey. It described the context for what we were doing and the “methodical march” to increasing levels of information interoperability that we proposed. I don’t think this was well understood by the public, who might have thought some of what we did was arbitrary. But here’s what was in our mind (well, I can only speak for myself – mine at least).

Key principles:

• We aren’t starting with a blank sheet of paper. The existing state of adoption (immature as it might be) must be considered, so we started each year with an “environmental scan” on the state of the HIT industry.
• Adequate time must be provided for EHR products to make the transition to a more desirable state. We aimed for 18 months lead time from our first “signal to the market” (new draft criteria) until certification testing of those criteria.

Given these principles, our methodical march followed the path below.

• Data Capture precedes interoperability. You can’t share what you don’t have! When we started CCHIT, not all EHRs even captured all the data that other systems would want, so functionality criteria (ambulatory, inpatient, ED) needed to capture the data elements that could then be shared. Thus in the first certification year, 2006, there were no interoperability criteria required (though several criteria, such as Lab Results and e-prescribing, were roadmapped for later years).
• Information Liquidity (a term we didn’t use, but which implicit in our criteria) is beneficial even prior to semantic interoperability. Thus, the first clinical summaries that CCHIT certified (Continuity of Care Documents, CCDs) in 2008 required human-readable sections for meds, allergies, problems, etc., but not structured data. We were later criticized for this, as if we didn’t care about structured data, but this was always intended as a stepping stone to later stages (as explained in the white paper). Even today (3 years later than the 2008 criteria), many voices espouse liquidity as a more important and realistic for the near-term than semantic interoperability (“don’t let the perfect be the enemy of the good”).  
• Standardized Content (formats and vocabulary). Human-readability is fine for humans, but doesn’t help most EHR systems to “understand” (semantics) and process the information they receive from others. Because of the existing Tower of Babel syndrome most EHRs have been “confounded” in their ability to share information with other EHRs, and to “consume” (not just display) it. So CCHIT supported the goals of ONC and HITSP to progress toward standardization of vocabularies. But it realized that changing an EHR’s vocabulary overnight wasn’t realistic, and that mapping from one vocabulary to another is not simple or foolproof and must be done carefully for patient safety and other reasons. Thus a “glide path” from narrative to standard vocabularies was roadmapped.
• Discrete Data Import (Consumption). When liquidity and standardized content are established, the EHRs that receive or retrieve clinical data can “do something” with the data, more than just file and display it. As early as 2007 criteria, we published certification criteria for ambulatory EHR discrete import of Laboratory Results that referenced the HITSP-endorsed HL7 2.5.1 Lab Results Reporting Implementation Guide, including a common subset of LOINC observation codes. Thus, many of us were disappointed when the ONC Final Rule for certification, published in 2010, lacked a specific HL7 standard and LOINC requirement, not going as far as what CCHIT had been certifying since 2007. There are 127 CCHIT Certified® ambulatory EHRs that already meet this requirement. For CCDs, the challenge of discrete data import is  greater because of the wider variety of data, and multiple applicable vocabularies. Rather than declaring the problem too big to tackle in one shot, we proposed chipping away a few sections at a time, starting in 2010 with discrete import of medications (using RxNorm as specified by HITSP), allergies (using RxNorm and UNII), Immunizations (using CVX) and demographics, then moving on to problems (using SNOMED-CT). These criteria are a matter of public record, and can be found on the CCHIT website: 2008 criteria. See 2008 Criterion IO-AM 11.06 for example, which originally roadmapped discrete data import for 2009, though the date changed to 2010 in the 2009 criteria (published but then withdrawn)***. You can decide based on the facts whether we were too aggressive, or too laggardly, or about right.

So what’s the point of this retrospective? I’ll sum up lessons learned in the next post. I’ll also speak more about how CCHIT’s interoperability criteria compared to ONC’s criteria.

I’ll also discuss some of the agonizing controversies that we faced, particularly in 2008 and 2009 over issues like standardized transport, discrete data import, data reconciliation turf (is it interoperability or functionality or both?) and the feasibility of requiring SNOMED-CT. We wrestled with each of these, and did our best. We might not have succeeded perfectly in all of them, and our criteria development work was superseded by ONC before we could resolve them all, but we learned much in the methodical march!

David

*** (Long Footnote)

The 2009 Ambulatory Certification criteria, along with the CCHIT 2009 certification program, were not launched, since they were in limbo awaiting decisions on ONC-ATCB certification. But eventually most of these criteria (without the roadmap) were relabeled and launched as the 2011 CCHIT Certified® ambulatory criteria. If the 2009 program had proceeded, Criterion IO-AM 11.06 would have required the following in 2010, adding much specificity beyond what had been proposed in the same criterion in 2008:

The system shall provide the ability to display CCD documents, using a subset of the HITSP C32 specification for the Medication and Immunization History module, file them as intact documents in the EHR, and import the discrete data from one or more of the entries in a structured form into the patient record. The system shall provide the ability to import as discrete data the following data elements from the C32 Medications Module: Free text SIG, free text product name, free text brand name, coded product name, coded brand name, product concentration, status of medication, quantity ordered, order date/time, order expiration date/time, prescription number, quantity dispensed, and fill number. (Note: fill number means the number of fills dispensed, not the number of fills prescribed). The “required if known” (R2) data elements will not always be known, and therefore will not be present in every instance of a C32 document, but if present, the system shall be able to import them

More was explained in the Comments column:

Medication and Immunization History-Coded Subset includes the following content modules of the HITSP C32: Person Information, Healthcare Provider, Medications-Prescriptions and Non-Prescription, Information Source, Comments

The intent for 10 Certification is not to require specific capability to use the imported data but only to show the system is capable of processing discrete data elements from a CCD. In future certification years, more functionality criteria to “use” the data may be added. Fulfillment of the 10 Certification criteria may be demonstrated in a variety of ways, including but not limited to any one of the following:

- Reconciliation functionality that allows CCD data to be compared with existing data, and selectively imported into a list

- A display of data captured from external sources, including the imported discrete data. The physical storage location (database table, file, etc.) is at the vendor’s discretion and does not need to be the same location as the active allergies.

- Creation of an CCD that includes the elements previously imported as discrete data

Vendors may innovate and differentiate their products by how well such data are used, but 10 Certification will not limit vendors to any particular approach

Certification Retrospective -- Part 1

Note: this blog post will discuss my experiences in CCHIT. The opinions are mine, and do not necessarily reflect the views of CCHIT or Siemens.

It’s been a long time since I blogged, even though there have been many HIT topics to blog about, such as Stage 2 meaningful use, the ONC Standards and Interoperability Framework, the ONC Strategic Plan, the NPRM on Medicare Shared Savings (Accountable Care Organizations), and more. I’m getting back on the wagon. One topic I’ve wanted to blog about for a while, even though it’s not a current “hot topic” in the news, is my experience in the Certification Commission for Health Information Technology (CCHIT) prior to the establishment of the ONC certification program. This bubbled to the surface when I read a HISTalk “Time Capsule” posting. So I’ll write another mini-series (probably two or three parts). This first post will simply introduce the “cast of characters” and how they interacted. The next posts will talk about what the Interoperability Workgroup of CCHIT did, and how its work through 2009 related to the ONC-ATCB certification program as we know it today. There are useful lessons learned.

As a matter of disclosure, I’ve been a CCHIT volunteer for more than five years, and am still officially a co-chair of CCHIT’s Interoperability Workgroup, but this group has been inactive for about a year though it still nominally exists. The reason for its inactivity is that it was chartered as a group to propose certification criteria EHRs, a role that was displaced by ONC and the HIT Standards Committee. CCHIT still offers its branded certification program called “CCHIT Certified®” that is completely distinct from the ONC certification for which CCHIT is an authorized testing and certification body (ATCB). The branded program includes criteria that I and many others worked on until 2009. In interoperability, CCHIT is currently performing testing based on the NIST test scripts and some optional tools, but it is not developing any new interoperability criteria at this time.

Industry collaboration – whether in standards organizations or groups like CCHIT – has been a rewarding and mind-expanding aspect of my career. It’s about the people who work together to benefit many more people (e.g., patients and providers), trying to leave their agendas and “organization hats” at the door. So this first post focuses on some facts about the volunteers of CCHIT interoperability.

I found it fascinating to compile a list of the people I’ve served with on Interoperability workgroups at CCHIT. See this link for a table of volunteer names. It’s a pretty diverse and impressive list, IMHO. The list of CCHIT workgroup members has always been publically posted, though I don’t know if it’s preserved historically, so I had to dig through my archives. My apologies if there are some typos in my list. The organizations listed were as of the time the people were on the workgroup, not necessarily where they are now. And occasionally there were people who had to resign for lack of time and a few were replaced but that may not be reflected in my table.

Each year, the Interop Workgroup had between 13 and 24 members (the 24 divided into two groups). Over 60 members served on the workgroup at some time. Of these, 9 were from EHR vendors (only counting vendors of products subject to certification, not vendors of different product types like e-prescribing, lab systems, or HIE tools). The EHR vendor reps were generally active participants, but they didn’t dominate the workgroup. Of course, there are many other interoperability experts who could have been on CCHIT but weren’t, since there was limited space. In that sense, CCHIT was not open like a Standards Development Organization, but it tried to establish a small committed group of volunteers, selected by an application process according to guidelines for stakeholder balance. Sometimes, we knew what we didn’t know, and invited guests to educate us at our meetings, on topics like e-prescribing, vocabularies, and clinical documents.

Sometimes the group was split into an Inpatient Interop WG and and Ambulatory Interop WG (in which case I was on the Inpatient side). In the last year, it divided into an Advanced Interoperability WG and a “regular” Interoperability WG (I co-chaired the latter). Usually a small subset of volunteers would work on specific subsets of criteria (e.g., Lab, eRx, clinical documents) and the entire WG would review and refine.

I have very positive feelings towards these people. They’re some of the finest I’ve worked with, and I am grateful to them. There was a strong sense of collegiality and teamwork, and the sense that we were doing something important. While there were downsides to the group being selected rather than open, that also helped motivate most members since they couldn’t passively leave the work to “someone else” as tends to happen in larger groups. Collaboration was real across different stakeholders and even among vendor competitors. Sure we wouldn’t always agree. Some members thought criteria should be much more aggressive. Others couldn’t understand why features they regarded as only “nice-to-haves” should be mandatory for certification. Sometimes (as co-chair) I’d feel frustrated if I thought someone wasn’t pulling their weight (e.g., not showing up for meetings). It’s inevitable that in volunteer organizations not everyone contributes equally, but everyone had the opportunity.

In acknowledgments, there’s always a risk of omitting someone important, but I especially want to thank Amit Trivedi and Anita Samarth who alternated as CCHIT staff leads for interoperability. Their support was tireless and outstanding: I still remember 4am e-mails from both of them as they worked towards tight deadlines! And I really appreciated the commitment, passion, and the desire to do the right thing, from my fellow-co-chairs such as Dr. Alan Zuckerman, Dr. Pat Hale, and George Robinson.

The Board of Commissioners was mostly “out of sight” for the workgroup. They respected and endorsed the expertise of the volunteers, but would attend our meetings if we needed them (Dr. Leavitt did this several times), offering guidance but not dictating what we should do. I only remember being “overridden” by the Board on one substantive issue.

In the subsequent post(s), I’ll talk more about the interoperability work products that we developed, how it related to standards organizations, ONC, and what I would have done differently if I could start over.

David

An Aid to Survey the River of Clinical Information

Recently, I’ve had discussions with many people in the HIT industry about the future direction for Meaningful Use (MU), especially about exchanging clinical information. It’s clear that Stage 1 is only the start of a journey, and that increasingly robust sets of clinical information will need to be exchanged. However, such exchanges need to be helpful rather than burdensome, and much work needs to occur to figure out how to display what’s significant, accurate, reconciled, and usable for clinicians and patients (see my recent blog series Part 1, Part 2, Finale).

I’m sharing a spreadsheet (a work in progress) showing a library of standardized reusable data sections and comparing many types of electronic clinical documents/records, including the two most “famous” that are required in Meaningful Use Stage 1, CCD and CCR. This spreadsheet may eventually be published in a more final form from an organization, but even now it may be useful to you if you have any of these questions:

• What must be shared in Stage 1 as a clinical summary with providers and/or patients? (See spreadsheet columns B and D green cells)
• Are certification and MU clinical summaries the same? There are some interesting differences that can be found between the standards for certification and the language for meaningful use (See column A and the CMS FR – Clinical Summary tab).
• What’s the difference between a full C32 and what’s required in Stage 1 certification? Quite a bit, actually! (See column D, white check marks)
• What data are not included in CCR and CCD but might be desirable to share in the future? (See all rows and columns). Stage 1 requires only the most commonly requested clinical summary data (meds, allergies, problems, results, etc.).
• How much reusability exists among today’s electronic clinical documents? (See column Q, which also indicates which sections are most often used). There are worthy efforts afoot to improve commonality and reduce divergence, such as the ONC sponsored CDA Consolidation Project. 

Here’s my compulsory “liquid” or “music” analogy of the day. A mighty rushing river is growing, of clinical information and the standards thereof. I hope my high-level spreadsheet serves as an aid, a “Bridge Over Troubled Water” where you can safely survey the river before you plunge in. Although I created this spreadsheet, with plenty of review from Health Story and EHRA members, I’m far from being the authority on clinical documents. Robin Raiford did wonderful work on a similar spreadsheet in 2009, which was actually more detailed, but was limited to HITSP CDA documents and preceded the ONC Final Rules. I recommend that you read the original specs, and follow Keith Boone’s blog and his upcoming CDA book.

Making HIT MUsic

This post assumes that you have a general understanding of ARRA HITECH legislation and the concepts of Certification and Meaningful Use (MU) of Electronic Health Records (EHRs).

So what would make HIT successful? Let’s keep our eye on the nationally stated goals of quality, safety and efficiency, i.e., better and safer health for consumers at a reasonable cost. As my first blog article compared interoperability specifications to musical scores, I’ll recommend where detailed specifications are necessary, and where they aren’t, so that we end up with a HIT MUsical, not an expensive flop/bomb. (I warned you about my puns in the first article!)

As people wonder what MU should be in the future, some alternative approaches are being considered by groups such as the HIT Policy Committee Meaningful Use Workgroup. One approach is to focus on outcomes achieved more than specific functional requirements. The HIT Policy Committee’s Certification and Adoption Workgroup made recommendations in August, 2009, that “Criteria on functions/features should be high level; however, criteria on interoperability should be more explicit.”

Interoperability: yes, be very specific! As one who has pored over many committee recommendations, interoperability specifications, CCHIT criteria, HHS regulations, NIST test scripts, certification handbooks, and HHS FAQs, I agree that interoperability criteria need to be explicit. I enjoy the opportunity to listen to various performances and recordings of masterpieces such as Beethoven’s Symphony #7. They allow some innovation in tempo, dynamics, and expressivity, but they’re all still the same symphony, because they conform to Beethoven’s musical notation “spec.” I don’t think anyone would suggest that the symphony be performed without all musicians following the same score. Because many players’ efforts need to fit together, it wouldn’t work to let musicians make up their own parts. Interoperability likewise requires many systems to fit together. But let's not skimp on quality: specs need to be implemented successfully in the real world before requiring their use. Also, evaluating return on investment is also key to avoid going overboard: the more frequently information is exchanged, the more people who need the info, and the higher the impact, the more it is worth standardizing. But for something on the other end of the spectrum (infrequent exchange, few people, little impact), is standardization worth the effort and cost?

Functionality: don’t be too prescriptive! In contrast, “functions/features” criteria, excluding interoperability, can be stated at a high (non-prescriptive) level, because they’re within a single system. Other organizations’ systems don’t interact directly with them. There’s the dilemma of wanting to ensure success without “micromanaging” and stifling innovation. Everyone “doing their own thing” re interoperability might be innovative but counterproductive. But when it comes to clinical functionality and workflows, won’t providers have enough information from other providers, market reports, professional associations, user groups, Regional Extension Centers, and test-drive opportunities to figure out what will work for them? Bear in mind that an EHR’s customers already have a starting base, and will lean towards whatever adds value to what they have, rather than something that conforms to generic criteria but doesn’t fit their environment.

We all want a health system that produces the best possible health outcomes for us. I’d rather see our policies, and the efforts of EHR developers, focused on achieving those outcomes, rather than meeting the “letter of the law” of detailed functionality criteria. Such criteria can lead to developers diverting time from features their customers really want, in order to “check the box” and deliver features to meet certification criteria and test scripts that won’t ever be used. So for functionality, excessive specificity of criteria produces distraction and waste! But in contrast, lack of precision and clarity of interoperability specs has produced waste, resulting from people being confused and the same question being asked over and over again.

So I think the original recommendations of the Certification and Adoption Workgroup regarding high-level functionality criteria and explicit interoperability criteria, and the MU Workgroup’s thoughts of more outcome-oriented objectives, are excellent paths to follow as we proceed into MU Stages 2 and beyond.