Engaging clinicians: building EHR specifications

There is a methodology that is pragmatically evolving from my experience in openEHR clinical modelling work over the past few years. It has developed in a rather ad hoc way, and totally in response to working directly with clinicians. The simplicity and apparent effectiveness – both for me and the clinicians involved – continues to surprise me each time I use it.

The clinical content specifications for specialised health records and care plans that we are building are being developed with a sequence of expert input and clinical verification:

1. Identifying the clinical requirements and business rules in conjunction with a selected initial domain expert group;
2. Broader abstract verification of the notion of ‘maximal data set’ for ‘universal use case’ during formal archetype review cycles;
3. Contextual validation during template review by ‘on-the-ground’ clinicians; and finally, although to a lesser degree,
4. Validation during mapping and migration of legacy data.

With each project I am refining this process. Starting off a project with face-to-face meetings has been a ‘no brainer’ – after all, it takes a while for everyone to understand the get the idea of what we are doing. However after initial workshops, pretty much everything else can be done via web conference, online collaboration via CKM and email.

I find the initial workshops are usually greatly satisfying. Within hours we can be creating two outputs – a mind map that reflects the clinicians evolving conversation about their requirements and, in parallel, an equally agile template of clinical content specifications that can be verified by the clinicians in real time.

The mind map is displayed on a shared screen or via a data projector and acts as a living document, evolving as we talk through the clinical requirements, and identify the complexities, dependencies and relationships of all the components. The final mind map may be surprisingly different to how it started, and at the end of the conversation, the clinicians can verify that what they’ve said if accurately reflected in the mind map. It is an open source tool, so we can also share this around after the workshop for further comments.

Subsection of a mind map

Most recently I have begun building a template on the fly during the workshop, using any existing archetypes that are available, and identifying gaps or the need for new archetypes on the mind map as we go. In this way we are actually building the content specification in front of the clinicians as well. They get an understanding of how the abstract discussion will actually shape the resulting EHR content and they can verify it as we gradually pull it together. The domain experts are immediately equipped to answer the question: “Does this specification match what you have been telling me you do in practice?”

This methodology seems to bring the clinicians along with us on the clinical modelling journey, and most are able to understand at least some of the implications of some of our requirements discussions and, in particular, the ‘shape’ of the data that we can collect. It is a process seems to suit the thinking process of many clinicians and the overwhelmingly consistent feedback from recent workshops is that they have all actually enjoyed the experience and want to know what are the next steps for them to be involved. So that’s certainly a winner.

And the funders/jurisdictions are anecdotally confirming for me that they are finding that this approach is supporting higher quality specifications in a much shorter time frame.

For example, at a project kickoff workshop for a new project recently, in two days we:

• developed a series of mind maps capturing a consensus view of the clinical requirements and business processes;
• identified all the archetypes required for the entire project, including those that existed and were ‘fit for use’, those that needed some extension to meet requirements and new archetypes that needed to be created;
• identified sources of information or mind mapped the requirements for each new archetype identified; and
• built 3 templates comprising all of the existing archetypes available from a number of sources – the NEHTA CKM http://dcm.nehta.org.au/ckm/, the openEHR international CKM http://www.openehr.org/ckm/ and local drafts that I had on my own computer. For a number of the new archetypes we also collectively identified source information that would inform or be the basis for the archetype development.

All of this described above took 8 medical practitioners clinicians away from their everyday practice for only 1-2 days, each according to their availability. Yet it provided the foundation for development of a new clinical application.

Then I go home. Next steps are to refine the mind map, modify/update/specialise any archetypes for which we have identified new requirements and build the new ones. And in parallel start the collaborative process through a CKM project to ensure that existing and modified archetypes are ‘fit for (our project’s) purpose’, and to upload and initiate reviews on the new draft archetypes.

All work to progress these archetypes to maturity (ie aiming for clinical consensus) and then validate the templates as ready for handover to the implementers can be done online, asynchronously and at a time convenient to the clinicians work/life balance!

I live over 2000 kilometres away from these clinicians. Yet the combination of web conference and CKM enables us to operate as an ongoing collaborative team. It seems to be working well at the moment… No doubt I’ll continue to learn how to do it better.

The Archetype Journey…

I’m surprised to realise I’ve been building archetypes for over 7 years. It honestly doesn’t feel that long. It still feels like we are in the relatively early days of understanding how to model clinical archetypes, to validate them and to govern them. I am learning more with each archetype I build. They are definitely getting better and the process more refined. But we aren’t there yet. We have a ways to go!

Let me try to share some idea of the challenges and complexities I see…

We can build all kinds of archetypes for different purposes.

There are the ones we just want to use for our own project or purpose, to be used in splendid isolation. Yes, anyone can build an archetype for any reason. Easy as. No design constraints, no collaboration, just whatever you want to model and as large or complex as you like.

But if you want to build them so that they will be re-used and shared, then a whole different approach is required. Each archetype needs to fit with the others around it, to complement but not duplicate or overlap; to be of the same granularity; to be consistent with the way similar concepts are modelled; to have the same principles regarding the level of detail modelled; the same approach to defining scope; and of course the same approach to defining a clinical concept versus a data element or group of data elements… The list goes on.

Some archetypes are straightforward to design and build, for example all the very prescriptive and well recognised scales like the Braden Scale or Glasgow Coma Scale. These are the ‘no brainers’ of clinical modelling.

Some are harder and more abstract, such as those underpinning a clinical decision support system of orders and activities to ensure that care plans are carried out, clinical outcomes achieved and patients don’t ‘fall through the cracks’ from transitions of care.

Then there are the repositories of archetypes that are intended to work as single, cohesive pool of models – each archetype for a single clinical concept that all sits closely aligned to the next one, but minimising any duplication or overlap.

That is a massive coordination task, and one that I underestimated hugely when we embarked on the development of the openEHR Clinical Knowledge Manager, and especially more recently, the really active development and coordination required to manage the model development, collaboration and management process within the Australian CKM – where the national eHealth program and jurisdictions are working within the same domain of models, developing new ones for specific purposes and re-using common, shared models for different use cases and clinical contexts.

The archetype ecosystems are hard, numbers of archetypes that need to work together intimately and precisely to enable the accurate and safe modelling of clinical data. Physical examination is the perfect example that has been weighing on my mind now for some time. I’ve dabbled with small parts of this over the years, as specific projects needed to model a small part of the physical exam here and there. My initial focus was on modelling generic patterns for inspection, palpation, auscultation and percussion – four well identified pillars of the art of clinical examination. If you take a look at the Inspection archetype clinicians will recognise the kind of pattern that we were taught in First Year of our Medical or Nursing degrees. And I built huge mind maps to try to anticipate how the basic generic pattern could be specialised or adapted for use in all aspects of recording the inspection component of clinical examination.Over time, I have convinced myself that this would not work, and so the ongoing dilemma was how to approach it to create a standardised, yet extraordinarily flexible solution.

Consider the dilemma of modelling physical examination. How can we capture the fractal nature of physical examination? How can we represent the art of every clinician’s practice in standardised archetypes? We need models that can be standardised, yet we also need to be able to respond to the massive variability in the requirements and approach of each and every clinician. Each profession will record the same concept in different levels of detail, and often in a slightly different context. Each specialty will record different combinations of details. Specialists need all the detail; generalists only want to record the bare basics, unless they find something significant in which case they want to drill down to the n^th degree. And don’t forget the ability to just quickly note ‘NAD’ as you fly past to the next part of the examination; for rheumatologists to record a homunculus; for the requirement for addition of photos or annotated diagrams! Ha – modelling physical examination IS NOT SIMPLE!

I think I might have finally broken the back of the physical examination modelling dilemma just this week. Seven years after starting this journey, with all this modelling experience behind me! The one sure thing I have learned – a realisation of how much we don’t know. Don’t let anyone tell you it is easy or we know enough. IMO we aren’t ready to publish standards or even specifications about this work, yet. But we are making good, sound, robust progress. We can start to document our experience and sound principles.

This new domain of clinical knowledge management is complex; nobody should be saying we have it sorted…

Archetype quality I

Up until recently clinical content models, such as archetypes, have been regarded as a novelty; watched from the sidelines with interest from many but not regarded as mainstream. However now that they are increasingly being adopted by jurisdictions and used in real systems, modellers need to change their approach to include processes, methodologies and quality criteria that ensure that the models are robust, credible and fit for purpose.

There has been some work done identifying quality criteria for clinical models but there is no doubt that establishing quality criteria for clinical content models is still very much in its infancy:

• Kalra D, Tapuria A, Freriks G, Mennerat F, Devlies J (2008). Management and maintenance policies for EHR interoperability resources [36 pages] (Q-REC Project IST 027370 3.3 ). The European Commission: Brussels. (Last accessed May 28, 2011)

• There has been some slowly progressing work in ISO TC 215 – ISO 13972 Health Informatics: Detailed Clinical Models. Recently it has been split into two separate components, not yet publicly available:
• Part 1. Quality processes regarding detailed clinical model development, governance, publishing and maintenance; and
• Part II – Quality attributes of detailed clinical models

Most of the work on quality of clinical models has been based largely on theory, with few groups having practical experience in developing and managing collections of clinical models, other than in local implementations.

In 2007, Ocean Informatics participated in a significant pilot project. The recommendations were published in the NHS CFH Pilot Study 2007 Summary Report. My own analysis, conducted in December 2007, revealed that there were 691 archetypes within the NHS repository. Of these, 570 were archetypes for unique clinical concepts, with the remainder reflecting multiple versions of the same concept. In fact, for 90 unique concepts there were 207 archetypes that needed rationalisation – most of these had only two versions however one archetype was represented in five versions! We needed better processes!

Towards the end of 2007 a small team within Ocean commenced building an online tool, the Clinical Knowledge Manager to:

• function as a clinical knowledge repository for openEHR archetypes and templates and, later, terminology subsets;
• manage the life-cycle of registered artefacts, especially the archetype content – from draft, through team review to published, deprecated and rejected. Also terminology binding and language translations;
• governance of the artefacts.

In July 2008 we started uploading archetypes to the openEHR CKM, including many of the best from the NHS pilot project. Over the following months we added archetypes and templates; recruited users; and started archetype reviews. All activity was voluntary – both from reviewers and editors. Progress has thus been slower than we would have liked and somewhat episodic but provided early evidence that a transparent, crowd sourced verification of the archetypes was achievable.

In early 2010, Sweden’s Clinical Knowledge Manager had its first archetypes uploaded.

In November 2010, a NEHTA instance of the CKM was launched, supporting Australia’s development of Detailed Clinical Models for the national eHealth priorities. This is where most collaborative activity is occurring internationally at present.

In this context, I have pondered the issues around clinical knowledge governance now for a number of years, and gradually our team has developed considerable insight into clinical knowledge governance – the requirements, solutions and thorny issues. To be perfectly honest, the more we delve into knowledge governance, the more complicated we realise it to be – the challenge and the journey continues; a lot is yet to be solved 🙂

It is relatively easy to identify the high level processes in the development of clinical knowledge artifacts, each of which requires identification of quality criteria and measurable indicators to ensure that the final artifacts are fit for purpose and safe to use in our EHR systems. The process is similar for both archetypes and templates; plus the Requirements gathering and Analysis components are applicable to any single overarching project as well.

For archetypes:

The harder task is that for each of these steps, there are multiple quality criteria that need to be determined, and for each criterion it will be necessary to be able to assess and/or measure them through identifiable quality indicators.

Ideally a quality indicator is a measurement or fact about the clinical model. In some situations it will be necessary to include additional assessments manually performed by qualified experts.

If an indicator can be automatically derived from the Clinical Knowledge Manager (CKM), it ensures that up-to-date assessments of the models are instantly available as the models evolve (such as this Blood Pressure archetype example), and more importantly, without reliance on manual human intervention. However while assessments that do need to be assessed by an expert human – for example, compliance to existing specifications or standards – add valuable depth and richness to the overall quality assessment, they also add a vulnerability due to the need for skilled human resources to not only conduct the assessment, but to apply consistent methodologies during the assessment; these will be much more difficult to sustain.

Assessment of whether the indicators actually satisfy the quality criteria should also ideally be as objective as possible, however our reality is that it will probably more often be subjective and vary depending on the nature of the archetype concept itself. The process cannot be automated, nor can there be a single set of indicators or criteria that will determine the quality of every archetype. We need to ensure appropriate oversight to archetype development, ensuring that a quality process has actually been followed and utilise quality indicators to determine if the quality criteria have been met – on an archetype by archetype basis.

And it continues on…