ISO/TS 18864:2017

Health informatics — Quality metrics for detailed clinical models

The purpose of ISO/TS 18864:2017 is to define objective, reliable and reproducible quality metrics for detailed clinical models (DCM). ISO/TS 18864:2017 specifies the principal metrics which are necessary and sufficient to evaluate DCMs. The intended audiences of this document are: - DCM developers, all users of information represented using DCMs, and evaluators of DCM quality; - clinical and IT professionals of healthcare institutions; - technical staff in the healthcare technology industry; - experts involved in standards development; - national and regional healthcare information technology leadership including certification bodies. ISO/TS 18864:2017 defines a set of quality metrics required to evaluate DCMs objectively. These quality metrics can be used to support rational decision making by DCM developers who will have the essential qualitative and quantitative quality requirements to use as guidelines as they create new content. Clinical users can then use the quantitative assessments as they select models for specific use cases and implement them in their clinical systems.

Informatique de santé — Indicateurs de qualité pour modèles cliniques détaillés

General Information

• Status: Published
• Publication Date: 28-Nov-2017
• ICS: 35.240.80 - IT applications in health care technology
• Technical Committee: ISO/TC 215 - Health informatics
• Drafting Committee: ISO/TC 215/WG 1 - Architecture, Frameworks and Models
• Current Stage: 9093 - International Standard confirmed
• Completion Date: 09-Jul-2021

Standards Content (Sample)

TECHNICAL ISO/TS
SPECIFICATION 18864
First edition
2017-11
Health informatics — Quality metrics
for detailed clinical models
Informatique de santé — Indicateurs de qualité pour modèles
cliniques détaillés
Reference number
ISO/TS 18864:2017(E)
©
ISO 2017

---------------------- Page: 1 ----------------------
ISO/TS 18864:2017(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO 2017 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/TS 18864:2017(E)

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 5
5 Relationship with other activities . 6
5.1 ISO/TS 13972 . 6
5.2 ISO 13606-2 . 6
5.3 EuroRec quality criteria for archetypes . 6
5.4 Delphi study . 6
6 The purpose and importance of DCM quality . 6
7 Quality aspect of DCMs . 7
8 DCM quality framework . 7
8.1 DCM quality framework component . 7
8.2 DCM Quality characteristics . 8
8.3 DCM Quality metrics . 8
8.4 Expected benefits of high quality DCMs . 9
9 Quality metrics for DCMs . 9
9.1 Design and development . 9
9.1.1 General. 9
9.1.2 Stakeholders participated in the development (or design) of DCM .10
9.1.3 Stakeholders participated in the verification/approval of DCM .10
9.1.4 Translations, only if applicable .10
9.1.5 Relationships .11
9.2 Per detailed clinical model .11
9.2.1 Compliance to standard .11
9.3 Metadata .13
9.3.1 DCM Version .13
9.3.2 Purpose of DCM .13
9.3.3 Appropriate description of application target patient or population of DCM .14
9.3.4 Description of multiple uses .14
9.3.5 Appropriate description of discipline of DCM user .15
9.3.6 DCM author(s) .15
9.3.7 Authoring date .15
9.3.8 Modification date .15
9.3.9 Initial review round date .16
9.3.10 Number of review rounds .16
9.3.11 Last review round date . .16
9.3.12 Status of content publication .17
9.3.13 Mention of reference(s) used in DCM development, only if applicable .17
9.4 Per data element .17
9.4.1 Valid value of DCM .17
9.4.2 Terminology binding .18
9.5 Governance .18
9.5.1 Maintenance organization of DCM .18
9.5.2 Existence of user feedback mechanism for DCM .18
9.5.3 Realm-specific specializations and extensions .19
9.5.4 Multiple outputs .19
9.5.5 Clinicians participated in the verification/validation of DCM for
determining customer satisfaction .19
© ISO 2017 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/TS 18864:2017(E)

Annex A (informative) Some published quality requirements and criteria for DCMs .21
Annex B (informative) Comparison of Detailed Clinical Modelling approach .24
Bibliography .28
iv © ISO 2017 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/TS 18864:2017(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 215, Health informatics.
© ISO 2017 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/TS 18864:2017(E)

Introduction
Normalization and formalization of evidence-based clinical data are fundamental requirements towards
enabling a high quality and safe healthcare system. Normalized data elements and clinical models can
help to ensure data consistency and comparability among users and systems, and formalized data
specifications can support the sharing of clinical knowledge among diverse jurisdictions with higher
efficiency and lower cost.
The goal of shareable, executable clinical guidelines is both worthwhile and challenging. One of the
greatest hurdles is that of representing the clinical information in a precise and shareable manner. The
Detailed Clinical Model (DCM) is an information model that is an essential part of the infrastructure
supporting the codification and standardization of evidence-based knowledge elements across
EHR systems. Each DCM is a logical model that shows the meaning, structure, possible content and
relationships between data elements for a given subject of EHR documentation. The coded elements
have explicit bindings to allowed coded values. Models are independent of any specific programming
language or type of database. DCMs support explicit, unambiguous query statements against data
instances. The DCM aids decision making by medical staff, and supports the valid reuse of medical data
for example for epidemiology, quality assurance and managerial goals.
The DCM for supporting evidence-based clinical practice is already in common use in the EHR systems of
some countries and regions. The Clinical Element Model of Intermountain Healthcare, templates defined
by HL7 International, archetypes defined by the openEHR Foundation and ISO 13606, the Clinical
Information Model in the Netherlands, and the Clinical Content Model by the Center for Interoperable
EHR in South Korea are examples of published DCM formalisms that can be used to organize the
clinical content of an EHR interoperability message and an EHR repository, share decision logic, and
build a data capture form of a clinical application. Each instance of a DCM defines how a corresponding
generic EHR representation is used to represent particular types of clinical information. Example DCMs
include representations for allergies, problem lists, laboratory tests, medication and diagnostic orders,
medication administration, adverse reaction, physical examination and clinical measurements, signs,
symptoms, diagnosis, clinical documents, procedure, family history, medical history and review of
symptoms.
According to the open EHR clinical community, medication, problem/diagnosis, adverse reaction,
vital signs group, laboratory report, alert, blood group, procedure, admission/episode and clinical
synopsis are the top ten priority archetype modelling areas. Examples of DCM instances might include
representations for documenting a pain symptom, heart sounds, liver function tests, a prescribed drug,
or a chest x-ray report. The standardization of EHR content through DCMs enables consistency and
interoperability in many possible application areas within healthcare, quality assurance, and research.
DCMs support multiple contexts as in the following:
— EHR data storage;
— message payload and service payload;
— exchanging clinical content;
— querying and analytics over clinical content;
— decision support over clinical content;
— entering clinical content;
— displaying clinical content;
— healthcare quality measures;
— clinical trials data (clinical research);
— normalization of data for secondary use;
— capture of coding output from NLP.
vi © ISO 2017 – All rights reserved

---------------------- Page: 6 ----------------------
ISO/TS 18864:2017(E)

Whereas DCMs have the potential to improve clinical decision support and clinical documentation in
EHR systems, the critical challenge is to identify the qualitative and quantitative requirements of DCMs.
A few studies have suggested some quality requirements for DCMs (Table A.1). For the flexibility and
scalability of DCMs, general requirements for the system in which clinical data models are implemented
demand the following:
a) the addition of elements and attributes to the clinical model without the necessity of changing
the underlying software or database schema;
b) use of an existing formalism/syntax for the representation of the model;
c) binding of model attributes to standard terminology systems; and
d) the existence of a mechanism for stating ‘negation’.
General principles of good modelling include:
1) adoption of standard terminologies for use in the models;
2) representing the models in standard modelling languages;
3) sharing and approving the DCMs with a community of clinical experts;
4) defining data elements for decision support analysis of EHR clinical content.
DCM quality criteria have also been proposed where the following qualities were identified as being
important requirements of a good DCM: usefulness, desirability, the degree of use/acceptance in clinical
services, reusability, the degree of clinician introduction/validation, the proper use of vocabulary,
easy mapping to information models, applicability, application to other technologies, and ease of
maintenance.
Principles for the development of DCMs can be classified as principles pertaining to the structure of
the DCM, principles for creating the DCM content, and principles for the DCM development process. The
principles that pertain to the structure of DCMs contains information about the language formalism,
description of binding of attributes to standard terminologies, a strategy for supporting semantic links
among DCM instances, the definition of standard data types, and the description of standard units of
measure. The principles for DCM content creation emphasize the granularity, reusability, correctness,
and comprehensiveness of the models. Principles for the DCM development process emphasize
evidence-based model development, the need for proper use cases, use of metadata to track changes,
and compliance to the syntax of the modelling language.
There is common knowledge and understanding about the creation and utilization of DCMs. The
Clinical Information Modeling Initiative (CIMI) is an international collaboration that is dedicated to
providing a common format for DCMs so that semantically interoperable information may be created
and shared in health records, messages and documents. The strategic goal of CIMI is to enable sharing
of data, applications, reports, alerts, protocols, and decision support modules with anyone in the world.
“Plug-n-play” interoperability is the primary goal of CIMI. To accomplish these goals, CIMI is now
developing a shared repository of detailed clinical information models. CIMI is committed to making
these specifications openly available in a number of formats, beginning with the Archetype Definition
Language (ADL) from the openEHR Foundation (ISO 13606-2) and the Unified Modeling Language (UML)
from the Object Management Group (OMG), with the intent that the users of these specifications can
convert the models into their local formats. These formalisms are based on a common set of base data
types with formal bindings of the models to standard coded terminologies. The CIMI repository is open
to everyone and models are free for use at no cost.
The CIMI Reference Model Taskforce has established a set of requirements for the CIMI Reference
Model (RM) and Clinical Modeling Language respectively. The CIMI RM is the underlying Reference
Model on which CIMI’s clinical models are defined. The CIMI RM requirements define or recommend the
candidate elements that should be in the CIMI RM. The Clinical Modeling Taskforce is working to test the
candidate reference model in the development of a set of initial clinical information models. The Clinical
Modeling Language document lists the set of known requirements of the Clinical Modeling Language.
© ISO 2017 – All rights reserved vii

---------------------- Page: 7 ----------------------
ISO/TS 18864:2017(E)

Many parties that have been involved in the various clinical modelling activities (13606/OpenEHR/HL7
templates/UML-DCM/CEML/Korean examples) are jointly working in CIMI.
Most of these aspects are specified in ISO/TS 13972, which provides characteristics, principles and
processes for development of detailed clinical models. With increasing research and implementation
of DCMs in international standards organizations and local health information systems as well, it is
important to ask how the models’ quality can be objectively measured. The last version of the CIMI RM
is 99,9 % consistent with ISO/TS 13972 characteristics. The remaining differences deal with archetype
specification typicalities which are not relevant to ISO/TS 13972, since the Technical Specification
covers multiple technical formalisms.
To illustrate the level of detail for a DCM, one example (Figure 1) is presented for heart rate, using the
ISO/TS 13972 format and the HL7 DCM project.
Figure 1 — DCM for heart rate example for the details required
Some quality requirements and criteria (see Table A.1) for DCMs proposed general principles of good
modelling, including:
i. adoption of standard terminologies for use in the models;
ii. representing the models in standard modelling languages;
iii. sharing and approving the DCMs with a community of clinical experts;
iv. defining models that can serve as unambiguous references for data used in decision modules;
v. allowing different styles of models that represent the same clinical data (though only one style
of model should be selected as the preferred style);
viii © ISO 2017 – All rights reserved

---------------------- Page: 8 ----------------------
ISO/TS 18864:2017(E)

vi. applying the DCM Quality Management system (ISO/TS 13972) for DCM maintenance and
governance.
Most of these requirements also include metadata of DCM, design and development process, governance
and management as quality aspects of DCMs. Published requirements also specify process, product and
provenance-related DCM quality. However, one critical limitation of these attempts is that there is no
measurable metric to identify qualified DCMs for intended use.
If DCMs are to adequately support the EHR documentation needs of clinical practice, be endorsed by
clinical professional bodies and health services, and be adopted by vendors, these models have to be of
good quality, trusted, and in the future, certified. This document defines a set of quality metrics which
are required to evaluate the DCMs objectively. Quality metric for DCMs specifies:
— definition;
— evaluation target;
— evaluation method;
— evaluation result to evaluate DCMs.
This set of quality metrics, which constitutes essential qualitative and quantitative quality requirements
for DCMs, can be used to support rational decision-making by DCM developers and clinical users.
© ISO 2017 – All rights reserved ix

---------------------- Page: 9 ----------------------
ISO/TS 18864:2017(E)

x © ISO 2017 – All rights reserved

---------------------- Page: 10 ----------------------
TECHNICAL SPECIFICATION ISO/TS 18864:2017(E)
Health informatics — Quality metrics for detailed
clinical models
1 Scope
The purpose of this document is to define objective, reliable and reproducible quality metrics for
detailed clinical models (DCM).
This document specifies the principal metrics which are necessary and sufficient to evaluate DCMs.
The intended audiences of this document are:
— DCM developers, all users of information represented using DCMs, and evaluators of DCM quality;
— clinical and IT professionals of healthcare institutions;
— technical staff in the healthcare technology industry;
— experts involved in standards development;
— national and regional healthcare information technology leadership including certification bodies.
This document defines a set of quality metrics required to evaluate DCMs objectively. These quality
metrics can be used to support rational decision making by DCM developers who will have the essential
qualitative and quantitative quality requirements to use as guidelines as they create new content.
Clinical users can then use the quantitative assessments as they select models for specific use cases and
implement them in their clinical systems.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
3.1
archetype
knowledge-related data structure that strongly supports semantic interoperability of EHRs
Note 1 to entry: They help to ensure reliable and easy sharing of meaningful sets of data between different
healthcare providers while allowing the re-use of their 'atomic' data components separately or within other
clinical models.
© ISO 2017 – All rights reserved 1

---------------------- Page: 11 ----------------------
ISO/TS 18864:2017(E)

3.2
archetype model
information model of the metadata to represent the domain-specific characteristics of electronic health
record entries by specifying values or value constraints for classes and attributes in the electronic
health record reference model
[SOURCE: ISO 13606-1:—, 3.14]
3.3
archetype modeling language
formal language for expressing archetypes
Note 1 to entry: It provides a formal, textual syntax for describing constraints on a domain entity whose data are
described by an information model.
3.4
attribute
characteristic of an object or entity
Note 1 to entry: In the context of this standard: a specific characteristic of a data element.
[SOURCE: ISO/IEC 11179-1:2015, 3.1.1, modified]
3.5
concept definition
description of the attributes of a concept to delineate its meaning
3.6
conceptual model
description of common concepts and their relationships, particularly in order to facilitate exchange of
information between parties within a specific domain of healthcare
[SOURCE: ENV 1613 CR 12587, modified]
3.7
content quality
quality of DCM content as determined by a set of international standard-based or evidence-based
quality criteria
3.8
data
re-interpretable representation of information in a formalized manner suitable for communication,
interpretation or processing
[SOURCE: ISO/IEC 2382:2015, 2121272, modified]
3.9
data element
unit of data that is considered, in context, to be indivisible
[SOURCE: ISO/IEC 2382:2015, 2121599, modified]
3.10
data type
set of distinct values, characterized by properties of those values, and by operations on those values
[SOURCE: ISO/IEC 11404:2007, 3.12]
2 © ISO 2017 – All rights reserved

---------------------- Page: 12 ----------------------
ISO/TS 18864:2017(E)

3.11
Detailed Clinical Model
DCM
relatively small, standalone information model designed to express a clinical concept in a standardized
and reusable manner
[SOURCE: ISO/TS 13972:2015, 2.22, modified]
3.12
logical Model
information model that specifies the structures and relationships between data elements but is
independent of any particular technology or implementation environment
3.13
electronic health record
EHR
logical representation of information regarding or relevant to the health of a subject of care
[SOURCE: ISO 18308:2011, 3.20, modified]
3.14
electronic health record architecture
EHRA
logical generic components from which electronic health records are designed, defined in terms of an
information model and computational services
[SOURCE: ISO 18308:2011, 3.21, modified]
3.15
entity
concrete or abstract thing of interest, including associations among things
[SOURCE: ISO/IEC 2382:2015, 2121433, modified]
3.16
entry
documentation of a discrete item of health information
Note 1 to entry: An entry may for example represent the documentation of a clinical observation, an inference, an
intention, a plan or an action.
[SOURCE: ISO 18308:2011, 3.24]
3.17
governance for Detailed Clinical Models
system by which development, distribution, and maintenance of detailed clinical models are directed
and controlled
[SOURCE: ISO/TS 13972:2015, 2.28, modified]
3.18
HL7 template
expression of a set of constraints on a Reference Information Model (RIM) or a RIM derived model that
is used to apply additional constraints to a portion of an instance of data which is expressed in terms of
some other Static Model, to further define and refine these existing models to specify a narrower and
more focused scope
Note 1 to entry: A template is represented by:
— a formal definition in one or more human readable languages or notations;
— [optionally] a formal definition as a static model;
© ISO 2017 – All rights reserved 3

---------------------- Page: 13 ----------------------
ISO/TS 18864:2017(E)

— [optionally] one or more i
...