Health Informatics -- Terminology resource map quality measures (MapQual)

This document provides quality requirements for producing a quality map between terminological systems. This document establishes measures which can be used to assess the quality and utility of a map between terminological resources in order to determine the types and levels of measure required for common use cases in healthcare. NOTE Examples of such cases include conformity assessment.

Titre manque

General Information

Status
Published
Publication Date
17-Jun-2019
Current Stage
6060 - International Standard published
Start Date
17-May-2019
Completion Date
18-Jun-2019
Ref Project

Buy Standard

Technical specification
ISO/TS 21564:2019 - Health Informatics -- Terminology resource map quality measures (MapQual)
English language
25 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

TECHNICAL ISO/TS
SPECIFICATION 21564
First edition
2019-06
Health Informatics — Terminology
resource map quality measures
(MapQual)
Reference number
ISO/TS 21564:2019(E)
ISO 2019
---------------------- Page: 1 ----------------------
ISO/TS 21564:2019(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2019

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2019 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TS 21564:2019(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Determinants of map quality ................................................................................................................................................................... 4

4.1 General ........................................................................................................................................................................................................... 4

4.2 Terminological resource capacity ........................................................................................................................................... 4

4.2.1 General...................................................................................................................................................................................... 4

4.2.2 Determinant 1: Common categorial structure ....................................................................................... 4

4.2.3 Determinant 2: Shared semantic domain ................................................................................................... 5

4.2.4 Determinant 3: Language and translation ................................................................................................. 6

4.3 Equivalence of individual maps ................................................................................................................................................ 7

4.3.1 General...................................................................................................................................................................................... 7

4.3.2 Determinant 4: Equivalence identification/Publication ............................................................... 7

4.3.3 Determinant 5: Equivalence assessment .................................................................................................... 7

4.3.4 Determinant 6: Map set outliers ........................................................................................................................ 8

4.4 Building a map set ................................................................................................................................................................................ 9

4.4.1 Map development process ....................................................................................................................................... 9

4.4.2 Determinant 7: Clear documentation of the purpose of the map .......................................... 9

4.4.3 Determinant 8: Currency of the map .............................................................................................................. 9

4.4.4 Determinant 9: Business arrangements ...................................................................................................10

4.4.5 Determinant 10: Methodology documentation ..................................................................................10

4.4.6 Validation ............................................................................................................................................................................11

4.4.7 Determinant 13: Decision making — Consensus building process ..................................12

4.4.8 Determinant 14: Tools used to develop or maintain the map ...............................................13

4.4.9 Determinant 15: Workforce ................................................................................................................................14

4.5 Map governance and maintenance .....................................................................................................................................16

4.5.1 Determinant 16: Governance .............................................................................................................................16

4.5.2 Determinant 17: Map maintenance ..............................................................................................................16

5 Using map quality determinants ......................................................................................................................................................17

5.1 Required determinants .................................................................................................................................................................17

5.2 Level of quality .....................................................................................................................................................................................17

5.2.1 Step 1: Establish map quality requirement ...........................................................................................17

5.2.2 Step 2: Assess map against requirement .................................................................................................17

5.2.3 Step 3: Calculate the score and actions required ..............................................................................17

6 Use cases ....................................................................................................................................................................................................................18

6.1 General ........................................................................................................................................................................................................18

6.2 Determining requirements for a purpose ......... ............................................................................................................18

6.2.1 General...................................................................................................................................................................................18

6.3 Direct patient care use case ......................................................................................................................................................19

6.3.1 General...................................................................................................................................................................................19

6.3.2 Direct patient care level of conformance required and rationale .......................................19

6.4 Administrative, financial or service planning use case .....................................................................................20

6.4.1 General...................................................................................................................................................................................20

6.5 Administrative, financial or service planning level of conformance required and

rationale .....................................................................................................................................................................................................21

6.6 Other use cases ....................................................................................................................................................................................22

Annex A (informative) Example of map quality evaluation .......................................................................................................23

Bibliography .............................................................................................................................................................................................................................24

© ISO 2019 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TS 21564:2019(E)
Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards

bodies (ISO member bodies). The work of preparing International Standards is normally carried out

through ISO technical committees. Each member body interested in a subject for which a technical

committee has been established has the right to be represented on that committee. International

organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.

ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of

electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the

different types of ISO documents should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of

any patent rights identified during the development of the document will be in the Introduction and/or

on the ISO list of patent declarations received (see www .iso .org/patents).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso

.org/iso/foreword .html.

This document was prepared by Technical Committee ISO/TC 215, Health Informatics.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO 2019 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TS 21564:2019(E)
Introduction
0.1 General

Healthcare organizations and software vendors are increasingly using maps to convert data from one

code system to another code system. In the past, data in health information systems was largely used

for organizations’ administrative planning and decision making. Data captured in Electronic Health

Records (EHR) systems for patient care has a significant impact on patient safety. The use of this data

as the source of data for other purposes and for information exchange in clinical care through the use of

information technology is an emerging problem. Where that data is translated through maps from one

code system to another, the safety and quality issues associated with data use can be significant. The

increasing use of maps is costly.

The objective of this work is to support the definition of quality requirements for map sets to

a) establish standard quality conformance requirements for a map for a purpose,
b) assess the quality of a map for a purpose,
c) guide decision makers in map project requirements and processes, and
d) establish pathways to improvement.

Maps are widely used but the quality of these maps cannot be accurately and consistently assessed and

compared against their intended use. It is not currently possible for decision makers to assess whether

a map will be worth the cost of building and whether the scope and map processes will deliver a map

which is able to meet the intended business case.
[1]

This document is based upon ISO/TR 12300 . Some terminological resources are so different in their

content and purpose that they will never map closely to a resource designed and structured differently.

Therefore, the decision maker might need to consider whether to map at all or to move to a new

terminological resource.

Quality measures consider a wide range of requirements and processes relevant to the creation and

maintenance of data maps and their use (including manual and tool-based mapping), as well as for the

map sets delivered as a result of using that process.
0.2 Stakeholders and audience
This document is focused on the needs of
a) implementers and software vendors developing and implementing maps sets,
b) health information and data managers developing and using maps sets,
c) data users such as researchers, government, decision makers, and

d) developers of map sets including all in mapping teams including terminologists, coders, clinical

users, epidemiologists and statisticians, project managers.
Additionally, the target audience for this document might include

— procurement officers who establish requirements of map product capacity and quality, or

— decision makers to determine and assess resources needed in projects and services associated with

map produce, maintenance or use.
0.3 Challenges of mapping

Healthcare organizations and software vendors are increasingly using maps to convert data from one

terminological resource to another terminological resource. In the past, data in health information

systems was largely used for organizations’ administrative planning and decision making.

© ISO 2019 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/TS 21564:2019(E)

Today, maps are being used for a much broader range of use cases and the challenges of their use include

the following:

a) Map purpose — a map built for one purpose might or might not suit use for other purposes. It is

important to establish the purpose and use of a map at the beginning of a project to ensure the best

result when building a map from a source code to a target code. When the purpose changes, the

resultant map content is likely to need to be different.

b) Map accuracy — there are three broad aspects to accuracy. The first is whether the map

development and maintained. The second is how closely the results of applying the map deliver

an outcome consistent in meaning to that of original source data. The third is the ability of the

outcome of the map to be used for the purpose intended.

c) Map effectiveness — Information retrieval is a critical functionality of maps.

The actual consequence of assigned map links imposed between terms of different code schemes

impacts the effectiveness of information retrieval searches. Map purpose and accuracy might both

impact the potential safety and appropriateness of the use of that map in healthcare. If the original

meaning is changed through use of a map, this might impact clinical safety. There is also the

consideration of whether the map is applied consistently to defined data elements in the health record.

The data element in which the original source data is recorded might add meaning to the code allocated

(e.g. family history of condition versus clinical diagnosis of the individual).

Another significant issue is the cost of creation and maintenance of a map and the ongoing risk and

difficulties of maintaining currency of the map.
More information on this topic is available in ISO/TR 12300.

If map quality is neglected, maps will continue to be classified in non-standard ways, increasing barriers

to establishing the purpose, accuracy, effectiveness of the quality of terminological maps. The longer

the international community is without a publication in this area, the more expensive the problem will

be to resolve due to the persistence of legacy metadata and the cost of modifying existing mapping

processes to fit an agreed specification; therefore, a TS solution is highly desirable.

vi © ISO 2019 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL SPECIFICATION ISO/TS 21564:2019(E)
Health Informatics — Terminology resource map quality
measures (MapQual)
1 Scope

This document provides quality requirements for producing a quality map between terminological

systems.

This document establishes measures which can be used to assess the quality and utility of a map

between terminological resources in order to determine the types and levels of measure required for

common use cases in healthcare.
NOTE Examples of such cases include conformity assessment.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
3.1
auto-matching

computational mapping task, undertaken using an algorithm based upon the relationship between

concepts (3.5)
3.2
categorial structure

reduced system of concepts to describe the organization of the semantic categories in a particular

system of concepts

Note 1 to entry: A categorial structure for body structure representation could include the categories for body

system (e.g. skin, digestive) and anatomical location (upper body, abdomen).
3.3
classification

exhaustive set of mutually exclusive categories to aggregate data at a pre-prescribed level of

specialization for a specific purpose
3.4
code system

organized, managed collection of codes each of which has associated designations, meanings and in

some cases relationships, properties or rules

Note 1 to entry: Code systems are often described as collections of uniquely identifiable concepts such as ICD-10,

SNOMED CT and LOINC. Code systems are often established and maintained by authoritative sources such as

standards development organisations.
© ISO 2019 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/TS 21564:2019(E)
3.5
concept
unit of knowledge created by a unique combination of characteristics

Note 1 to entry: Concepts are not necessarily bound to particular languages. They are, however, influenced by

the social or cultural background, often leading to different categorizations.
3.6
context

related conditions and situations that provide a useful understanding and meaning of a subject

3.7
currency of the map
map currency

difference between the date of release of the target and source terminological resources and the map

set (3.16)
3.8
determinant
influencing element or factor
3.9
equivalence
semantic equivalence
condition of being equal or the same in value, worth or function

Note 1 to entry: In terminological systems, two concepts are (semantically) equivalent if their domain of meaning

overlaps and their semantic definitions are interpreted as identical. In the context of terminological resources

equivalence and semantic equivalence are often considered as synonyms.
3.10
individual map

index from one term to another, sometimes using rules that allow translation from one representation

to another indicating degree of equivalence

Note 1 to entry: Entry in a map which indicates how to translate from an individual source concept to a target

concept. The term map is often used to indicate a table of individual map entries. It is for this reason that the

individual and map tables are being differentiated. A map is often computable. A map is the outcome of the

mapping process. The use of this term is often used in ways which are confusing. It is essential to always make it

clear whether one is referring to an individual map, or a map table (or set). In SNOMED CT, each individual map

is represented as a row or group of rows in a map reference set. It links a single map source concept code (e.g.

SNOMED CT Concept ID) to one or more codes in a map target (e.g. ICD Code).
3.11
priority map content

individual maps (3.10) in a map set (3.16) which are most important for a use case

Note 1 to entry: Importance might reflect the frequency of use or impact of the specific concepts being mapped.

For example: a propriety map content set might be established to represent the most common diagnoses in

hospitals for morbidity reporting and this could be the priority map content for a map from SNOMED CT to ICD

based classification.
3.12
map
conversion
transformation
cross-map

device which provides an index from one term (3.19) to another, sometimes using rules that allow

translation from one representation to another indicating degree of equivalence (3.9)

Note 1 to entry: The index is used to convert concepts in one code system or representation into concepts in an

alternative code system or representation.
2 © ISO 2019 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/TS 21564:2019(E)
3.13
map quality determinant

attribute of a map (3.12), map development process, or map metadata that is considered a reliable

measure of the suitability of the map to a use case
3.14
map quality measure
quantitative measure of the characteristics and attributes of a map (3.12)
3.15
map source
source mapping

terminology (3.21), coding scheme or classification (3.3) used as the starting point for map production

3.16
map set
map table

group of individual maps (3.10) used to convert a range of entries from source to target code system (3.4)

3.17
mapping

process of defining, building or using a relationship between concepts (3.5) in one coding system to

concepts in another coding system in accordance with a documented rationale, for a given purpose

Note 1 to entry: Quality mapping will produce a usable map table, be a reproducible and understandable process. It

is the relation with the best semantic correspondence between an element in one set and an element in another set.

3.18
semantic domain
semantic space
area of meaning covered by a terminological resource (3.20)

Note 1 to entry: This is used to evaluate the lexical or formal overlap between such resources. In value set

specification this might also be called a value set domain.

Note 2 to entry: Terminology resources can include value sets, code systems, and subsets.

EXAMPLE One code system might have the domain of anatomy while another might have the domain of

disease. Though these are related concepts, the semantic domain of each code system is different.

3.19
term
linguistic representation of a concept (3.5)

Note 1 to entry: A term can contain symbols and have variants, e.g. different forms of spelling terms are members

of a terminology; a defined or limited vocabulary of terms or concepts, for example, ICD, SNOMED CT, LOINC.

3.20
terminological resource
controlled set of terms in healthcare

Note 1 to entry: These terms are usually designed and controlled for use with computers for specific healthcare

purpose, such as data entry, aggregation, retrieval and analysis.
3.21
terminology
structured, human and machine-readable representation of concepts (3.5)

Note 1 to entry: This includes the relationship of the terminology to the specifications for organizing,

communicating and interpreting such a set of concepts. The use of the term terminology in healthcare implies a

terminology that is designed for use in computer systems. The word 'vocabulary' or 'health' or 'medical language'

is used to indicate the broader idea of linguistic representation without the specification of computability.

© ISO 2019 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/TS 21564:2019(E)
4 Determinants of map quality
4.1 General

This document defines a set of quality determinants which cover the development and maintenance of

map content, and the precision of the map between source and target content. The precision represents

accuracy but also maintainability and usability.

Each determinant shall be measured separately in order to allow evaluation of purpose for use case,

and in that sense, stand alone. The specific measures are listed for each determinant under "Measure".

The lower the number allocated to the measure, the higher the quality of the map. These measures

are intended to be used to assess the quality of a map. The required level of conformance for each

determinant differs according to the use case for which the map is intended. Clause 5 covers map

quality measurement and requirement specification for specific use cases, explaining how to apply the

determinant measures of map quality.
4.2 Terminological resource capacity
4.2.1 General

To assess the quality of a map, it is necessary to understand the capacity and intent of the source code

system and target code system, and the relationship between how each of these code systems represent

concepts.
4.2.2 Determinant 1: Common categorial structure

Evaluate whether the target and the source terminological resource share the same categorial structure.

This determinant seeks to specify whether the structure of each system is common.

EXAMPLE
If one terminological resource has a structure which includes
— clinical findings,
— substances, and
— events
And the other has a structure of
— body systems
The terminological resources do not share a common categorial structure.

The degree of commonality affects the ability to produce a meaningful map. If there is no categorial

structure, or such a structure is not applied to the terminological resources, the ability to map between

terminological resources is less likely to deliver a high-quality product. The impact upon quality might

differ depending on the intended use case.

Categorial structure represents the structure within a semantic domain. Maps may be developed

across an entire categorial structure or a part of that structure, depending upon the semantic domain

intended for use in the map.
Measure:

0 — source and target terminological resources share the same categorial structure.

4 — source and target do not share the same categorial structure.
4 © ISO 2019 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/TS 21564:2019(E)
4.2.3 Determinant 2: Shared semantic domain

The ability to map one terminological resource to another assumes that each terminological resource

shares a common scope of meaning, i.e. that the code system for apples applies to apples and the second

code system for fruit also includes apples, if each code system does not share some concepts, it is not

possible to produce a meaningful map between these code systems. It is necessary to assess whether

the semantic domains are the same, overlapping, inclusive or without overlap. To evaluate the likely

utility of a map, it is necessary to consider the amount of overlap.

The same semantic domain is where both terminological resources describe the same content, though

they may describe it in different ways, with different categorial structures.
EXAMPLE

a) The categorial structure is different to the thing (the object the intangible concept of an apple – the apple

itself) being described by the code (a code which represents an apple).

b) The semantic domain is different — one is describing the thing and its uses while the second describes the

suitable uses — the intended meanings differ. This provides an overlap of semantic domain which might or

might not impact the quality of the map or the usability of the map.

Terminological resource 1 describes apples as eating or cooking apples, this might include additional attributes

such as apple colour, origin, cultivar, etc.
Terminological resource 2 describes apples by colour and suitable uses.

Each terminological resource describes apples though the categorial structures are different. In the

example of apples, each concept has the attributes of colour and use, which are areas of overlap, between

these terminological resources but do not share other key attributes such as origin, cultivar.

Where the attributes are represented differently in each code system but share a semantic domain, it

can be possible to map these concepts. Table 1 shows a shared semantic domain which would result in a

quality map. Table 2 indicates
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.