ISO/IEC 18305:2016

Information technology — Real time locating systems — Test and evaluation of localization and tracking systems

ISO/IEC 18305:2016 identifies appropriate performance metrics and test & evaluation scenarios for localization and tracking systems, and it provides guidance on how best to present and visualize the T&E results. It focuses primarily on indoor environments.

Technologies de l'information - Systèmes de localisation en temps réel - Essais et évaluation des systèmes de localisation et de suivi

General Information

• Status: Published
• Publication Date: 31-Oct-2016
• ICS: 35.040 - Information coding
• ICS: 35.040.50 - Automatic identification and data capture techniques
• Technical Committee: ISO/IEC JTC 1/SC 31 - Automatic identification and data capture techniques
• Drafting Committee: ISO/IEC JTC 1/SC 31/WG 4 - Radio communications
• Current Stage: 9093 - International Standard confirmed
• Completion Date: 11-Apr-2022

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 18305
First edition
2016-11-01
Information technology — Real time
locating systems — Test and evaluation
of localization and tracking systems
Technologies de l’information - Systèmes de localisation en temps réel
- Essais et évaluation des systèmes de localisation et de suivi
Reference number
ISO/IEC 18305:2016(E)
©
ISO/IEC 2016

---------------------- Page: 1 ----------------------
ISO/IEC 18305:2016(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2016 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC 18305:2016(E)

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 LTS taxonomy . 3
5.1 Types of location sensors . 3
5.1.1 Unimodal systems . 4
5.1.2 Multimodal systems . 5
5.2 Reliance on pre-existing networking / localization infrastructure . 5
5.2.1 LTSs requiring infrastructure . 5
5.2.2 LTSs capable of infrastructure-less operation . 5
5.2.3 Real-time deployment of nodes facilitating localization . 6
5.2.4 Opportunistic use of infrastructure/environment . 6
5.3 Off-line, building-specific training . 6
5.3.1 LTSs requiring off-line training . 6
5.3.2 LTSs not requiring off-line training . 7
5.4 Ultimate consumer(s) of location information . 7
5.4.1 Introduction . 7
5.4.2 The ELT . 8
5.4.3 The tracking authority . 8
5.4.4 Both the ELT and the tracking authority . 8
6 LTS privacy and security considerations . 8
6.1 Privacy . 8
6.2 Security . 9
7 T&E methodology taxonomy . 9
7.1 System vs. component testing . 9
7.1.1 System testing . 9
7.1.2 Component testing . 9
7.2 Knowledge about LTS inner-workings .10
7.2.1 T&E designed with full knowledge of LTS inner-workings .10
7.2.2 Black-box testing . .10
7.3 Repeatability .10
7.3.1 Repeatable testing .10
7.3.2 Non-repeatable testing .10
7.4 Test site .10
7.4.1 Building-wide testing . .10
7.4.2 Laboratory testing .11
7.5 Ground truth .11
7.5.1 Off-line surveyed test points .11
7.5.2 Reference LTS.11
8 LTS performance metrics .12
8.1 Introduction .12
8.2 Floor detection probability .13
8.3 Zone detection probability .13
8.4 Means of various errors .13
8.5 Covariance matrix of the error vector .14
8.6 Variances of magnitudes of various errors .14
8.7 RMS values of various errors .15
8.8 Absolute mean of the error vector .15
© ISO/IEC 2016 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC 18305:2016(E)

8.9 Circular Error 95% (CE95) and Circular Error Probable (CEP) .15
8.10 Vertical Error 95% (VE95) and Vertical Error Probable (VEP) .16
8.11 Spherical Error 95% (SE95) and Spherical Error Probable (SEP) .16
8.12 Coverage .16
8.13 Relative accuracy .17
8.14 Latency .17
8.15 Set-up time .18
8.16 Optional performance metrics .18
8.16.1 Location-specific accuracy .18
8.16.2 Availability.19
9 Optional performance metrics for LTS use in mission critical applications .19
9.1 Introduction .19
9.2 Susceptibility .20
9.3 Resilience .20
10 T&E considerations and scenarios .20
10.1 Building types .20
10.1.1 Introduction .20
10.1.2 Wooden structure single-family house .20
10.1.3 Medium-size brick & concrete office building .21
10.1.4 Warehouse/factory .21
10.1.5 High-rise steel structure.21
10.1.6 Subterranean structure .21
10.2 Effects of mobility .21
10.2.1 Introduction .21
10.2.2 Stationary object/person .22
10.2.3 Walking .22
10.2.4 Running .22
10.2.5 Backward walking .22
10.2.6 Sidestepping .22
10.2.7 Crawling.22
10.3 Failure modes and vulnerabilities of location sensors .23
10.4 T&E scenarios .23
11 T&E reporting requirements .30
11.1 Introduction .30
11.2 Test place and date .33
11.3 Environmental conditions .33
11.4 LTS product tested .33
11.5 Equipment used by the LTS .33
11.6 ELTD features .33
11.7 Location data format .34
11.8 Location update rate and system capacity .34
11.9 RF emission and interference issues .34
11.10 Set-up procedure .35
11.11 Building information needed by the LTS .35
11.12 LTS GUIs .36
11.12.1 ELTD GUI .36
11.12.2 Tracking authority GUI .36
11.13 Maintenance .36
11.14 Floor plans of test buildings .37
11.15 Characterization of T&E scenarios involving entities in motion .37
11.16 Presentation of numerical T&E results .38
11.17 Visualization of T&E results .43
Annex A (normative) Conversions between local Cartesian and WGS 84 coordinates .47
Annex B (informative) Location sensors and their failure modes .64
Bibliography .76
iv © ISO/IEC 2016 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC 18305:2016(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information.
ISO/IEC 18305 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 31, Automatic identification and data capture techniques.
© ISO/IEC 2016 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/IEC 18305:2016(E)

Introduction
There exists a potentially large market for personnel / asset Localization and Tracking Systems (LTSs)
in diverse application domains such as:
— emergency response;
— military;
— law enforcement;
— mining;
— E-911;
— offender tracking;
— personal vehicular navigation;
— smart phones / social networking;
— fleet management;
— asset tracking in factories / warehouses / hospitals;
— tracking the elderly / children; and
— personal navigation in museums / shopping malls.
Some applications of localization and tracking – such as personal navigation, fleet management, and
asset tracking in factories / warehouses / hospitals – are commonly referred to as Location-Based
Services (LBS). The use of LBS alone is expected to grow dramatically by 2020. Yet, lack of standardized
Test and Evaluation (T&E) procedures has been an impediment to market growth for LTSs, because:
i) potential users cannot easily determine whether these systems meet the users’ requirements;
ii) it is hard to interpret T&E results when different metrics and procedures are used to evaluate a
given system or even worse to evaluate different systems; and
iii) the use of disparate minimum performance requirements by various buyers / jurisdictions
forces manufacturers to develop jurisdiction-specific products, thereby raising manufacturing costs.
In contrast with LBS, there are many applications of localization and tracking that are essentially
governmental functions in the sense that the government is the entity that is most concerned about
the effectiveness of solutions for such applications. Examples of these applications include tracking
firefighters entering a burning structure for command and control purposes and to launch a rescue
mission if a firefighter becomes incapacitated, prevention of friendly fire when soldiers or Special
Weapons And Tactics (SWAT) team members enter a building where either hostile forces or armed
individuals threatening public safety have taken refuge, and guidance and navigation for missiles and
precision-guided munitions. Many of these applications have more stringent localization accuracy and
latency requirements than other applications of localization and tracking used by the general public,
such as navigation in museums / shopping malls, tracking the elderly in nursing homes, ensuring
children are not abducted from school grounds, and fleet management for a trucking company.
This document deals with T&E of LTSs. Once standardized T&E procedures have been established,
it is possible to set minimum performance requirements for various applications of localization and
tracking. For example, regulations promulgated by a government agency may require coal mine
operators to have the capability to track the miners on duty within 5 m accuracy during normal mine
operations and 100 m accuracy in the aftermath of a catastrophic incident in the mine, such as an
explosion or a roof collapse. It makes sense to separate the T&E issue from minimum performance
requirements, because the same T&E standard may be applicable to many applications of localization
and tracking, but the minimum performance requirements typically vary from one application to
vi © ISO/IEC 2016 – All rights reserved

---------------------- Page: 6 ----------------------
ISO/IEC 18305:2016(E)

another. This document deals with T&E only; it does not set minimum performance requirements for
any localization and tracking applications.
T&E of LTSs is challenging for several reasons:
i) Many systems work in a “networked” fashion. That is, several devices would have to
communicate with each other in order to estimate the location(s) of one or more such devices. Therefore,
the LTS performance is affected by how these devices are situated with respect to each other, i.e. by the
network topology.
ii) The physical environment in which the devices are situated affects communications between
them and functionalities such as ranging or estimating direction of another device and hence LTS
performance. For example, Radio Frequency (RF) communications in a single-family house with a
wooden structure is very different from that in a large high-rise building with a steel and concrete
structure.
iii) Even though it is best to take a “black-box” approach to LTS T&E, one needs to be cognizant of
the failure modes of various location sensors (such as Global Positioning System (GPS), RF ranging, RF
direction of arrival estimation, accelerometer, gyroscope, and altimeter) that “might” be used in an LTS
in order to design a comprehensive T&E procedure.
Yet another difficulty of a different nature is that some systems rely on the availability of a networking
infrastructure, such as a Wi-Fi network, or other devices, such as Radio Frequency IDentification
(RFID) or Real Time Locating System (RTLS) tags, to facilitate localization and tracking in a building or
structure. Some allow deployment of such devices – sometimes called “breadcrumbs” – as users enter a
building. Other systems are designed to function based on the assumption that they cannot get any help
with localization and tracking from the building and breadcrumb deployment is not allowed. Therefore,
the T&E procedure has to account for these possibilities or classes of LTSs.
The main purpose of this document is to develop performance metrics and T&E scenarios for LTSs. LBS
are envisaged in many application domains in both governmental operations and general public usage
scenarios. Therefore, industry, consumers, trade, governments, and distributors are all affected by this
document. Every effort has been made to write this document in such a way that it would be applicable
to as many applications of indoor localization and tracking as possible. This document provides explicit
instructions on how to report the T&E results, i.e. what information to document and what kind of tables
and figures/plots to include to best visualize the results of the T&E effort. LTS T&E is complicated even
once this document has been published, because there has to be a “network deployment” and testing
in at least a few types of buildings. One should not expect that LTS T&E can be done in a laboratory.
Performance results can depend on the particular building(s) used in the T&E procedure, but at least
there will be a standardized way of doing the T&E, and if multiple LTSs are evaluated according to
the standard in the same set of buildings, then the performance results can be compared. Localization
and tracking technology has not yet matured. New systems and approaches will be developed in the
next several years, but the T&E procedure can be standardized regardless of what takes place on the
technology front and it may in fact foster technology development. In the absence of a T&E standard, the
present uncertainties in the LTS market, where it is hard for users to asce
...