ISO/TS 17444-2:2017
(Main)Electronic fee collection — Charging performance — Part 2: Examination framework
Electronic fee collection — Charging performance — Part 2: Examination framework
ISO/TS 17444-2:2017 defines the Examination Framework for the measurement of charging performance metrics defined in ISO/TS 17444‑1 to be used during Evaluation and/or on-going Monitoring. It specifies a method for the specification and documentation of a Specific Examination Framework which can be used by the responsible entity to evaluate charging performance for a particular information exchange interface or for overall charging performance within a Toll Scheme. It provides a toolbox of Examination Tests for the roles of Toll Charger and Toll Service Provider for the following Scheme types: a) DSRC Discrete; b) Autonomous Discrete; c) Autonomous Continuous. The detailed choice of the set of examination tests to be used depends on the application and the respective context. Compliance with this specification is understood as using the definitions and prescriptions laid out in ISO/TS 17444-2:2017 whenever the respective system aspects are subjected to performance measurements, rather than using other definitions and examination methods than the ones specified in ISO/TS 17444-2:2017. The following aspects are outside the scope of ISO/TS 17444-2:2017. - ISO/TS 17444-2:2017 does not propose specific numeric performance bounds, or average or worst-case error bounds in percentage or monetary units. Those decisions are left to the Toll Charger (or to agreements between Toll Charger and Service Provider). ISO/TS 17444-2:2017 does not consider the evaluation of the expected performance of a system based on modelling and measured data from trial at another place. - ISO/TS 17444-2:2017 does not consider the specification of a common reference system which would be required for comparison of performance between systems. - ISO/TS 17444-2:2017 defines measurements only on standardized interfaces. Proprietary interfaces are excluded, because it is not possible to define standardized metrics on such system properties. These excluded interfaces are among others the link between Toll Charger RSE and central systems in DSRC systems, and the additional sensor input of GNSS modules (inertial sensors, CAN-bus for wheel ticks, etc.).
Perception du télépéage — Performance d'imputation — Partie 2: Cadre d'examen
General Information
Relations
Standards Content (Sample)
TECHNICAL ISO/TS
SPECIFICATION 17444-2
Second edition
2017-09
Electronic fee collection — Charging
performance —
Part 2:
Examination framework
Perception du télépéage — Performance d'imputation —
Partie 2: Cadre d'examen
Reference number
ISO/TS 17444-2:2017(E)
©
ISO 2017
---------------------- Page: 1 ----------------------
ISO/TS 17444-2:2017(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO 2017 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TS 17444-2:2017(E)
Contents Page
Foreword .vi
Introduction .vii
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 2
4 Symbols and abbreviated terms . 6
5 Examination Framework . 7
5.1 General . 7
5.2 Method for defining a Specific Examination Framework . 8
5.2.1 General. 8
5.2.2 Selection of metrics to be evaluated . 9
5.2.3 Definition of environmental conditions and associated
performance requirements . . 9
5.2.4 Determination of Required Sample Sizes . 9
5.2.5 Selection of methods for generating Charging Input and Reference Data .10
5.2.6 Determination of Test Routes/Subset of Charged Network for generating
representative trips .10
5.2.7 Documentation of the Specific Examination Framework .10
5.3 Sources of data .10
5.4 Methods of generating charging input .13
5.4.1 General.13
5.4.2 Predefined routes (identifier: “PVP”) .14
5.4.3 Reference System (used in combination with identifiers: “PVR” and “UVR”) .15
5.4.4 Simulated OBE/FE (identifier: “SO”) .16
5.4.5 Dedicated OBE Testing (identifier: “DO”) .17
5.5 Applicability of metrics scheme types .17
5.6 Charging Metric Selection Tables .22
5.6.1 General.22
5.6.2 DSRC Discrete .22
5.6.3 Autonomous Discrete .23
5.6.4 Autonomous Continuous .24
6 Examination Tests .26
6.1 Common (and DSRC Discrete) Examination Tests .26
6.1.1 General.26
6.1.2 ET-CM-E2E-1 E2E — Correct Charging Rate .26
6.1.3 ET-CM-E2E-2 E2E — Overcharging Rate .27
6.1.4 ET-CM-E2E-3 E2E — Undercharging Rate.27
6.1.5 ET-CM-E2E-4 E2E — Late Charging Rate .28
6.1.6 ET-CM-UA-1 UA — Correct Charging Rate .29
6.1.7 ET-CM-UA-2 UA — Overcharging Rate .29
6.1.8 ET-CM-UA-3 UA — Undercharging Rate .30
6.1.9 ET-CM-UA-4 UA — Accurate application of Payments and Refunds .31
6.1.10 ET-CM-UA-5 UA — Accurate Personalisation of OBUs .31
6.1.11 ET-CM-PC-1 PC — Correct Charging Rate .32
6.1.12 ET-CM-PC-2 PC — Overcharging Rate .32
6.1.13 ET-CM-PC-3 PC — Undercharging Rate .33
6.1.14 ET-CM-PC-4 PC — Latency — TC .34
6.1.15 ET-CM-PC-5 PC — Late Payment Claims Rate .34
6.1.16 ET-CM-PC-6 PC — Rejected Payment Claim Rate .35
6.1.17 ET-CM-BD-1 BD — Correct Charging Rate.35
6.1.18 ET-CM-BD-2 BD — Overcharging Rate .36
6.1.19 ET-CM-BD-3 BD — Undercharging Rate .37
© ISO 2017 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TS 17444-2:2017(E)
6.1.20 ET-CM-BD-4 BD — Incorrect Charging Rate .37
6.1.21 ET-CM-BD-5 BD — Latency — TC .38
6.1.22 ET-CM-BD-6 BD — Late Billing Details Rate .38
6.1.23 ET-CM-BD-7 BD — Rejected Billing Details Rate .39
6.1.24 ET-CM-BD-8 BD — Incorrectly rejected Billing Details Rate .39
6.1.25 ET-CM-BD-9 BD — Inferred Billing Details Rate .40
6.2 DSRC Discrete — Optional DSRC Toll Declaration Metrics .40
6.2.1 General.40
6.2.2 ET-CM-TD-1 TD — Correct Toll Declaration Generation Rate .40
6.2.3 ET-CM-TD-2 TD — Incorrect Toll Declaration Generation Rate .41
6.2.4 ET-CM-TD-3 TD — Late Toll Declarations Rate .41
6.2.5 ET-CM-TD-4 TD — TSP Charge Parameter Change Rate .42
6.2.6 ET-CM-TD-5 TD — TSP False Positive Rate .42
6.3 Autonomous Discrete Specific Examination Tests .43
6.3.1 General.43
6.3.2 ET-CM-TD-1 TD — Correct Toll Declaration Generation Rate .43
6.3.3 ET-CM-TD-2 TD — Incorrect Toll Declaration Generation Rate .43
6.3.4 ET-CM-TD-3 TD — Late Toll Declarations Rate .44
6.3.5 ET-CM-TD-4 TD — TSP Charge Parameter Change Rate .44
6.3.6 ET-CM-TD-5 TD — TSP False Positive Rate .45
6.3.7 ET-CM-DTD-1 DTD — Correct Charging Rate (charge object detections) .45
6.3.8 ET-CM-DTD-2 DTD — Incorrect Charge Event Recognition Rate .46
6.3.9 ET-CM-DTD-3 DTD — Missed Charge Object Detection Rate .46
6.3.10 ET-CM-DTD-4 DTD Overcharging Rate .47
6.3.11 ET-CM-CR-1 CR — Correct Charge Report Generation Rate .47
6.3.12 ET-CM-CR-2 CR — Incorrect Charge Report Generation Rate .48
6.3.13 ET-CM-CR-3 CR — Charge Report Latency .48
6.3.14 ET-CM-CR-4 CR — TSP Front End Charge Parameter Change Rate .49
6.3.15 ET-CM-CR-5 CR — TSP Front End False Positive Rate .49
6.3.16 ET-CM-DCR-1 DCR — Correct Charging Rate (charge object detections) .50
6.3.17 ET-CM-DCR-2 DCR — Incorrect Charge Event Recognition Rate .50
6.3.18 ET-CM-DCR-3 DCR — Missed Charge Object Detection Rate .51
6.3.19 ET-CM-DCR-4 DCR — Overcharging rate (Incorrect false positive Charge
Event Recognition) .51
6.4 Autonomous Continuous Specific Examination Tests .52
6.4.1 General.52
6.4.2 ET-CM-TD-1 TD — Correct Toll Declaration Generation Rate .52
6.4.3 ET-CM-TD-2 TD — Incorrect Toll Declaration Generation Rate .52
6.4.4 ET-CM-TD-3 TD — Late Toll Declarations Rate .53
6.4.5 ET-CM-TD-4 TD — TSP Charge Parameter Change Rate .53
6.4.6 ET-CM-TD-5 TD — TSP False Positive Rate .54
6.4.7 ET-CM-CTD-1 CTD Correct Charging Rate .54
6.4.8 ET-CM-CTD-2 CTD Overcharging Rate .55
6.4.9 ET-CM-CTD-3 CTD Accuracy of Distance/Time Measurement .55
6.4.10 ET-CM-CR-1 CR — Correct Charge Report Generation Rate .56
6.4.11 ET-CM-CR-2 CR — Incorrect Charge Report Generation Rate .56
6.4.12 ET-CM-CR-3 CR — Charge Report Latency .57
6.4.13 ET-CM-CR-4 CR — TSP Front End Charge Parameter Change Rate .57
6.4.14 ET-CM-CR-5 CR — TSP Front End False Positive Rate .58
6.4.15 ET-CM-CCR-1 CCR — Correct Charging Rate .58
6.4.16 ET-CM-CCR-2 CCR — Overcharging Rate .59
6.4.17 ET-CM-CCR-3 CCR — Accuracy of Distance/Time Measurement .59
Annex A (informative) Examination Test documentation template .61
Annex B (informative) Examination Framework considerations .62
Annex C (informative) Statistical considerations .66
iv © ISO 2017 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TS 17444-2:2017(E)
Annex D (informative) Methods for reducing sample sizes for very high/low probability
metrics during the Evaluation Phase .72
Annex E (informative) Example-specific Examination Frameworks .76
Bibliography .93
© ISO 2017 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/TS 17444-2:2017(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 204, Intelligent transport systems.
This second edition cancels and replaces the first edition (ISO/TS 17444-2:2013), which has been
revised with the following changes:
— editorial and formal corrections, as well as changes, to improve readability;
— updated terminology.
A list of all parts in the ISO/TS 17444 series can be found on the ISO website.
vi © ISO 2017 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/TS 17444-2:2017(E)
Introduction
Electronic tolling systems are complex distributed systems involving critical technology such as
dedicated short-range communication (DSRC) and global navigation satellite systems (GNSS), both
subject to a certain random behaviour that may affect the computation of the charges. Thus, in
order to protect the interests of the different involved stakeholders, in particular Service Users and
Toll Chargers, it is essential to define metrics that measure the performance of the system as far as
computation of charges is concerned and ensure that the potential resulting errors in terms of size and
probability are acceptable. These metrics will be an essential tool when establishing requirements for
the systems and also for examination of the system capabilities both during acceptance and during the
operational life of the system.
In addition, in order to ensure the interoperability of different systems, it will be necessary to agree on
common metrics to be used and on the actual values that define the required acceptable performances
although this is not covered in this document.
This document is defined as a toolbox standard of examination tests plus a method for defining and
documenting Specific Examination Frameworks to meet specific needs. The detailed choice of the set
of examination tests within an Examination Framework depends on the application and the respective
context. Compliance with this specification is understood as using the definitions and prescriptions
laid out in this document whenever the respective system aspects are subjected to performance
measurements, rather than using other definitions and examination methods than the ones specified in
this document.
ISO/TS 17444-1 defines a set of charging performance metrics with appropriate definitions, principles
and formulations, which together make up a reference framework for the establishment of requirements
for EFC systems and their later examination of the charging performance.
These charging performance metrics are intended for use with any toll scheme, regardless of its
technical underpinnings, system architecture, tariff structure, geographical coverage, or organizational
model. They are defined to treat technical details that may be different among technologies as a “black
box”. They focus solely on the outcome of the charging process, i.e. the amount charged in relation
to a pre-measured or theoretically correct amount, rather than intermediate variables from various
components as sensors, such as positioning accuracy, signal range, or optical resolution. This approach
ensures comparable results for each metric in all relevant situations.
The metrics are designed to cover the information exchanged on the front-end interface and the
interoperability interfaces between Toll Service Providers and Toll Chargers, as well as information on
the end-to-end level.
Metrics for the following information exchanges are defined:
— Charge Reports;
— Toll Declarations;
— Billing Details and associated event data;
— Payment Claims on the level of user accounts;
— End-to-End Metrics which assess the overall performance of the charging process.
The proposed metrics are specifically addressed to protect the interests of the actors in a toll system,
such as Toll Service Providers, Toll Chargers and Service Users. The metrics can be used to define
requirements (e.g. for requests for proposals) and for performance assessment.
© ISO 2017 – All rights reserved vii
---------------------- Page: 7 ----------------------
ISO/TS 17444-2:2017(E)
Toll schemes take on various forms as identified in the ISO 17575 series and ISO 14906. In order to
create a uniform performance metric specification, toll schemes are grouped into two classes based on
the character of their primary charging variable:
— charging based on discrete events (charges associated to the fact that a vehicle is crossing or
standing within a certain zone);
— those based on a continuous measurement (duration or distance).
In all these toll schemes, tolls may additionally vary as a function of vehicle class characteristics such
as trailer presence, number of axles, taxation class, operating function, and depending on time of day or
day of week, such that, for example, tariffs are higher in rush hour and lower on the weekends.
With this degree of complexity, it is not surprising to find that the attempts to evaluate and compare
technical solutions for Service User charging have been made uniquely each time a procurement
or study is initiated, and with only limited ability to reuse prior comparisons made by other testing
entities.
Examination Framework
The Examination Framework that is defined in this document is designed for measuring the metrics
defined in ISO/TS 17444-1. The general aim is to achieve a maximum comparability and reproducibility
of the results without restricting the technological choices in system design. Specific Examination
Frameworks may be defined for the Evaluation and Monitoring Phases of a project due to the differences
in the availability of equipped vehicles.
Evaluation Phase
This phase encompasses system evaluation and selection, as well as commissioning and ramp up during
implementation. Important aspects of this phase are
— relatively small sample sizes, and
— well controlled behaviour of test vehicles.
Monitoring Phase
After the system has gone into operation, its behaviour needs to be monitored for several reasons, such
as fine-tuning of the system performance, monitoring of SLAs between contractual partners (supplier,
Toll Charger, Toll Service Provider, etc.). In this phase, the following system aspects can be expected:
— very large sample sizes possible, but with unknown behaviour of the vehicles;
— in principle all measurements from implementation phase possible, too.
Readers Guide
To understand the content of this document, the reader should be aware of the methodology and
assumptions used to develop the Examination Framework and associated examination tests; therefore,
a suggested reading order is given below.
a) Annex B provides details of the underlying considerations for developing the Examination
Framework.
b) Annex C provides background statistical information which will enable the reader to determine
sample sizes and confidence limits based on the defined performance requirements.
c) Clause 5 provides the definition of the Examination Framework for the evaluation of Charging
Performance.
d) Clause 6 contains the toolbox of Examination Tests for the evaluation of charging performance for
the identified scheme types.
viii © ISO 2017 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/TS 17444-2:2017(E)
e) Annex D contains methods which can be used to reduce the required sample sizes for metrics with
high/low probabilities during the evaluation phase.
f) Annex E provides examples of Specific Examination Frameworks which have been developed in
accordance with the methodology in 5.2.
© ISO 2017 – All rights reserved ix
---------------------- Page: 9 ----------------------
TECHNICAL SPECIFICATION ISO/TS 17444-2:2017(E)
Electronic fee collection — Charging performance —
Part 2:
Examination framework
1 Scope
This document defines the Examination Framework for the measurement of charging performance
metrics defined in ISO/TS 17444-1 to be used during Evaluation and/or on-going Monitoring.
It specifies a method for the specification and documentation of a Specific Examination Framework
which can be used by the responsible entity to evaluate charging performance for a particular
information exchange interface or for overall charging performance within a Toll Scheme.
It provides a toolbox of Examination Tests for the roles of Toll Charger and Toll Service Provider for the
following Scheme types:
a) DSRC Discrete;
b) Autonomous Discrete;
c) Autonomous Continuous.
The detailed choice of the set of examination tests to be used depends on the application and the
respective context. Compliance with this specification is understood as using the definitions and
prescriptions laid out in this document whenever the respective system aspects are subjected to
performance measurements, rather than using other definitions and examination methods than the
ones specified in this document.
The following aspects are outside the scope of this document.
— This document does not propose specific numeric performance bounds, or average or worst-case
error bounds in percentage or monetary units. Those decisions are left to the Toll Charger (or
to agreements between Toll Charger and Service Provider).This document does not consider the
evaluation of the expected performance of a system based on modelling and measured data from
trial at another place.
— This document does not consider the specification of a common reference system which would be
required for comparison of performance between systems.
— This document defines measurements only on standardized interfaces. Proprietary interfaces
are excluded, because it is not possible to define standardized metrics on such system properties.
These excluded interfaces are among others the link between Toll Charger RSE and central systems
in DSRC systems, and the additional sensor input of GNSS modules (inertial sensors, CAN-bus for
wheel ticks, etc.).
2 Normative references
There are no normative references in this document.
© ISO 2017 – All rights reserved 1
---------------------- Page: 10 ----------------------
ISO/TS 17444-2:2017(E)
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
absolute charging error
difference betwe
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.