Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) - Metrics and Performance levels detailed definition

This document constitutes the main deliverable from WP1.1 of the GP-START project. It is devoted to a thorough review of the metrics defined in EN 16803-1 and proposes a performance classification for GNSS-based positioning terminals within designed for road applications. It will serve as one of the inputs to the elaboration of prEN 16803-2:2019 and prEN 16803-3:2019.
This document should serve as a starting point for discussion within CEN/CENELEC/JTC 5/WG1 on a consolidated set of performance metrics and associated classification logic. The proposals and conclusions appearing in this document are therefore only preliminary.

Detaillierte Definition von Metriken und Leistungsstufen

Espace - Utilisation de la localisation basée sur les GNSS pour les systèmes de transport routiers intelligents - Définition détaillée des mesures et niveaux de performance

Vesolje - Uporaba sistemov globalne satelitske navigacije (GNSS) za ugotavljanje položaja pri inteligentnih transportnih sistemih (ITS) v cestnem prometu - Podrobna opredelitev meritev in ravni uspešnosti

General Information

Status
Published
Public Enquiry End Date
11-Dec-2019
Publication Date
30-Mar-2020
Technical Committee
Current Stage
6060 - National Implementation/Publication (Adopted Project)
Start Date
12-Mar-2020
Due Date
17-May-2020
Completion Date
31-Mar-2020

Buy Standard

Technical report
SIST-TP CEN/TR 17448:2020 - BARVE na PDF-str 9,10,21,22,23,24,26,30,33
English language
41 pages
sale 10% off
Preview
sale 10% off
Preview

e-Library read for
1 day
Technical report
kSIST-TP FprCEN/TR 17448:2019 - BARVE na PDF-str 10,11,22,23,24,25,27,31,33
English language
42 pages
sale 10% off
Preview
sale 10% off
Preview

e-Library read for
1 day

Standards Content (sample)

SLOVENSKI STANDARD
SIST-TP CEN/TR 17448:2020
01-maj-2020
Vesolje - Uporaba sistemov globalne satelitske navigacije (GNSS) za ugotavljanje
položaja pri inteligentnih transportnih sistemih (ITS) v cestnem prometu -
Podrobna opredelitev meritev in ravni uspešnosti

Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) -

Metrics and Performance levels detailed definition
Detaillierte Definition von Metriken und Leistungsstufen
Espace - Utilisation de la localisation basée sur les GNSS pour les systèmes de
transport routiers intelligents - Définition détaillée des mesures et niveaux de
performance
Ta slovenski standard je istoveten z: CEN/TR 17448:2020
ICS:
03.220.20 Cestni transport Road transport
33.060.30 Radiorelejni in fiksni satelitski Radio relay and fixed satellite
komunikacijski sistemi communications systems
35.240.60 Uporabniške rešitve IT v IT applications in transport
prometu
SIST-TP CEN/TR 17448:2020 en,fr,de

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
SIST-TP CEN/TR 17448:2020
---------------------- Page: 2 ----------------------
SIST-TP CEN/TR 17448:2020
TECHNICAL REPORT
CEN/TR 17448
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
March 2020
ICS 03.220.20; 33.060.30; 35.240.60
English version
Space - Use of GNSS-based positioning for road Intelligent
Transport Systems (ITS) - Metrics and Performance levels
detailed definition

Espace - Utilisation de la localisation basée sur les Detaillierte Definition von Metriken und

GNSS pour les systèmes de transport routiers Leistungsstufen
intelligents - Définition détaillée des mesures et
niveaux de performance

This Technical Report was approved by CEN on 13 January 2020. It has been drawn up by the Technical Committee

CEN/CLC/JTC 5.

CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,

Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,

Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,

Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.
CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels

© 2020 CEN/CENELEC All rights of exploitation in any form and by any means Ref. No. CEN/TR 17448:2020 E

reserved worldwide for CEN national Members and for
CENELEC Members.
---------------------- Page: 3 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
Contents Page

European foreword .......................................................................................................................................... 3

1 Scope ........................................................................................................................................................ 4

2 Normative references ........................................................................................................................ 4

3 Terms and definitions ....................................................................................................................... 4

4 List of acronyms .................................................................................................................................. 4

5 Review of EN 16803-1 Performance Metrics ............................................................................ 5

5.1 Potential Improvements of unstable definitions .................................................................... 5

5.2 Completion With Additional Metrics ......................................................................................... 13

5.3 Justification of the choice of percentiles .................................................................................. 16

6 GBPT Performance Classification ............................................................................................... 22

6.1 General .................................................................................................................................................. 22

6.2 Classification logic ............................................................................................................................ 24

6.3 Identification of Performance Classes ....................................................................................... 26

6.4 Indicative Performance Figures For Main Categories Of Road Applications .............. 31

7 Conclusions and Recommendations .......................................................................................... 31

7.1 Purpose ................................................................................................................................................. 31

7.2 Improvements of Existing Definitions ....................................................................................... 31

7.3 Removal of Existing Definitions ................................................................................................... 33

7.4 Inclusion of New Definitions ......................................................................................................... 33

7.5 Choice of Percentiles ........................................................................................................................ 33

7.6 Performance Classification Logic ................................................................................................ 33

7.7 Performance Classes ........................................................................................................................ 34

Annex A (normative) Performance metrics as per EN 16803-1 .................................................... 36

Bibliography...................................................................................................................................................... 41

---------------------- Page: 4 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
European foreword

This document (CEN/TR 17448:2020) has been prepared by Technical Committee CEN/JTC 5 “Space”,

the secretariat of which is held by DIN.

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. CEN shall not be held responsible for identifying any or all such patent rights.

---------------------- Page: 5 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
1 Scope

This document constitutes the main deliverable from WP1.1 of the GP-START project. It is devoted to a

thorough review of the metrics defined in EN 16803-1 and proposes a performance classification for

GNSS-based positioning terminals within designed for road applications. It will serve as one of the inputs

to the elaboration of prEN 16803-2:2019 and prEN 16803-3:2019.

This document should serve as a starting point for discussion within CEN/CENELEC/JTC 5/WG1 on a

consolidated set of performance metrics and associated classification logic. The proposals and

conclusions appearing in this document are therefore only preliminary.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

EN 16803-1:2016, Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) -

Part 1: Definitions and system engineering procedures for the establishment and assessment of

performances
3 Terms and definitions

For the purposes of this document, the terms and definitions given in EN 16803-1 apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at http://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
4 List of acronyms
ADAS Advanced Driver Assistance Systems
CAN Controller Area Network
CDF Cumulative Distribution Function
CEN Comité Européen de Normalization — (European Committee for Standardization)

CENELEC Comité Européen de Normalization Électrotechnique — (European Committee for

Electrotechnical Standardization)
ECEF Earth Centred Earth Fixed
ETSI European Telecommunications Standards Institute
GBPT GNSS-Based Positioning Terminal
GNSS Global Navigation Satellite Systems
HPA Horizontal Position Error
HPL Horizontal Protection Level
IMU Inertial Measurement Unit
ITS Intelligent Transport Systems
KOM Kick-Off Meeting
MEMS Micro Electro-Mechanical Systems
---------------------- Page: 6 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
NMEA National Marine Electronics Association
PPP Precise Point Positioning
RTCA Radio Technical Commission for Aeronautics
RTK Real Time Kinematics
SPP Standard Point Positioning
TTFF Time To First Fix
5 Review of EN 16803-1 Performance Metrics
5.1 Potential Improvements of unstable definitions
5.1.1 Position accuracy metrics
5.1.1.1 Vectors vs their Norms

One thing that draws immediate attention when reviewing the metrics is some degree of ambiguity in

some of the definitions. For instance, the first Accuracy metric (EN 16803-1:2016, Table 1) refers to the

“3D position error”, which has not been explicitly defined anywhere along the document:

3D Position Accuracy is defined as the set of three statistical values given by the 50th, 75th and 95th

percentiles of the cumulative distribution of 3D position errors.

There is some discussion in EN 16803-1:2016, 3.2.1 regarding vector and scalar quantities, but no

explicit definition of the 3D position error is proposed. The position error (without the “3D” adjective)

is defined in EN 16803-1:2016, 4.3 as follows:

Position error: is the difference between the true position and the position provided by the positioning

terminal. It shall be understood as a vector expressed in some convenient local reference frame (e.g. local

horizontal frame).

This definition explicitly states that the position error shall be understood as a vector quantity. Then,

the use of the expression “3D position error” in the definition of the metric seems to emphasize the

vector character of the position error, which may be misleading since the metric actually refers to the

norm of the position error vector, which is actually a scalar quantity.

The same concern can be raised about the horizontal position error. It is therefore recommended to

include explanations on the meaning of expressions such as “3D position error” and “horizontal position

error”, making it clear that they refer to norms of vectors rather than vectors. Note that footnote 5 on

EN 16803-1:2016, A.2.1 of the document contains such a clarification for the case of the horizontal

position error, but a footnote in an annex may not be the best place for it (besides, the expression “it is

recalled” seems to indicate that the definition was written in some other, more prominent place within

the document and later removed).

NOTE The norm of a vector is not uniquely defined. To overcome this problem, it could be further specified

that the norm of interest is the Euclidean norm (square root of the sum of squared coordinates) of the vector when

expressed in a linear (and orthonormal) coordinate system. Suppose, for instance, that the position is expressed

in geodetic coordinates (latitude, longitude and height) and the position error is expressed as a latitude error, a

longitude error and a height error. The square root of the sum of the squares of these 3 quantities has no physical

meaning, and is not what is meant in the above proposed definition. It could be worth making this sort of

considerations in the standard.

A related remark (although not concerned with performance metrics) is on the identification of the

GBPT outputs made in EN 16803-1:2016, 4.2, which may require some review and perhaps include

attitude parameters (e.g. heading) or make some additional considerations on the reference frame used

to represent position and velocity (e.g. horizontal velocity could be represented in polar coordinates as

a pair consisting of speed and heading).
---------------------- Page: 7 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
5.1.1.2 Along Track and Cross Track Components

Another potential issue that has been detected is the fact that the expressions “along track” and “cross

track” are undefined, yielding the definitions of “along track” and “cross track” position accuracy a little

ambiguous. It is recommended to include the definitions of these terms somewhere in the document,

especially considering that there is no general agreement as to their meanings. Note that these terms

have their roots in aeronautics and astronautics, and have been widely use to describe the motion of

space vehicles, such as artificial satellites, especially when in orbit around the Earth. Each satellite is

assigned a body-centred orthogonal reference frame with axes pointing:
— in the satellite’s direction of motion;
— in the direction orthogonal to the orbital plane;
— in the direction orthogonal to the previous 2.

However, since most orbits are nearly circular, the third direction is roughly pointing to the centre of

the Earth, and in some cases, this is how the third axis is defined, implying a slight misalignment of the

first with respect to the satellite’s direction of motion. Besides, the direction of motion is not well defined

unless the satellite’s trajectory is referred to an external (not body-centred) reference frame, such as

one with origin at the centre of the Earth. Depending on how this external frame is chosen (e.g. an inertial

frame vs one which rotates with the Earth), the satellite’s direction of motion may be different.

In road applications the situation is also somewhat complicated. It may seem natural to define the along

track direction as the one parallel to the vehicle’s velocity vector, but caution shall be taken as to the

reference frame used to define the vehicle’s motion. A natural choice would be an Earth-centred, Earth-

fixed (ECEF) frame, such as WGS84. Of course, when the vehicle is standing still, the along track direction

is not well defined using the velocity vector (which in this case is the null vector), but still the last along

track direction computed before the vehicle stopped could be used (besides, there is no actual “track”

when the vehicle is not moving, so the along track and cross track errors may not make much sense in

that case either). However, there is still the problem of defining the cross-track direction, and now there

is no such thing as an orbital plane. Among all directions orthogonal to the along track axis, a natural

choice seems to be the one lying on the horizontal plane (well defined unless the vehicle’s motion is

purely vertical, which is an extremely unlikely situation in road applications). Another natural option

seems to be the one lying on the local road plane, which may differ from the horizontal plane due to road

banking. This second option may be of interest when an inertial measurement unit (IMU) is involved in

the navigation process, as the local road plane is nearly fixed with respect to the IMU axes. However, the

first option seems better for most implementations as it does not require any prior knowledge of the

road geometry or of the vehicle’s attitude. There’s yet a third option to be considered in which the cross-

track direction is the one defined by the normal acceleration vector, but this has an important drawback,

namely that the normal acceleration is nearly zero when in low-dynamics situations (such as driving

along a nearly straight road or a highway). Hence the first option continues to seem the most convenient

one. With this in mind, the following definition is proposed:

Along track and cross track components are coordinates in a reference frame whose definition is based

on the vehicle’s true velocity vector (relative to some ECEF reference frame) and the local upward unit

vector . Namely, the said reference frame is defined by the following 3 orthogonal unit vectors:

 
  
  
τ= vv/ , and . The along track and cross track components of a vector
b τ× n ε
nv=ηη××/ v

attached to the user’s position (such as the position error vector) are then defined as the scalar products

  
and , respectively.
ε⋅τ ε⋅ n

NOTE 1 The vector n as defined above corresponds to the first of the 3 options previously discussed: it is

orthogonal to the along track direction (given by v ) and lies on the horizontal plane (as it is orthogonal to η ).

NOTE 2 The notation used to define the reference frame is commonly used to denote the so-called Frenet

trihedron, although the reference frame defined above and the Frenet trihedron are not exactly the same (rather,

the Frenet trihedron would correspond to the third option, which has been readily discarded).

---------------------- Page: 8 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
5.1.2 Velocity Accuracy Metrics

The same considerations made in 5.1.1 with regard to position accuracy metrics can be directly applied

to velocity accuracy metrics, in particular those regarding the ambiguity in the use of expressions such

as 3D or horizontal, along track and cross track, etc. Also, identical recommendations are made and

analogous rewordings are proposed.

In addition, it has been pointed out that 3D and horizontal velocity accuracy metrics may not be relevant

and could be deleted. It has also been pointed out that there may be some redundancy between 3D

velocity accuracy and speed accuracy. At this point it is worth discussing the difference between both.

Suppose that the true velocity vector (expressed in some orthonormal coordinate system, such as the

local horizontal system with coordinates along the East, North and up axes) is v = 100,, and that the

( )
estimated velocity is v −100,, . Then the speed error would be , whereas the
( ) vv− =−=1 1 0
3D velocity error (in norm) would be , thus illustrating the difference between
vv−=−2,,0 0=2
( )

both concepts. The underlying idea is that the norm of the error is not the same thing as the error of the

norm, which is a consequence of the Triangle Inequality illustrated in Figure 1. Speed accuracy refers to

the error of the norm, whereas 3D velocity accuracy refers to the norm of the error, so this shows that

3D Velocity and Speed metrics proposed in EN 16803-1 are not redundant.
Figure 1 — Triangle Inequality

The question remains as to whether all of them (an in particular those of 3D and horizontal velocity

accuracy) are of relevance. Relevance is hard to assess, and rather subjective. Many of the metrics

proposed in EN 16803-1 could be questioned in terms of their relevance (it seems difficult to think of

an application which makes use of the East Velocity Protection Level, to give an example). However, they

have been included for completeness. Whether or not they should be deleted is a relevant discussion

but goes beyond the scope of this study, but we can envisage:
— the 3D Velocity Accuracy metric could be removed;

— the Horizontal Velocity Accuracy metric could be transformed into the Horizontal Speed Accuracy

metric (then moved to the “Speed” section of the table, in which now it would make sense to make

a distinction between 3D and Horizontal), where the horizontal speed is to be understood as the

norm of the horizontal projection of the velocity vector.
5.1.3 Integrity Metrics
5.1.3.1 General
Similar issues have been detected as with accuracy metrics. Namely:

— 3D/Horizontal Protection Levels have not been explicitly defined (neither for position nor for

velocity). However, if 3D and horizontal errors are properly defined as norms of the corresponding

vectors (following the recommendations made in 5.1.1), then expressions such as 3D and Horizontal

Protection Levels can be assumed to be self-explanatory without the need of explicit definitions;

---------------------- Page: 9 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)

— 3D/Horizontal Integrity Risk definitions contain references to such things as 3D/Horizontal

position (or velocity) errors, which are undefined. This would be solved by implementing the

recommendations stated in 5.1.1;

— along track, cross track, etc. are undefined. This would be solved by implementing the

recommendations stated in 5.1.1;

— there is a 3D Velocity Protection Level metric, but there is no Speed Protection Level Metric, which

is probably more interesting. In this regard it is proposed to turn 3D and Horizontal Velocity

Protection Level metrics into 3D and Horizontal Speed Protection Level metrics, and then move

them to a new “Speed” section within the table (much in the same way as in the Accuracy Metrics

table).
5.1.3.2 Percentile Computation Procedure

It is stated in EN 16803-1:2016, A.2.1 that Accuracy metrics shall not take into account those epochs in

which the output of interest (e.g. horizontal position) is not provided by the GBPT. However, in

EN 16803-1:2016, A.2.2.2 it is said that Protection Level performance metrics shall include those epochs

in which there is no protection level, which shall be understood as if the protection level was infinite.

These two approaches are exactly opposite and it seems contradictory to adopt one of them for accuracy

and the other one for protection levels. We briefly discus here both approaches using an example and

show a few of their advantages and drawbacks. The example addresses the case of protection level

percentile computation, but the same ideas apply to error percentiles (and hence to Accuracy metrics).

Suppose that 10 % of the time there is no position output or no associated protection level and that the

CDF of the protection levels is computed taking into account all epochs. Then the maximum value of the

protection level, which would normally correspond to the 100 percentile, will rather correspond to

the 90 percentile (smaller or equal protection levels than the maximum have been obtained only 90 %

of the time, since there is another 10 % without protection levels). Actually, this way of reasoning shows

that the whole CDF plot shrinks by a factor 0,9 (with respect to the one computed using only epochs

with a protection level) along the ordinate axis. This is illustrated in Figure 2. As a result, all percentiles

th th
up to the 90 yield higher values. In particular, the 95 percentile is undefined.
Figure 2 — CDF considering all epochs or only those with Protection Levels
---------------------- Page: 10 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)

Each approach presents different advantages and drawbacks. If all epochs are considered, the CDF can

be used to assess the Protection Level size and Protection Level Availability in a single plot (note that

Protection Level Availability metrics have not been defined in EN 16803-1:2016, see 5.2.1). Likewise, in

the case of error percentiles, Accuracy and Availability could be assessed in one single plot, which may

be seen as an advantage. As a drawback, both metrics (accuracy and availability) get coupled,

complicating validation, certification and comparison of different solutions. Besides, some of the

percentiles defining the accuracy metric (such as the 95 in the preceding example) may be not

computable. Benefits of both approaches are summarized in Table 1.
Table 1 — Benefits of both percentile computation approaches
Metrics Use all epochs Only epochs with a valid output
PL size and availability are
PL size and availability are
assessed in a single plot.
decoupled.
Protection Level Performance
PL availability can be assessed
All PL size percentiles are well-
without the need of additional
defined.
availability metrics.
Accuracy and availability are
Accuracy and availability are
assessed in a single plot.
decoupled.
Accuracy
Availability can be assessed
All error percentiles are well-
without the need of additional
defined.
availability metrics.

Regardless of the final decision, it is recommended to emphasize in the document the approach to be

taken in each case, perhaps in a more prominent place than the Annex A (where such explanations are

currently placed), in order to avoid misunderstandings. This is especially encouraged if it is decided to

keep using different approaches for Accuracy and Integrity metrics.

As to the question how a “valid” output is to be understood (in order to filter out epochs without a valid

output when that is the selected approach), the proposed answer is to consider an output valid when no

flag indicates otherwise. For instance, assuming that the NMEA standard is used to output the data, any

value of the “fix quality” flag in a GGA sentence other than 0 would indicate a valid position.

5.1.3.3 Integrity Risk Computation Procedure

Similar to what has been pointed out previously, a minor concern can be raised about Integrity Risk

metrics, which refer to probabilities whose computation procedure has not been clearly specified. For

the sake of clarity let us focus on position (rather than velocity) Integrity Risk metrics, although all what

is said here can be applied to any of the Integrity Risk metrics. As previously, we are faced with the

decision whether to consider all epochs or those with a valid position and an associated Protection Level.

When no position and no PL exist, that could be counted as a “safe” epoch (one with no integrity event

taking place). The same could be said of an epoch with a position and without a PL. Therefore, those

epochs could either be discarded or considered in the IR computation as safe epochs. If they are

discarded, the results will be slightly worse (by a factor 1/0,9 approximately 1,1 taking the example

of 5.1.3.2) than if they are taken into consideration. However, the impact in the case of the Integrity Risk

is somewhat smaller, as a 1,1 factor (taking again the example of 5.1.3.2) applied to an IR figure which

is already very small (e.g. 1 -6) does not make much of a difference, especially considering that such

small figures can hardly be measured to a high degree of accuracy. Nonetheless, it is recommended to

specify whether or not epochs with a valid position and PL are to be considered in the computation of

the Integrity Risk.

Although either approach can be acceptable, we are inclined in this case to consider only epochs with

valid outputs, which is a slightly more conservative approach than the other one (as it yields slightly

worse integrity risk figures).
---------------------- Page: 11 ----------------------
SIST-TP CEN/TR 17448:2020
CEN/TR 17448:2020 (E)
5.1.4 Availability Metrics

Two availability metrics have been defined in EN 16803-1, both identical except that one refers to

position and the other one to velocity/speed. In both cases Availability is defined in terms of the

existence of the output of interest (either position or velocity/speed) but nothing is said as to the criteria

that an output shall meet to be acceptable and thus be taken into account when computing Availability

figures. Since no criteria are specified, it can be interpreted as if all outputs are acceptable, which would

allow for unwanted situations such us a GNSS-standalone GBPT which keeps outputting (at its nominal

rate) the last known position after a complete loss of GNSS signal (e.g. inside a tunnel) and those outputs

being added to the availability count.

In order to avoid this kind of situations it is proposed to reword these metrics specifying that only valid

outputs are to be taken into account. For example, Position Availability could be reworded as follows:

Position Availability (T) is the percentage of operating time intervals of length T during which the

positioning terminal provides at least one valid position output

Note that This wording differs from the original one only in the appearance of the word “valid”. This

should be accompanied by an explanation somewhere along the document as to what is meant by a

“valid” output. In line with what was said at the end of 5.1.3.2, this could be understood as an output that

is accompanied by a flag (similar to the “fix quality” flag that can be found in the GGA sentence of the

NMEA protocol) indicating that the output is healthy (or not indicating otherwise). Similar criteria could

be followed in the case of velocity and/or speed.

In order to make things simple, it could be established that one single flag be used to indicate the health

status of all provided outputs (position, velocity, speed or protection levels if they are also provided)

and that a value of such flag indicating good health shall be understood as all outputs being okay.

Otherwise, maybe the existing standard protocols (such as NMEA) would have to be modified to include

health status flags for the different outputs (not only for the position). However, for terminals which

provide protection levels the NMEA protocol may already be insufficient, so maybe the protocol needs

to some evolution anyway.

It may also be worth specifying how the time intervals of length T are to be handled when computing

Availability (T) figures, namely whether or not such time intervals shall be con
...

SLOVENSKI STANDARD
kSIST-TP FprCEN/TR 17448:2019
01-december-2019
Vesolje - Ugotavljanje položaja z uporabo sistema globalne satelitske navigacije

(GNSS) pri inteligentnih transportnih sistemih (ITS) v cestnem prometu - Podrobna

opredelitev meritev in ravni uspešnosti

Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) -

Metrics and Performance levels detailed definition
Detaillierte Definition von Metriken und Leistungsstufen
Espace - Utilisation de la localisation basée sur les GNSS pour les systèmes de
transport routiers intelligents - Définition détaillée des mesures et niveaux de
performance
Ta slovenski standard je istoveten z: FprCEN/TR 17448
ICS:
03.220.20 Cestni transport Road transport
33.060.30 Radiorelejni in fiksni satelitski Radio relay and fixed satellite
komunikacijski sistemi communications systems
35.240.60 Uporabniške rešitve IT v IT applications in transport
prometu
kSIST-TP FprCEN/TR 17448:2019 en,fr,de

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
kSIST-TP FprCEN/TR 17448:2019
---------------------- Page: 2 ----------------------
kSIST-TP FprCEN/TR 17448:2019
TECHNICAL REPORT
FINAL DRAFT
FprCEN/TR 17448
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
September 2019
ICS
English version
Space - Use of GNSS-based positioning for road Intelligent
Transport Systems (ITS) - Metrics and Performance levels
detailed definition

Espace - Utilisation de la localisation basée sur les Detaillierte Definition von Metriken und

GNSS pour les systèmes de transport routiers Leistungsstufen
intelligents - Définition détaillée des mesures et
niveaux de performance

This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee

CEN/CLC/JTC 5.

CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,

Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,

Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,

Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.

Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are

aware and to provide supporting documentation.

Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without

notice and shall not be referred to as a Technical Report.
CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels

© 2019 CEN/CENELEC All rights of exploitation in any form and by any means Ref. No. FprCEN/TR 17448:2019 E

reserved worldwide for CEN national Members and for
CENELEC Members.
---------------------- Page: 3 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)
Contents Page

European foreword .......................................................................................................................................... 4

1 Scope ........................................................................................................................................................ 5

2 Normative references ........................................................................................................................ 5

3 Terms and definitions ....................................................................................................................... 5

4 List of acronyms .................................................................................................................................. 5

5 Review of EN 16803-1 Performance Metrics ............................................................................ 6

5.1 Potential Improvements of unstable definitions .................................................................... 6

5.1.1 Position accuracy metrics ................................................................................................................ 6

5.1.2 Velocity Accuracy Metrics ................................................................................................................ 8

5.1.3 Integrity Metrics .................................................................................................................................. 8

5.1.4 Availability Metrics .......................................................................................................................... 11

5.1.5 Timing Metrics ................................................................................................................................... 11

5.1.6 Additional Considerations ............................................................................................................. 13

5.2 Completion With Additional Metrics ......................................................................................... 14

5.2.1 Protection Level Availability Metrics ........................................................................................ 14

5.2.2 Continuity Metrics ............................................................................................................................ 15

5.3 Justification of the choice of percentiles .................................................................................. 17

5.3.1 Overview .............................................................................................................................................. 17

5.3.2 Number of points used to describe the CDF ............................................................................ 18

5.3.3 Values of the percentiles ................................................................................................................ 18

5.3.4 Stability of the Metrics .................................................................................................................... 19

6 GBPT Performance Classification ............................................................................................... 23

6.1 General .................................................................................................................................................. 23

6.2 Classification logic ............................................................................................................................ 24

6.2.1 Introduction ........................................................................................................................................ 24

6.2.2 Multi-parametric Classification ................................................................................................... 27

6.3 Identification of Performance Classes ....................................................................................... 27

6.3.1 General .................................................................................................................................................. 27

6.3.2 Accuracy Performance Classes..................................................................................................... 28

6.3.3 Availability Performance Classes ................................................................................................ 30

6.3.4 Integrity Performance Classes ..................................................................................................... 30

6.4 Indicative Performance Figures For Main Categories Of Road Applications .............. 32

7 Conclusions and Recommendations .......................................................................................... 32

7.1 Purpose ................................................................................................................................................. 32

7.2 Improvements of Existing Definitions ....................................................................................... 32

7.3 Removal of Existing Definitions ................................................................................................... 33

7.4 Inclusion of New Definitions ......................................................................................................... 33

7.5 Choice of Percentiles ........................................................................................................................ 34

7.6 Performance Classification Logic ................................................................................................ 34

7.7 Performance Classes ........................................................................................................................ 34

7.7.1 General .................................................................................................................................................. 34

7.7.2 Horizontal Position Accuracy Classification ........................................................................... 35

7.7.3 Position Availability Classification ............................................................................................. 35

7.7.4 Horizontal Protection Level Classification .............................................................................. 35

7.7.5 Integrity Risk Classification .......................................................................................................... 35

7.7.6 Protection Level Availability Classification ............................................................................. 36

---------------------- Page: 4 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)

Annex A (normative) Performance Metrics As Per EN 16803-1 ................................................... 37

Bibliography ..................................................................................................................................................... 42

---------------------- Page: 5 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)
European foreword

This document (FprCEN/TR 17448:2019) has been prepared by Technical Committee CEN/JTC 5

“Space”, the secretariat of which is held by DIN.
This document is currently submitted to the Vote on TR.
---------------------- Page: 6 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)
1 Scope

This document constitutes the main deliverable from WP1.1 of the GP-START project. It is devoted to a

thorough review of the metrics defined in EN 16803-1 and proposes a performance classification for

GNSS-based positioning terminals within designed for road applications. It will serve as one of the inputs

to the elaboration of prEN 16803-2:2019 and prEN 16803-3:2019.

This document should serve as a starting point for discussion within CEN/CENELEC/JTC 5/WG1 on a

consolidated set of performance metrics and associated classification logic. The proposals and

conclusions appearing in this document are therefore only preliminary.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

EN 16803-1:2016, Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) -

Part 1: Definitions and system engineering procedures for the establishment and assessment of

performances
3 Terms and definitions

For the purposes of this document, the terms and definitions given in EN 16803-1 apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at http://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
4 List of acronyms
ADAS Advanced Driver Assistance Systems
CAN Controller Area Network
CDF Cumulative Distribution Function
CEN Comité Européen de Normalization — (European Committee for Standardization)

CENELEC Comité Européen de Normalization Électrotechnique — (European Committee for

Electrotechnical Standardization)
ECEF Earth Centred Earth Fixed
ETSI European Telecommunications Standards Institute
GBPT GNSS-Based Positioning Terminal
GNSS Global Navigation Satellite Systems
HPA Horizontal Position Error
HPL Horizontal Protection Level
IMU Inertial Measurement Unit
ITS Intelligent Transport Systems
KOM Kick-Off Meeting
MEMS Micro Electro-Mechanical Systems
---------------------- Page: 7 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)
NMEA National Marine Electronics Association
PPP Precise Point Positioning
RTCA Radio Technical Commission for Aeronautics
RTK Real Time Kinematics
SPP Standard Point Positioning
TTFF Time To First Fix
5 Review of EN 16803-1 Performance Metrics
5.1 Potential Improvements of unstable definitions
5.1.1 Position accuracy metrics
5.1.1.1 Vectors vs their Norms

One thing that draws immediate attention when reviewing the metrics is some degree of ambiguity in

some of the definitions. For instance, the first Accuracy metric (EN 16803-1:2016, Table 1) refers to the

“3D position error”, which has not been explicitly defined anywhere along the document:

3D Position Accuracy is defined as the set of three statistical values given by the 50th, 75th and 95th

percentiles of the cumulative distribution of 3D position errors.

There is some discussion in EN 16803-1:2016, 3.2.1 regarding vector and scalar quantities, but no

explicit definition of the 3D position error is proposed. The position error (without the “3D” adjective)

is defined in EN 16803-1:2016, 4.3 as follows:

Position error: is the difference between the true position and the position provided by the positioning

terminal. It shall be understood as a vector expressed in some convenient local reference frame (e.g. local

horizontal frame).

This definition explicitly states that the position error shall be understood as a vector quantity. Then,

the use of the expression “3D position error” in the definition of the metric seems to emphasize the

vector character of the position error, which may be misleading since the metric actually refers to the

norm of the position error vector, which is actually a scalar quantity.

The same concern can be raised about the horizontal position error. It is therefore recommended to

include explanations on the meaning of expressions such as “3D position error” and “horizontal position

error”, making it clear that they refer to norms of vectors rather than vectors. Note that footnote 5 on

EN 16803-1:2016, A.2.1 of the document contains such a clarification for the case of the horizontal

position error, but a footnote in an annex may not be the best place for it (besides, the expression “it is

recalled” seems to indicate that the definition was written in some other, more prominent place within

the document and later removed).

NOTE The norm of a vector is not uniquely defined. To overcome this problem, it could be further specified

that the norm of interest is the Euclidean norm (square root of the sum of squared coordinates) of the vector when

expressed in a linear (and orthonormal) coordinate system. Suppose, for instance, that the position is expressed

in geodetic coordinates (latitude, longitude and height) and the position error is expressed as a latitude error, a

longitude error and a height error. The square root of the sum of the squares of these 3 quantities has no physical

meaning, and is not what is meant in the above proposed definition. It could be worth making this sort of

considerations in the standard.

A related remark (although not concerned with performance metrics) is on the identification of the

GBPT outputs made in EN 16803-1:2016, 4.2, which may require some review and perhaps include

attitude parameters (e.g. heading) or make some additional considerations on the reference frame used

to represent position and velocity (e.g. horizontal velocity could be represented in polar coordinates as

a pair consisting of speed and heading).
---------------------- Page: 8 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)
5.1.1.2 Along Track and Cross Track Components

Another potential issue that has been detected is the fact that the expressions “along track” and “cross

track” are undefined, yielding the definitions of “along track” and “cross track” position accuracy a little

ambiguous. It is recommended to include the definitions of these terms somewhere in the document,

especially considering that there is no general agreement as to their meanings. Note that these terms

have their roots in aeronautics and astronautics, and have been widely use to describe the motion of

space vehicles, such as artificial satellites, especially when in orbit around the Earth. Each satellite is

assigned a body-centred orthogonal reference frame with axes pointing:
— in the satellite’s direction of motion;
— in the direction orthogonal to the orbital plane;
— in the direction orthogonal to the previous 2.

However, since most orbits are nearly circular, the third direction is roughly pointing to the centre of

the Earth, and in some cases, this is how the third axis is defined, implying a slight misalignment of the

first with respect to the satellite’s direction of motion. Besides, the direction of motion is not well defined

unless the satellite’s trajectory is referred to an external (not body-centred) reference frame, such as

one with origin at the centre of the Earth. Depending on how this external frame is chosen (e.g. an inertial

frame vs one which rotates with the Earth), the satellite’s direction of motion may be different.

In road applications the situation is also somewhat complicated. It may seem natural to define the along

track direction as the one parallel to the vehicle’s velocity vector, but caution shall be taken as to the

reference frame used to define the vehicle’s motion. A natural choice would be an Earth-centred, Earth-

fixed (ECEF) frame, such as WGS84. Of course, when the vehicle is standing still, the along track direction

is not well defined using the velocity vector (which in this case is the null vector), but still the last along

track direction computed before the vehicle stopped could be used (besides, there is no actual “track”

when the vehicle is not moving, so the along track and cross track errors may not make much sense in

that case either). However, there is still the problem of defining the cross-track direction, and now there

is no such thing as an orbital plane. Among all directions orthogonal to the along track axis, a natural

choice seems to be the one lying on the horizontal plane (well defined unless the vehicle’s motion is

purely vertical, which is an extremely unlikely situation in road applications). Another natural option

seems to be the one lying on the local road plane, which may differ from the horizontal plane due to road

banking. This second option may be of interest when an inertial measurement unit (IMU) is involved in

the navigation process, as the local road plane is nearly fixed with respect to the IMU axes. However, the

first option seems better for most implementations as it does not require any prior knowledge of the

road geometry or of the vehicle’s attitude. There’s yet a third option to be considered in which the cross-

track direction is the one defined by the normal acceleration vector, but this has an important drawback,

namely that the normal acceleration is nearly zero when in low-dynamics situations (such as driving

along a nearly straight road or a highway). Hence the first option continues to seem the most convenient

one. With this in mind, the following definition is proposed:

Along track and cross track components are coordinates in a reference frame whose definition is based

on the vehicle’s true velocity vector v (relative to some ECEF reference frame) and the local upward

unit vector η . Namely, the said reference frame is defined by the following 3 orthogonal unit vectors:

 
  
  
τ= vv/
, and b τ× n . The along track and cross track components of a vector ε
nv=ηη××/ v

attached to the user’s position (such as the position error vector) are then defined as the scalar products

  
and , respectively.
ε⋅τ ε⋅ n

NOTE 1 The vector n as defined above corresponds to the first of the 3 options previously discussed: it is

orthogonal to the along track direction (given by v ) and lies on the horizontal plane (as it is orthogonal to η ).

NOTE 2 The notation used to define the reference frame is commonly used to denote the so-called Frenet

trihedron, although the reference frame defined above and the Frenet trihedron are not exactly the same (rather,

the Frenet trihedron would correspond to the third option, which has been readily discarded).

---------------------- Page: 9 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)
5.1.2 Velocity Accuracy Metrics

The same considerations made in 5.1.1 with regard to position accuracy metrics can be directly applied

to velocity accuracy metrics, in particular those regarding the ambiguity in the use of expressions such

as 3D or horizontal, along track and cross track, etc. Also, identical recommendations are made and

analogous rewordings are proposed.

In addition, it has been pointed out that 3D and horizontal velocity accuracy metrics may not be relevant

and could be deleted. It has also been pointed out that there may be some redundancy between 3D

velocity accuracy and speed accuracy. At this point it is worth discussing the difference between both.

Suppose that the true velocity vector (expressed in some orthonormal coordinate system, such as the

local horizontal system with coordinates along the East, North and up axes) is v = 100,, and that the

( )
estimated velocity is v −100,, . Then the speed error would be , whereas the
vv− =−=1 1 0
( )
3D velocity error (in norm) would be , thus illustrating the difference between
vv−=−2,,0 0=2
( )

both concepts. The underlying idea is that the norm of the error is not the same thing as the error of the

norm, which is a consequence of the Triangle Inequality illustrated in Figure 1. Speed accuracy refers to

the error of the norm, whereas 3D velocity accuracy refers to the norm of the error, so this shows that

3D Velocity and Speed metrics proposed in EN 16803-1 are not redundant.
Figure 1 — Triangle Inequality

The question remains as to whether all of them (an in particular those of 3D and horizontal velocity

accuracy) are of relevance. Relevance is hard to assess, and rather subjective. Many of the metrics

proposed in EN 16803-1 could be questioned in terms of their relevance (it seems difficult to think of

an application which makes use of the East Velocity Protection Level, to give an example). However, they

have been included for completeness. Whether or not they should be deleted is a relevant discussion

but goes beyond the scope of this study, but we can envisage:
— the 3D Velocity Accuracy metric could be removed;

— the Horizontal Velocity Accuracy metric could be transformed into the Horizontal Speed Accuracy

metric (then moved to the “Speed” section of the table, in which now it would make sense to make

a distinction between 3D and Horizontal), where the horizontal speed is to be understood as the

norm of the horizontal projection of the velocity vector.
5.1.3 Integrity Metrics
5.1.3.1 General
Similar issues have been detected as with accuracy metrics. Namely:

— 3D/Horizontal Protection Levels have not been explicitly defined (neither for position nor for

velocity). However, if 3D and horizontal errors are properly defined as norms of the corresponding

vectors (following the recommendations made in 3.1.1), then expressions such as 3D and

Horizontal Protection Levels can be assumed to be self-explanatory without the need of explicit

definitions;
---------------------- Page: 10 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)

— 3D/Horizontal Integrity Risk definitions contain references to such things as 3D/Horizontal

position (or velocity) errors, which are undefined. This would be solved by implementing the

recommendations stated in 5.1.1;

— along track, cross track, etc. are undefined. This would be solved by implementing the

recommendations stated in 5.1.1;

— there is a 3D Velocity Protection Level metric, but there is no Speed Protection Level Metric, which

is probably more interesting. In this regard it is proposed to turn 3D and Horizontal Velocity

Protection Level metrics into 3D and Horizontal Speed Protection Level metrics, and then move

them to a new “Speed” section within the table (much in the same way as in the Accuracy Metrics

table).
5.1.3.2 Percentile Computation Procedure

It is stated in EN 16803-1:2016, A.2.1 that Accuracy metrics shall not take into account those epochs in

which the output of interest (e.g. horizontal position) is not provided by the GBPT. However, in

EN 16803-1:2016, A.2.2.2 it is said that Protection Level performance metrics shall include those epochs

in which there is no protection level, which shall be understood as if the protection level was infinite.

These two approaches are exactly opposite and it seems contradictory to adopt one of them for accuracy

and the other one for protection levels. We briefly discus here both approaches using an example and

show a few of their advantages and drawbacks. The example addresses the case of protection level

percentile computation, but the same ideas apply to error percentiles (and hence to Accuracy metrics).

Suppose that 10 % of the time there is no position output or no associated protection level and that the

CDF of the protection levels is computed taking into account all epochs. Then the maximum value of the

protection level, which would normally correspond to the 100 percentile, will rather correspond to

the 90 percentile (smaller or equal protection levels than the maximum have been obtained only 90 %

of the time, since there is another 10 % without protection levels). Actually, this way of reasoning shows

that the whole CDF plot shrinks by a factor 0,9 (with respect to the one computed using only epochs

with a protection level) along the ordinate axis. This is illustrated in Figure 2. As a result, all percentiles

th th
up to the 90 yield higher values. In particular, the 95 percentile is undefined.
Figure 2 — CDF considering all epochs or only those with Protection Levels
---------------------- Page: 11 ----------------------
kSIST-TP FprCEN/TR 17448:2019
FprCEN/TR 17448:2019 (E)

Each approach presents different advantages and drawbacks. If all epochs are considered, the CDF can

be used to assess the Protection Level size and Protection Level Availability in a single plot (note that

Protection Level Availability metrics have not been defined in EN 16803-1:2016, see 5.2.1). Likewise, in

the case of error percentiles, Accuracy and Availability could be assessed in one single plot, which may

be seen as an advantage. As a drawback, both metrics (accuracy and availability) get coupled,

complicating validation, certification and comparison of different solutions. Besides, some of the

percentiles defining the accuracy metric (such as the 95 in the preceding example) may be not

computable. Benefits of both approaches are summarized in Table 1.
Table 1 — Benefits of both percentile computation approaches
Metrics Use all epochs Only epochs with a valid output
PL size and availability are
PL size and availability are
assessed in a single plot.
decoupled.
Protection Level Performance
PL availability can be assessed
All PL size percentiles are well-
without the need of additional
defined.
availability metrics.
Accuracy and availability are
Accuracy and availability are
assessed in a single plot.
decoupled.
Accuracy
Availability can be assessed
All error percentiles are well-
without the need of additional
defined.
availability metrics.

Regardless of the final decision, it is recommended to emphasize in the document the approach to be

taken in each case, perhaps in a more prominent place than the Annex A (where such explanations are

currently placed), in order to avoid misunderstandings. This is especially encouraged if it is decided to

keep using different approaches for Accuracy and Integrity metrics.

As to the question how a “valid” output is to be understood (in order to filter out epochs without a valid

output when that is the selected approach), the proposed answer is to consider an output valid when no

flag indicates otherwise. For instance, assuming that the NMEA standard is used to output the data, any

value of the “fix quality” flag in a GGA sentence other than 0 would indicate a valid position.

5.1.3.3 Integrity Risk Computation Procedure

Similar to what has been pointed out previously, a minor concern can be raised about Integrity Risk

metrics, which refer to probabilities whose computation procedure has not been clearly specified. For

the sake of clarity let us focus on position (rather than velocity) Integrity Risk metrics, although all what

is said here can be applied to any of the Integrity Risk metrics. As previously, we are faced with the

decision whether to consider all epochs or those with a valid position and
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.