SIST-TS CEN/TS 17458:2021
(Main)Ambient air - Methodology for the assessment of the performance of source apportionment modelling system applications
Ambient air - Methodology for the assessment of the performance of source apportionment modelling system applications
The European Air Quality Directive (Directive on ambient air quality and cleaner air for Europe) identifies different uses for modelling: Assessment, planning, forecast and source apportionment (SA). This CEN/TS addresses source apportionment modelling and specifies performance tests to check whether given criteria for receptor oriented source apportionment (RM) are met. The scope of the tests set out in this CEN/TS is performance assessment of SA of particulate matter using RM in the context of the European Directives 2004/107/EC and 2008/50/EC (AQD) including the Commission Implementing Decision 2011/850/EU of 12 December 2011. The application of RM does not quantify the spatial origin of particulate matter hence this CEN/TS does not test spatial SA.
This CEN/TS addresses RM users: participants and organisers of source apportionment intercomparison studies as well as practitioners of individual source apportionment studies. This CEN/TS is suitable for the evaluation of results of a specific SA modelling system with respect to intercomparison reference values (a-priori known or calculated on the basis of participants' values, see 3.12) in the following application areas:
- Assessment of performance and uncertainties of a modelling system or modelling system set up using the indicators laid down in this CEN/TS.
- Testing and comparing different source apportionment outputs in a specific situation (applying an evaluation dataset) using the indicators laid down in this CEN/TS.
- QA/QC tests every time practitioners run a modelling system.
It should be noted for clarity that the procedures and calculations presented in this CEN/TS cannot be used to check the performance of a specific SA modelling result without having any a-priori reference information about the contributions of sources/source categories.
The principles of receptor oriented models are summarised in Annex A. An overview of uncertainty sources and recommendations about steps to follow in SA studies are provided in Annex B and Annex C.
There are different methodologies than RM widely used to accomplish SA, e.g. source oriented models. These other methodologies cover aspects of SA which are required in the AQD and are not addressed by RM. Performance assessment of such methodologies is out of the scope of this CEN/TS.
Außenluft - Methodik zur Erfassung der Leistungsfähigkeit von Systemanwendungen zur Quellenzuordnung
Die Europäische Luftqualitätsrichtlinie (Directive on ambient air quality and cleaner air for Europe) (2008/50/EC; AQD) definiert verschiedene Anwendungsgebiete für die Modellierung: Beurteilung, Planung, Vorhersage und Quellzuordnung (engl. source apportionment, SA). Diese CEN/TS bezieht sich auf die Quellzuordnungs-Modellierung und spezifiziert Leistungsfähigkeitsprüfungen zur Feststellung, ob festgelegte Kriterien für die rezeptororientierten Quellzuordnungsmodelle (Rezeptormodelle, RM) erfüllt werden. Der Anwendungsbereich für die Prüfungen in dieser CEN/TS beschreibt die Leistungsbeurteilung der Quellzuordnung partikulären Materials (PM) mithilfe rezeptororientierter Quellmodellierung im Kontext der Europäischen Richtlinien 2004/107/EC und AQD einschließlich des Durchführungsbeschlusses der europäischen Kommission 2011/850/EU vom 12. Dezember 2011. Die Anwendung der RM quantifiziert nicht die räumliche Quelle des PM, somit umfasst diese CEN/TS keine Prüfung der räumlichen Quellzuordnung.
Diese CEN/TS wendet sich an Nutzer der RM: Anwender individueller Quellzuordnungsstudien sowie Teilnehmer und Organisatoren von Quellzuordnungsvergleichsstudien. Diese CEN/TS ist für die Auswertung der Ergebnisse eines spezifischen Modellierungssystems zur Quellzuordnung mit Hinsicht auf Referenzwerte (a priori bekannt oder auf Grundlage der Vergleichswerte der Teilnehmer berechnet) in den folgenden Anwendungsbereichen geeignet:
— Erfassung der Leistungsfähigkeit und Unsicherheiten eines Modellierungssystems oder der Einrichtung eines Modellierungssystems durch Verwendung der in dieser CEN/TS dargelegten Indikatoren.
— Prüfen und Vergleichen verschiedener Quellzuordnungsergebnisse in einer spezifischen Situation (Verwenden eines Evaluierungsdatensatzes) mithilfe der in dieser CEN/TS dargelegten Indikatoren.
— QA/QC-Prüfungen bei jeder Verwendung eines Modellierungssystems durch einen Anwender.
Es sollte zur Verdeutlichung beachtet werden, dass die in dieser CEN/TS dargestellten Verfahren und Berechnungen nicht für die Prüfung der Leistungsfähigkeit eines spezifischen Quellzuordnungs-Modellierungsergebnisses verwendet werden können, ohne a priori Referenzinformation über die Beiträge der Quellen/Quellenkategorien zu haben.
ANMERKUNG Die Anwendung dieser CEN/TS impliziert, dass die Vergleichsprüfung von einer Institution organisiert und koordiniert ist, die über die nötige technische Ausstattung und Unabhängigkeit verfügt; die Definition einer solchen Institution geht über den Anwendungsbereich dieser TS hinaus.
Die Grundsätze der RM sind in Anhang A zusammengefasst. Eine Übersicht über Messunsicherheitsquellen und Empfehlungen zu Schritten, die in Quellzuordnungsstudien zu befolgen sind, sind in Anhang B und Anhang C dargestellt. Für weitere Informationen zu Quellzuordnungsmethodiken, siehe [1; 2; 3].
Es gibt neben RM auch andere Methodiken, die häufig zur Quellzuordnung verwendet werden, z. B. quellorientierte Modelle. Diese anderen Methodiken decken Aspekte der Quellzuordnung ab, die von der AQD gefordert und von der RM nicht behandelt werden (z. B. Zuordnung von Schadstoffen zu geographischen Emissionsgebieten). Die Leistungserfassung dieser Methodiken liegt außerhalb des Anwendungsbereichs dieser CEN/TS.
Air ambiant - Méthodologie d'évaluation des performances dapplications de modélisation de l'attribution des sources et des récepteurs
La Directive européenne concernant la qualité de l’air ambiant et un air pur pour l’Europe (2008/50/CE ; AQD) identifie différents usages de la modélisation : évaluation, planification, prévision et contribution des sources (SA). Le présent document concerne la modélisation de la contribution des sources et spécifie les essais de performance permettant de s’assurer que les critères donnés pour les modèles de contribution des sources de type « récepteur-orienté » (RM) sont remplis. Le domaine d’application des essais décrits dans le présent document est l’évaluation de la performance de SA de la matière particulaire à l’aide de RM dans le contexte de la Directive européenne 2004/107/CE et de l’AQD, y compris la Décision d’exécution de la Commission 2011/850/UE du 12 décembre 2011. L’application de RM ne permet pas de quantifier l’origine spatiale de la matière particulaire ; par conséquent, le présent document n’évalue pas la SA spatiale.
Le présent document s’adresse aux utilisateurs de RM et opérateurs d’études de contribution des sources ainsi qu’aux participants et organisateurs de comparaisons interlaboratoires de la contribution des sources. Le présent document est applicable à l’évaluation des résultats d’un système de modélisation SA spécifique par rapport à des valeurs de référence (a priori connues ou calculées d’après les valeurs des participants à la comparaison interlaboratoires) dans les secteurs d’application suivants :
- Évaluation de la performance et des incertitudes d’un système de modélisation ou d’une configuration du système de modélisation à l’aide des indicateurs énoncés dans le présent document.
- Essais et comparaison des différents résultats de contribution des sources dans une situation spécifique (application d’un jeu de données d’évaluation) à l’aide des indicateurs énoncés dans le présent document.
- Essais AQ/CQ à chaque fois qu’un opérateur utilise un système de modélisation.
Il convient de noter que, pour plus de clarté, les modes opératoires et les calculs présentés dans le présent document ne peuvent pas être utilisés pour vérifier la performance d’un résultat de modélisation de SA spécifique sans disposer a priori d’informations de référence sur les contributions des sources/catégories de sources.
NOTE L’application du présent document implique que la comparaison interlaboratoires soit organisée et coordonnée par un organisme indépendant disposant des compétences techniques nécessaires ; sa définition ne fait pas partie du domaine d’application du présent document.
Les principes des RM sont résumés à l’Annexe A. Une vue d’ensemble des sources d’incertitude et des recommandations relatives aux étapes à respecter lors des études de SA sont fournies aux Annexes B et C. Pour plus d’informations sur les méthodes de SA, voir par exemple [1 ; 2 ; 3].
D’autres méthodes que les RM sont couramment utilisées pour effectuer la SA, par exemple les modèles de type « source-orienté ». Ces autres méthodes couvrent certains aspects de SA qui sont requis dans l’AQD et ne sont pas abordés par les RM (par exemple, attribution de polluants à des zones géographiques d’émission). L’évaluation de la performance de ces méthodes ne fait pas partie du domaine d’application du présent document.
Zunanji zrak - Metodologija za oceno lastnosti aplikacij sistemov za modeliranje porazdelitve virov
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-TS CEN/TS 17458:2021
01-februar-2021
Zunanji zrak - Metodologija za oceno lastnosti aplikacij sistemov za modeliranje
porazdelitve virov
Ambient air - Methodology for the assessment of the performance of source
apportionment modelling system applications
Außenluft - Methodik zur Erfassung der Leistungsfähigkeit von Systemanwendungen zur
Quellenzuordnung
Air ambiant - Méthodologie d'évaluation des performances dapplications de modélisation
de l'attribution des sources et des récepteurs
Ta slovenski standard je istoveten z: CEN/TS 17458:2020
ICS:
13.040.20 Kakovost okoljskega zraka Ambient atmospheres
SIST-TS CEN/TS 17458:2021 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-TS CEN/TS 17458:2021
---------------------- Page: 2 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458
TECHNICAL SPECIFICATION
SPÉCIFICATION TECHNIQUE
December 2020
TECHNISCHE SPEZIFIKATION
ICS 13.040.01
English Version
Ambient air - Methodology to assess the performance of
receptor oriented source apportionment modelling
applications for particulate matter
Air ambiant - Méthode d'évaluation de la performance Außenluft - Methodik zur Erfassung der
d'applications d'un système de modélisation de la Leistungsfähigkeit von Systemanwendungen zur
contribution des sources de particules en suspension Quellenzuordnung
de type " récepteur-orienté "
This Technical Specification (CEN/TS) was approved by CEN on 24 February 2020 for provisional application.
The period of validity of this CEN/TS is limited initially to three years. After two years the members of CEN will be requested to
submit their comments, particularly on the question whether the CEN/TS can be converted into a European Standard.
CEN members are required to announce the existence of this CEN/TS in the same way as for an EN and to make the CEN/TS
available promptly at national level in an appropriate form. It is permissible to keep conflicting national standards in force (in
parallel to the CEN/TS) until the final decision about the possible conversion of the CEN/TS into an EN is reached.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2020 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TS 17458:2020 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
Contents Page
European foreword . 3
1 Scope . 4
2 Normative references . 4
3 Terms and definitions . 5
4 Symbols and abbreviations . 7
5 Fundamentals of the evaluation methodology . 8
6 Arrays of tests . 9
7 The performance assessment method . 9
7.1 Evaluation data sets . 9
7.2 Determination of the SCE reference values and their standard uncertainties . 9
7.2.1 Consensus value from participants . 9
7.2.2 Formulation of a consensus value from a synthetic data set . 10
8 Performance indicators and other indicators . 10
8.1 Complementary indicators . 10
8.1.1 General . 10
8.1.2 Apportioned pollutant mass . 10
8.1.3 Number of identified sources . 11
8.2 Similarity indicators . 11
8.2.1 General . 11
8.2.2 Pearson product-moment correlation coefficient . 12
8.2.3 Standardized Identity Distance (SID, Annex I) . 12
8.2.4 Weighted Distance (WD) . 12
8.3 Performance indicators . 13
8.3.1 General . 13
8.3.2 z-scores for the average SCEs . 13
8.3.3 RMSE for the SCEs time series . 14
u
8.3.4 Zeta-score for the uncertainty of the SCEs . 14
Annex A (informative) Principle of receptor oriented models . 15
Annex B (informative) Recommended steps for SA studies with receptor oriented models . 16
Annex C (informative) Sources of uncertainty in receptor oriented models . 17
Annex D (informative) Array of tests for individual SA runs (Responsible: Practitioner) . 18
Annex E (normative) Array of tests for intercomparisons (Responsible: Coordinator) . 19
Annex F (normative) Calculation of the SCE reference values X and X in real-world data
kt k
sets . 20
Annex G (informative) Robust average and standard deviation. 21
Annex H (informative) Acceptability range for complementary tests . 22
Annex I (informative) Geometric meaning of the Standardized Identity Distance . 23
Annex J (normative) Acceptability range for the similarity and performance tests . 25
Annex K (informative) Intercomparison organization and test reports . 26
Bibliography . 28
2
---------------------- Page: 4 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
European foreword
This document (CEN/TS 17458:2020) has been prepared by Technical Committee CEN/TC 264 “Air
quality”, the secretariat of which is held by DIN.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
According to the CEN/CENELEC Internal Regulations, the national standards organisations of the
following countries are bound to announce this Technical Specification: Austria, Belgium, Bulgaria,
Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland,
Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of
North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United
Kingdom.
3
---------------------- Page: 5 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
1 Scope
The European Directive on ambient air quality and cleaner air for Europe (2008/50/EC; AQD) identifies
different uses for modelling: Assessment, planning, forecast and source apportionment (SA). This
document addresses source apportionment modelling and specifies performance tests to check whether
given criteria for receptor oriented source apportionment models (RM) are met. The scope of the tests
set out in this document is the performance assessment of SA of particulate matter using RM in the
context of the European Directives 2004/107/EC and AQD, including the Commission Implementing
Decision 2011/850/EU of 12 December 2011. The application of RM does not quantify the spatial origin
of particulate matter; hence, this document does not test spatial SA.
This document addresses RM users: practitioners of individual source apportionment studies as well as
participants and organizers of source apportionment intercomparison studies. This document is suitable
for the evaluation of results of a specific SA modelling system with respect to reference values (a priori
known or calculated on the basis of intercomparison participants' values) in the following application
areas:
— Assessment of performance and uncertainties of a modelling system or modelling system set up using
the indicators laid down in this document.
— Testing and comparing different source apportionment outputs in a specific situation (applying an
evaluation data set) using the indicators laid down in this document.
— QA/QC tests every time practitioners run a modelling system.
It should be noted for clarity that the procedures and calculations presented in this document cannot be
used to check the performance of a specific SA modelling result without having any a priori reference
information about the contributions of sources/source categories.
NOTE The application of this document implies that the intercomparison is organized and coordinated by an
institution with the necessary technical capabilities and independence; the definition of which is beyond the scope
of this document.
The principles of RM are summarized in Annex A. An overview of uncertainty sources and
recommendations about steps to follow in SA studies are provided in Annex B and Annex C. For further
information about SA methodologies, refer to e.g. [1; 2; 3].
There are methodologies different from RM which are widely used to accomplish SA, e.g. source oriented
models. These other methodologies cover aspects of SA which are required in the AQD and are not
addressed by RM (e.g. allocation of pollutants to geographic emission areas). Performance assessment of
such methodologies is out of the scope of this document.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 13528:2015, Statistical methods for use in proficiency testing by interlaboratory comparison
4
---------------------- Page: 6 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
3.1
candidate modelling system
modelling system or modelling system set up that is being tested in the intercomparison or being applied
in isolation
3.2
candidate source
every source or source category present in the result of a SA candidate modelling system which is tested
with the methodology described in this document
3.3
coordinator
organisation with responsibility for coordinating all activities involved in the operation of an
intercomparison exercise
3.4
contribution-to-species
mass of each single chemical element or compound forming part of a (composite) pollutant apportioned
to a source or source category, expressed as a percentage
Note 1 to entry: Depending on the type of SA modelling applications, this concept could be referred to as:
“contribution by species” in CMB 8.2, “explained variation” in PMF 2, “percentage of species total matrix” in EPA
PMF v3, and “factor fingerprint” in EPA-PMF v5.
3.5
input uncertainty
statistical dispersion of the values that can be reasonably attributed to every single species ambient
concentration in the input matrix of receptor oriented models
3.6
derived source profile
relative chemical composition of the source or source category, resulting from a SA model run, expressed
as parts per unit mass of every chemical species with respect to the total source or source category mass
3.7
factor
underlying latent variable identified by a multivariate analysis SA model (e.g. factor analysis or singular
value decomposition) associated with an air pollution source or source category
3.8
maximum accepted distance
maximum Euclidean distance, on a Cartesian plane, between the point representing the values
(concentration ratios) of a chemical species in two source profiles, one on the abscissa and the other on
the ordinate, and the identity line, which is considered acceptable to indicate the similarity of the two
values
5
---------------------- Page: 7 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
3.9
output uncertainty
uncertainty attributed to the average source contribution estimate (SCE), expressed in the same units,
for a given time window of a source or source category in a source apportionment application result
3.10
receptor oriented models
models that, using as input the measured concentrations of pollutants at a given site (receptor) in a
defined time window, provide source contribution estimates (SCEs) by applying multivariate analysis
3.11
reference source profile
source profile used as reference for the similarity tests of derived source profiles. In general, the reference
source profiles are chemical profiles obtained from samples collected specifically to characterise the
source or source category and which are available in the literature or in public repositories (e.g.
SPECIATE, SPECIEUROPE)
3.12
reference value
a priori known or calculated source contribution estimate (SCE) attributed to a source or source category,
represented by an evaluation dataset
3.13
results
output obtained by running a candidate modelling system using an evaluation dataset. The source
apportionment results consist of e.g.: source contribution estimates (SCEs) including their output
uncertainty, derived source profiles, source contribution time series and contribution-to-species
3.14
source apportionment
SA
practice of deriving quantitative information about the contribution of sources and/or source categories
to the concentration of pollutants at a given point or area in a defined time window
3.15
source category
group of air pollution sources of the same type that are characterised by similar chemical composition
and/or temporal profile
3.16
source contribution estimate
SCE
−3
amount of the pollutant expressed as mass concentration (e.g. µg m ), attributed to a specific source or
source category in SA model application results
3.17
source of air pollution
any human activity or natural process that causes pollutants to be released into the atmosphere or to be
produced in the atmosphere
6
---------------------- Page: 8 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
3.18
source profile
relative chemical composition of a (composite) pollutant emitted from an air pollution source expressed
as the ratio of the mass of every single chemical species to the total mass of the pollutant
3.19
source oriented models
models that, starting from emission inventories, meteorological fields and boundary conditions, simulate
the physical and/or chemical atmospheric processes affecting the emitted pollutants
3.20
species
single chemical elements or compounds that compose the studied pollutant when it is a family of
compounds or an aggregate (composite)
3.21
pollutant
collective term meaning the total mass of the substance for which source contribution estimates (SCEs)
are determined using source apportionment methods
Note 1 to entry: It could either be a single chemical compound (e.g. Hg), a family of similar chemical compounds
(e.g. polycyclic aromatic hydrocarbons) or composite: an aggregation of different chemical compounds (e.g.
particulate matter).
4 Symbols and abbreviations
fr tests found-reference tests
ff tests found-found tests
M sum of the SCEs of all the candidate sources k in time step t of a given result
t
M sum of the time averaged SCEs of all the candidate sources k of a given result
O observed mass concentration of the pollutant in time step t
t
O time averaged observed mass concentration of the pollutant
Pearson Pearson product-moment correlation coefficient (r )
xy
RM receptor oriented source apportionment model
RMSE uncertainty normalized root mean square error
u
RMSE root mean square error weighted by the standard deviation of the observations
σ
o
Pearson product-moment correlation coefficient (Pearson)
r
xy
SA source apportionment
SCE source contribution estimate (mass concentration)
SID standardized identity distance
WD weighted distance
xkt SCE of candidate source k in time step t
x average SCE of candidate source k
k
X reference value for the SCE of source k in time step t
kt
7
---------------------- Page: 9 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
X reference value for the average SCE of source k
k
u standard uncertainty of X
Xkt kt
u standard uncertainty of X
Xk k
z z-score
σ uncertainty for proficiency test
p
ζ zeta-score
5 Fundamentals of the evaluation methodology
The goal of the procedures and tests described in this document is to assess the performance of receptor
oriented SA modelling applications. This is done by comparing the results of the tested SA candidate
modelling system using as input pre-defined evaluation data sets with the reference values for the used
data set (details in 7.1 and 7.2).
The evaluation methodology applied to the SA results encompasses three types of tests: complementary
tests, similarity tests and performance tests. The similarity and performance tests are carried out
independently for every candidate source or source category reported in the results. Hereon, the term
“source” includes both source and source category, unless explicitly mentioned.
The complementary tests provide ancillary information about the overall consistency of the SA results.
They do not assess the accuracy of the factor/source contributions’ identification and quantification. The
complementary tests include a) the apportioned pollutant mass test, consisting of the comparison
between the pollutant reference mass (e.g. gravimetric mass) and the sum of the SCEs of all the sources
and b) the comparison among the number of sources in every reported SA result or, if available, with the
reference number of sources (see 8.1).
NOTE For the purposes of this document, the following SCEs are considered: a) the values for every sample or
time step and b) their average for the entire time window represented by the evaluation data set.
The similarity tests assess whether a candidate source in a SA result obtained with a candidate modelling
system can be allocated to a source category. The outcome of these tests is used to select the candidates
for the determination of the reference values (see 7.2.1). To that end, the candidate sources are compared
with reference source profiles (fr tests). In the absence of reference source profiles, all the candidate
sources attributed to a source category are compared among each other (ff tests). The similarity tests
compare the sources based on their chemical profiles, the time series of their SCEs, and their
contribution-to-species. When provided, the uncertainty of the derived source profiles in the reported
results is tested by comparison with the uncertainty of the reference source profiles (see 8.2 on similarity
indicators).
The performance of a candidate SA modelling system is evaluated by comparing the SCEs of the candidate
sources, resulting from runs with a given evaluation data set, with the reference values for that specific
data set. The outcome of the performance test depends on whether the difference between the candidate
and the reference meets a pre-established quality objective (see 8.3).
There are two types of performance tests: a) tests based on the z-scores (ISO 13528:2015), for the
average SCEs of the time window represented in the evaluation data set (x ) and b) tests based on the
k
uncertainty normalized root mean square error (RMSE ) [4; 5] for the time series of the SCEs (x ).
u kt
Candidates passing the z-score test are considered to have an average SCE for the entire modelled period
comparable with the reference value. Candidates passing the RMSE test are considered to have a SCE
u
time trend for the modelled period comparable with the reference value. Candidates passing both tests
have a performance which is considered “sufficient” for the purposes of the present document.
8
---------------------- Page: 10 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
6 Arrays of tests
The tests to apply vary according to the objective of the application: individual run or intercomparison
exercise.
In the individual runs a practitioner executes a source apportionment application using as input a data
set for which the SCE reference values are not available. The array of tests for individual runs is reported
in Annex D.
In intercomparison exercises, more practitioners execute SA models using as input the same evaluation
data set (see 7.1) and the SCE reference values (X) are either available in advance (SA applications using
synthetic evaluation data sets) or are calculated from the participant results. SA applications using
synthetic evaluation data sets are included in this category. The array of tests for intercomparison
exercises are reported in Annex E.
7 The performance assessment method
7.1 Evaluation data sets
The evaluation data sets used as input for the candidate modelling systems are derived from observed
data or are synthetic.
The evaluation data sets consist of a matrix with the concentrations of selected chemical species in the
ambient air measured at a receptor site over a given time-window and with a given time-resolution. The
input uncertainties of the entries in the matrix are provided. The principles for the development of state-
of-the-art data sets to be used as input for RMs are described in [1]. Distributing additional information
about the study site or area (e.g. measured source profiles, pollutant concentrations, meteorological data,
etc.) is optional. Siting information about the monitoring location of the observed data shall be also
distributed.
Synthetic evaluation data sets are created by fixing the contribution of sources in every sample or time
step. The concentration of the chemical species in the synthetic evaluation data sets is mathematically
coherent with the source contributions. To create synthetic evaluation data sets, noise is added using
randomization techniques and attributing an input uncertainty to the entries of the evaluation data set
proportional to the noise e.g. [6].
The result of the performance assessment described in this document is associated with the classification
of the monitoring site location represented in the evaluation data set (e.g. urban background, regional
background, sites close to one specific source) and, therefore, it is not providing evidence about the
performance of the candidate modelling system for different site classifications.
NOTE Estimating the transferability of the performance assessment obtained with one evaluation data set to
classes of sites not present in the evaluation data set is beyond the scope of this document.
7.2 Determination of the SCE reference values and their standard uncertainties
7.2.1 Consensus value from participants
When executing a SA model using real-world evaluation data sets, the true SCEs values are unknown.
Therefore, for this kind of evaluation data set the results reported by participants in intercomparison
exercises are used to calculate the reference values.
9
---------------------- Page: 11 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
The reference values for the averaged SCE (X ) and for the SCE in every time step (X ) and their standard
k kt
uncertainties u and u are calculated for every source (k) separately. Such reference values are
Xk Xkt
associated with the evaluation data set of observations that was used as input for the intercomparison.
The best estimators of the reference values and their uncertainties are the robust average and standard
deviation, respectively, of the results that pass the similarity tests and are calculated using the method
described in Annex F and Annex G.
7.2.2 Formulation of a consensus value from a synthetic data set
When the evaluation data set is synthetic, the values of X and u are defined a priori and the
kt Xkt
corresponding reference values X and their uncertainties u are best estimators of the mean and
k Xk
standard deviation of the X population, respectively, for each source (k).
kt
8 Performance indicators and other indicators
8.1 Complementary indicators
8.1.1 General
The application of the complementary tests is outlined in the flow chart in Figure 1. The acceptability
ranges for the complementary tests are reported in Annex H and their use in the arrays of tests is
described in Annex D and Annex E.
Figure 1 — Flow diagram illustrating the complementary tests
8.1.2 Apportioned pollutant mass
This test is applied when the SA aims at estimating all the contributing sources. It aims at comparing the
sum of the mass contributed by the sources with the measured pollutant mass. There are different
methods for the apportioned mass tests:
— direct comparison of the SCEs sum with the pollutant mass,
— analysis of the linear regression parameters between the sum of the SCEs and the mass concentration
of the pollutant in every sample or time step, and
— calculation of the RMSE [6].
σ
o
10
---------------------- Page: 12 ----------------------
SIST-TS CEN/TS 17458:2021
CEN/TS 17458:2020 (E)
n
1 2
M −O
( )
∑ t t
1
n
RMSE = (1)
σ
o
σ
o
where M is the sum of the modelled SCEs of all the candidate sources in every time step t of a given result,
t
O is the observed (measured) mass concentration in every time step, σ is the standard deviation of the
t O
observed values and n is the number of considered time steps.
8.1.3 Number of identified sources
The comparison of the number of sources estimated may be accomplished visually using a histogram plot.
A reference number of sources is available only for synthetic evaluation data sets.
8.2 Similarity indicators
8.2.1 General
In the similarity tests, the comparison between sources is accomplished using similarity indicators:
Pearson product-moment correlation coefficient (Pearson) and the standardized identity distance (SID).
In addition, the weighted distance (WD) is used to assess whether the candidates’ declared output
uncertainty of the source profiles in the reported results is coherent with the one of the reference source
profile.
The reference source profiles used for the tests shall be coherent with the site represented in the
evaluation data set and the candidate
...
SLOVENSKI STANDARD
kSIST-TS FprCEN/TS 17458:2019
01-december-2019
[Not translated]
Ambient air - Methodology for the assessment of the performance of source
apportionment modelling system applications
Außenluft - Methodik zur Erfassung der Leistungsfähigkeit von Systemanwendungen zur
Quellenzuordnung
Air ambiant - Méthodologie d'évaluation des performances dapplications de modélisation
de l'attribution des sources et des récepteurs
Ta slovenski standard je istoveten z: FprCEN/TS 17458
ICS:
13.040.20 Kakovost okoljskega zraka Ambient atmospheres
kSIST-TS FprCEN/TS 17458:2019 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
kSIST-TS FprCEN/TS 17458:2019
---------------------- Page: 2 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FINAL DRAFT
TECHNICAL SPECIFICATION
FprCEN/TS 17458
SPÉCIFICATION TECHNIQUE
TECHNISCHE SPEZIFIKATION
October 2019
ICS
English Version
Ambient air - Methodology for the assessment of the
performance of source apportionment modelling system
applications
Air ambiant - Méthodologie d'évaluation des Außenluft - Methodik zur Erfassung der
performances d¿applications de modélisation de Leistungsfähigkeit von Systemanwendungen zur
l'attribution des sources et des récepteurs Quellenzuordnung
This draft Technical Specification is submitted to CEN members for Vote. It has been drawn up by the Technical Committee
CEN/TC 264.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are
aware and to provide supporting documentation.
Warning : This document is not a Technical Specification. It is distributed for review and comments. It is subject to change
without notice and shall not be referred to as a Technical Specification.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2019 CEN All rights of exploitation in any form and by any means reserved Ref. No. FprCEN/TS 17458:2019 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
Contents
European foreword . 3
1 Scope . 4
2 Normative references . 4
3 Terms and definitions . 5
4 Symbols and abbreviations . 7
5 Fundamentals of the evaluation methodology . 8
6 Arrays of tests . 9
7 The performance assessment method . 9
7.1 Evaluation data sets . 9
7.2 Determination of the SCE reference values and their standard uncertainties . 9
7.2.1 Consensus value from participants . 9
7.2.2 Formulation of a consensus value from a synthetic data set . 10
8 Performance indicators and other indicators . 10
8.1 Complementary indicators . 10
8.1.1 General . 10
8.1.2 Apportioned pollutant mass . 10
8.1.3 Number of identified sources . 11
8.2 Similarity indicators . 11
8.2.1 General . 11
8.2.2 Pearson product-moment correlation coefficient . 11
8.2.3 Standardized Identity Distance (SID, Annex I) . 12
8.2.4 Weighted Distance (WD) . 12
8.3 Performance indicators . 12
8.3.1 General . 12
8.3.2 z-scores for the average SCEs . 13
8.3.3 RMSE for the SCEs time series . 13
u
8.3.4 Zeta-score for the uncertainty of the SCEs . 14
Annex A (informative) Principle of receptor oriented models . 15
Annex B (informative) Recommended steps for SA studies with receptor oriented models . 16
Annex C (informative) Sources of uncertainty in receptor oriented models . 17
Annex D (informative) Array of tests for individual SA runs (Responsible: Practitioner) . 18
Annex E (normative) Array of tests for intercomparisons (Responsible: Coordinator) . 19
Annex F (normative) Calculation of the SCE reference values X and X in real-world data
kt k
sets . 20
Annex G (informative) Robust average and standard deviation. 21
Annex H (informative) Acceptability range for complementary tests . 22
Annex I (informative) Geometric meaning of the Standardized Identity Distance . 23
Annex J (normative) Acceptability range for the similarity and performance tests . 25
Annex K (informative) Intercomparison organization and test reports . 26
Bibliography . 28
2
---------------------- Page: 4 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
European foreword
This document (FprCEN/TS 17458:2019) has been prepared by Technical Committee CEN/TC 264 “Air
quality”, the secretariat of which is held by DIN.
This document is currently submitted to the vote.
3
---------------------- Page: 5 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
1 Scope
The European Directive on ambient air quality and cleaner air for Europe (2008/50/EC; AQD) identifies
different uses for modelling: Assessment, planning, forecast and source apportionment (SA). This
document addresses source apportionment modelling and specifies performance tests to check whether
given criteria for receptor oriented source apportionment models (RM) are met. The scope of the tests
set out in this document is the performance assessment of SA of particulate matter using RM in the
context of the European Directives 2004/107/EC and AQD, including the Commission Implementing
Decision 2011/850/EU of 12 December 2011. The application of RM does not quantify the spatial origin
of particulate matter; hence, this document does not test spatial SA.
This document addresses RM users: practitioners of individual source apportionment studies as well as
participants and organizers of source apportionment intercomparison studies. This document is suitable
for the evaluation of results of a specific SA modelling system with respect to reference values (a priori
known or calculated on the basis of intercomparison participants' values) in the following application
areas:
— Assessment of performance and uncertainties of a modelling system or modelling system set up using
the indicators laid down in this document.
— Testing and comparing different source apportionment outputs in a specific situation (applying an
evaluation data set) using the indicators laid down in this document.
— QA/QC tests every time practitioners run a modelling system.
It should be noted for clarity that the procedures and calculations presented in this document cannot be
used to check the performance of a specific SA modelling result without having any a priori reference
information about the contributions of sources/source categories.
NOTE The application of this document implies that the intercomparison is organized and coordinated by an
institution with the necessary technical capabilities and independence; the definition of which is beyond the scope
of this document.
The principles of RM are summarized in Annex A. An overview of uncertainty sources and
recommendations about steps to follow in SA studies are provided in Annex B and Annex C. For further
information about SA methodologies, refer to e.g. [1; 2; 3].
There are methodologies different from RM which are widely used to accomplish SA, e.g. source oriented
models. These other methodologies cover aspects of SA which are required in the AQD and are not
addressed by RM (e.g. allocation of pollutants to geographic emission areas). Performance assessment of
such methodologies is out of the scope of this document.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 13528:2015, Statistical methods for use in proficiency testing by interlaboratory comparison
4
---------------------- Page: 6 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
3.1
candidate modelling system
modelling system or modelling system set up that is being tested in the intercomparison or being applied
in isolation
3.2
candidate source
every source or source category present in the result of a SA candidate modelling system which is tested
with the methodology described in this document
3.3
coordinator
organisation with responsibility for coordinating all activities involved in the operation of an
intercomparison exercise
3.4
contribution-to-species
mass of each single chemical element or compound forming part of a (composite) pollutant apportioned
to a source or source category, expressed as a percentage
Note 1 to entry: Depending on the type of SA modelling applications, this concept could be referred to as:
“contribution by species” in CMB 8.2, “explained variation” in PMF 2, “percentage of species total matrix” in EPA
PMF v3, and “factor fingerprint” in EPA-PMF v5.
3.5
input uncertainty
statistical dispersion of the values that can be reasonably attributed to every single species ambient
concentration in the input matrix of receptor oriented models
3.6
derived source profile
relative chemical composition of the source or source category, resulting from a SA model run, expressed
as parts per unit mass of every chemical species with respect to the total source or source category mass
3.7
factor
underlying latent variable identified by a multivariate analysis SA model (e.g. factor analysis or singular
value decomposition) associated with an air pollution source or source category
3.8
maximum accepted distance
maximum Euclidean distance, on a cartesian plane, between the point representing the values
(concentration ratios) of a chemical species in two source profiles, one on the abscissa and the other on
the ordinate, and the identity line, which is considered acceptable to indicate the similarity of the two
values
5
---------------------- Page: 7 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
3.9
output uncertainty
uncertainty attributed to the average source contribution estimate (SCE), expressed in the same units,
for a given time window of a source or source category in a source apportionment application result
3.10
receptor oriented models
models that, using as input the measured concentrations of pollutants at a given site (receptor) in a
defined time window, provide source contribution estimates (SCEs) by applying multivariate analysis
3.11
reference source profile
source profile used as reference for the similarity tests of derived source profiles. In general, the reference
source profiles are chemical profiles obtained from samples collected specifically to characterise the
source or source category and which are available in the literature or in public repositories (e.g.
SPECIATE, SPECIEUROPE)
3.12
reference value
a priori known or calculated source contribution estimate (SCE) attributed to a source or source category,
represented by an evaluation dataset
3.13
results
output obtained by running a candidate modelling system using an evaluation dataset. The source
apportionment results consist of e.g.: source contribution estimates (SCEs) including their output
uncertainty, derived source profiles, source contribution time series and contribution-to-species
3.14
source apportionment (SA)
practice of deriving quantitative information about the contribution of sources and/or source categories
to the concentration of pollutants at a given point or area in a defined time window
3.15
source category
group of air pollution sources of the same type that are characterised by similar chemical composition
and/or temporal profile
3.16
source contribution estimate (SCE)
−3
amount of the pollutant expressed as mass concentration (e.g. µg m ), attributed to a specific source or
source category in SA model application results
3.17
source of air pollution
any human activity or natural process that causes pollutants to be released into the atmosphere or to be
produced in the atmosphere
3.18
source profile
relative chemical composition of a (composite) pollutant emitted from an air pollution source expressed
as the ratio of the mass of every single chemical species to the total mass of the pollutant
6
---------------------- Page: 8 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
3.19
source oriented models
models that, starting from emission inventories, meteorological fields and boundary conditions, simulate
the physical and/or chemical atmospheric processes affecting the emitted pollutants
3.20
species
single chemical elements or compounds that compose the studied pollutant when it is a family of
compounds or an aggregate (composite)
3.21
pollutant
collective term meaning the total mass of the substance for which source contribution estimates (SCEs)
are determined using source apportionment methods
Note 1 to entry: It could either be a single chemical compound (e.g. Hg), a family of similar chemical
compounds (e.g. polycyclic aromatic hydrocarbons) or composite: an aggregation of different chemical compounds
(e.g. particulate matter).
4 Symbols and abbreviations
fr tests found-reference tests
ff tests found-found tests
M sum of the SCEs of all the candidate sources k in time step t of a given result
t
M sum of the time averaged SCEs of all the candidate sources k of a given result
O observed mass concentration of the pollutant in time step t
t
O time averaged observed mass concentration of the pollutant
Pearson Pearson product-moment correlation coefficient (r )
xy
RM receptor oriented source apportionment model
RMSE uncertainty normalized root mean square error
u
RMSE root mean square error weighted by the standard deviation of the observations
σ
o
Pearson product-moment correlation coefficient (Pearson)
r
xy
SA source apportionment
SCE source contribution estimate (mass concentration)
SID standardized identity distance
WD weighted distance
x SCE of candidate source k in time step t
kt
x average SCE of candidate source k
k
X reference value for the SCE of source k in time step t
kt
X reference value for the average SCE of source k
k
u standard uncertainty of X
Xkt kt
u standard uncertainty of X
Xk k
7
---------------------- Page: 9 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
z z-score
σ uncertainty for proficiency test
p
ζ zeta-score
5 Fundamentals of the evaluation methodology
The goal of the procedures and tests described in this document is to assess the performance of receptor
oriented SA modelling applications. This is done by comparing the results of the tested SA candidate
modelling system using as input pre-defined evaluation data sets with the reference values for the used
data set (details in 7.1 and 7.2).
The evaluation methodology applied to the SA results encompasses three types of tests: complementary
tests, similarity tests and performance tests. The similarity and performance tests are carried out
independently for every candidate source or source category reported in the results. Hereon, the term
“source” includes both source and source category, unless explicitly mentioned.
The complementary tests provide ancillary information about the overall consistency of the SA results.
They do not assess the accuracy of the factor/source contributions’ identification and quantification. The
complementary tests include a) the apportioned pollutant mass test, consisting of the comparison
between the pollutant reference mass (e.g. gravimetric mass) and the sum of the SCEs of all the sources
and b) the comparison among the number of sources in every reported SA result or, if available, with the
reference number of sources (see 8.1).
NOTE For the purposes of this document, the following SCEs are considered: a) the values for every sample or
time step and b) their average for the entire time window represented by the evaluation data set.
The similarity tests assess whether a candidate source in a SA result obtained with a candidate modelling
system can be allocated to a source category. The outcome of these tests is used to select the candidates
for the determination of the reference values (see 7.2.1). To that end, the candidate sources are compared
with reference source profiles (fr tests). In the absence of reference source profiles, all the candidate
sources attributed to a source category are compared among each other (ff tests). The similarity tests
compare the sources based on their chemical profiles, the time series of their SCEs, and their
contribution-to-species. When provided, the uncertainty of the derived source profiles in the reported
results is tested by comparison with the uncertainty of the reference source profiles (see 8.2 on similarity
indicators).
The performance of a candidate SA modelling system is evaluated by comparing the SCEs of the candidate
sources, resulting from runs with a given evaluation data set, with the reference values for that specific
data set. The outcome of the performance test depends on whether the difference between the candidate
and the reference meets a pre-established quality objective (see 8.3).
There are two types of performance tests: a) tests based on the z-scores (ISO 13528:2015), for the
average SCEs of the time window represented in the evaluation data set (x ) and b) tests based on the
k
uncertainty normalized root mean square error (RMSE ) [4; 5] for the time series of the SCEs (x ).
u kt
Candidates passing the z-score test are considered to have an average SCE for the entire modelled period
comparable with the reference value. Candidates passing the RMSE test are considered to have a SCE
u
time trend for the modelled period comparable with the reference value. Candidates passing both tests
have a performance which is considered “sufficient” for the purposes of the present document.
8
---------------------- Page: 10 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
6 Arrays of tests
The tests to apply vary according to the objective of the application: individual run or intercomparison
exercise.
In the individual runs a practitioner executes a source apportionment application using as input a data
set for which the SCE reference values are not available. The array of tests for individual runs is reported
in Annex D.
In intercomparison exercises, more practitioners execute SA models using as input the same evaluation
data set (see 7.1) and the SCE reference values (X) are either available in advance (SA applications using
synthetic evaluation data sets) or are calculated from the participant results. SA applications using
synthetic evaluation data sets are included in this category. The array of tests for intercomparison
exercises are reported in Annex E.
7 The performance assessment method
7.1 Evaluation data sets
The evaluation data sets used as input for the candidate modelling systems are derived from observed
data or are synthetic.
The evaluation data sets consist of a matrix with the concentrations of selected chemical species in the
ambient air measured at a receptor site over a given time-window and with a given time-resolution. The
input uncertainties of the entries in the matrix are provided. The principles for the development of state-
of-the-art data sets to be used as input for RMs are described in [1]. Distributing additional information
about the study site or area (e.g. measured source profiles, pollutant concentrations, meteorological data,
etc.) is optional. Siting information about the monitoring location of the observed data shall be also
distributed.
Synthetic evaluation data sets are created by fixing the contribution of sources in every sample or time
step. The concentration of the chemical species in the synthetic evaluation data sets is mathematically
coherent with the source contributions. To create synthetic evaluation data sets, noise is added using
randomization techniques and attributing an input uncertainty to the entries of the evaluation data set
proportional to the noise e.g. [6].
The result of the performance assessment described in this document is associated with the classification
of the monitoring site location represented in the evaluation data set (e.g. urban background, regional
background, sites close to one specific source) and, therefore, it is not providing evidence about the
performance of the candidate modelling system for different site classifications.
NOTE Estimating the transferability of the performance assessment obtained with one evaluation data set to
classes of sites not present in the evaluation data set is beyond the scope of this document.
7.2 Determination of the SCE reference values and their standard uncertainties
7.2.1 Consensus value from participants
When executing a SA model using real-world evaluation data sets, the true SCEs values are unknown.
Therefore, for this kind of evaluation data set the results reported by participants in intercomparison
exercises are used to calculate the reference values.
The reference values for the averaged SCE (X ) and for the SCE in every time step (X ) and their standard
k kt
uncertainties uXk and uXkt are calculated for every source (k) separately. Such reference values are
associated with the evaluation data set of observations that was used as input for the intercomparison.
The best estimators of the reference values and their uncertainties are the robust average and standard
9
---------------------- Page: 11 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
deviation, respectively, of the results that pass the similarity tests and are calculated using the method
described in Annex F and Annex G.
7.2.2 Formulation of a consensus value from a synthetic data set
When the evaluation data set is synthetic, the values of X and u are defined a priori and the
kt Xkt
corresponding reference values X and their uncertainties u are best estimators of the mean and
k Xk
standard deviation of the Xkt population, respectively, for each source (k).
8 Performance indicators and other indicators
8.1 Complementary indicators
8.1.1 General
The application of the complementary tests is outlined in the flow chart in Figure 1. The acceptability
ranges for the complementary tests are reported in Annex H and their use in the arrays of tests is
described in Annex D and Annex E.
Figure 1 — Flow diagram illustrating the complementary tests
8.1.2 Apportioned pollutant mass
This test is applied when the SA aims at estimating all the contributing sources. It aims at comparing the
sum of the mass contributed by the sources with the measured pollutant mass. There are different
methods for the apportioned mass tests:
— direct comparison of the SCEs sum with the pollutant mass,
— analysis of the linear regression parameters between the sum of the SCEs and the mass concentration
of the pollutant in every sample or time step, and
— calculation of the RMSE [6].
σ
o
1 n 2
M −O
( )
∑ t t
1
n
RMSE = (1)
σ
o
σ
o
10
---------------------- Page: 12 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
where M is the sum of the modelled SCEs of all the candidate sources in every time step t of a given result,
t
O is the observed (measured) mass concentration in every time step, σ is the standard deviation of the
t O
observed values and n is the number of considered time steps.
8.1.3 Number of identified sources
The comparison of the number of sources estimated may be accomplished visually using a histogram plot.
A reference number of sources is available only for synthetic evaluation data sets.
8.2 Similarity indicators
8.2.1 General
In the similarity tests, the comparison between sources is accomplished using similarity indicators:
Pearson product-moment correlation coefficient (Pearson) and the standardized identity distance (SID).
In addition, the weighted distance (WD) is used to assess whether the candidates’ declared output
uncertainty of the source profiles in the reported results is coherent with the one of the reference source
profile.
The reference source profiles used for the tests shall be coherent with the site represented in the
evaluation data set and the candidate sources.
The application of the similarity tests is outlined in the flow chart in Figure 2. The acceptability ranges
for the similarity tests are reported in Annex J and their use in the arrays of tests is described in Annex D
and Annex E. The overall organization of an intercomparison is described in Annex K.
Figure 2 — Flow diagram illustrating the similarity tests
8.2.2 Pearson product-moment correlation coefficient
n
()x −−xy y
( )
∑ jj
1
r = (2)
xy
22
nn
xx−−y y
( ) ( )
∑∑j j
11
11
---------------------- Page: 13 ----------------------
kSIST-TS FprCEN/TS 17458:2019
FprCEN/TS 17458:2019 (E)
Where n is the number of items (samples or species) in the sources under comparison x and y [7]. Pearson
values equal or above 0,6 are generally considered acceptable [8]. This indicator is applied to calculate
the similarity between sources on the basis three different types of data: their chemical profiles, their
time tr
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.