CEN/TS 17458:2020

Ambient air - Methodology to assess the performance of receptor oriented source apportionment modelling applications for particulate matter

The European Directive on ambient air quality and cleaner air for Europe (2008/50/EC; AQD) identifies different uses for modelling: Assessment, planning, forecast and source apportionment (SA). This document addresses source apportionment modelling and specifies performance tests to check whether given criteria for receptor oriented source apportionment models (RM) are met. The scope of the tests set out in this document is the performance assessment of SA of particulate matter using RM in the context of the European Directives 2004/107/EC and AQD, including the Commission Implementing Decision 2011/850/EU of 12 December 2011. The application of RM does not quantify the spatial origin of particulate matter; hence, this document does not test spatial SA.
This document addresses RM users: practitioners of individual source apportionment studies as well as participants and organizers of source apportionment intercomparison studies. This document is suitable for the evaluation of results of a specific SA modelling system with respect to reference values (a priori known or calculated on the basis of intercomparison participants' values) in the following application areas:
-   Assessment of performance and uncertainties of a modelling system or modelling system set up using the indicators laid down in this document.
-   Testing and comparing different source apportionment outputs in a specific situation (applying an evaluation data set) using the indicators laid down in this document.
-   QA/QC tests every time practitioners run a modelling system.
It should be noted for clarity that the procedures and calculations presented in this document cannot be used to check the performance of a specific SA modelling result without having any a priori reference information about the contributions of sources/source categories.
NOTE   The application of this document implies that the intercomparison is organized and coordinated by an institution with the necessary technical capabilities and independence; the definition of which is beyond the scope of this document.
The principles of RM are summarized in Annex A. An overview of uncertainty sources and recommendations about steps to follow in SA studies are provided in Annex B and Annex C. For further information about SA methodologies, refer to e.g. [1; 2; 3].
There are methodologies different from RM which are widely used to accomplish SA, e.g. source oriented models. These other methodologies cover aspects of SA which are required in the AQD and are not addressed by RM (e.g. allocation of pollutants to geographic emission areas). Performance assessment of such methodologies is out of the scope of this document.

Außenluft - Methodik zur Erfassung der Leistungsfähigkeit von Systemanwendungen zur Quellenzuordnung

Die Europäische Luftqualitätsrichtlinie (Directive on ambient air quality and cleaner air for Europe) (2008/50/EG; AQD) definiert verschiedene Anwendungsgebiete für die Modellierung: Beurteilung, Planung, Vorhersage und Quellzuordnung (engl. source apportionment, SA). Dieses Dokument bezieht sich auf die Quellzuordnungs-Modellierung und spezifiziert Methoden zur Beurteilung, ob festgelegte Gütekriterien für die rezeptororientierten Quellzuordnungsmodelle (Rezeptormodelle, RM) erfüllt werden. Der Anwendungsbereich für die Methoden nach diesem Dokument beschreibt die Leistungsbeurteilung der Quellzuordnung partikulären Materials (PM) mithilfe rezeptororientierter Quellmodellierung (RM) im Kontext der Europäischen Richtlinien 2004/107/EG und AQD einschließlich des Durchführungsbeschlusses der europäischen Kommission 2011/850/EU vom 12. Dezember 2011. Die Anwendung der RM quantifiziert nicht die räumliche Quelle des PM, somit umfasst dieses Dokument keine Prüfung der räumlichen Quellzuordnung.
Dieses Dokument wendet sich an Nutzer der RM: Anwender individueller Quellzuordnungsstudien sowie Teilnehmer und Organisatoren von Quellzuordnungsvergleichsstudien. Dieses Dokument ist für die Auswertung der Ergebnisse eines spezifischen Modellierungssystems zur Quellzuordnung mit Hinsicht auf Referenzwerte (a priori bekannt oder auf Grundlage der Vergleichswerte der Teilnehmer berechnet) in den folgenden Anwendungsbereichen geeignet:
-   Erfassung der Leistungsfähigkeit und Unsicherheiten eines Modellierungssystems oder der Einrichtung eines Modellierungssystems durch Verwendung der in diesem Dokument dargelegten Indikatoren.
-   Prüfen und Vergleichen verschiedener Quellzuordnungsergebnisse in einer spezifischen Situation (Verwenden eines Evaluierungsdatensatzes) mithilfe der in dieser CEN/TS dargelegten Indikatoren.
-   QA/QC-Prüfungen bei jeder Verwendung eines Modellierungssystems durch einen Anwender.
Es sollte zur Verdeutlichung beachtet werden, dass die in diesem Dokument dargestellten Verfahren und Berechnungen nicht für die Prüfung der Leistungsfähigkeit eines spezifischen Quellzuordnungs-Modellierungsergebnisses verwendet werden können, ohne a priori Referenzinformation über die Beiträge der Quellen/Quellenkategorien zu haben.
ANMERKUNG   Die Anwendung dieses Dokuments impliziert, dass die Vergleichsprüfung von einer Institution organisiert und koordiniert ist, die über die nötige technische Ausstattung und Unabhängigkeit verfügt; die Definition einer solchen Institution geht über den Anwendungsbereich dieses Dokuments hinaus.
Die Grundsätze der RM sind in Anhang A zusammengefasst. Eine Übersicht über Messunsicherheitsquellen und Empfehlungen zu Schritten, die in Quellzuordnungsstudien zu befolgen sind, sind in Anhang B und Anhang C dargestellt. Für weitere Informationen zu Quellzuordnungsmethodiken, siehe zum Beispiel [1; 2; 3].
Es gibt neben RM auch andere Methodiken, die häufig zur Quellzuordnung verwendet werden, z. B. quellorientierte Modelle. Diese anderen Methodiken decken Aspekte der Quellzuordnung ab, die von der AQD gefordert und von der RM nicht behandelt werden (z. B. Zuordnung von Schadstoffen zu geographischen Emissionsgebieten). Die Einschätzung der Güte dieser Methodiken liegt außerhalb des Anwendungsbereichs dieses Dokuments.

Air ambiant - Méthode d’évaluation de la performance d’applications d’un système de modélisation de la contribution des sources de particules en suspension de type « récepteur-orienté »

La Directive européenne concernant la qualité de l’air ambiant et un air pur pour l’Europe (2008/50/CE ; AQD) identifie différents usages de la modélisation : évaluation, planification, prévision et contribution des sources (SA). Le présent document concerne la modélisation de la contribution des sources et spécifie les essais de performance permettant de s’assurer que les critères donnés pour les modèles de contribution des sources de type « récepteur-orienté » (RM) sont remplis. Le domaine d’application des essais décrits dans le présent document est l’évaluation de la performance de SA de la matière particulaire à l’aide de RM dans le contexte de la Directive européenne 2004/107/CE et de l’AQD, y compris la Décision d’exécution de la Commission 2011/850/UE du 12 décembre 2011. L’application de RM ne permet pas de quantifier l’origine spatiale de la matière particulaire ; par conséquent, le présent document n’évalue pas la SA spatiale.
Le présent document s’adresse aux utilisateurs de RM et opérateurs d’études de contribution des sources ainsi qu’aux participants et organisateurs de comparaisons interlaboratoires de la contribution des sources. Le présent document est applicable à l’évaluation des résultats d’un système de modélisation SA spécifique par rapport à des valeurs de référence (a priori connues ou calculées d’après les valeurs des participants à la comparaison interlaboratoires) dans les secteurs d’application suivants :
-   Évaluation de la performance et des incertitudes d’un système de modélisation ou d’une configuration du système de modélisation à l’aide des indicateurs énoncés dans le présent document.
-   Essais et comparaison des différents résultats de contribution des sources dans une situation spécifique (application d’un jeu de données d’évaluation) à l’aide des indicateurs énoncés dans le présent document.
-   Essais AQ/CQ à chaque fois qu’un opérateur utilise un système de modélisation.
Il convient de noter que, pour plus de clarté, les modes opératoires et les calculs présentés dans le présent document ne peuvent pas être utilisés pour vérifier la performance d’un résultat de modélisation de SA spécifique sans disposer a priori d’informations de référence sur les contributions des sources/catégories de sources.
NOTE   L’application du présent document implique que la comparaison interlaboratoires soit organisée et coordonnée par un organisme indépendant disposant des compétences techniques nécessaires ; sa définition ne fait pas partie du domaine d’application du présent document.
Les principes des RM sont résumés à l’Annexe A. Une vue d’ensemble des sources d’incertitude et des recommandations relatives aux étapes à respecter lors des études de SA sont fournies aux Annexes B et C. Pour plus d’informations sur les méthodes de SA, voir par exemple [1 ; 2 ; 3].
D’autres méthodes que les RM sont couramment utilisées pour effectuer la SA, par exemple les modèles de type « source-orienté ». Ces autres méthodes couvrent certains aspects de SA qui sont requis dans l’AQD et ne sont pas abordés par les RM (par exemple, attribution de polluants à des zones géographiques d’émission). L’évaluation de la performance de ces méthodes ne fait pas partie du domaine d’application du présent document.

Zunanji zrak - Metodologija za oceno lastnosti aplikacij sistemov za modeliranje porazdelitve virov

General Information

• Status: Published
• Publication Date: 22-Dec-2020
• ICS: 13.040.01 - Air quality in general
• Technical Committee: CEN/TC 264 - Air quality
• Drafting Committee: CEN/TC 264/WG 44 - Source apportionment
• Directive: 2008/50/EC - Directive 2008/50/EC of the European Parliament and of the Council of 21 May 2008 on ambient air quality and cleaner air for Europe
• Ref Project: SIST-TS CEN/TS 17458:2021 - Ambient air - Methodology for the assessment of the performance of source apportionment modelling system applications

Standards Content (Sample)

SLOVENSKI STANDARD
01-februar-2021
Zunanji zrak - Metodologija za oceno lastnosti aplikacij sistemov za modeliranje
porazdelitve virov
Ambient air - Methodology for the assessment of the performance of source
apportionment modelling system applications
Außenluft - Methodik zur Erfassung der Leistungsfähigkeit von Systemanwendungen zur
Quellenzuordnung
Air ambiant - Méthodologie d'évaluation des performances dapplications de modélisation
de l'attribution des sources et des récepteurs
Ta slovenski standard je istoveten z: CEN/TS 17458:2020
ICS:
13.040.20 Kakovost okoljskega zraka Ambient atmospheres
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

CEN/TS 17458
TECHNICAL SPECIFICATION
SPÉCIFICATION TECHNIQUE
December 2020
TECHNISCHE SPEZIFIKATION
ICS 13.040.01
English Version
Ambient air - Methodology to assess the performance of
receptor oriented source apportionment modelling
applications for particulate matter
Air ambiant - Méthode d'évaluation de la performance Außenluft - Methodik zur Erfassung der
d'applications d'un système de modélisation de la Leistungsfähigkeit von Systemanwendungen zur
contribution des sources de particules en suspension Quellenzuordnung
de type " récepteur-orienté "
This Technical Specification (CEN/TS) was approved by CEN on 24 February 2020 for provisional application.

The period of validity of this CEN/TS is limited initially to three years. After two years the members of CEN will be requested to
submit their comments, particularly on the question whether the CEN/TS can be converted into a European Standard.

CEN members are required to announce the existence of this CEN/TS in the same way as for an EN and to make the CEN/TS
available promptly at national level in an appropriate form. It is permissible to keep conflicting national standards in force (in
parallel to the CEN/TS) until the final decision about the possible conversion of the CEN/TS into an EN is reached.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION

EUROPÄISCHES KOMITEE FÜR NORMUNG

CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2020 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TS 17458:2020 E
worldwide for CEN national Members.

Contents Page
European foreword . 3
1 Scope . 4
2 Normative references . 4
3 Terms and definitions . 5
4 Symbols and abbreviations . 7
5 Fundamentals of the evaluation methodology . 8
6 Arrays of tests . 9
7 The performance assessment method . 9
7.1 Evaluation data sets . 9
7.2 Determination of the SCE reference values and their standard uncertainties . 9
7.2.1 Consensus value from participants . 9
7.2.2 Formulation of a consensus value from a synthetic data set . 10
8 Performance indicators and other indicators . 10
8.1 Complementary indicators . 10
8.1.1 General . 10
8.1.2 Apportioned pollutant mass . 10
8.1.3 Number of identified sources . 11
8.2 Similarity indicators . 11
8.2.1 General . 11
8.2.2 Pearson product-moment correlation coefficient . 12
8.2.3 Standardized Identity Distance (SID, Annex I) . 12
8.2.4 Weighted Distance (WD) . 12
8.3 Performance indicators . 13
8.3.1 General . 13
8.3.2 z-scores for the average SCEs . 13
8.3.3 RMSE for the SCEs time series . 14
u
8.3.4 Zeta-score for the uncertainty of the SCEs . 14
Annex A (informative) Principle of receptor oriented models . 15
Annex B (informative) Recommended steps for SA studies with receptor oriented models . 16
Annex C (informative) Sources of uncertainty in receptor oriented models . 17
Annex D (informative) Array of tests for individual SA runs (Responsible: Practitioner) . 18
Annex E (normative) Array of tests for intercomparisons (Responsible: Coordinator) . 19
Annex F (normative) Calculation of the SCE reference values X and X in real-world data
kt k
sets . 20
Annex G (informative) Robust average and standard deviation. 21
Annex H (informative) Acceptability range for complementary tests . 22
Annex I (informative) Geometric meaning of the Standardized Identity Distance . 23
Annex J (normative) Acceptability range for the similarity and performance tests . 25
Annex K (informative) Intercomparison organization and test reports . 26
Bibliography . 28
European foreword
This document (CEN/TS 17458:2020) has been prepared by Technical Committee CEN/TC 264 “Air
quality”, the secretariat of which is held by DIN.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
According to the CEN/CENELEC Internal Regulations, the national standards organisations of the
following countries are bound to announce this Technical Specification: Austria, Belgium, Bulgaria,
Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland,
Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of
North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United
Kingdom.
1 Scope
The European Directive on ambient air quality and cleaner air for Europe (2008/50/EC; AQD) identifies
different uses for modelling: Assessment, planning, forecast and source apportionment (SA). This
document addresses source apportionment modelling and specifies performance tests to check whether
given criteria for receptor oriented source apportionment models (RM) are met. The scope of the tests
set out in this document is the performance assessment of SA of particulate matter using RM in the
context of the European Directives 2004/107/EC and AQD, including the Commission Implementing
Decision 2011/850/EU of 12 December 2011. The application of RM does not quantify the spatial origin
of particulate matter; hence, this document does not test spatial SA.
This document addresses RM users: practitioners of individual source apportionment studies as well as
participants and organizers of source apportionment intercomparison studies. This document is suitable
for the evaluation of results of a specific SA modelling system with respect to reference values (a priori
known or calculated on the basis of intercomparison participants' values) in the following application
areas:
— Assessment of performance and uncertainties of a modelling system or modelling system set up using
the indicators laid down in this document.
— Testing and comparing different source apportionment outputs in a specific situation (applying an
evaluation data set) using the indicators laid down in this document.
— QA/QC tests every time practitioners run a modelling system.
It should be noted for clarity that the procedures and calculations presented in this document cannot be
used to check the performance of a specific SA modelling result without having any a priori reference
information about the contributions of sources/source categories.
NOTE The application of this document implies that the intercomparison is organized and coordinated by an
institution with the necessary technical capabilities and independence; the definition of which is beyond the scope
of this document.
The principles of RM are summarized in Annex A. An overview of uncertainty sources and
recommendations about steps to follow in SA studies are provided in Annex B and Annex C. For further
information about SA methodologies, refer to e.g. [1; 2; 3].
There are methodologies different from RM which are widely used to accomplish SA, e.g. source oriented
models. These other methodologies cover aspects of SA which are required in the AQD and are not
addressed by RM (e.g. allocation of pollutants to geographic emission areas). Performance assessment of
such methodologies is out of the scope of this document.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 13528:2015, Statistical methods for use in proficiency testing by interlaboratory comparison
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
3.1
candidate modelling system
modelling system or modelling system set up that is being tested in the intercomparison or being applied
in isolation
3.2
candidate source
every source or source category present in the result of a SA candidate modelling system which is tested
with the methodology described in this document
3.3
coordinator
organisation with responsibility for coordinating all activities involved in the operation of an
intercomparison exercise
3.4
contribution-to-species
mass of each single chemical element or compound forming part of a (composite) pollutant apportioned
to a source or source category, expressed as a percentage
Note 1 to entry: Depending on the type of SA modelling applications, this concept could be referred to as:
“contribution by species” in CMB 8.2, “explained variation” in PMF 2, “percentage of species total matrix” in EPA
PMF v3, and “factor fingerprint” in EPA-PMF v5.
3.5
input uncertainty
statistical dispersion of the values that can be reasonably attributed to every single species ambient
concentration in the input matrix of receptor oriented models
3.6
derived source profile
relative chemical composition of the source or source category, resulting from a SA model run, expressed
as parts per unit mass of every chemical species with respect to the total source or source category mass
3.7
factor
underlying latent variable identified by a multivariate
...