kSIST-TP FprCEN/CLC/TR 17894:2024
(Main)Artificial Intelligence - Artificial Intelligence Conformity Assessment
Artificial Intelligence - Artificial Intelligence Conformity Assessment
This document sets out a review of the current methods and practices (including tools, assets, and conditions of acceptability) for
conformity assessment in respect to, among others, products, services, processes, management systems, organizations, or persons,
as relevant for the development and use of AI systems. It includes an industry horizontal (vertical agnostic) perspective as well as an
industry vertical perspective.
This document focuses only on the process of assessment and gap analysis of conformity. It defines the objects of conformity
related to AI systems and all other related aspects of the process of conformity assessment. The document also reviews to what
extent AI poses specific challenges with respect to assessment of, for example, software engineering, data quality and engineering
processes.
This document takes into account requirements and orientations from policy frameworks such as the EU AI strategy and those from
CEN and CENELEC member countries.
This document is intended for technologists, standards bodies, regulators and interested parties.
Künstliche Intelligenz - Konformitätsbewertung von Künstlicher Intelligenz
Intelligence Artificielle - Évaluation de la conformité liée à l'Intelligence Artificielle
Umetna inteligenca - Ugotavljanje skladnosti z umetno inteligenco
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
01-oktober-2024
Umetna inteligenca - Ugotavljanje skladnosti z umetno inteligenco
Artificial Intelligence - Artificial Intelligence Conformity Assessment
Künstliche Intelligenz - Konformitätsbewertung von Künstlicher Intelligenz
Intelligence Artificielle - Évaluation de la conformité liée à l'Intelligence Artificielle
Ta slovenski standard je istoveten z: FprCEN/CLC/TR 17894
ICS:
03.120.20 Certificiranje proizvodov in Product and company
podjetij. Ugotavljanje certification. Conformity
skladnosti assessment
35.240.01 Uporabniške rešitve Application of information
informacijske tehnike in technology in general
tehnologije na splošno
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
TECHNICAL REPORT FINAL DRAFT
RAPPORT TECHNIQUE
TECHNISCHER REPORT
August 2024
ICS 03.120.20; 35.240.01
English version
Artificial Intelligence - Artificial Intelligence Conformity
Assessment
Intelligence Artificielle - Évaluation de la conformité Künstliche Intelligenz - Konformitätsbewertung von
liée à l'Intelligence Artificielle Künstlicher Intelligenz
This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee
CEN/CLC/JTC 21.
CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,
Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,
Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,
Slovakia, Slovenia, Spain, Sweden, Switzerland, Türkiye and United Kingdom.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are
aware and to provide supporting documentation.
Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without
notice and shall not be referred to as a Technical Report.
CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels
© 2024 CEN/CENELEC All rights of exploitation in any form and by any means
Ref. No. FprCEN/CLC/TR 17894:2024 E
reserved worldwide for CEN national Members and for
CENELEC Members.
Contents Page
European foreword . 4
1 Scope . 5
2 Normative references . 5
3 Terms and definitions . 5
4 Framework of conformity assessment and objects of conformity assessment . 6
4.1 General . 6
4.2 International accreditation and conformity assessment framework . 6
4.2.1 General . 6
4.2.2 Level 5 . 7
4.2.3 Level 4 . 8
4.2.4 Level 3 . 8
4.2.5 Level 2 . 9
4.2.6 Level 1 . 9
4.3 Conformity assessment modules . 9
4.3.1 Conformity assessment modules of Decision No 768/2008/EC . 9
4.3.2 Conformity assessment modules of the EU AI Act .11
4.3.3 Conformity assessment modules following the EU AI Act Annex II Section A .14
4.3.4 Conformity assessment modules under sectorial legislation .14
4.4 Considerations on the interplay of conformity assessment under EU AI Act and
sectorial legislation .21
4.4.1 General .21
4.4.2 Interplay between notified body requirements under EU AI Act and its Annex II
Section A legislation .21
4.4.3 Possible interplay between conformity assessment in the Machinery Regulation and
the EU AI Act .22
4.4.4 Possible interplay between conformity assessment in the medical devices sectorial
legislation and the EU AI Act .23
4.4.5 Conformity assessment in the automotive sectorial legislation .23
4.4.6 Conformity assessment of representative AI system (aka ‘sampling’) .24
5 Mapping of horizontal and vertical standard items to the level system and
assignment to conformity assessment activities .26
5.1 Mapping of AI horizontal standard items to conformity assessment activities .26
5.1.1 General .26
5.1.2 Management system certification according to EN ISO/IEC 17021-1 .27
5.1.3 Inspection according to EN ISO/IEC 17020 .27
5.1.4 Testing according to EN ISO/IEC 17025 .28
5.1.5 Verification and Validation according to EN ISO/IEC 17029 .28
5.1.6 Product, process or service certification according to EN ISO/IEC 17065 .29
6 Supporting compliance to EU AI Act .29
6.1 Analysis of conformity assessment elements in EU AI Act .29
6.1.1 Conformity assessment requirements for high-risk AI systems according to EU
C(2023)3215 – Standardization request M/593 .29
6.1.2 Interdependencies of EU AI Act provisions .30
6.1.3 Article 17 “quality management system” in the EU AI Act .32
6.2 EN ISO/IEC 17065 certification approach related to EU AI Act .33
6.2.1 General .33
6.2.2 Potential certification process according to EN ISO/IEC 17065 .34
6.2.3 Accreditation of certification bodies according to EN ISO/IEC 17065 within the field
of AI . 36
6.3 Role of testing for Conformity Assessments . 37
6.3.1 General . 37
6.3.2 Testing of general purpose AI models and general purpose AI models with systemic
risk . 38
6.4 Measurement, measures and metrics . 38
7 Existing horizontal certifications possibly relevant for the AI area . 41
7.1 General . 41
7.2 Data related certifications: the example of GDPR-CARPA national level certification
................................................................................................................................................................... 41
7.3 Cybersecurity related certification . 42
8 Observations and Identified Gaps . 42
8.1 General . 42
8.2 Challenges of terms and definitions operationalisation for AI conformity assessment
................................................................................................................................................................... 44
8.2.1 General . 44
8.2.2 Identified differences of terms definition . 44
Annex A (informative) Tools to support operationalisation of AI Conformity assesment . 45
Bibliography . 46
European foreword
This document (FprCEN/CLC/TR 17894:2024) has been prepared by Technical Committee
CEN/CENELEC JTC 21 “Artificial Intelligence”, the secretariat of which is held by Danish Standards (DS).
This document is currently submitted to Vote on TR.
1 Scope
This document sets out a review of the current methods and practices (including tools, assets, and
conditions of acceptability) for conformity assessment as relevant for the development and use of AI
systems. Among others, it addresses the conformity assessment for products, services, processes,
management systems and organizations. It includes an industry horizontal (vertical agnostic)
perspective and an industry vertical perspective.
This document focuses only on the process and gap analysis of conformity assessments. It defines the
objects of conformity related to AI systems and all other aspects of the conformity assessment process.
The document also reviews to what extent AI poses specific challenges with respect to assessment of, for
example, software engineering, data quality and engineering processes.
This document takes into account requirements and orientations from policy frameworks such as the EU
AI strategy and those from CEN and CENELEC member countries.
This document is intended for technologists, standards bodies, regulators and interest groups.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
conformity assessment
demonstration that specified requirements (3.2) relating to a product, process, system, person or body are
fulfilled
Note 1 to entry: The process of conformity assessment as described in the functional approach in Annex A can have
a negative outcome, i.e. demonstrating that the specified requirements are not fulfilled.
Note 2 to entry: Conformity assessment includes activities defined elsewhere in this document, such as but not
limited to testing (6.2), inspection (6.3), validation (6.5), verification (6.6), certification (7.6), and accreditation (7.7).
Note 3 to entry: Conformity assessment is explained in Annex A as a series of functions. Activities contributing to
any of these functions can be described as conformity assessment activities.
[SOURCE: EN ISO/IEC 17000:2020]
3.2
specified requirements
need or expectation that is stated
Note 1 to entry: Specified requirements can be stated in normative documents such as regulations, standards and
technical specifications.
Note 2 to entry: Specified requirements can be detailed or general.
[SOURCE: EN ISO/IEC 17000:2020
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.