FprCEN/CLC ISO/IEC/TS 12791

Information technology - Artificial intelligence - Treatment of unwanted bias in classification and regression machine learning tasks (ISO/IEC DTS 12791:2024)

This document describes how to address unwanted bias in AI systems that use machine learning to conduct classification and regression tasks. This document provides mitigation techniques that can be applied throughout the AI system life cycle in order to treat unwanted bias. This document is applicable to all types and sizes of organization.

Technologies de l'information - Intelligence artificielle - Traitement des biais indésirables dans les tâches d'apprentissage automatique de classification et de régression (ISO/IEC DTS 12791:2024)

Informacijska tehnologija - Umetna inteligenca - Obravnava neželene pristranskosti pri nalogah strojnega učenja klasifikacije in regresije (ISO/IEC DTS 12791:2023)

General Information

• Status: Not Published
• Publication Date: 12-Nov-2024
• ICS: 35.020 - Information technology (IT) in general
• Technical Committee: CEN/CLC/JTC 21 - Artificial Intelligence
• Drafting Committee: CEN/CLC/JTC 21/WG 3 - Engineering aspects
• Current Stage: 6055 - CEN Ratification completed (DOR) - Publishing
• Start Date: 12-Oct-2024
• Completion Date: 12-Oct-2024
• Ref Project: kSIST-TS FprCEN/CLC ISO/IEC/TS 12791:2024 - Information technology - Artificial intelligence - Treatment of unwanted bias in classification and regression machine learning tasks (ISO/IEC DTS 12791:2023)

Standards Content (Sample)

SLOVENSKI STANDARD
kSIST-TS FprCEN/CLC ISO/IEC/TS
12791:2024
01-januar-2024
Informacijska tehnologija - Umetna inteligenca - Obravnava neželene
pristranskosti pri nalogah strojnega učenja klasifikacije in regresije (ISO/IEC DTS
12791:2023)
Information technology - Artificial intelligence - Treatment of unwanted bias in
classification and regression machine learning tasks (ISO/IEC DTS 12791:2023)
Technologies de l'information - Intelligence artificielle - Traitement des biais indésirables
dans les tâches d'apprentissage automatique de classification et de régression (ISO/IEC
DTS 12791:2023)
Ta slovenski standard je istoveten z: FprCEN/CLC ISO/IEC/TS 12791
ICS:
35.020 Informacijska tehnika in Information technology (IT) in
tehnologija na splošno general
kSIST-TS FprCEN/CLC ISO/IEC/TS en,fr,de
12791:2024
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

FINAL
TECHNICAL ISO/IEC DTS
DRAFT
SPECIFICATION 12791
ISO/IEC JTC 1/SC 42
Information technology — Artificial
Secretariat: ANSI
intelligence — Treatment of unwanted
Voting begins on:
2023-11-09 bias in classification and regression
machine learning tasks
Voting terminates on:
2024-02-01
Technologies de l'information — Intelligence artificielle —
Traitement des biais indésirables dans les tâches d'apprentissage
automatique de classification et de régression
ISO/CEN PARALLEL PROCESSING
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO-
ISO/IEC DTS 12791:2023(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN-
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. © ISO/IEC 2023

ISO/IEC DTS 12791:2023(E)
FINAL
TECHNICAL ISO/IEC DTS
DRAFT
SPECIFICATION 12791
ISO/IEC JTC 1/SC 42
Information technology — Artificial
Secretariat: ANSI
intelligence — Treatment of unwanted
Voting begins on:
bias in classification and regression
machine learning tasks
Voting terminates on:
Technologies de l'information — Intelligence artificielle —
Traitement des biais indésirables dans les tâches d'apprentissage
automatique de classification et de régression
© ISO/IEC 2023
ISO/CEN PARALLEL PROCESSING
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
RECIPIENTS OF THIS DRAFT ARE INVITED TO
ISO copyright office
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
CP 401 • Ch. de Blandonnet 8
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
CH-1214 Vernier, Geneva
DOCUMENTATION.
Phone: [phone]
IN ADDITION TO THEIR EVALUATION AS
Reference number
Email: [e-mail]
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO­
ISO/IEC DTS 12791:2023(E)
Website: www.iso.org
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
Published in Switzerland
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN­
DARDS TO WHICH REFERENCE MAY BE MADE IN
ii
© ISO/IEC 2023 – All rights reserved
NATIONAL REGULATIONS. © ISO/IEC 2023

ISO/IEC DTS 12791:2023(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
3.1 General . 1
3.2 Artificial intelligence . 3
3.3 Bias . 4
3.4 Testing . 5
4 Abbreviated terms . 7
5 Treating bias in the AI system life cycle . 7
5.1 Inception . 7
5.1.1 Stakeholder needs and requirements definition . 7
5.1.2 Procurement . 7
5.1.3 Stakeholder identification . 8
5.1.4 Selection and documentation of data sources . 9
5.1.5 Integration with risk management . 9
5.1.6 External change . 10
5.1.7 Acceptance criteria . 10
5.2 Design and development . 11
5.2.1 Data representation and labelling . 11
5.2.2 Labelling sufficiency . 11
5.2.3 Labelling quality . 11
5.2.4 Training data . 11
5.2.5 Methods for preventing identified risks . 11
5.3 V erification and validation .12
5.3.1 General .12
5.3.2 Static testing of training data .12
5.3.3 Dynamic testing . 13
5.4 Re­evaluation, continuous validation, operations and monitoring . 14
5.5 Disposal . . . 14
6 Techniques to address unwanted bias .14
6.1 Algorithmic and training techniques . 15
6.1.1 General .15
6.1.2 Pre­trained models .15
6.2 Data techniques . 16
7 Handling bias in a distributed AI system life cycle .16
Annex A (informative) Life cycle processes map .18
Annex B (informative) Potential impacts of unwanted bias on different types of specific
user .19
Bibliography .21
iii
© ISO/IEC 2023 – All rights reserved

ISO/IEC DTS 12791:2023(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non­governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance
are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria
needed for the different types of documents should be noted. This document was drafted in
accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or
www.iec.ch/members_experts/refdocs).
ISO and IEC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of
any claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC
had not received notice of (a) patent(s) which may be required to implement this document. However,
implementers are cautioned that this may not represent the latest information, which may be obtained
from the patent database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall
not be held responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see
www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding­standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 42, Artificial intelligence.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national­committees.
iv
© ISO/IEC 2023 – All rights reserved

ISO/IEC DTS 12791:2023(E)
Introduction
This document describes appropriate steps that can be taken to treat unwanted bias during the
development or use of AI systems.
[1]
This document is based on ISO/IEC TR 24027 and pr
...