ISO/IEC JTC 1/SC 42 - Artificial intelligence

This document specifies a taxonomy of information elements to assist AI stakeholders with identifying and addressing the needs for transparency of AI systems. The document describes the semantics of the information elements and their relevance to the various objectives of different stakeholders. This document is applicable to any kind of organization and application involving an AI system.

This document provides requirements and guidance on the application of the ISO/IEC/IEEE 29119 series to the testing of AI systems. This document follows a risk-based approach and uses risks associated with AI systems, and their development and maintenance, to identify suitable test practices, approaches and techniques applicable to AI systems and their components. When the test practices, approaches and techniques are already specified in the ISO/IEC/IEEE 29119 series, this document provides additional detail and describes their application in the context of AI systems.

This document describes the delivery of functional, economic, environmental, social, societal, cultural, intellectual and personal benefits by AI systems as perceived by their stakeholders. The document includes illustrative use cases of AI systems.

This document describes approaches and methods that can be used to achieve explainability objectives of stakeholders with regard to machine learning (ML) models and artificial intelligence (AI) systems’ behaviours, outputs and results. Stakeholders include but are not limited to, academia, industry, policy makers and end users. It provides guidance concerning the applicability of the described approaches and methods to the identified objectives throughout the AI system’s life cycle, as defined in ISO/IEC 22989.

This document specifies additional requirements to ISO/IEC 17021-1. The requirements contained in this document, when implemented, support the demonstration of competence, consistency and reliability by the bodies performing auditing and certification of an artificial intelligence management system (AIMS) according to ISO/IEC 42001 for organizations that provide, develop or use AI systems. Certification of AIMS is a third-party conformity assessment activity (as described in ISO/IEC 17000:2020, 4.5), and bodies performing this activity are third-party conformity assessment bodies. This document also provides the necessary information and confidence to customers about the way certification has been granted. NOTE This document can be used as a criteria document for accreditation or peer assessment.

This document provides an overview of the environmental sustainability aspects (e.g. workload, resource and asset utilization, carbon impact, pollution, waste, transportation, location) of AI systems during their life cycle, and related potential metrics. NOTE 1 This document does not identify opportunities on how AI, AI applications and AI systems can improve environmental, social or economic sustainability outcomes. NOTE 2 This document can help other projects related to AI system environmental sustainability.

This document provides guidance for organizations performing artificial intelligence (AI) system impact assessments for individuals and societies that can be affected by an AI system and its foreseeable applications. It includes considerations for how and when to perform such assessments and at what stages of the AI system life cycle, as well as guidance for AI system impact assessment documentation. Additionally, this guidance includes how this AI system impact assessment process can be integrated into an organization’s AI risk management and AI management system. This document is intended for use by organizations developing, providing or using AI systems. This document is applicable to any organization, regardless of size, type and nature.

This document provides a data quality governance framework for analytics and machine learning (ML) to enable governing bodies of organizations to direct and oversee the implementation and operation of data quality measures, management, and related processes with adequate controls throughout the data life cycle (DLC) model according to ISO/IEC 5259-1. This document can be applied to any analytics and ML. This document does not define specific management requirements or process requirements according to ISO/IEC 5259-3 and ISO/IEC 5259-4 respectively.

This document specifies a data quality model, data quality measures and guidance on reporting data quality in the context of analytics and machine learning (ML). This document is applicable to all types of organizations who want to achieve their data quality objectives.

This document describes how to address unwanted bias in AI systems that use machine learning to conduct classification and regression tasks. This document provides mitigation techniques that can be applied throughout the AI system life cycle in order to treat unwanted bias. This document is applicable to all types and sizes of organization.

This document establishes general common organizational approaches, regardless of the type, size or nature of the applying organization, to ensure data quality for training and evaluation in analytics and machine learning (ML). It includes guidance on the data quality process for: — supervised ML with regard to the labelling of data used for training ML systems, including common organizational approaches for training data labelling; — unsupervised ML; — semi-supervised ML; — reinforcement learning; — analytics. This document is applicable to training and evaluation data that come from different sources, including data acquisition and data composition, data preparation, data labelling, evaluation and data use. This document does not define specific services, platforms or tools.

This document specifies requirements and provides guidance for establishing, implementing, maintaining and continually improving the quality of data used in the areas of analytics and machine learning. This document does not define a detailed process, methods or metrics. Rather it defines the requirements and guidance for a quality management process along with a reference process and methods that can be tailored to meet the requirements in this document. The requirements and recommendations set out in this document are generic and are intended to be applicable to all organizations, regardless of type, size or nature.

This document provides the means for understanding and associating the individual documents of the ISO/IEC 5259 series and is the foundation for conceptual understanding of data quality for analytics and machine learning. It also discusses associated technologies and examples (e.g. use cases and usage scenarios).

This document surveys machine learning (ML) computing devices, including the following: — ML computing device terminology and characteristics; — existing approaches to the setting and use of characteristics for optimizing ML computing device performance. The information provided in this document is relevant for organizations of all types and sizes.

This document specifies a basic framework with principles, characteristics and approaches for the realization and enhancement for automated artificial intelligence (AI) systems’ controllability. The following areas are covered: — state observability and state transition; — control transfer process and cost; — reaction to uncertainty during control transfer; — verification and validation approaches. This document is applicable to all types of organizations (e.g. commercial enterprises, government agencies, not-for-profit organizations) developing and using AI systems during their whole life cycle.

This document provides a collection of representative use cases of AI applications in a variety of domains.

This document defines a reference architecture of knowledge engineering (KE) in artificial intelligence (AI). The reference architecture describes KE roles, activities, constructional layers, components and their relationships amongst themselves and other systems from systemic user and functional views. This document also provides a common KE vocabulary by defining KE terms.

This document provides guidance for evaluation of artificial intelligence (AI) systems using an AI system quality model. The document is applicable to all types of organizations engaged in the development and use of AI.

This document provides guidance for identifying the context, opportunities and processes for developing and applying AI applications. The guidance provides a macro-level view of the AI application context, the stakeholders and their roles, relationship to the life cycle of the system, and common AI application characteristics and considerations.

This document describes the properties, related risk factors, available methods and processes relating to: — use of AI inside a safety related function to realize the functionality; — use of non-AI safety related functions to ensure safety for an AI controlled equipment; — use of AI systems to design and develop safety related functions.

This document defines a set of processes and associated concepts for describing the life cycle of AI systems based on machine learning and heuristic systems. It is based on ISO/IEC/IEEE 15288 and ISO/IEC/IEEE 12207 with modifications and additions of AI-specific processes from ISO/IEC 22989 and ISO/IEC 23053. This document provides processes that support the definition, control, management, execution and improvement of the AI system in its life cycle stages. These processes can also be used within an organization or a project when developing or acquiring AI systems. When an element of an AI system is traditional software or a traditional system, the software life cycle processes in ISO/IEC/IEEE 12207 and the system life cycle processes in ISO/IEC/IEEE 15288 can be used to implement that element.

This document specifies the requirements and provides guidance for establishing, implementing, maintaining and continually improving an AI (artificial intelligence) management system within the context of an organization. This document is intended for use by an organization providing or using products or services that utilize AI systems. This document is intended to help the organization develop, provide or use AI systems responsibly in pursuing its objectives and meet applicable requirements, obligations related to interested parties and expectations from them. This document is applicable to any organization, regardless of size, type and nature, that provides or uses products or services that utilize AI systems.

This document provides methodology for the use of formal methods to assess robustness properties of neural networks. The document focuses on how to select, apply and manage formal methods to prove robustness properties.

This document defines the stages and identifies associated actions for data processing throughout the artificial intelligence (AI) system life cycle, including acquisition, creation, development, deployment, maintenance and decommissioning. This document does not define specific services, platforms or tools. This document is applicable to all organizations, regardless of type, size or nature, that use data in the development and use of AI systems.

This document outlines a quality model for AI systems and is an application-specific extension to the standards on SQuaRE. The characteristics and sub-characteristics detailed in the model provide consistent terminology for specifying, measuring and evaluating AI system quality. The characteristics and sub-characteristics detailed in the model also provide a set of quality characteristics against which stated quality requirements can be compared for completeness.

This document provides guidance on how organizations that develop, produce, deploy or use products, systems and services that utilize artificial intelligence (AI) can manage risk specifically related to AI. The guidance also aims to assist organizations to integrate risk management into their AI-related activities and functions. It moreover describes processes for the effective implementation and integration of AI risk management. The application of this guidance can be customized to any organization and its context.

This document provides a framework for developing processes to effectively leverage big data analytics across the organization irrespective of the industries or sectors. This document specifies process management for big data analytics with its various process categories taken into account along with their interconnectivities. These process categories are organization stakeholder processes, competency development processes, data management processes, analytics development processes and technology integration processes. This document describes processes to acquire, describe, store and process data at an organization level which provides big data analytics services.

This document specifies methodologies for measuring classification performance of machine learning models, systems and algorithms.

This document provides a high-level overview of AI ethical and societal concerns. In addition, this document: — provides information in relation to principles, processes and methods in this area; — is intended for technologists, regulators, interest groups, and society at large; — is not intended to advocate for any specific set of values (value systems). This document includes an overview of International Standards that address issues arising from AI ethical and societal concerns.

This document establishes terminology for AI and describes concepts in the field of AI. This document can be used in the development of other standards and in support of communications among diverse, interested parties or stakeholders. This document is applicable to all types of organizations (e.g. commercial enterprises, government agencies, not-for-profit organizations).

This document establishes an Artificial Intelligence (AI) and Machine Learning (ML) framework for describing a generic AI system using ML technology. The framework describes the system components and their functions in the AI ecosystem. This document is applicable to all types and sizes of organizations, including public and private companies, government entities, and not-for-profit organizations, that are implementing or using AI systems.

This document provides guidance for members of the governing body of an organization to enable and govern the use of Artificial Intelligence (AI), in order to ensure its effective, efficient and acceptable use within the organization. This document also provides guidance to a wider community, including: — executive managers; — external businesses or technical specialists, such as legal or accounting specialists, retail or industrial associations, or professional bodies; — public authorities and policymakers; — internal and external service providers (including consultants); — assessors and auditors. This document is applicable to the governance of current and future uses of AI as well as the implications of such use for the organization itself. This document is applicable to any organization, including public and private companies, government entities and not-for-profit organizations. This document is applicable to an organization of any size irrespective of their dependence on data or information technologies.

This document provides an overview of the state of the art of computational approaches for AI systems, by describing: a) main computational characteristics of AI systems; b) main algorithms and approaches used in AI systems, referencing use cases contained in ISO/IEC TR 24030.

This document addresses bias in relation to AI systems, especially with regards to AI-aided decision-making. Measurement techniques and methods for assessing bias are described, with the aim to address and treat bias-related vulnerabilities. All AI system lifecycle phases are in scope, including but not limited to data collection, training, continual learning, design, testing, evaluation and use.

This document provides background about existing methods to assess the robustness of neural networks.

This document describes the framework of the big data reference architecture and the process for how a user of the document can apply it to their particular problem domain.

This document surveys topics related to trustworthiness in AI systems, including the following: — approaches to establish trust in AI systems through transparency, explainability, controllability, etc.; — engineering pitfalls and typical associated threats and risks to AI systems, along with possible mitigation techniques and methods; and — approaches to assess and achieve availability, resiliency, reliability, accuracy, safety, security and privacy of AI systems. The specification of levels of trustworthiness for AI systems is out of the scope of this document.

This document specifies the big data reference architecture (BDRA). The reference architecture includes concepts and architectural views. The reference architecture specified in this document defines two architectural viewpoints: — a user view defining roles/sub-roles, their relationships, and types of activities within a big data ecosystem; — a functional view defining the architectural layers and the classes of functional components within those layers that implement the activities of the roles/sub-roles within the user view. The BDRA is intended to: — provide a common language for the various stakeholders; — encourage adherence to common standards, specifications, and patterns; — provide consistency of implementation of technology to solve similar problem sets; — facilitate the understanding of the operational intricacies in big data; — illustrate and understand the various big data components, processes, and systems, in the context of an overall big data conceptual model; — provide a technical reference for government departments, agencies and other consumers to understand, discuss, categorize and compare big data solutions; and — facilitate the analysis of candidate standards for interoperability, portability, reusability, and extendibility.

This document provides a set of terms and definitions needed to promote improved communication and understanding of this area. It provides a terminological foundation for big data-related standards. This document provides a conceptual overview of the field of big data, its relationship to other technical areas and standards efforts, and the concepts ascribed to big data that are not new to big data.

ISO/IEC TR 20547-5:2018 describes big data relevant standards, both in existence and under development, along with priorities for future big data standards development based on gap analysis.

ISO/IEC TR 20547-2:2018 provides examples of big data use cases with application domains and technical considerations derived from the contributed use cases.

This document describes approaches and provides guidance on processes for the verification and validation analysis of AI systems (comprising AI system components and the interaction of non-AI components with the AI system components) including formal methods, simulation and evaluation. This document is applicable for AI systems verification and validation in the context of the AI system life cycle stages described in ISO/IEC 22989. This document is applicable to all types of organizations engaged in the development, deployment and use of AI systems.

This document provides a collection of representative use cases of AI applications in a variety of domains.