ISO/TC 184/SC 4/WG 13 - Industrial Data Quality

This document specifies requirements that support the exchange of data governance policy statements and automated conformance testing of data sets to the data specifications referenced by policy statements. The following are within the scope of this document: — requirements for the syntax and semantics of identifiers for organizations issuing data governance policy statements; — requirements for the syntax and semantics of identifiers for data governance policy statements; — data specifications referenced by data governance policy statements, where those specifications are computer processable. The following are outside the scope of this document: — general processes, roles and responsibilities for performing data governance; EXAMPLE An approach to data governance is covered by ISO/IEC 38505-1 and ISO/IEC TR 38505-2. — requirements for the syntax and semantics of data specifications referenced by a data governance policy statement; — requirements for the syntax and semantics of data governance policy statements; — methods used for the creation of data governance policy statements; — methods used for measuring conformance with the requirements referenced by data governance policy statements; — methods used for monitoring conformance with the requirements referenced by data governance policy statements.

This document describes how data rules apply to various types of data. Such rules exist to sustain the integrity and reliability of data by capturing requirements into a form that can be processed by databases and other information systems. The following are within the scope of this document: — fundamental concepts of data rules; — key characteristics of data rules for common types of data, where these types are identifier, currency value, quantity, date or time, rate, free‑text entry, code and key; — how data profiling contributes to formulating effective data rules. The following is outside the scope of this document: — specific rules for specific sets of data. This document can be used in conjunction with or independently of standards for quality management systems. EXAMPLE 1 ISO 9001 specifies requirements for quality management systems. This document can also be used in conjunction with or independently of standards for more detailed definitions of data types. EXAMPLE 2 ISO/IEC 11404 specifies the nomenclature and shared semantics for a collection of datatypes commonly occurring in programming languages and software interfaces. EXAMPLE 3 IEC 61360‑1 specifies principles for the definition of the properties and associated attributes and explains the methods for representing verbally defined concepts.

This document specifies how to apply the Test Process Improvement method to the assessment of organizational process maturity. This document refines the language in the method, which has an original purpose of more than just data quality management. This document supports improving data quality management when an organization uses the process reference model in ISO 8000‑61 as the foundation for specifying the measurement goals for the process measurement necessary when performing maturity assessment. The following are within scope of this document: — the relationship between the Test Process Improvement method[11][12] and the procedure for assessing process maturity of data quality management as specified by ISO 8000‑61; — the role of process measurement in accordance with ISO 8000‑63 when performing assessment of process maturity; — the outputs from the procedure. The following are outside the scope of this document: — the purpose, outcomes and activities of processes for data quality management; — specific content for the scope of individual quality management areas, the maturity levels and the maturity level targets; — specific content for each inspection order including the frequency of inspection points; — instructions on how to develop a process improvement strategy in response to the outputs of the procedure. This document can be used in conjunction with or independently of quality management systems standards.

This document specifies the key considerations for organizations that are establishing appropriate roles and responsibilities for data quality management. The following are within the scope of this document: — implementing roles and responsibilities for data quality management; — providing documentary evidence of this implementation; — a framework for roles and responsibilities; — a functional model of roles and responsibilities; — example deployment scenarios for the framework of roles and responsibilities; — comparison with the processes specified by ISO 8000‑61. The following are outside the scope of this document: — process reference models for data quality management (ISO 8000‑61 specifies a process reference model for data quality management); — methods for data quality evaluation and certification; — models for assessing the maturity of data quality management (ISO 8000‑62 and ISO 8000‑64 specify approaches to assessing the maturity of data quality management). This document can be used in conjunction with or independently of standards for quality management systems.

This document provides an overview of the ISO 8000 series. The following are within the scope of this document: — stating the scope of the ISO 8000 series as a whole; — establishing the principles of information and data quality; — describing the path to data quality; — describing the structure of the ISO 8000 series; — providing a summary of the content of each part in the ISO 8000 series; — establishing the relationship of the ISO 8000 series to other international standards.

This document specifies requirements for the exchange of messages that contain master data consisting of characteristic data. These requirements can be checked by computer. The messages are suitable for exchange between organizations and between systems. EXAMPLE 1 A supplier sends a message to a customer. The message contains characteristic data describing an item that the customer is considering buying. The following are within the scope of this document: — conformance of master data messages to a formal syntax; — semantic encoding of master data messages; — conformance of master data messages to data specifications; — requirements on access to the data dictionaries that enable decoding of master data messages. The following are outside the scope of this document: — master data that are not characteristic data; — data that are not in messages; — messages that do not exchange master data between organizations or systems; EXAMPLE 2 A merchant sends a message to a credit card company. The message represents a credit charge transaction and does not exchange master data between the organizations. — recording the provenance of master data; EXAMPLE 3 ISO 8000‑120 addresses the capture and exchange of data provenance information. — accuracy of master data; EXAMPLE 4 ISO 8000‑130 addresses the representation and exchange of information about the accuracy of master data that consists of characteristic data. — exchange of data that are not master data; EXAMPLE 5 ISO 8000‑140 addresses the representation and exchange of information about the completeness of master data that consists of characteristic data. — management of master data internally within an organization; EXAMPLE 6 Data within an organization's enterprise resource planning or product data management system is out of scope. EXAMPLE 7 Making backup copies of data files containing master data is out of scope. — quality of data dictionaries; — a specific formal syntax for the exchange of master data. EXAMPLE 8 The ISO 9735 series, the ISO 13584 series, the ISO 15926 series and the ISO 22745 series specify formats that enable exchange of master data. The requirements in this document are considered necessary but not sufficient to achieve data quality with respect to exchange of master data. Issues such as the accuracy and provenance of master data also need to be addressed as part of an overall data quality strategy.

This document specifies assessment indicators to support the assessment of organizational process maturity for data quality management in the context of manufacturing operations management as specified by IEC 62264‑1. The following are within scope of this document: — assessment indicators that are work products generated by data processing (as specified by ISO 8000‑61) and, thus, enable rating of process performance, a process attribute specified by ISO/IEC 33020; — the role played by each work product in the processes of manufacturing operations management; — the connection of each work product to the outcomes of the processes of manufacturing operations management. The following are outside the scope of this document: — assessment indicators for any of the other process attributes specified by ISO/IEC 33020; — methods or procedures to measure process capability.

This document specifies a procedure for data profiling to generate the foundation for performing data quality assessment. This profiling is applicable to data sets that are either originally in a structure of tables and columns or are the output from a transformation to create such a structure. NOTE 1 Data profiling is applicable to all types of database technology. The following are within the scope of this document: — performing structure analysis to determine data element concepts; — performing column analysis to identify relevant data elements, including statistics about a data set; — performing relationship analysis to identify dependencies in a data set. The following are outside the scope of this document: — methods for extracting and sampling data to be profiled from a data set; — deriving data rules; — measuring the extent of nonconformities in a data set. NOTE 2 ISO 8000‑8 specifies approaches to measuring data and information quality. This document can be used in conjunction with, or independently of, quality management systems standards.

This document specifies a questionnaire to audit the performance of the processes specified by the process reference model in ISO 8000‑61. NOTE 1 This questionnaire is applicable to all types of business process, technology, information system, data and data processing. This questionnaire can be used as part of a continuous improvement process. The following are within the scope of this document: — guiding principles for generating questions from the process outcomes specified by ISO 8000‑61; — one or more questions for each outcome of every process in ISO 8000‑61; — a measurement method based on a simple indicator and measurement scale for each question; — guidance on how to present the results generated by the questionnaire. NOTE 2 The questions and corresponding indicators in this document conform to the requirements of ISO 8000‑63. The following is outside the scope of this document: — defining how the questions relate to models of organizational process maturity. NOTE 3 Such models define an overall scale by which to understand the degree to which an organization is performing effectively and efficiently. EXAMPLE ISO 8000‑62 and ISO 8000‑64 [1]specify how to use maturity models with ISO 8000‑61. [1] Under preparation.

This document specifies a structure for process measurement stacks that organizations can instantiate to measure the characteristics of processes for data quality management. This structure consists of goal, sub goal, question, indicator and metric. The instantiated stack consists of content that is determined by a chosen model for assessing the maturity of the processes under consideration. The following are within the scope of this document: — the fundamentals of process measurement; — the inspection plan and inspection order by which to perform measurement; — the structure of the measurement stack; — the role of the chosen maturity model in instantiating the stack. The following are outside the scope of this document: — methods for inspecting implemented processes; — methods for generating measured values from the inspection of processes; — how to choose a suitable maturity model. This document can be used in conjunction with, or independently of, quality management systems standards. NOTE ISO 8000‑8 specifies concepts and methods for measuring information and data quality.

This document specifies the requirements for representing authoritative legal entity identifiers (ALEI). These requirements supplement those of ISO 8000‑115. The following are within the scope of this document: — the requirements for the prefix element, a single sub-domain element and the identifier element to represent authoritative legal entity identifiers. NOTE ALEIs are applicable to all types of legal entity, whether individuals or organizations. The following are outside the scope of this document: — the methods used to create a legal entity identifier; — the syntax of the query and of the response used in the resolution of the representation of an authoritative legal entity identifier; — the methods used for the resolution of the representation of an authoritative legal entity identifier.

This document specifies particular elements of a maturity model. These elements conform to ISO/IEC 33004. Organizations can use these elements in combination with their own assessment indicators to determine the maturity level of processes for data quality management as specified by ISO 8000‑61. The following are within the scope of this document: - some of the elements of a model for assessing organizational process maturity; - identifying those elements that exist in other standards (process capability levels, process attributes, ordinal scale for measuring process attributes and the scheme for derivation of process capability levels from process attribute rating); - specifying six maturity levels and process profiles to indicate when organizations have achieved each of the maturity levels; - providing guidance on how to assess the maturity level of an organization. Methods or procedures to improve organizational maturity are outside the scope of this document. This document can be used by the organization itself or by another party (including certification bodies) to perform assessment of the maturity. This document can be used in conjunction with, or independently of, quality management systems standards.

ISO 8000-115:2018 specifies the requirements for the quality identifiers that form part of an exchange of master data. These requirements supplement those of ISO 8000‑110. The following are within the scope of ISO 8000-115:2018: · the syntax and semantics of quality identifiers to allow the unambiguous identification of the owner of the identifier and any restrictions on the use of the identifier; · the principles of resolving quality identifiers to the data set they represent; · the characteristics that define quality identifiers. The following are outside the scope of ISO 8000-115:2018: · the methods used for the creation of identifiers; · the syntax of the query and of the response used in the resolution of identifiers; · the methods used for the resolution of identifiers.

ISO/TS 8000‑60:2017 introduces the concepts within the parts of ISO 8000 related to implementing, assessing and improving data quality management. The following are within the scope of ISO/TS 8000‑60:2017: · a statement of the overall scope of those parts of ISO 8000 related to data quality management; · an overview of ISO 8000‑61, which specifies a process reference model for data quality management; · an overview of ISO 8000‑62, which specifies a model that conforms to some of the requirements of ISO/IEC 33004 and that establishes a basis on which to determine the maturity of an organization with respect to data quality management; · an overview of ISO 8000-63, which specifies the measurement procedure and measurement stack by which to measure the characteristics of processes for data quality management; · an overview of ISO 8000‑64, which specifies a model that applies the Test Process Improvement method as a basis on which to determine the maturity of an organization with respect to data quality management. The details for implementing the parts of ISO 8000 related to data quality management are outside the scope of ISO/TS 8000‑60:2017.

ISO 8000-61:2016 specifies the processes required for data quality management. The processes are used as a reference to enhance data quality and assess process capability or organizational maturity for data quality management.

ISO 8000-140:2016 is an optional addition to ISO 8000‑120 and specifies requirements for representation and exchange of information about completeness of master data that consists of characteristic data. NOTE 1 ISO 8000‑110 specifies that such data be represented as property values. ISO 8000‑120 provides additional requirements for property values when data provenance information needs to be captured. ISO 8000-140:2016 does not specify a complete model for characteristic data, nor does it specify an exchange format for characteristic data with data completeness information. NOTE 2 This is done in other standards that reference this part of ISO 8000, e.g. ISO/TS 22745‑40. The following are within the scope of this part of ISO 8000: - requirements for capture and exchange of data completeness information in the form of statements and assertions of data completeness; - conceptual data model for data completeness information in the form of statements and assertions of data completeness. The following are outside the scope of this part of ISO 8000: - requirements for data completeness; NOTE 3 The requirements for data completeness depend on many factors, e.g. the kind of data, how the data are being used, industry, and needs of the partners exchanging the data. It is not possible to state general requirements for data completeness. - exchange format for data completeness information; - scheme for registering and resolving organization identifiers and person identifiers; - completeness of data that are not characteristic data represented as property values; - syntax of identifiers; - resolution of identifiers. Some of the requirements in ISO 8000-140:2016 can apply to exchange of data that is not master data which consists of characteristic data represented as property values.

ISO 8000-120:2016 specifies requirements for the representation and exchange of information about the provenance of master data that consists of characteristic data, and supplements the requirements of ISO 8000‑110. ISO 8000-120:2016 does not specify a complete model for characteristic data, nor does it specify an exchange format for characteristic data with provenance information. The following are within the scope of ISO 8000-120:2016: · scenarios for data provenance; · requirements for capture and exchange of data provenance information; · data model for data provenance information. The following are outside the scope of ISO 8000-120:2016: · exchange format for data provenance information; · scheme for registering and resolving organization identifiers and person identifiers; · provenance of data that are not characteristic data represented as property values; · configuration management; · change control; · syntax of identifiers; · resolution of identifiers. Some of the requirements in ISO 8000-120:2016 can apply to exchange of data that is not master data which consists of characteristic data represented as property values.

ISO 8000-100:2016 contains an overview of the master data quality series of parts of ISO 8000, which addresses master data quality. The following are within the scope of the master data quality series of parts of ISO 8000: · master data-specific aspects of quality management systems; · master data quality metrics. The approach of the master data quality series of parts of ISO 8000 is to address data quality: · from the bottom up, i.e. from the smallest meaningful element, the property value; · at the interface of master data management systems, not within the systems. The master data quality series of parts of ISO 8000 contains requirements that can be checked by computer for the exchange, between organizations and systems, of master data that consists of characteristic data. These parts address the quality of property values that are exchanged within master data messages. ISO 8000-100:2016 describes fundamentals of master data quality and specifies requirements on both data and organizations to enable master data quality. The following are within the scope of ISO 8000-100:2016: · specification of the scope of the master data quality series of parts of ISO 8000; · introduction to master data; · description of the data architecture; · overview of the content of the other parts of the series. The following are outside the scope of ISO 8000-100:2016: · aspects of data quality that apply to all data regardless of whether they are master data; · aspects of data quality that apply to data that are not master data.

ISO 8000-130:2016 is an optional addition to ISO 8000‑120 and specifies requirements for representation and exchange of information about accuracy of master data that consists of characteristic data. ISO 8000-130:2016 does not specify a complete model for characteristic data, nor does it specify an exchange format for characteristic data with data accuracy information. The following are within the scope of ISO 8000-130:2016: · requirements for capture and exchange of data accuracy information in the form of statements and assertions of data accuracy; · conceptual data model for data accuracy information in the form of statements and assertions of data accuracy. The following are outside the scope of ISO 8000-130:2016: · requirements for data accuracy; · exchange format for data accuracy information; · scheme for registering and resolving organization identifiers and person identifiers; · accuracy of data that are not characteristic data represented as property values; · syntax of identifiers; · resolution of identifiers. Some of the requirements in ISO 8000-130:2016 can apply to exchange of data that is not master data which consists of characteristic data represented as property values.

ISO 8000-8:2015 describes fundamental concepts of information and data quality, and how these concepts apply to quality management processes and quality management systems. It also specifies prerequisites for measuring information and data quality when executed within quality management processes and quality management systems.

ISO/TS 8000-311:2012 provides guidance for the application of product data quality for shape, as described in ISO 10303-59.

ISO 22745-13:2010 specifies rules for identification of concepts, terms, definitions and images within an open technical dictionary (OTD). ISO 22745-13:2010 is a specialization of ISO 29002-5.

ISO/TS 22745-10:2010 is a specialization of ISO/TS 29002-6 that specifies the requirements for exchange of elements of an OTD. It is intended for use with the web services specified in ISO/TS 22745-14.

ISO/TS 29002-6:2010 specifies a conceptual information model and an exchange file format for terminology information from a concept dictionary. The conceptual model is in Unified Modeling Language (UML). The physical file format is based on Extensible Markup Language (XML). The XML structures defined in ISO/TS 29002-6:2010 are intended to be returned by calls to the Terminology Service defined in ISO/TS 29002-20. The Terminology Service is an interface for retrieving terminological information about a concept from a concept dictionary. ISO/TS 29002-6:2010 was developed as a common resource to provide a level of interoperability between implementations of ISO 13584 and ISO 22745 for exchange of information from a concept dictionary. Its use is not restricted to these standards, however. It can be restricted through implementation profiles by standards that reference it. It can also be implemented without restriction.

ISO/TS 29002-20:2010 specifies an interface to retrieve information from a concept dictionary. Three sets of capabilities, called "services", are defined: Location Service: given a registration authority identifier, returns location on the Internet of Terminology and Ontology servers for the registration authority; Terminology Service: searches for concepts, languages, and other elements of the dictionary based on a rich set of keywords, search criteria, and expressions; given an international registration data identifier (IRDI) in the format defined in ISO/TS 29002-5, retrieves terminological information; Ontology Service: retrieves ontological information about a concept based on the concept's IRDI.

ISO/TS 22745-14:2010 is a specialization of ISO/TS 29002-20 that specifies an interface to retrieve information from an open technical dictionary (OTD). The interface can be used to retrieve terminology information and to search for concepts.

ISO/TS 22745-35:2010 is a specialization of ISO/TS 29002-31 that specifies the requirements for queries for ISO 22745 compliant master data. The conceptual model is in Unified Modeling Language (UML). The physical file format is based on Extensible Markup Language (XML).

ISO/TS 22745-40:2010 is a specialization of ISO/TS 29002-10 that specifies a conceptual information model and an exchange file format for catalogues. A catalogue is a collection of master data records. A master data record is made up of characteristic data expressed as property value pairs, where the property is identified by reference to a concept in an open technical dictionary. The conceptual information model is in Unified Modeling Language (UML). The physical file format is specified via an Extensible Markup Language (XML) schema.

ISO 22745-11:2010 provides guidelines that are recommended for the development of terminology for concepts in an open technical dictionary (OTD). Compliance with these guidelines is not required in order for a term or definition to be included in an OTD. These guidelines are provided as a framework for developers of industrial terminology to ensure that definitions are consistent and understandable. Terminology that complies with these guidelines will be marked as such in the OTD. This will allow a user to perform a search that filters terminology based on compliance with ISO 22745-11:2010.

ISO 22745-2:2010 contains vocabulary for ISO 22745.

ISO 22745-20:2010 contains procedures for maintenance of an open technical dictionary (OTD). It includes procedures to: register a consensus organization so that it can contribute terminology to an OTD; add a new concept, with associated terminology, to an OTD; add terminology to an existing concept in an OTD; mark terminology for an existing concept as deleted; merge concepts.

ISO 22745-1:2010 contains an introduction to ISO 22745. It includes: the scope of ISO 22745; requirements that all open technical dictionaries must follow; a description of the architecture of ISO 22745; an overview of ISO 22745, including a summary of the content of the other parts of ISO 22745; a discussion of the differences between ISO 22745 and ISO 13584, and between ISO 22745 and ISO 15926; and the location of a website with additional information for implementers.

ISO/TS 29002-10:2009 specifies a conceptual information model and an exchange file format for characteristic data. The conceptual model is in Unified Modeling Language (UML). The physical file format is based on Extensible Markup Language (XML).

ISO/TS 29002-31:2009 specifies a conceptual information model and an exchange file format for a query that requests product and service characteristic data and supplier identification from item and information suppliers. The conceptual model is in Unified Modeling Language (UML). The physical file format is based on Extensible Markup Language (XML).

ISO/TS 29002-4:2009 specifies a conceptual information model and an exchange file format for basic entities and types that are used as a resource by other parts of ISO/TS 29002. The conceptual model is in Unified Modeling Language (UML). The physical data format is based on Extensible Markup Language (XML). ISO/TS 29002-4:2009 defines types for language code, country code, currency code, uniform resource identifier, and Abstract Syntax Notation One (ASN.1) identifier. It defines an entity for human-readable text that has been localized into one or more languages. The resources in ISO/TS 29002-4:2009 are used by the models in ISO/TS 29002-6 and ISO/TS 29002-20.

ISO/TS 22745-30:2009 specifies a conceptual information model for identification guides and the datatypes necessary to support identification guides. ISO/TS 22745-30:2009 also specifies an Extensible Markup Language (XML) exchange structure for identification guides.

ISO/TS 29002-5:2009 specifies the data elements and a syntax for an unambiguous identifier for an administered item. The identifier is called an "international registration data identifier" (IRDI), as specified in ISO/IEC 11179-5. An administered item is any registry item for which administrative information is recorded. For the purposes of ISO/TS 29002-5:2009, an administered item can be a concept or concept information element in a concept dictionary. "Concept information element" includes things such as terminology (terms, abbreviations, definitions, images, symbols), assignment of a concept to a class of similar concepts (the concept type), and references to source documents. ISO/TS 29002-5:2009 was developed as a common resource to provide a level of interoperability between implementations of ISO 13584 and ISO/TS 22745. Its use is not restricted to these standards, however. It can be restricted through implementation profiles by standards that reference it. It can also be implemented without restriction.

This document defines terms relating to data quality used in the ISO 8000 series of parts.

This document defines terms relating to data quality used in the ISO 8000 series of parts. Terms and definitions related to quality in general are outside the scope of this document.

ISO/TS 8000-1:2011 contains an introduction to ISO 8000. It contains a statement of the scope of ISO 8000 as a whole, principles of data quality, the high-level data architecture of ISO 8000, a description of the structure of ISO 8000, and a summary of the content of the other parts of the general data quality series of parts of ISO 8000. It also describes the relationship between ISO 8000 and other standards.

ISO/TS 8000-150:2011 specifies fundamental principles of master data quality management, and requirements for implementation, data exchange and provenance. ISO/TS 8000-150:2011 also contains an informative framework that identifies processes for data quality management. This framework can be used in conjunction with, or independently of, quality management systems standards, for example, ISO 9001.

ISO 8000-110:2009 specifies requirements that can be checked by computer for the exchange, between organizations and systems, of master data that consists of characteristic data. The focus of ISO 8000-110:2009 is on requirements that can be checked by computer.