ISO 15746-3:2020
(Main)Automation systems and integration — Integration of advanced process control and optimization capabilities for manufacturing systems — Part 3: Verification and validation
Automation systems and integration — Integration of advanced process control and optimization capabilities for manufacturing systems — Part 3: Verification and validation
This document defines the principle of verification and validation according to the activity models and workflow of an advanced process control and optimization (APC-O) system, analyses and defines the general process for verification and validation of APC-O systems, and specifies a set of indicators and checkpoints used for verification and validation.
Systèmes d'automatisation et intégration — Intégration de contrôles de processus avancés et capacités d'optimisation des systèmes de fabrication — Partie 3: Vérification et validation
General Information
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO
STANDARD 15746-3
First edition
2020-02
Automation systems and
integration — Integration of advanced
process control and optimization
capabilities for manufacturing
systems —
Part 3:
Verification and validation
Systèmes d'automatisation et intégration — Intégration de contrôles
de processus avancés et capacités d'optimisation des systèmes de
fabrication —
Partie 3: Vérification et validation
Reference number
©
ISO 2020
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Principle and purpose . 2
5.1 Principle of verification and validation . 2
5.2 Structures of indicators . 3
5.2.1 General. 3
5.2.2 Structure of quantitative indicators . 3
5.2.3 Structure of judgement indicator . 3
5.3 General process for verification and validation. 4
5.4 Verification . 5
5.5 Checkpoints in requirement analysis phase . 5
5.6 Checkpoints of design phase . 8
5.7 Checkpoints in development phase . 9
5.8 Checkpoints in execution phase .11
5.9 Checkpoints in support phase .14
6 Validation .15
Bibliography .19
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 184, Automation systems and integration,
Subcommittee SC 5, Interoperability, integration and architectures for enterprise systems and automation
applications.
A list of all parts in the ISO 15746 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO 2020 – All rights reserved
Introduction
As a crucial part of the increasingly complex manufacturing systems, automation and control
applications which are enabled by advanced process control and optimization (APC-O) methodology
and solutions are implemented under the direction of production planning and scheduling. This task
involves initially the specific use of APC-O that will eventually enable the integration of manufacturing
operations management (MOM) with the automation and control of manufacturing process and
equipment.
Automation solutions equipped with both software and hardware components are provided by
different suppliers to accomplish APC-O functions. Due to the diversity of development environments
and the variety of demand focus, the automation solutions from various suppliers tend to be isolated
and relatively independent, which make it harder for the automation solutions to be integrated.
Consequently, various automation solution components that the customers can have access to are filled
with redundant and duplicated functions, resulting in a waste of resources and limited interoperability.
This document offers an interoperability framework for advanced process control and optimization
with the intention of maximizing both the integration and the interoperability of automation solutions.
It is not the intent of this document to suggest that there is only one way of implementing APC-O or to
force users to abandon their current way of implementing APC-O.
The target users of this document include users and providers of advanced process control and
optimization solutions, such as, project solution suppliers, automation systems integrators, production
departments of companies, process engineers, independent software testing organizations,
implementation and consulting service organizations of advanced process control and optimization
software, and relevant government and academic organizations.
INTERNATIONAL STANDARD ISO 15746-3:2020(E)
Automation systems and integration — Integration of
advanced process control and optimization capabilities for
manufacturing systems —
Part 3:
Verification and validation
1 Scope
This document defines the principle of verification and validation according to the activity models and
workflow of an advanced process control and optimization (APC-O) system, analyses and defines the
general process for verification and validation of APC-O systems, and specifies a set of indicators and
checkpoints used for verification and validation.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
checkpoint
point where verification (3.6) and validation (3.5) activities needed to be performed throughout the
APC-O lifecycle
3.2
indicator
measurement of an aspect of the system or component
Note 1 to entry: There are two types of indicators: quantitative indicators (3.3) and judgement indicators (3.4).
3.3
quantitative indicator
indicator (3.2) that is calculated using formulae
3.4
judgement indicator
indicator (3.2) that is evaluated using the evaluation method
3.5
validation
process of evaluating an APC-O system to determine whether it satisfies the stakeholders’ requirements
for that system
3.6
verification
process of evaluating an APC-O system to determine whether the output of a phase satisfies the
conditions imposed at the start of that phase
4 Abbreviated terms
APC-O Advanced Process Control and Optimization
MOM Manufacturing Operations Management
V&V Verification and Validation
5 Principle and purpose
5.1 Principle of verification and validation
This document provides a specification for both users and suppliers of the APC-O systems. The APC-O
system that conforms to the specification will satisfy the customer requirements and facilitate the
integration between different APC-O systems, as illustrated in Figure 1.
Figure 1 — Verification and validation of APC-O systems
Workflow: The workflow of an APC-O system relates to the phases of the lifecycle.
Output of lifecycle phase: Each work phase has an object.
Work phase: The lifecycle of an APC-O system consists of the following phases:
a) Requirements analysis;
b) Design;
2 © ISO 2020 – All rights reserved
c) Development;
d) Execution;
e) Support.
5.2 Structures of indicators
5.2.1 General
Both the quantitative indicators and the judgement indicators are expressed in the manner of the
structure specified in ISO 22400-2. The structure identifies the descriptive element of indicator in the
left column and gives a description of each element in the right column.
5.2.2 Structure of quantitative indicators
Table 1 presents an overview of the main elements of quantitative indicator structure.
Table 2 is an example of quantitative indicator.
Table 1 — Quantitative indicator structure
Name Name of the indicator
ID Unique identification
Description A brief description of the indicator
Scope The object that the indicator is used for, including the lifecycle phases or
the elements/activities within a lifecycle phase
Formula The mathematical formula of the indicator defined in terms of elements
Unit of measure The basic unit or dimension of the indicator
Range Defines the upper and lower logical limits of the indicator
Trend The information that indicates the direction for improvement, e.g. if a
higher or lower value is better
Audience Users and providers of APC-O solutions, such as project solution suppli-
ers, automation systems integrators, production departments of com-
panies, process engineers, independent software testing organizations,
implementation and consulting service organizations of APC-O software,
and relevant government and academic organizations
Table 2 — Example of quantitative indicator
Name Mean squared error (MSE)
ID
Description This checkpoint is used to describe the stable rate
NOTE Definition of MSE
Formula Mean[(SV − PV) ^2]
Unit of measure %
Range Min: 0 %
Max: 100 %
Trend The lower, the better
Audience Engineers and operators of process
5.2.3 Structure of judgement indicator
Table 3 presents an overview of the main elements of judgement indicator structure.
Table 4 is an example of judgement indicator
...
INTERNATIONAL ISO
STANDARD 15746-3
First edition
2020-02
Automation systems and
integration — Integration of advanced
process control and optimization
capabilities for manufacturing
systems —
Part 3:
Verification and validation
Systèmes d'automatisation et intégration — Intégration de contrôles
de processus avancés et capacités d'optimisation des systèmes de
fabrication —
Partie 3: Vérification et validation
Reference number
©
ISO 2020
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Principle and purpose . 2
5.1 Principle of verification and validation . 2
5.2 Structures of indicators . 3
5.2.1 General. 3
5.2.2 Structure of quantitative indicators . 3
5.2.3 Structure of judgement indicator . 3
5.3 General process for verification and validation. 4
5.4 Verification . 5
5.5 Checkpoints in requirement analysis phase . 5
5.6 Checkpoints of design phase . 8
5.7 Checkpoints in development phase . 9
5.8 Checkpoints in execution phase .11
5.9 Checkpoints in support phase .14
6 Validation .15
Bibliography .19
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 184, Automation systems and integration,
Subcommittee SC 5, Interoperability, integration and architectures for enterprise systems and automation
applications.
A list of all parts in the ISO 15746 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO 2020 – All rights reserved
Introduction
As a crucial part of the increasingly complex manufacturing systems, automation and control
applications which are enabled by advanced process control and optimization (APC-O) methodology
and solutions are implemented under the direction of production planning and scheduling. This task
involves initially the specific use of APC-O that will eventually enable the integration of manufacturing
operations management (MOM) with the automation and control of manufacturing process and
equipment.
Automation solutions equipped with both software and hardware components are provided by
different suppliers to accomplish APC-O functions. Due to the diversity of development environments
and the variety of demand focus, the automation solutions from various suppliers tend to be isolated
and relatively independent, which make it harder for the automation solutions to be integrated.
Consequently, various automation solution components that the customers can have access to are filled
with redundant and duplicated functions, resulting in a waste of resources and limited interoperability.
This document offers an interoperability framework for advanced process control and optimization
with the intention of maximizing both the integration and the interoperability of automation solutions.
It is not the intent of this document to suggest that there is only one way of implementing APC-O or to
force users to abandon their current way of implementing APC-O.
The target users of this document include users and providers of advanced process control and
optimization solutions, such as, project solution suppliers, automation systems integrators, production
departments of companies, process engineers, independent software testing organizations,
implementation and consulting service organizations of advanced process control and optimization
software, and relevant government and academic organizations.
INTERNATIONAL STANDARD ISO 15746-3:2020(E)
Automation systems and integration — Integration of
advanced process control and optimization capabilities for
manufacturing systems —
Part 3:
Verification and validation
1 Scope
This document defines the principle of verification and validation according to the activity models and
workflow of an advanced process control and optimization (APC-O) system, analyses and defines the
general process for verification and validation of APC-O systems, and specifies a set of indicators and
checkpoints used for verification and validation.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
checkpoint
point where verification (3.6) and validation (3.5) activities needed to be performed throughout the
APC-O lifecycle
3.2
indicator
measurement of an aspect of the system or component
Note 1 to entry: There are two types of indicators: quantitative indicators (3.3) and judgement indicators (3.4).
3.3
quantitative indicator
indicator (3.2) that is calculated using formulae
3.4
judgement indicator
indicator (3.2) that is evaluated using the evaluation method
3.5
validation
process of evaluating an APC-O system to determine whether it satisfies the stakeholders’ requirements
for that system
3.6
verification
process of evaluating an APC-O system to determine whether the output of a phase satisfies the
conditions imposed at the start of that phase
4 Abbreviated terms
APC-O Advanced Process Control and Optimization
MOM Manufacturing Operations Management
V&V Verification and Validation
5 Principle and purpose
5.1 Principle of verification and validation
This document provides a specification for both users and suppliers of the APC-O systems. The APC-O
system that conforms to the specification will satisfy the customer requirements and facilitate the
integration between different APC-O systems, as illustrated in Figure 1.
Figure 1 — Verification and validation of APC-O systems
Workflow: The workflow of an APC-O system relates to the phases of the lifecycle.
Output of lifecycle phase: Each work phase has an object.
Work phase: The lifecycle of an APC-O system consists of the following phases:
a) Requirements analysis;
b) Design;
2 © ISO 2020 – All rights reserved
c) Development;
d) Execution;
e) Support.
5.2 Structures of indicators
5.2.1 General
Both the quantitative indicators and the judgement indicators are expressed in the manner of the
structure specified in ISO 22400-2. The structure identifies the descriptive element of indicator in the
left column and gives a description of each element in the right column.
5.2.2 Structure of quantitative indicators
Table 1 presents an overview of the main elements of quantitative indicator structure.
Table 2 is an example of quantitative indicator.
Table 1 — Quantitative indicator structure
Name Name of the indicator
ID Unique identification
Description A brief description of the indicator
Scope The object that the indicator is used for, including the lifecycle phases or
the elements/activities within a lifecycle phase
Formula The mathematical formula of the indicator defined in terms of elements
Unit of measure The basic unit or dimension of the indicator
Range Defines the upper and lower logical limits of the indicator
Trend The information that indicates the direction for improvement, e.g. if a
higher or lower value is better
Audience Users and providers of APC-O solutions, such as project solution suppli-
ers, automation systems integrators, production departments of com-
panies, process engineers, independent software testing organizations,
implementation and consulting service organizations of APC-O software,
and relevant government and academic organizations
Table 2 — Example of quantitative indicator
Name Mean squared error (MSE)
ID
Description This checkpoint is used to describe the stable rate
NOTE Definition of MSE
Formula Mean[(SV − PV) ^2]
Unit of measure %
Range Min: 0 %
Max: 100 %
Trend The lower, the better
Audience Engineers and operators of process
5.2.3 Structure of judgement indicator
Table 3 presents an overview of the main elements of judgement indicator structure.
Table 4 is an example of judgement indicator
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.