oSIST prEN IEC 63270-2:2026

• Status: Not Published
• Public Enquiry End Date: 29-Sep-2026
• ICS: 25.040.01 - Industrial automation systems in general
• Technical Committee: MOV - Measuring equipment for electromagnetic quantities
• Current Stage: 4020 - Public enquire (PE) (Adopted Project)
• Start Date: 14-Jul-2026
• Due Date: 01-Dec-2026
• Ref Project: prEN IEC 63270-2:2026 - Industrial automation equipment and systems - Part 2: Algorithm verification methods

Overview

oSIST prEN IEC 63270-2:2026, developed by the International Electrotechnical Commission (IEC) and CENELEC (CLC), defines standard practices for verifying algorithms used in industrial automation equipment and systems. Focused on predictive maintenance, this standard outlines a unified framework for classifying, testing, and validating algorithms that underpin condition monitoring, fault diagnosis, and life prediction. As industrial sectors increasingly rely on automated systems and data-driven maintenance strategies, standardized algorithm verification assures users, manufacturers, and solution providers of consistent performance and reliability.

This document is part of a broader series addressing predictive maintenance in industrial automation and builds on general requirements set out in IEC 63270-1:2025. By implementing robust algorithm verification methods, practitioners can objectively evaluate the accuracy, effectiveness, and applicability of predictive maintenance algorithms for diverse industrial contexts.

Key Topics

• Algorithm Classification: The standard details categories of algorithms pivotal to industrial predictive maintenance, mainly:

  • Condition monitoring algorithms
  • Fault diagnosis algorithms
  • Life (remaining useful life) prediction algorithms
  • Signal processing algorithms (supporting condition analysis)

• Verification Process: A structured verification workflow is defined, comprising:

  • Test preparation (requirements analysis, data readiness)
  • Algorithm verification (testing and validation using industry-accepted indicators)
  • Algorithm debugging (iterative performance improvement as needed)

• Performance Indicators: Objective metrics are prescribed for measuring algorithm output quality, including:

  • Condition discrimination accuracy rate
  • Abnormal condition omission rate
  • Fault identification and category identification accuracy (for expert systems)
  • Machine learning performance metrics: accuracy, precision, recall
  • Prediction accuracy, mean absolute error, and additional indicators relevant to Remaining Useful Life (RUL) estimation

• Verification Data Requirements: Clear guidelines on test data preparation, formatting, calibration, conversion, and validity checking ensure results are reproducible and industry-relevant.

Applications

Applying oSIST prEN IEC 63270-2:2026 brings significant practical value to stakeholders in industrial automation:

• Equipment Users: Gain confidence in predictive maintenance solutions by referencing consistent, validated metrics for algorithm performance before deployment.
• Manufacturers and Suppliers: Rely on standardized verification to demonstrate compliance and technical superiority when providing predictive maintenance technologies.
• Solution Integrators: Use standardized methods to benchmark new and existing algorithms, facilitating competitive selection and technology integration.
• Certification and Procurement: The standard acts as an objective reference when purchasing or certifying predictive maintenance systems, reducing risks associated with unproven algorithmic methods.

Industries benefiting from this standard include manufacturing, process industries, utilities, and any sector leveraging automation for equipment health monitoring and predictive maintenance.

Related Standards

• IEC 63270-1:2025
  • Predictive maintenance of industrial automation equipment and systems - Part 1: General requirements
• ISO 17359
  • Condition monitoring and diagnostics of machines - General guidelines
• IEC 62342:2007
  • Reference for predictive maintenance definitions and concepts
• ISO 13381-1:2015
  • Condition monitoring and diagnostics of machines – Prognostics – Part 1: General guidelines

Additional references within the IEC and ISO framework offer guidance on condition monitoring, diagnostics, failure analysis, and performance assessment, supporting comprehensive predictive maintenance strategies.

---

By leveraging oSIST prEN IEC 63270-2:2026's standardized algorithm verification methods, the industrial automation community can ensure reliable, technically sound, and interoperable predictive maintenance across global operations. This standard supports digital transformation, reduces unplanned downtime, and fosters innovation by providing a trusted benchmark for algorithm evaluation within industrial environments.

SLOVENSKI STANDARD
01-september-2026
Oprema in sistemi za industrijsko avtomatizacijo - 2. del: Metode preverjanja
algoritmov
Industrial automation equipment and systems - Part 2: Algorithm verification methods
Ta slovenski standard je istoveten z: prEN IEC 63270-2:2026
ICS:
25.040.01 Sistemi za avtomatizacijo v Industrial automation
industriji na splošno systems in general
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

65E/1222/CDV
COMMITTEE DRAFT FOR VOTE (CDV)
PROJECT NUMBER:
IEC 63270-2 ED1
DATE OF CIRCULATION: CLOSING DATE FOR VOTING:
2026-07-10 2026-10-02
SUPERSEDES DOCUMENTS:
65E/1195/CD, 65E/1217/CC
IEC SC 65E : DEVICES AND INTEGRATION IN ENTERPRISE SYSTEMS
SECRETARIAT: SECRETARY:
United States of America Mr David Richmond
OF INTEREST TO THE FOLLOWING COMMITTEES: HORIZONTAL FUNCTION(S):

ASPECTS CONCERNED:
SUBMITTED FOR CENELEC PARALLEL VOTING NOT SUBMITTED FOR CENELEC PARALLEL VOTING
Attention IEC-CENELEC parallel voting
The attention of IEC National Committees, members of
CENELEC, is drawn to the fact that this Committee Draft
for Vote (CDV) is submitted for parallel voting.
The CENELEC members are invited to vote through the
CENELEC online voting system.
This document is still under study and subject to change. It should not be used for reference purposes.
Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of
which they are aware and to provide supporting documentation.
Recipients of this document are invited to submit, with their comments, notification of any relevant “In Some Countries”
clauses to be included should this proposal proceed. Recipients are reminded that the CDV stage is the final stage for
submitting ISC clauses. (SEE AC/22/2007 OR NEW GUIDANCE DOC).

TITLE:
Industrial automation equipment and systems – Part 2: Algorithm Verification Methods

PROPOSED STABILITY DATE: 2031
NOTE FROM TC/SC OFFICERS:
this electronic file, to make a copy and to print out the content for the sole purpose of preparing National Committee
positions. You may not copy or "mirror" the file or printed version of the document, or any part of it, for any other
purpose without permission in writing from IEC.

IEC CDV 63270-2 © IEC 2026
1 CONTENTS
2 FOREWORD . 3
3 1. Scope . 6
4 2. Normative References . 6
5 3. Terms and definitions . 6
6 3.1 Terms, definitions, and abbreviated terms . 6
7 3.1.1 condition monitoring . 6
8 3.1.2 Fault Diagnosis . 6
9 3.1.3 Predictive Maintenance . 6
10 3.1.4 Confidence Level . 6
11 3.2 Abbreviated Terms . 7
12 4. General . 7
13 5. Classification . 7
14 6. Algorithm Verification Process . 8
15 7. Algorithm Test Indicator . 9
16 7.1 Condition Monitoring Algorithm Indicator . 9
17 7.1.1 Overview. 9
18 7.1.2 Condition Discrimination Accuracy Rate . 9
19 7.1.3 Abnormal Condition Omission Rate . 9
20 7.2 Fault Diagnosis Algorithm Indicators . 10
21 7.2.1 Expert System Algorithm Indicators . 10
22 7.2.2 Artificial Based Algorithm Indicators . 10
23 7.3 Prediction Algorithm Indicators . 12
24 7.3.1 Overview. 12
25 7.3.2 Prediction Accuracy Rate . 12
26 7.3.3 Mean Absolute Error . 13
27 7.3.4 Root Mean Squared Error . 13
28 7.3.5 Coefficient of Determination . 13
29 7.3.6 Score of Prediction Error Indicator . 13
30 8 Algorithm Verification Method . 14
31 8.1 Condition Monitoring Algorithm Test . 14
32 8.1.1 Test Data Requirements . 14
33 8.1.2 Algorithm Test . 14
34 8.1.3 Test Process . 14
35 8.1.4 Test Results . 15
36 8.2 Fault Diagnosis Algorithm Test . 15
37 8.2.1 Test Data Requirements . 15
38 8.2.2 Test Method . 15
39 8.2.3 Test Result . 15
40 8.3 Prediction Algorithm Test . 15
41 8.3.1 Test Data Requirements . 15
42 8.3.2 Test Method . 16
43 8.3.3 Test Result . 17
44 9 Algorithm Verification Index System . 17
45 9.1 Overview. 17
46 9.2 Verification Requirements . 18
47 9.2.1 Condition Monitoring Algorithms . 18
48 9.2.2 Fault Diagnosis Algorithm . 18
IEC CDV 63270-2 © IEC 2026
49 9.2.3 Life Prediction Algorithm . 18
50 9.2.4 Verification Description . 18
51 10 Verification Data Requirements . 18
52 10.1 Overview. 18
53 10.2 Data Entry Requirements . 18
54 10.2.1 Format and Content Requirements . 18
55 10.2.2 Calibration Requirements . 18
56 10.2.3 Data Conversion . 19
57 10.2.4 Data Entry . 19
58 10.2.5 Post-entry Checks . 19
59 Annex A (informative)  Signal Processing Algorithm Test Indicators . 20
60 A.1 Waveform Quality Verification Indicator . 20
61 A.2 Spectrum Quality Verification Indicator . 20
62 A.3 Quality Verification Indicator for Time-frequency Distribution . 21
63 A.4 Computational Complexity Verification Indicator. 21
64 Annex B (informative) Format of the Algorithm Verification Report . 22
65 B.1 Algorithm Verification Report . 22
66 Annex C (informative) Verification of the Prediction Algorithm . 23
67 C.1 Verification of the Prediction Algorithm . 23
68 Annex D (informative) Database Construction and System Requirements . 25
69 D.1 Overview. 25
70 D.2 Data Quality Specifications . 26
71 D.3 Database System Functions . 26
72 Annex E (Normative) Supplementary Indicators of Prediction Algorithm Test . 28
73 E.1 Mean Squared Error . 28
74 E.2 Precision . 28
75 E.3 Uncertainty Quantification Indicator . 28
IEC CDV 63270-2 © IEC 2026
77 INTERNATIONAL ELECTROTECHNICAL COMMISSION
78 ____________
80 PREDICTIVE MAINTENANCE OF INDUSTRIAL AUTOMATION EQUIPMENT
81 AND SYSTEMS – PART 2: ALGORITHM VERIFICATION METHODS
84 FOREWORD
85 1) The International Electrotechnical Commission (IEC) is a worldwide organization for standardization comprising
86 all national electrotechnical committees (IEC National Committees). The object of IEC is to promote international
87 co-operation on all questions concerning standardization in the electrical and electronic fields. To this end and
88 in addition to other activities, IEC publishes International Standards, Technical Specifications, Technical Reports,
89 Publicly Available Specifications (PAS) and Guides (hereafter referred to as “IEC Publication(s)”). Their
90 preparation is entrusted to technical committees; any IEC National Committee interested in the subject dealt with
91 may participate in this preparatory work. International, governmental and non-governmental organizations liaising
92 with the IEC also participate in this preparation. IEC collaborates closely with the International Organization for
93 Standardization (ISO) in accordance with conditions determined by agreement between the two organizations.
94 2) The formal decisions or agreements of IEC on technical matters express, as nearly as possible, an international
95 consensus of opinion on the relevant subjects since each technical committee has representation from all
96 interested IEC National Committees.
97 3) IEC Publications have the form of recommendations for international use and are accepted by IEC National
98 Committees in that sense. While all reasonable efforts are made to ensure that the technical content of IEC
99 Publications is accurate, IEC cannot be held responsible for the way in which they are used or for any
100 misinterpretation by any end user.
101 4) In order to promote international uniformity, IEC National Committees undertake to apply IEC Publications
102 transparently to the maximum extent possible in their national and regional publications. Any divergence between
103 any IEC Publication and the corresponding national or regional publication shall be clearly indicated in the latter.
104 5) IEC itself does not provide any attestation of conformity. Independent certification bodies provide conformity
105 assessment services and, in some areas, access to IEC marks of conformity. IEC is not responsible for any
106 services carried out by independent certification bodies.
107 6) All users should ensure that they have the latest edition of this publication.
108 7) No liability shall attach to IEC or its directors, employees, servants or agents including individual experts and
109 members of its technical committees and IEC National Committees for any personal injury, property damage or
110 other damage of any nature whatsoever, whether direct or indirect, or for costs (including legal fees) and
111 expenses arising out of the publication, use of, or reliance upon, this IEC Publication or any other IEC Publications.
112 8) Attention is drawn to the Normative references cited in this publication. Use of the referenced publications is
113 indispensable for the correct application of this publication.
114 9) Attention is drawn to the possibility that some of the elements of this IEC Publication may be the subject of patent
115 rights. IEC shall not be held responsible for identifying any or all such patent rights.
116 International Standard IEC XXXXX has been prepared by IEC technical committee 65: Industrial
117 process measurement, control and automation.
118 The text of this International Standard is based on the following documents:
FDIS Report on voting
XX/XX/FDIS XX/XX/RVD
120 Full information on the voting for the approval of this International Standard can be found in the
121 report on voting indicated in the above table.
122 This document has been drafted in accordance with the ISO/IEC Directives, Part 2.
123 The committee has decided that the contents of this document will remain unchanged until the
124 stability date indicated on the IEC website under "http://webstore.iec.ch" in the data related to
125 the specific document. At this date, the document will be
126 • reconfirmed,
IEC CDV 63270-2 © IEC 2026
127 • withdrawn,
128 • replaced by a revised edition, or
129 • amended.
131 The National Committees are requested to note that for this document the stability date
132 is 20XX.
133 THIS TEXT IS INCLUDED FOR THE INFORMATION OF THE NATIONAL COMMITTEES AND WILL BE DELETED
134 AT THE PUBLICATION STAGE.
IEC CDV 63270-2 © IEC 2026
137 INTRODUCTION
138 The use of predictive maintenance has increased in many industries as it has evolved into the
139 predominant method of operation and maintenance services. Algorithms are at the core of
140 effective implementation of predictive maintenance. Moreover, there is an urgent need for a
141 standardized measurement system to evaluate the accuracy, effectiveness, and applicability of
142 the predictive maintenance algorithm in a scientific and reasonable manner.
143 Predictive maintenance is a new equipment operation and maintenance model for equipment
144 or components in various industries. It uses new-generation information technology in
145 conjunction with vibration, image, current, vocal print, and other signal analysis for condition
146 monitoring, fault diagnosis, remaining life prediction, developing operation and maintenance
147 decision plans, and arranging reasonable maintenance activities. By using the standard test
148 verification indications and procedures used in industry, a general algorithm verification system
149 is developed in this document with condition monitoring, fault diagnosis, life prediction, and
150 other predictive maintenance algorithms as objects. This standard can provide support for its
151 application in different industries after issuance and implementation.
152 Generally, the practitioners of predictive maintenance are divided into equipment users,
153 equipment manufacturers, and solution suppliers. This document can be used by the
154 aforementioned practitioners as a reference manual and verification basis, as well as an
155 important supporting material for the procurement and acceptance of equipment. It can also be
156 used as a basis for equipment manufacturers and solution suppliers to explain the advantages
157 of the technical solutions for predictive maintenance that they provide.
IEC CDV 63270-2 © IEC 2026
160 PREDICTIVE MAINTENANCE OF INDUSTRIAL AUTOMATION EQUIPMENT
161 AND SYSTEMS – PART 2: ALGORITHM VERIFICATION METHODS
163 1. Scope
164 This document specifies the classification, the verification process, test indicators, test methods,
165 the verification index system, and the verification data requirements for predictive maintenance.
166 This document applies to the test verification of condition monitoring, fault diagnosis, life
167 prediction, and other algorithms in predictive maintenance.
168 2. Normative References
169 The following documents are referred to in the text in such a way that some or all of their content
170 constitutes requirements of this document. For dated references, only the edition cited applies.
171 For undated references, the latest edition of the referenced document (including any
172 amendments) applies.
173 IEC 63270-1:2025: Predictive maintenance of industrial automation equipment and systems —
174 Part 1: General requirements
175 3. Terms and definitions
176 For the purposes of this document, the following terms and definitions apply.
177 ISO and IEC maintain terminological databases for use in standardization at the following
178 addresses:
179 1 IEC Electropedia: available at http://www.electropedia.org/
180 2 ISO Online browsing platform: available at http://www.iso.org/obp
181 3.1 Terms, definitions, and abbreviated terms
182 3.1.1 condition monitoring
183 Recording and verification of technical entity status for maintenance purposes.
184 [SOURCE: IEC 63270-1, 3.1.3]
185 3.1.2 Fault Diagnosis
186 Examine the symptoms and symptom complexes in order to determine the nature (type,
187 condition, and degree) of the fault or failure.
188 3.1.3 Predictive Maintenance
189 It is a form of preventive maintenance that is carried out continuously or at predetermined
190 intervals governed by observed conditions in order to monitor, diagnose, or trend the condition
191 indicators of a structure, system, or component. Results indicate functional capacity for the
192 present and the future, or the nature and timing of planned maintenance.
193 [SOURCE: IEC 62342:2007, 3.14, modified – notes have been deleted]
194 3.1.4 Confidence Level
195 Quality criteria represent the accuracy degree of the diagnosis or prediction.
196 Note 1: It is expressed as a percentage.
197 Note 2: A computational or weighted verification system may compute this value, which is
198 basically a number that represents the cumulative impact of the error source on the ultimate
199 reliability or confidence in the accuracy of the output result. Abbreviated terms.
IEC CDV 63270-2 © IEC 2026
200 [SOURCE: ISO 13381-1-2015, 3.3]
201 3.2 Abbreviated Terms
RUL Remaining Use Life
MAE Mean Absolute Error
RMSE Root Mean Squared Error
SPE Score of Prediction Error
MSE Mean Squared Error
UQ Uncertainty Quantification
202 4. General
203 Based on IEC 63270-1:2025, predictive maintenance can realize condition monitoring, fault
204 diagnosis, and life prediction. In this document, condition monitoring, fault diagnosis, life
205 prediction, and other algorithms are evaluated, of which the life prediction algorithm is
206 abbreviated as the prediction algorithm.
207 The verification method presented in this document is made up of two parts: testing and
208 verification. The test is primarily based on the algorithm test indicators in Chapter 7 and the
209 algorithm test methods in Chapter 8. However, the verification is primarily based on the
210 algorithm verification index system in Chapter 9. These three elements together constitute the
211 algorithm verification requirements.
212 5. Classification
213 In this document, condition monitoring, fault diagnosis, life prediction, and other algorithms are
214 evaluated. Moreover, one or several can be selected to be tested in the actual test. Signal
215 processing is a significant part of monitoring, diagnosis, and prediction algorithms. In addition,
216 signal test indicators include signal processing quality verification and algorithm complexity
217 verification. It is feasible to decide whether to perform the test of signal processing algorithms
218 during the testing of monitoring, diagnosis, and prediction algorithms based on specific
219 application scenarios and requirements. Check Annex A for the test indicators of signal
220 processing algorithms such as atlas analysis.
221 Condition monitoring algorithm verification includes:
222 - Threshold-based monitoring algorithm: Condition monitoring algorithms can be divided into
223 two categories, which are fixed threshold-based discrimination methods and relative threshold-
224 based discrimination methods. Clause 7.1 describes the test indicator system, while Clause 8.1
225 describes the test methods.
226 - Algorithms that do not fall into these categories may be evaluated by referring to the indicators
227 in Clause 7.1.
228 Fault diagnosis algorithm verification consists of:
229 - Expert system verification: Based on the knowledge and experience of experts in the field,
230 expert systems can conduct reasoning and judgement by applying artificial intelligence
231 technology and computer technology in order to obtain the results of fault diagnosis according
232 to the characteristic data matching degree of expert systems. Clause 7.2.1 describes the test
233 indicator system of the expert system, while Clause 8.2 describes the test methods.
234 - Verification of Artificial Intelligence based methods such as artificial based algorithm: Artificial
235 Intelligence based methods such as Machine learning algorithms can obtain the structure of
236 fault diagnosis through the classification of condition characteristics.
IEC CDV 63270-2 © IEC 2026
237 - Algorithms that do not fall into these categories may be evaluated using the indicators
238 specified in Clause 8.2.
239 Prediction algorithm verification consists of the following:
240 - Prediction algorithm verification: Based on the user-defined failure threshold, the life
241 prediction algorithm can determine where the trajectory of predictive features intersects with
242 the failure threshold, obtain the end time of life, and then obtain the results of life prediction .
243 Clause 7.3 describes the test indicator system for the prediction algorithm test, while Clause
244 8.3 describes the test methods.
245 6. Algorithm Verification Process
246 As shown in Fig. 1, the algorithm verification process can be divided into three stages: test
247 preparation, algorithm verification, and algorithm debugging.
249 Figure 1: Algorithm Verification Process
250 The algorithm verification process is described as follows:
251 - Test preparation stage: The test preparation stage consists of activities such as customer
252 application, assessment type selection, and determining whether a sample database (referred
253 to as a database) supports judgement. First, determine the verification type based on the
254 verification application submitted by the customer. Second, determine whether the database
255 supports the test. If so, proceed to sampling. If not, ask the customer to provide sample data
256 (referred to as data) and update the database.
257 - Algorithm verification stage: The algorithm test stage consists of activities such as sampling,
258 model preparation, test environment setup, algorithm testing, and algorithm verification. The
259 sampled model is prepared for training, etc., and then a test environment is set up based on
260 the algorithm’s public availability, which shall be either an interface call or direct deployment of
261 the algorithm.
262 - Algorithm debugging stage: The algorithm debugging stage consists of activities such as
263 debugging judgement, algorithm debugging, and report issuance. The algorithm will be allowed
264 to be debugged and updated for re-verification if the verification results are unsatisfactory. The
IEC CDV 63270-2 © IEC 2026
265 debugging shall be limited 2 times. Moreover, the verification shall be completed if customer
266 satisfaction is reached.
267 7. Algorithm Test Indicator
268 7.1 Condition Monitoring Algorithm Indicator
269 7.1.1 Overview
270 Condition monitoring algorithms can be divided into fixed threshold-based discrimination
271 methods and relative threshold-based discrimination methods.
272 The fixed threshold-based discrimination method obtains the status discrimination results by
273 analyzing and processing the test samples and comparing them to the current standard's pre-
274 defined alarm thresholds. For example, the setting of vibration data alarm thresholds can be
275 referred to in ISO 17359: 2011.
276 The relative threshold-based discrimination method obtains status discrimination results by
277 learning from standard data, analysing, and processing the sample data, as well as comparing
278 them with customized alarm thresholds. The relative threshold-based discrimination method
279 can be applied in situations where existing standards are available, or to scenarios where
280 existing standards cannot be referenced.
281 7.1.2 Condition Discrimination Accuracy Rate
282 Condition discrimination accuracy rate refers to the accuracy with which equipment condition
283 monitoring results are classified. There are two types of equipment status: normal and abnormal.
284 Under normal conditions, the equipment is allowed to operate for an unlimited period of time.
285 However, an abnormal condition indicates that the equipment’s condition has changed
286 significantly from its normal condition and that maintenance is required soon.
287 As shown in Eq. (1), the condition discrimination accuracy rate can be evaluated as a
288 percentage of the number of samples whose condition is correctly discriminated relative to the
289 total number of samples.
𝐶
𝑠
290 𝐴 = ………………………………………… (1)
𝑐𝑚
𝑆
291 where:
292 𝐴 – Condition discrimination accuracy rate.
𝑐𝑚
293 𝐶 – The number of samples whose conditions are correctly discriminated, including the number
𝑠
294 of normal samples judged to be normal and the number of abnormal samples judged to be
295 abnormal.
296 S– The number of all samples.
297 The condition discrimination accuracy rate is in the range [0, 1], and the larger, the better. The
298 value 𝐴 can be expressed as a percentage by multiplying by 100%.
𝑐𝑚
299 7.1.3 Abnormal Condition Omission Rate
300 As shown in Eq. (2), the omission rate of abnormal condition identification and equipment’s
301 determination can be tested as a percentage of the number of unidentified abnormal samples
302 in given samples in the total number of abnormal samples.
𝑀𝑠
303 𝑈𝑟 = ………………………………………… (2)
𝐴𝑠
304 where:
305 Ur – Abnormal condition omission rate.
306 Ms – Number of unidentified abnormal samples.
307 As – Total number of abnormal samples.
308 The abnormal condition omission rate takes the value range of [0, 1], and the smaller, the better.
309 The value 𝑈𝑟 can be expressed as a percentage by multiplying by 100%.
IEC CDV 63270-2 © IEC 2026
310 7.2 Fault Diagnosis Algorithm Indicators
311 7.2.1 Expert System Algorithm Indicators
312 Expert system-based fault diagnosis algorithms shall be tested for fault identification accuracy
313 rate first, followed by the fault category identification accuracy rate test. The mean of the
314 confidence levels can be tested for expert system algorithms with confidence level outputs.
315 As shown in Eq. (3), the fault identification accuracy rate is the percentage of the number of
316 samples with or without fault correctly identified in the total number of samples for a given set
317 of diagnostic test samples.
𝑁
𝐶
318 𝐴 = ………………………………………… (3)
𝑐𝑓
𝑁
319 where:
320 𝐴 – Fault identification accuracy rate.
𝑐𝑓
321 𝑁 – Number of samples with fault correctly identified .
𝐶
322 𝑁- Total number of samples.
323 The fault identification accuracy rate is in the range [0,1], and the larger, the better. The value
324 𝐴 can be expressed as a percentage by multiplying by 100%.
𝑐𝑓
325 As shown in Eq. (4), the fault category identification accuracy rate is the percentage of the
326 number of samples correctly classified in the total number of samples for a given set of
327 diagnostic test samples.
𝑛
∑ 𝑇
𝑖
𝑖=1
328 𝐴 = ………………………………………… (4)
𝑐𝑑
𝑁
329 where:
330 𝐴 – Fault category identification accuracy rate.
𝑐𝑑
331 𝑇 – Number of the samples correctly identified as the i category.
𝑖
332 𝑛 – Number of samples in this category.
333 𝑁 – Total number of samples.
334 The fault category identification accuracy rate is in the range [0,1], and the larger, the better.
335 The value 𝐴 can be expressed as a percentage by multiplying by 100%.
𝑐𝑑
336 The mean of the confidence levels can be tested for the expert system where confidence levels
337 are output. As shown in Eq. (5), the mean of the confidence levels is not tested if the expert
338 system does not output a confidence level.
𝑘
∑ 𝐶
𝑗=1 𝑗
339 𝐶 = ………………………………………… (5)
𝑚
𝑁
340 where:
341 C – Mean of the confidence levels output by the expert system.
𝑚
342 C – Confidence level output by the expert system for the first correctly identified category of
𝑗
343 samples.
344 𝑘 – Number of samples whose categories are correctly identified.
345 𝑁 – Total number of samples.
346 The mean of confidence levels is in the range [0,1], and the larger, the better. The value 𝐶 can
𝑚
347 be expressed as a percentage by multiplying by 100%.
348 7.2.2 Artificial Based Algorithm Indicators
349 7.2.2.1 General
350 The main verification metrics of artificial based algorithm are accuracy rate, precision rate, and
351 recall rate. In addition, the calculation methods can be categorised as either macro-averaging
352 or micro-averaging. Macro-averaging involves calculating the metric values for each category
353 first and then finding the arithmetic mean for all categories. However, micro-averaging involves
IEC CDV 63270-2 © IEC 2026
354 calculating the final metric values by aggregating all instances in the dataset together without
355 classifying them.
356 7.2.2.2 Accuracy Rate
357 The accuracy rate refers to the ratio of the number of correctly diagnosed samples to the total
358 number of diagnosed samples, which reflects the overall performance of the algorithm. As
359 shown in Eq. (6), the accuracy rate is calculated for n categories.
𝑛
∑ 𝑇𝑃
𝑖=1 𝑖
360 A = ………………………………………… (6)
𝑐𝑙
𝑁
361 where:
362 A – Accuracy rate.
𝑐𝑙
363 𝑇𝑃 – Number of the samples correctly identified as the i category.
𝑖
364 𝑛 – Number of categories.
365 𝑁 – Total number of samples.
366 The accuracy rate takes a range of [0, 1], which represents the degree of consistency between
367 the results of the algorithm's diagnosis and the real situation. Also, the higher the accuracy rate,
368 the more consistent the overall results of the algorithm's diagnosis and the real situation. The
369 value A can be expressed as a percentage by multiplying by 100%.
𝑐𝑙
370 7.2.2.3 Precision Rate
371 Precision rate refers to the ratio of the number of positive samples correctly diagnosed to the
372 number of all samples diagnosed as positive. The macro-averaging of precision rate for n
373 categories is shown in Eq. (7).
1 𝑇𝑃
𝑖
𝑛
∑
374 𝑃 = × ………………………………… (7)
𝑟𝑚𝑎 𝑖=1
n 𝑇𝑃 +𝐹𝑃
𝑖 𝑖
375 where:
376 𝑃 – Macro-average of precision rate.
𝑟𝑚𝑎
377 𝑇𝑃 – Number of the samples correctly identified as the i category.
𝑖
378 𝐹𝑃 – Number of the samples wrongly identified as i category.
𝑖
379 𝑛 – - Number of categories.
380 The micro-averaging of precision rate is shown in Eq. (8):
𝑛
∑ 𝑇𝑃
𝑖=1 𝑖
381 𝑃 = …………………………………… (8)
𝑟𝑚𝑖 𝑛
∑ (𝑇𝑃 +𝐹𝑃 )
𝑖 𝑖
𝑖=1
382 where:
383 𝑃 – Micro-average of precision rate.
𝑟𝑚𝑖
384 𝑇𝑃 – Number of the samples correctly identified as i category.
𝑖
385 𝐹𝑃 – Number of the samples wrongly identified as i category.
𝑖
386 𝑛 – Number of categories.
387 The precision rate takes a range of [0,1]. This represents the precision of the samples identified
388 as faults in the algorithm’s diagnosis result. The higher the precision rate, the more consistent
389 the samples identified as faults by the algorithm and the real situation. The value 𝑃 and
𝑟𝑚𝑎
390 𝑃 can be expressed as a percentage by multiplying by 100%.
𝑟𝑚𝑖
391 7.2.2.4 Recall Rate
392 The recall rate refers to the ratio of the number of positive samples correctly diagnosed to the
393 total number of authentic positive samples, also known as the true positive rate. For n
394 categories, the macro-averaging of recall rate is shown in Eq. (9).
1 𝑇𝑃
𝑛 𝑖
395 𝑅 = × ∑ ………………………………… (9)
𝑒𝑚𝑎
𝑖=1
n 𝑇𝑃 +𝐹𝑁
𝑖 𝑖
396 where:
IEC CDV 63270-2 © IEC 2026
397 𝑅 – Macro-average of recall rate.
𝑒𝑚𝑎
398 𝑇𝑃 – Number of the samples correctly identified as the i category.
𝑖
399 𝐹𝑁 – Number of the i category samples identified as other categories.
𝑖
400 𝑛 – Number of categories.
401 The micro-averaging of recall rate is shown in Eq. (10):
𝑛
∑
𝑇𝑃
𝑖
𝑖=1
402 𝑅 = ……………………………………… (10)
𝑒𝑚𝑖 𝑛
∑
(𝑇𝑃 +𝐹𝑁 )
𝑖 𝑖
𝑖=1
403 where:
404 𝑅 – Micro-average of recall rate.
𝑒𝑚𝑖
405 𝑇𝑃 – Number of the samples correctly identified as the i category.
𝑖
406 𝐹𝑁 – Number of the i-category samples identified as other category.
𝑖
407 𝑛 – Number of categories.
408 The recall rate takes a range of [0,1]. This represents the degree of completeness of the
409 diagnostic algorithm to determine the fault samples from the total diagnostic samples. Also, the
410 higher the recall rate, the more complete the diagnostic algorithm can determine the real fault
411 samples. The value 𝑅 and 𝑅 can be expressed as a percentage by multiplying by 100%.
𝑒𝑚𝑎 𝑒𝑚𝑖
412 7.3 Prediction Algorithm Indicators
413 7.3.1 Overview
414 Prediction algorithm indicators measure algorithm performance in four dimensions: prediction
415 accuracy, prediction error, algorithm fitting degree, and prediction error score. The prediction
416 algorithm indicator is used to measure the accuracy of the algorithm's prediction results. The
417 prediction error indicator is used to measure the degree of error between the algorithm's
418 predicted Remaining Useful Life and the actual Remaining Useful Life. The algorithm fitting
419 degree indicator is used to measure the fitting degree between the predicted results and the
420 actual Remaining Useful Life. The prediction error score scores the algorithm's prediction
421 results and tends to favor retarded prediction over premature prediction in engineering. This
422 indicator has a higher penalty for retarded prediction than for premature prediction and can be
423 used as an error scoring indicator. Furthermore, the supplemental indicators for the prediction
424 algorithm test are shown in Annex E.
425 7.3.2 Prediction Accuracy Rate
426 As shown in Eqs. (11) and (12), the prediction accuracy rate is a measure of the algorithm's
427 accuracy in predicting the Remaining Useful Life.
428 𝑒(𝑡 ) = 𝑟(𝑡 ) − 𝑟 (𝑡 )………………………………………… (11)
𝑖 𝑖 ∗ 𝑖
|𝑒(𝑡 )|
𝑖
−
𝑁
𝑟(𝑡 )
∑ 𝑖
429 𝐴𝑐(𝑟, 𝑟 ) = 𝑒𝑥𝑝 ……………………………… (12)
∗ 𝑖=1
𝑁
430 where:
431 𝐴𝑐(𝑟, 𝑟 ) – Prediction accuracy rate.
∗
432 𝑖 – ith sample.
433 𝑁 – Total number of samples.
434 𝑒(𝑡 )- Difference between the actual Remaining Useful Life and the predicted Remaining Useful
𝑖
435 Life at the moment of 𝑡 .
𝑖
436 𝑟(𝑡 ) – Actual Remaining Useful Life at the moment of 𝑡 .
𝑖 𝑖
437 𝑟 (𝑡 ) – Predicted Remaining Useful Life at the moment of 𝑡 .
∗ 𝑖 𝑖
438 The prediction accuracy rate takes a range of (0,1]. Also, the closer the prediction accuracy
439 rate is to 1, the better the algorithm’s accuracy rate is. The value 𝐴𝑐(𝑟, 𝑟 )can be expressed as
∗
440 a percentage by multiplying by 100%.
IEC CDV 63270-2 © IEC 2026
441 7.3.3 Mean Absolute Error
442 As shown in Eq. (13), the mean absolute error is a measure of the proximity between the
443 predicted Remaining Useful Life and the actual Remaining Useful Life.
𝑁
∑
444 𝑀𝐴𝐸(𝑟, 𝑟 ) = |𝑟(𝑡 ) − 𝑟 (𝑡 )|……………………………… (13)
∗ 𝑖=1 𝑖 ∗ 𝑖
𝑁
445 where:
446 𝑀𝐴𝐸(𝑟, 𝑟 ) – Mean absolute error.
∗
447 𝑖 – i sample.
th
448 𝑁 – Total number of samples.
449 𝑟(𝑡 ) – Actual Remaining Useful Life at the moment of 𝑡 .
𝑖 𝑖
450 𝑟 (𝑡 ) – Predicted Remaining Useful Life at the moment of 𝑡 .
∗ 𝑖 𝑖
451 MAE takes a range of [0, +∞), and the smaller the MAE value, the smaller the algorithm error.
452 7.3.4 Root Mean Squared Error
453 The root mean squared error is defined as the square root of the squared mean of the error.
454 This indicator will have a higher weight for values with large errors, and any excessive error will
455 give a poor value of RMSE, as shown in Eq. (14).
𝑁
456 𝑅𝑀𝑆𝐸(𝑟, 𝑟 ) = √ ∑ (𝑟(𝑡 ) − 𝑟 (𝑡 )) …………………………… (14)
∗ 𝑖 ∗ 𝑖
𝑖=1
𝑁
457 where:
458 𝑅𝑀𝑆𝐸(𝑟, 𝑟 ) − Root mean squared error.
∗
459 𝑖 – i sample.
th
460 𝑁 – Total number of samples.
461 𝑟(𝑡 ) – Actual Remaining Useful Life at the moment of 𝑡 .
𝑖 𝑖
462 𝑟 (𝑡 ) – Predicted Remaining Useful Life at the moment of 𝑡 .
∗ 𝑖 𝑖
463 RMSE takes a range of [0, +∞), and the smaller the RMSE value, the smaller the algorithm error.
464 7.3.5 Coefficient of Determination
465 The coefficient of determination is the ratio of the regression sum of squares to the total sum
466 of squares. It is used to measure how well the predicted Remaining Useful Life fits the actual
467 Remaining Useful Life. The coefficient of determination is calculated in Equations. (15) and (16).
𝑁 2
∑ (𝑟(𝑡 )−𝑟 (𝑡 ))
∗
2 𝑖=1 𝑖 𝑖
468 𝑅 = 1 − ……………………………………… (15)
𝑁
∑ (𝑟(𝑡 )−𝑟̄)
𝑖
𝑖=1
𝑁
469 𝑟̄= ∑ 𝑟(𝑡 )…………………………………………… (16)
𝑖
𝑖=1
𝑁
470 where:
471 𝑅 – Coefficient of determination.
472 𝑖 – i sample.
th
473 𝑁 – Total number of samples.
474 𝑟(𝑡 ) – Actual Remaining Useful Life at the moment of 𝑡 .
𝑖 𝑖
475 𝑟 (𝑡 ) – Predicted Remaining Useful Life at the moment of 𝑡 .
∗ 𝑖 𝑖
476 𝑟̄ – Mean 𝑟(𝑡 )of actual Remaining Useful Life.
𝑖
477 The coefficient of determination takes the range of (-∞, 1). Also, the closer the coefficient of
478 determination is to 1, the better the algorithm fitting degree is.
479 7.3.6 Score of Prediction Error Indicator
480 The score of prediction error indicator is the weighted sum of the RUL errors. The score function
481 is an asymmetric function. As shown in Equations (11) and (17), the indicator has a greater
IEC CDV 63270-2 © IEC 2026
482 penalty for retarded prediction than for premature prediction for RUL, with lower scores
483 representing better algorithm performance.
𝑒(𝑡 )
𝑖
( )
𝑛 𝛼⋅𝑟(𝑡 )
∑ 𝑖
(𝑒 − 1) 𝑒(𝑡 ) ≥ 0
𝑖=1 𝑖
484 𝑆𝑃𝐸(𝑡 ) = { ………………………………… (17)
𝑖 𝑒(𝑡 )
𝑖
−( )
𝑛
𝛽⋅𝑟(𝑡 )
𝑖
∑ (𝑒 − 1) 𝑒(𝑡 ) < 0
𝑖=1 𝑖
485 where:
486 𝑆𝑃𝐸(𝑡 ) – Score of prediction error indicator.
𝑖
487 𝑖 – i sample.
th
488 𝛼 – The penalty factor for the premature prediction shall take a desirable range of values in
489 [10,15], which is recommended to be 13.
490 𝛽 – The penalty factor for the retarded prediction shall have a value less than 𝛼 and the
491 appropriate range of values in [7, 12], which is recommended to be 10.
492 SPE takes a range of [0, +∞), and the closer the SPE value is to 0, the better the algorithm
493 predicts the results.
494 8 Algorithm Verification Method
495 8.1 Condition Monitoring Algorithm Test
496 8.1.1 Test Data Requirements
497 The following are the requirements of the condition monitoring algorithm for the test:
498 - Standard data: including data, source of data generation, data collection method, metadata
499 for the corresponding equipment, including equipment number, equipment type, working
500 conditions, and data labels (abnormal or normal).
501 - Test samples: including data, source of data generation, data collection method, metadata for
502 the corresponding equipment, including equipment number, equipment type, working conditions
503 (the condition shall fall within the range of working conditions in the standard data), data labels
504 (abnormal or normal).
505 8.1.2 Algorithm Test
506 The following is the test methodology for the different condition monitoring algorithm types :
507 - Test based on a fixed threshold monitoring algorithm. The participants in verification judged
508 the test sample’s condition on the basis of the thresholds set by international or national
509 standards: abnormal or normal. The state judgments made by the participants in verification on
510 the test samples will be used to compare with the real condition. Moreover, the conclusions of
511 passing or failing the test will be given by the condition discrimination accuracy rate and the
512 abnormal condition omission rate.
513 - Test based on the relative threshold monitoring algorithm. It provides standard data during
514 the equipment’s normal operation. It also includes corresponding control information about the
515 equipment, such as processing information, etc., in order to
...