kSIST-TP FprCEN/TR 17833:2022

Railway applications - Guidance for the use of simulations - Guidance for the use of simulations to demonstrate compliance with technical and regulatory requirements and on the introduction and development of simulation requirements into standards

The aim of this document is to help CEN/CENELEC Working Group convenors and experts to promote/develop simulation in their standards as an alternative to physical tests on the real system for proving conformity. It can also provide useful guidance to assessors in the railway sector in approving simulations where they are not yet specifically defined or where physical tests on the real system are not defined in standards. Consequently, this document is also relevant to companies developing and applying simulations with the intention to achieve their acceptance for the purpose of system validation. It is not intended to provide technical guidance on applying simulations in general.
Where simulations are already introduced in existing standards, this guide is not intended to modify the specified requirements. However, technical harmonisation between standards might benefit from this guide for the introduction of additional alternative methods for simulations.
This document principally covers:
•   Numerical simulation, using complex methods or using simple spreadsheets methods
•   Hardware and software in the loop
•   Mathematical models solved using numerical methods or iteration, including spreadsheets.
It does not cover the following, although the general principles outlined can be applied to these methods:
•   Laboratory tests of components
•   Fatigue rig tests
•   Model scale tests
•   Mathematical models solved analytically.
NOTE: Due to the limited experience in the railway sector in the application of data-based (as opposed to model-based) simulations, for example using artificial intelligence (AI), neural networks, big data, etc., this approach is not further developed at this stage in this document.

Bahnanwendungen - Leitfaden für den Einsatz von Simulationen - Leitfaden für den Einsatz von Simulationen zum Nachweis der Einhaltung technischer und regulatorischer Anforderungen sowie zur Einführung und Entwicklung von Simulationsanforderungen in Normen

Železniške naprave - Navodilo za uporabo simulacij - Navodilo o uporabi simulacij za dokazovanje skladnosti s tehničnimi in regulativnimi zahtevami ter o vnašanju in razvoju simulacijskih zahtev v standarde

General Information

• Status: Not Published
• Public Enquiry End Date: 13-Apr-2022
• ICS: 01.120 - Standardization. General rules
• ICS: 45.020 - Railway engineering in general
• Technical Committee: IŽNP - Reilway applications
• Current Stage: 5520 - Unique Acceptance Procedure (UAP) (Adopted Project)
• Start Date: 01-Feb-2022
• Due Date: 21-Jun-2022
• Mandate: M/483 - Mandate for programming and standardisation addressed to the European standardisation bodies under directive 2008/57/EC in the field of the interoperability of the rail system within the European Union
• Ref Project: FprCEN/TR 17833 - Railway applications - Guidance for the use of simulations - Guidance for the use of simulations to demonstrate compliance with technical and regulatory requirements and on the introduction and development of simulation requirements into standards

Standards Content (sample)

SLOVENSKI STANDARD
kSIST-TP FprCEN/TR 17833:2022
01-april-2022

Železniške naprave - Navodilo za uporabo simulacij - Navodilo o uporabi simulacij

za dokazovanje skladnosti s tehničnimi in regulativnimi zahtevami ter o vnašanju
in razvoju simulacijskih zahtev v standarde

Railway applications - Guidance for the use of simulations - Guidance for the use of

simulations to demonstrate compliance with technical and regulatory requirements and

on the introduction and development of simulation requirements into standards
Bahnanwendungen - Leitfaden für den Einsatz von Simulationen - Leitfaden für den

Einsatz von Simulationen zum Nachweis der Einhaltung technischer und regulatorischer

Anforderungen sowie zur Einführung und Entwicklung von Simulationsanforderungen in

Normen
Ta slovenski standard je istoveten z: FprCEN/TR 17833
ICS:
01.120 Standardizacija. Splošna Standardization. General
pravila rules
45.020 Železniška tehnika na Railway engineering in
splošno general
kSIST-TP FprCEN/TR 17833:2022 en,fr,de

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
kSIST-TP FprCEN/TR 17833:2022
---------------------- Page: 2 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FINAL DRAFT
TECHNICAL REPORT
FprCEN/TR 17833
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
January 2022
ICS
English Version
Railway applications - Guidance for the use of simulations
- Guidance for the use of simulations to demonstrate
compliance with technical and regulatory requirements
and on the introduction and development of simulation
requirements into standards
Bahnanwendungen - Leitfaden für den Einsatz von
Simulationen - Leitfaden für den Einsatz von
Simulationen zum Nachweis der Einhaltung
technischer und regulatorischer Anforderungen sowie
zur Einführung und Entwicklung von
Simulationsanforderungen in Normen

This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee CEN/TC

256.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,

Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,

Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and

United Kingdom.

Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are

aware and to provide supporting documentation.

Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without

notice and shall not be referred to as a Technical Report.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels

© 2022 CEN All rights of exploitation in any form and by any means reserved Ref. No. FprCEN/TR 17833:2022 E

worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
Contents Page

European foreword ....................................................................................................................................................... 3

1 Scope .................................................................................................................................................................... 4

2 Normative references .................................................................................................................................... 4

3 Terms and definitions ................................................................................................................................... 4

4 Introduction ...................................................................................................................................................... 6

4.1 Background ....................................................................................................................................................... 6

4.2 Context ................................................................................................................................................................ 7

5 Principles governing the use of simulation ........................................................................................... 7

5.1 General ................................................................................................................................................................ 7

5.2 Verification of simulation tools.................................................................................................................. 8

5.3 User capabilities/qualification .................................................................................................................. 9

5.4 Verification and validation of simulation models ............................................................................... 9

5.4.1 Verification of models ................................................................................................................................... 9

5.4.2 Validation of models ...................................................................................................................................... 9

5.5 Specific additional conditions for Hardware- and Software in the Loop .................................... 9

5.6 Documentation when using simulations ............................................................................................. 10

6 Guidance for technical assessors (acceptance of simulation results) ...................................... 10

7 Guidance for WG Convenors ..................................................................................................................... 11

Annex A (informative) Examples where simulations have been substituted for physical

tests on the real system ............................................................................................................................. 15

Annex B (informative) Example of replacement of physical testing on the real system by

simulation – aerodynamic pressures at the trackside ................................................................... 17

B.1 Introduction ................................................................................................................................................... 17

B.2 Analysis of uncertainty and impact on output parameter Δp2σ ................................................. 17

Bibliography ................................................................................................................................................................. 19

---------------------- Page: 4 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
European foreword

This document (FprCEN/TS 17833:2022) has been prepared by Technical Committee CEN/TC 256

“Railway applications”, the secretariat of which is held by DIN.
This document is currently submitted to the Vote on TR.
---------------------- Page: 5 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
1 Scope

The aim of this document is to help CEN/CENELEC Working Group convenors and experts to

promote/develop simulation in their standards as an alternative to physical tests on the real system for

proving conformity. It can also provide useful guidance to assessors in the railway sector in approving

simulations where they are not yet specifically defined or where physical tests on the real system are not

defined in standards. Consequently, this document is also relevant to companies developing and applying

simulations with the intention to achieve their acceptance for the purpose of system validation. It is not

intended to provide technical guidance on applying simulations in general.

Where simulations are already introduced in existing standards, this guide is not intended to modify the

specified requirements. However, technical harmonisation between standards might benefit from this

guide for the introduction of additional alternative methods for simulations.
This document principally covers:

— numerical simulation, using complex methods or using simple spreadsheets methods;

— hardware and software in the loop;

— mathematical models solved using numerical methods or iteration, including spreadsheets.

It does not cover the following, although the general principles outlined can be applied to these methods:

— laboratory tests of components;
— fatigue rig tests;
— model scale tests;
— mathematical models solved analytically.

NOTE Due to the limited experience in the railway sector in the application of data-based (as opposed to model-

based) simulations, for example using artificial intelligence (AI), neural networks, big data, etc., this approach is not

further developed at this stage in this document.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
simulation (action and object)

(action) use of a similar or equivalent system to imitate a real system so that it behaves like or appears

to be the real system; (object) similar or equivalent system used to imitate a real system

Note 1 to entry: Simulation can be mathematical, analogue or scale modelling. Mathematical simulation includes

analytical and numerical calculation.
[SOURCE, ISO 16781:2013, 2.9, modified, Note 1 has been added.]
---------------------- Page: 6 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
3.2
numerical simulation
simulation based on numerical methods
3.3
test

technical operation that consists of applying to the object a set of environmental and operating conditions

under a specified procedure.

Note 1 to entry: A test can be carried out to determine one or more characteristics of a given object, process or

service according to a specified procedure. It can be used for different purposes (verifying requirements, calibration,

test cases and correct implementation of a model, etc.).

Note 2 to entry: A test can be conducted on the real system, or by entirely or partially using simulation

(simulation testing)
[SOURCE: IEV modified, ISO/IEC Guide 2 (13.1), notes 1 and 2 have been added]
3.4
simulation tool

in house, vendor or open source framework in which one may develop or embed models enabling the

execution of tests. It can be software and/or hardware and parts of the real system can be installed in the

tool.
3.5
model
mathematical and/or physical representation of a system or a process
3.6
numerical model
numerical representation of a mathematical model
3.7
verification (of simulation)

process of determining that a simulation in its tool environment produces expected results according to

the underlying model
3.8
validation (of simulation)

process of determining the degree to which a model is an accurate representation of the real system in

its environment
[ASME V&V 10 2006, modified] [ASME V&V 40 2018, modified]
3.9
system validation

process of proving conformity to system requirements, ensuring that the system is fit for its intended use

in its intended operational environment
---------------------- Page: 7 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
3.10
regression testing

testing required to determine that a change to a system (e.g. a model or a tool) has not adversely affected

functionality, reliability or performance and has not introduced additional defects

[SOURCE, ISO/IEC 27034-7:2018, 3.15, modified.]
3.11
environment
external aspects influencing the behaviour of a system
3.12
user
entity using the simulation
3.13
certification
third-party attestation related to components, sub-systems or systems
3.14
hardware in the loop simulation

type of simulation, in which some parts of the system or its environment are implemented or modelled by

real equipment

Note 1 to entry: Hardware in the loop simulations are characterised by a two-way coupling between the

simulated and the real components.
[SOURCE: IEC 16781:2013, 2.5, modified, note 1 added]
3.15
software in the loop simulation

type of simulation, in which a software that is executable on the real system is interfaced with simulation

models
4 Introduction
4.1 Background

The rationale for producing this document is the perception that physical testing on the real system for

train, infrastructure and command and control system certification leads to:
— excessive costs;
— delays bringing products to market.

The use of simulation is widespread in the automobile and the aerospace sectors, both for design and

validation. The challenge is for the European railway sector to examine its certification processes and

allow for using simulation methods as well as physical testing on the real system for system validation,

where it is possible and safe. In the majority of instances, the demonstration methods are defined in CEN

and CENELEC standards.
---------------------- Page: 8 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)

At the JPC Rail (Sector Forum Rail) meeting in March 2018, the issue of promoting the use of simulation

within the Railway Sector was raised. In response, three steps were proposed by CEN:

a) to set up a survey group to identify and ideally respond to transversal questions and needs to support

WG Conveners and experts to introduce or to further define simulation requirements in their

standards. It was foreseen that the outcome of the Survey Group would be a preliminary issue of a

guide;

b) to urge current CEN and CENELEC Working Groups’ conveners and experts to consider either

introducing or further defining existing simulation requirements within standards under their

responsibilities;

c) to invite the ERA and the EC to promote simulation approaches throughout the regulatory framework

whenever possible.
4.2 Context

Historically, the demonstration of safety and conformity to standards for obtaining the certification of

rolling stock, fixed installations, control-command systems and infrastructure has been mainly based on

physical tests on the real system.

Although already widely used in the design and pre-validation of sub-systems, simulation is still relatively

rarely applied to improve and accelerate the system validation phase, where physical tests on the real

system are often required for compliance assessments.

There are several ways in which simulations can help improve the system validation phase. They can be

used to better understand certain phenomena, enabling experts to study/explore a wider range of cases

than those practicably covered by physical tests on the real system (which are limited by environment

parameters such as weather, geographical range and configuration, boundary conditions etc.), and hence

complement them. Another possibility is to use simulation in order to reduce the amount of physical

testing on the real system, and to reduce delays bringing products to market. Use of simulation should

not be limited to exploring system behaviour in fault free conditions; it may also be extended to consider

failures or degraded modes.

Driven by the increase in simulation quality and reliability, the trend of evolution in the railway sector is

towards the use of more simulation and to less physical testing on the real system. The regulations and

the standards set requirements which have to be fulfilled. These may stipulate compulsory physical

testing on the real system or leave significant room for simulation or fail to specify the method of

demonstration at all.
5 Principles governing the use of simulation
5.1 General

Simulation can be used fully or partially to prove conformity (see example 1 in Annex A). For full

simulation proofs, only results from simulation testing are used as the final means to prove conformity.

In partial simulation proofs, physical tests on the real system or parts of it are required for some test

cases.

It should be noted that, in most cases, the initial validation of the simulation model might require physical

tests on the real system or parts of it (see Subclause 5.4).

Especially for design evolutions, where physical tests on the real system have already been conducted in

a previous similar case, it may be feasible to fully prove conformity by simulation if changes to the system

subject to testing remain within certain limits. For changes exceeding these limits, a partial proof by

simulation can be feasible, (see example 2 in Annex A).
---------------------- Page: 9 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)

NOTE It is also possible to fully prove conformity by simulation without starting from a pre existing design.

It should be ensured that the simulation tool is compatible with the intended purpose of the simulation

and that the models are representative of the real system in its environment. It is good practice to

evaluate qualitatively and quantitatively the uncertainty and the sensitivity of the models (more

information can be found in [1] and [2]). The user is responsible for critically interpreting the results

obtained.

Despite being separated in the following paragraphs, the concepts of tool verification, user capabilities

and model verification and validation are highly interconnected. Additional guidance can be found in

Clauses 7 and 8.

The simulation tool should be verified for the particular use to which it is being applied. Good practice

recommendations for the verification of simulation tools are given in Subclause 5.2.

Once the simulation tool is verified, it is necessary that measures are in place to ensure that the users of

the tools have the expertise and knowledge to apply them. Good practice recommendations for

demonstration of the skills necessary for users to use simulation tools and associated quality processes

are given in Subclause 5.3.

The primary purpose of validating a simulation model is to generate sufficient confidence in it, in order

to replace physical tests on the real system by simulations. This is further developed in Subclause 5.4.

Specific additional conditions for Hardware- and Software in the Loop are introduced in Subclause 5.5.

In Subclause 5.6, documentation requirements when using simulations are given.
5.2 Verification of simulation tools

Simulation tools consist of one or more components, which are interfaced in order to enable embedding

the model(s) of the system to be tested and possibly parts of the real system. Those components can be

software and/or hardware and their complexity may vary depending on the tool.

NOTE Software tools generally provide a library of tool-specific elementary models. They can range from

simple models, such as mathematical functions, to complex ones, such as a set of physics.

In order to determine that a simulation in its tool environment produces expected results according to

the underlying model, as a first step, the simulation tool should be verified for the particular use to which

it is being applied. For tools consisting of several components, each component should be verified

separately, and in combination, in order to verify their interfaces.

Verification should consider the accuracy, range of validity, boundary conditions and limitations of tool

components and their interfaces, which should be appropriate for the intended purpose of the simulation

tool.

The organisation performing tool component verification usually depends on its origin:

— for in-house simulation tools, it is the user organisation;
— for third-party simulation tools, it is the providing organisation;

— for open source simulation tools, it is, depending on the case, either an identified organisation or the

user organisation.

The verification process should also cover configuration and change management of tool components, for

example by performing regression testing during version changes and following updates to operating

systems and to host machines.

In every case, the user should verify the complete simulation tool by applying specific reference cases,

and checking the outputs against known results.

The part of the verification process performed in the user organisation should be traced and documented.

---------------------- Page: 10 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
It is good practice to maintain a log of user experiences of the tool.
5.3 User capabilities/qualification

As well as verifying the simulation tool for the intended use, measures should be in place to ensure that

the users of the simulations have the expertise and knowledge to set up and/or to apply them, and

critically analyse and interpret the results. This is the responsibility of the user organisation.

The user should ensure that the adopted simulation tool is fit for purpose.
NOTE The user in this context can refer to a range of different people.

It is good practice to verify the user skills for setting up and/or applying simulations, for instance, by

undertaking standard simulation test cases and recording skill levels.

Where relevant, it is good practice to maintain sufficient independence between designers and validation

experts, to ensure confidence in the results. The degree of independence should be based on the safety

impact of the intended use.
5.4 Verification and validation of simulation models
5.4.1 Verification of models

The second step of determining that a simulation in its tool environment produces expected results

involves verifying the consistency of the different choices of modelling and particularly checking:

— the individual sub models of the system and its environment;
— the full model of the system in its environment;
— the simulation method and its numerical convergence.
5.4.2 Validation of models

A validation process should demonstrate that the simulation model sufficiently represents a reference

system for different reference scenarios. This reference system might consist, preferably, of a physical

test on the real system, or of a reduced scale test or a generic case (e.g. analytical, benchmark, etc.). If a

model is an adaptation of, or similar to, a previously validated model, it may be possible to conduct a

reduced validation. In this case, the rationale for the reduced validation should be made and documented.

Generally, validation should be performed for the entire model, where necessary after validating

particular sub models individually.

The comparison between the simulation results and the reference system will give the level of confidence

in the simulation. The validation includes performing investigations concerning uncertainties (e.g.

accuracy, robustness and reproducibility). The domain of validity of the simulation should be given. (See

e.g. examples 3 and 4 in Annex A).

Best practice for validation involves proper consideration of the uncertainties in both the simulation and

in the reference system.

Validation of the simulation models should be documented as detailed in Subclause 5.6.

Particular attention should be paid to simulations of systems that have a safety function or a safety

impact.
5.5 Specific additional conditions for Hardware- and Software in the Loop

The concept of 'Hardware in the Loop' (HiL) simulations is based on using real components or pieces of

equipment directly interfaced with numerical simulation models (see example 5 in Annex A). The system

and the environment are partly real and partly modelled. The real part of the system can be mechanical,

electrical, electronic, etc.
---------------------- Page: 11 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)

Interfaces between components are physical and virtual; numerical data being converted into electrical

signals and vice versa. The 'time step' of the simulation should be adapted to the reaction time of the

equipment being studied and should be consistent between the components; HiL simulation is

considered to be in 'real time'. This should be considered when performing verification and validation.

The concept of 'Software in the Loop' (SiL) simulations is based on running a software that is executable

on the real system interfaced with a simulated system and environment model. Interfaces between

components are virtual.

As the real components are neither part of the tool, nor of the model, it is not necessary to verify and

validate them as part of this process. However, special attention should be paid to the interfaces to SiL or

HiL.
5.6 Documentation when using simulations
The following elements should be included in the documentation:

— a summary of the verification and validation of the simulation (tool and models) including:

— the domain of validity of the simulation;
— proof of numerical convergence;

— a comparison of the simulation to a reference system for one or more reference scenarios as

defined in 5.4, including details such as the test conditions, the degree of precision/uncertainty

of the measurements and the representativeness of the boundary conditions considering the

uncertainties;

— other evidence supporting the validity of the simulation may be presented as an alternative;

— a statement confirming the validity of the simulation (tool and models) for the given application;

— a statement confirming the qualification and capabilities of the user;

— a simulation report containing detailed information on the simulated scenarios and analyses of their

results, including:

— a statement of conformance of the system to its requirements (e.g. as defined in standards);

— a statement that the simulation was executed in its domain of validity.

It is good practice to document in detail the different steps in performing the simulation.

6 Guidance for technical assessors (acceptance of simulation results)

If the simulation tool and the user's aptitude are verified and the simulation models have been validated,

the simulation results can be considered as representative for the behaviour of the real system within the

scope of application specified.

For simulation results to be acknowledged as proof of compliance, either the applicable reference

documents and/or standards define the conditions in which simulation can be used, or they do not

directly deal with the use of simulations. In the first case, if the standards and/or regulatory requirements

have been respected, then the simulation results can be accepted.
In the second case:

— there may be generally accepted principles in fields where the knowledge and tools available provide

a sufficient degree of reliability to dispense with physical tests on the real system, allowing the

simulation results to be accepted; such principles should be documented;
---------------------- Page: 12 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)

— in other cases, a risk analysis (e.g. as outlined in CSM-RA [3]) should be conducted with an emphasis

on the following considerations:

— scope of the simulation: can the simulation cover all of the conformity demonstrations required

in the physical test on the real system in question or is it to be considered in conjunction with

results of other physical tests on the real system in cases involving complex configurations (e.g.

fail-safe modes, excess speed, values measured, etc.)?

— criticality of the simulation: The safety considerations associated with each demonstration need

to be specified. While the evidence provided should still comply with the applicable reference

standards, the degree of reliability required from simulation performed to establish specific

parameters may differ depending on the safety level associated with the parameter considered.

EXAMPLE Reliability of simulations for noise levels may be different from that required of a simulation

of braking behaviour.
7 Guidance for WG Convenors

This Clause gives particular guidance for CEN/CENELEC Working Group (WG) convenors when they are

considering and introducing simulation as an alternative to physical tests on the real system in their

standards. The following are general considerations to take into account, but are not exhaustive.

1) Clearly identify the reasons for introducing simulation in the standard.

— e.g. reduction in time/cost, difficulty of physical testing on the real system, e.g. in failure

modes.

— It should be noted that insufficient maturity in simulation at the time should not impede the

WG from considering the introduction of simulation within the standard.

2) Assess if the WG could benefit from external expertise in the field of simulation. If necessary, WG

members who currently use simulation can perform a common benchmark simulation to

establish best practice, and to have a basis for comparing different simulation tools and different

users. Alternatively, known experts in the field can be invited to contribute as guests or as

members (e.g. by opening up to universities, other research institutions, etc.), thus broadening

the expertise in the WG.
3) Use the princ
...