SIST-TP CEN/TR 17833:2022
(Main)Railway applications - Guidance for the use of simulations - Guidance for the use of simulations to demonstrate compliance with technical and regulatory requirements and on the introduction and development of simulation requirements into standards
Railway applications - Guidance for the use of simulations - Guidance for the use of simulations to demonstrate compliance with technical and regulatory requirements and on the introduction and development of simulation requirements into standards
The aim of this document is to help CEN/CENELEC Working Group convenors and experts to promote/develop simulation in their standards as an alternative to physical tests on the real system for proving conformity. It can also provide useful guidance to assessors in the railway sector in approving simulations where they are not yet specifically defined or where physical tests on the real system are not defined in standards. Consequently, this document is also relevant to companies developing and applying simulations with the intention to achieve their acceptance for the purpose of system validation. It is not intended to provide technical guidance on applying simulations in general.
Where simulations are already introduced in existing standards, this guide is not intended to modify the specified requirements. However, technical harmonisation between standards might benefit from this guide for the introduction of additional alternative methods for simulations.
This document principally covers:
• Numerical simulation, using complex methods or using simple spreadsheets methods
• Hardware and software in the loop
• Mathematical models solved using numerical methods or iteration, including spreadsheets.
It does not cover the following, although the general principles outlined can be applied to these methods:
• Laboratory tests of components
• Fatigue rig tests
• Model scale tests
• Mathematical models solved analytically.
NOTE: Due to the limited experience in the railway sector in the application of data-based (as opposed to model-based) simulations, for example using artificial intelligence (AI), neural networks, big data, etc., this approach is not further developed at this stage in this document.
Bahnanwendungen - Leitfaden für den Einsatz von Simulationen - Leitfaden für den Einsatz von Simulationen zum Nachweis der Einhaltung technischer und regulatorischer Anforderungen sowie zur Einführung und Entwicklung von Simulationsanforderungen in Normen
Železniške naprave - Navodilo za uporabo simulacij - Navodilo o uporabi simulacij za dokazovanje skladnosti s tehničnimi in regulativnimi zahtevami ter o vnašanju in razvoju simulacijskih zahtev v standarde
Namen tega dokumenta je pomagati vodjem in strokovnjakom delovne skupine CEN/CENELEC pri spodbujanju/razvoju simulacije v njihovih standardih kot alternativo fizičnim preskusom na dejanskem sistemu za dokazovanje skladnosti. Poleg tega ocenjevalcem v železniškem sektorju zagotavlja uporabne smernice pri odobritvi simulacij, kjer še niso posebej opredeljene ali kjer fizični preskusi na dejanskem sistemu niso opredeljeni v standardih. Posledično je ta dokument relevanten tudi za podjetja, ki razvijajo in uporabljajo simulacije z namenom, da bi dosegla njihovo sprejetje za namene sistemske validacije. Ta dokument ni namenjen zagotavljanju tehničnih navodil za uporabo simulacij na splošno.
Če so simulacije že uvedene v obstoječih standardih, ta navodila niso namenjena spreminjanju zahtev, ki so v njih določene. Ta navodila pa bi lahko bila koristna za tehnično usklajevanje med standardi za uvedbo dodatnih alternativnih metod za simulacije.
Ta dokument obravnava predvsem:
• numerično simulacijo z uporabo kompleksnih metod ali enostavnih metod, ki temeljijo na preglednicah;
• strojno opremo in programsko opremo v zanki;
• matematične modele, ki se rešujejo z uporabo numeričnih metod ali ponavljanja, vključno s preglednicami.
Ta dokument ne obravnava naslednjega, čeprav se lahko v njem opisana splošna načela uporabijo pri teh metodah:
• laboratorijski preskusi komponent;
• preskusi utrujenosti opreme;
• sorazmerno preskušanje modelov;
• matematični modeli, ki se rešujejo analitično.
OPOMBA: Zaradi omejenih izkušenj v železniškem sektorju pri uporabi simulacij, ki temeljijo na podatkih (v nasprotju s simulacijami, ki temeljijo na modelih), na primer z uporabo umetne inteligence (AI), nevronskih omrežij, masovnih podatkov itd., se ta pristop na tej stopnji v tem dokumentu ne razvija naprej.
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-TP CEN/TR 17833:2022
01-september-2022
Železniške naprave - Navodilo za uporabo simulacij - Navodilo o uporabi simulacij
za dokazovanje skladnosti s tehničnimi in regulativnimi zahtevami ter o vnašanju
in razvoju simulacijskih zahtev v standarde
Railway applications - Guidance for the use of simulations - Guidance for the use of
simulations to demonstrate compliance with technical and regulatory requirements and
on the introduction and development of simulation requirements into standards
Bahnanwendungen - Leitfaden für den Einsatz von Simulationen - Leitfaden für den
Einsatz von Simulationen zum Nachweis der Einhaltung technischer und regulatorischer
Anforderungen sowie zur Einführung und Entwicklung von Simulationsanforderungen in
Normen
Ta slovenski standard je istoveten z: CEN/TR 17833:2022
ICS:
01.120 Standardizacija. Splošna Standardization. General
pravila rules
45.020 Železniška tehnika na Railway engineering in
splošno general
SIST-TP CEN/TR 17833:2022 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-TP CEN/TR 17833:2022
---------------------- Page: 2 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833
TECHNICAL REPORT
RAPPORT TECHNIQUE
June 2022
TECHNISCHER BERICHT
ICS
English Version
Railway applications - Guidance for the use of simulations
- Guidance for the use of simulations to demonstrate
compliance with technical and regulatory requirements
and on the introduction and development of simulation
requirements into standards
Bahnanwendungen - Leitfaden für den Einsatz von
Simulationen - Leitfaden für den Einsatz von
Simulationen zum Nachweis der Einhaltung
technischer und regulatorischer Anforderungen sowie
zur Einführung und Entwicklung von
Simulationsanforderungen in Normen
This Technical Report was approved by CEN on 24 May 2022. It has been drawn up by the Technical Committee CEN/TC 256.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2022 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TR 17833:2022 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
Contents Page
European foreword . 3
1 Scope . 4
2 Normative references . 4
3 Terms and definitions . 4
4 Introduction . 6
4.1 Background . 6
4.2 Context . 7
5 Principles governing the use of simulation . 7
5.1 General . 7
5.2 Verification of simulation tools. 8
5.3 User capabilities/qualification . 9
5.4 Verification and validation of simulation models . 9
5.4.1 Verification of models . 9
5.4.2 Validation of models . 9
5.5 Specific additional conditions for Hardware- and Software in the Loop . 10
5.6 Documentation when using simulations . 10
6 Guidance for technical assessors (acceptance of simulation results) . 10
7 Guidance for WG Convenors . 12
Annex A (informative) Examples where simulations have been substituted for physical
tests on the real system . 15
Annex B (informative) Example of replacement of physical testing on the real system by
simulation – aerodynamic pressures at the trackside . 17
B.1 Introduction . 17
B.2 Analysis of uncertainty and impact on output parameter Δp . 17
2
σ
Bibliography . 20
2
---------------------- Page: 4 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
European foreword
This document (CEN/TR 17833:2022) has been prepared by Technical Committee CEN/TC 256 “Railway
applications”, the secretariat of which is held by DIN.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
Any feedback and questions on this document should be directed to the users’ national standards body.
A complete listing of these bodies can be found on the CEN website.
3
---------------------- Page: 5 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
1 Scope
The aim of this document is to help CEN/CENELEC Working Group convenors and experts to
promote/develop simulation in their standards as an alternative to physical tests on the real system for
proving conformity. It can also provide useful guidance to assessors in the railway sector in approving
simulations where they are not yet specifically defined or where physical tests on the real system are not
defined in standards. Consequently, this document is also relevant to companies developing and applying
simulations with the intention to achieve their acceptance for the purpose of system validation. It is not
intended to provide technical guidance on applying simulations in general.
Where simulations are already introduced in existing standards, this guide is not intended to modify the
specified requirements. However, technical harmonisation between standards might benefit from this
guide for the introduction of additional alternative methods for simulations.
This document principally covers:
— numerical simulation, using complex methods or using simple spreadsheets methods;
— hardware and software in the loop;
— mathematical models solved using numerical methods or iteration, including spreadsheets.
It does not cover the following, although the general principles outlined can be applied to these methods:
— laboratory tests of components;
— fatigue rig tests;
— model scale tests;
— mathematical models solved analytically.
NOTE Due to the limited experience in the railway sector in the application of data-based (as opposed to model-
based) simulations, for example using artificial intelligence (AI), neural networks, big data, etc., this approach is not
further developed at this stage in this document.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
simulation (action and object)
(action) use of a similar or equivalent system to imitate a real system so that it behaves like or appears
to be the real system; (object) similar or equivalent system used to imitate a real system
Note 1 to entry: Simulation can be mathematical, analogue or scale modelling. Mathematical simulation includes
analytical and numerical calculation.
[SOURCE, ISO 16781:2013, 2.9, modified, Note 1 has been added.]
4
---------------------- Page: 6 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
3.2
numerical simulation
simulation based on numerical methods
3.3
test
technical operation that consists of applying to the object a set of environmental and operating conditions
under a specified procedure.
Note 1 to entry: A test can be carried out to determine one or more characteristics of a given object, process or
service according to a specified procedure. It can be used for different purposes (verifying requirements, calibration,
test cases and correct implementation of a model, etc.).
Note 2 to entry: A test can be conducted on the real system, or by entirely or partially using simulation
(simulation testing)
[SOURCE: IEV modified, ISO/IEC Guide 2 (13.1), notes 1 and 2 have been added]
3.4
simulation tool
in house, vendor or open source framework in which one may develop or embed models enabling the
execution of tests. It can be software and/or hardware and parts of the real system can be installed in the
tool.
3.5
model
mathematical and/or physical representation of a system or a process
3.6
numerical model
numerical representation of a mathematical model
3.7
verification (of simulation)
process of determining that a simulation in its tool environment produces expected results according to
the underlying model
3.8
validation (of simulation)
process of determining the degree to which a model is an accurate representation of the real system in
its environment
[ASME V&V 10 2006, modified] [ASME V&V 40 2018, modified]
3.9
system validation
process of proving conformity to system requirements, ensuring that the system is fit for its intended use
in its intended operational environment
5
---------------------- Page: 7 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
3.10
regression testing
testing required to determine that a change to a system (e.g. a model or a tool) has not adversely affected
functionality, reliability or performance and has not introduced additional defects
[SOURCE, ISO/IEC 27034-7:2018, 3.15, modified.]
3.11
environment
external aspects influencing the behaviour of a system
3.12
user
entity using the simulation
3.13
certification
third-party attestation related to components, sub-systems or systems
3.14
hardware in the loop simulation
type of simulation, in which some parts of the system or its environment are implemented or modelled by
real equipment
Note 1 to entry: Hardware in the loop simulations are characterised by a two-way coupling between the
simulated and the real components.
[SOURCE: IEC 16781:2013, 2.5, modified, note 1 added]
3.15
software in the loop simulation
type of simulation, in which a software that is executable on the real system is interfaced with simulation
models
4 Introduction
4.1 Background
The rationale for producing this document is the perception that physical testing on the real system for
train, infrastructure and command and control system certification leads to:
— excessive costs;
— delays bringing products to market.
The use of simulation is widespread in the automotive and the aerospace sectors, both for design and
validation. The challenge is for the European railway sector to examine its certification processes and
allow for the use of simulation methods as well as physical testing on the real system for system
validation, where it is possible and safe. In the majority of instances, the demonstration methods are
defined in CEN and CENELEC standards.
6
---------------------- Page: 8 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
At the JPC Rail (Sector Forum Rail) meeting in March 2018, the issue of promoting the use of simulation
within the Railway Sector was raised. In response, three steps were proposed by CEN:
a) to set up a survey group to identify and ideally respond to transversal questions and needs to support
WG Convenors and experts to introduce or to further define simulation requirements in their
standards. It was foreseen that the outcome of the Survey Group would be a preliminary issue of a
guide;
b) to urge current CEN and CENELEC Working Groups’ conveners and experts to consider either
introducing or further defining existing simulation requirements within standards under their
responsibilities;
c) to invite the ERA and the EC to promote simulation approaches throughout the regulatory framework
whenever possible.
4.2 Context
Historically, the demonstration of safety and conformity to standards for obtaining the certification of
rolling stock, fixed installations, control-command systems and infrastructure has been mainly based on
physical tests on the real system.
Although already widely used in the design and pre-validation of sub-systems, simulation is still relatively
rarely applied to improve and accelerate the system validation phase, where physical tests on the real
system are often required for compliance assessments.
There are several ways in which simulations can help improve the system validation phase. They can be
used to better understand certain phenomena, enabling experts to study/explore a wider range of cases
than those practicably covered by physical tests on the real system (which are limited by environment
parameters such as weather, geographical range and configuration, boundary conditions etc.), and hence
complement them. Another possibility is to use simulation in order to reduce the amount of physical
testing on the real system, and to reduce delays bringing products to market. Use of simulation should
not be limited to exploring system behaviour in fault free conditions; it may also be extended to consider
failures or degraded modes.
Driven by the increase in simulation quality and reliability, the trend of evolution in the railway sector is
towards the use of more simulation and to less physical testing on the real system. The regulations and
the standards set requirements which have to be fulfilled. These may stipulate compulsory physical
testing on the real system or leave significant room for simulation or fail to specify the method of
demonstration at all.
5 Principles governing the use of simulation
5.1 General
Simulation can be used fully or partially to prove conformity (see example 1 in Annex A). For full
simulation proofs, only results from simulation testing are used as the final means to prove conformity.
In partial simulation proofs, physical tests on the real system or parts of it are required for some test
cases.
It should be noted that, in most cases, the initial validation of the simulation model might require physical
tests on the real system or parts of it (see Subclause 5.4).
Especially for design evolutions, where physical tests on the real system have already been conducted in
a previous similar case, it may be feasible to fully prove conformity by simulation if changes to the system
subject to testing remain within certain limits. For changes exceeding these limits, a partial proof by
simulation can be feasible, (see example 2 in Annex A).
7
---------------------- Page: 9 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
NOTE It is also possible to fully prove conformity by simulation without starting from a pre existing design.
It should be ensured that the simulation tool is compatible with the intended purpose of the simulation
and that the models are representative of the real system in its environment. It is good practice to
evaluate qualitatively and quantitatively the uncertainty and the sensitivity of the models (more
information can be found in [1] and [2]). The user is responsible for critically interpreting the results
obtained.
Despite being separated in the following paragraphs, the concepts of tool verification, user capabilities
and model verification and validation are highly interconnected. Additional guidance can be found in
Clauses 6 and 7.
The simulation tool should be verified for the particular use to which it is being applied. Good practice
recommendations for the verification of simulation tools are given in Subclause 5.2.
Once the simulation tool is verified, it is necessary that measures are in place to ensure that the users of
the tools have the expertise and knowledge to apply them. Good practice recommendations for
demonstration of the skills necessary for users to use simulation tools and associated quality processes
are given in Subclause 5.3.
The primary purpose of validating a simulation model is to generate sufficient confidence in it, in order
to replace physical tests on the real system by simulations. This is further developed in Subclause 5.4.
Specific additional conditions for Hardware- and Software in the Loop are introduced in Subclause 5.5.
In Subclause 5.6, documentation requirements when using simulations are given.
5.2 Verification of simulation tools
Simulation tools consist of one or more components, which are interfaced in order to enable embedding
the model(s) of the system to be tested and possibly parts of the real system. Those components can be
software and/or hardware and their complexity may vary depending on the tool.
NOTE Software tools generally provide a library of tool-specific elementary models. They can range from
simple models, such as mathematical functions, to complex ones, such as a set of physics.
In order to determine that a simulation in its tool environment produces expected results according to
the underlying model, as a first step, the simulation tool should be verified for the particular use to which
it is being applied. For tools consisting of several components, each component should be verified
separately, and in combination, in order to verify their interfaces.
Verification should consider the accuracy, range of validity, boundary conditions and limitations of tool
components and their interfaces, which should be appropriate for the intended purpose of the simulation
tool.
The organisation performing tool component verification usually depends on its origin:
— for in-house simulation tools, it is the user organisation;
— for third-party simulation tools, it is the providing organisation;
— for open source simulation tools, it is, depending on the case, either an identified organisation or the
user organisation.
The verification process should also cover configuration and change management of tool components, for
example by performing regression testing during version changes and following updates to operating
systems and to host machines.
In every case, the user should verify the complete simulation tool by applying specific reference cases,
and checking the outputs against known results.
8
---------------------- Page: 10 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
The part of the verification process performed in the user organisation should be traced and documented.
It is good practice to maintain a log of user experiences of the tool.
5.3 User capabilities/qualification
As well as verifying the simulation tool for the intended use, measures should be in place to ensure that
the users of the simulations have the expertise and knowledge to set up and/or to apply them, and
critically analyse and interpret the results. This is the responsibility of the user organisation.
The user should ensure that the adopted simulation tool is fit for purpose.
NOTE The user in this context can refer to a range of different people.
It is good practice to verify the user skills for setting up and/or applying simulations, for instance, by
undertaking standard simulation test cases and recording skill levels.
Where relevant, it is good practice to maintain sufficient independence between designers and validation
experts, to ensure confidence in the results. The degree of independence should be based on the safety
impact of the intended use.
5.4 Verification and validation of simulation models
5.4.1 Verification of models
The second step of determining that a simulation in its tool environment produces expected results
involves verifying the consistency of the different choices of modelling and particularly checking:
— the individual sub models of the system and its environment;
— the full model of the system in its environment;
— the simulation method and its numerical convergence.
5.4.2 Validation of models
A validation process should demonstrate that the simulation model sufficiently represents a reference
system for different reference scenarios. This reference system might consist, preferably, of a physical
test on the real system, or of a reduced scale test or a generic case (e.g. analytical, benchmark, etc.). If a
model is an adaptation of, or similar to, a previously validated model, it may be possible to conduct a
reduced validation. In this case, the rationale for the reduced validation should be made and documented.
Generally, validation should be performed for the entire model, where necessary after validating
particular sub models individually.
The comparison between the simulation results and the reference system will give the level of confidence
in the simulation. The validation includes performing investigations concerning uncertainties (e.g.
accuracy, robustness and reproducibility). The domain of validity of the simulation should be given. (See
e.g. examples 3 and 4 in Annex A).
Best practice for validation involves proper consideration of the uncertainties in both the simulation and
in the reference system.
Validation of the simulation models should be documented as detailed in Subclause 5.6.
Particular attention should be paid to simulations of systems that have a safety function or a safety
impact.
9
---------------------- Page: 11 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
5.5 Specific additional conditions for Hardware- and Software in the Loop
The concept of 'Hardware in the Loop' (HiL) simulations is based on using real components or pieces of
equipment directly interfaced with numerical simulation models (see example 6 in Annex A). The system
and the environment are partly real and partly modelled. The real part of the system can be mechanical,
electrical, electronic, etc.
Interfaces between components are physical and virtual; numerical data being converted into electrical
signals and vice versa. The 'time step' of the simulation should be adapted to the reaction time of the
equipment being studied and should be consistent between the components; HiL simulation is
considered to be in 'real time'. This should be considered when performing verification and validation.
The concept of 'Software in the Loop' (SiL) simulations is based on running a software that is executable
on the real system interfaced with a simulated system and environment model. Interfaces between
components are virtual.
As the real components are neither part of the tool, nor of the model, it is not necessary to verify and
validate them as part of this process. However, special attention should be paid to the interfaces to SiL or
HiL.
5.6 Documentation when using simulations
The following elements should be included in the documentation:
— a summary of the verification and validation of the simulation (tool and models) including:
— the domain of validity of the simulation;
— proof of numerical convergence;
— a comparison of the simulation to a reference system for one or more reference scenarios as
defined in 5.4, including details such as the test conditions, the degree of precision/uncertainty
of the measurements and the representativeness of the boundary conditions considering the
uncertainties;
— other evidence supporting the validity of the simulation may be presented as an alternative;
— a statement confirming the validity of the simulation (tool and models) for the given application;
— a statement confirming the qualification and capabilities of the user;
— a simulation report containing detailed information on the simulated scenarios and analyses of their
results, including:
— a statement of conformance of the system to its requirements (e.g. as defined in standards);
— a statement that the simulation was executed in its domain of validity.
It is good practice to document in detail the different steps in performing the simulation.
6 Guidance for technical assessors (acceptance of simulation results)
If the simulation tool and the user's aptitude are verified and the simulation models have been validated,
the simulation results can be considered as representative for the behaviour of the real system within the
scope of application specified.
10
---------------------- Page: 12 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
For simulation results to be acknowledged as proof of compliance, either the applicable reference
documents and/or standards define the conditions in which simulation can be used, or they do not
directly deal with the use of simulations.
In the first case, if the standards and/or regulatory requirements have been respected, then the
simulation results can be accepted.
11
---------------------- Page: 13 ----------------------
SIST-TP CEN/TR 17833:2022
CEN/TR 17833:2022 (E)
In the second case:
— there may be generally accepted principles in fields where the knowledge and tools available provide
a sufficient degree of reliability to dispense with physical tests on the real system, allowing the
simulation results to be accepted; such principles should be documented;
— in other cases, a risk analysis (e.g. as outlined in CSM-RA [3]) should be conducted with an emphasis
on the following considerations:
— scope of the simulation: can the simulation cover all of the conformity demonstrations required
in the physical test on the real system in question or is it to be considered in conjunction with
results of other physical tests on the real system in cases involving complex configurations (e.g.
fail-safe modes, excess speed, values measured, etc.)?
— criticality of the simulation: The safety considerations associated with each demonstration need
to be specified. While the evidence provided should still comply with the applicable reference
standards, the degree of reliability required from simulation performed to establish specific
parameters may differ depending on the safety level associated with the parameter considered.
EXAMPLE Reliability of simulations for noise levels may be different from that required of a simulation of
braking behaviour.
7 Guidance for WG Convenors
This Clause gives particular guidance for CEN/CENELEC Working Group (WG) convenors when they are
considering and introducing simulation as an alternative to physical tests on the real system in their
standards. The following are general considerations to take into account, but are not exhaustive.
1) Clearly identify the reasons for introducing simulation in the standard.
— e.g. reduction in time/cost, difficulty of physical testing on the real system, e.g. in failure modes.
— It should be noted that insufficient maturity in simulation at the time should not impede the WG
from considering the introduction of simulation within the standard.
2) Assess if the WG could benefit from external expertise in the field of simulation. If necessary, WG
members who currently use simulation can perform a common benchmark simulation to establish
best practice, and to have a basis for comparing different simulation tools and different users.
Alternatively, known experts in the field can be invited to contribute as guests or as members (e.g.
by opening up to universities, other research institutions, etc.), thus broadening the expertise in the
WG.
3) Use the principles outlined in Clause 5 to define the requir
...
SLOVENSKI STANDARD
kSIST-TP FprCEN/TR 17833:2022
01-april-2022
Železniške naprave - Navodilo za uporabo simulacij - Navodilo o uporabi simulacij
za dokazovanje skladnosti s tehničnimi in regulativnimi zahtevami ter o vnašanju
in razvoju simulacijskih zahtev v standarde
Railway applications - Guidance for the use of simulations - Guidance for the use of
simulations to demonstrate compliance with technical and regulatory requirements and
on the introduction and development of simulation requirements into standards
Bahnanwendungen - Leitfaden für den Einsatz von Simulationen - Leitfaden für den
Einsatz von Simulationen zum Nachweis der Einhaltung technischer und regulatorischer
Anforderungen sowie zur Einführung und Entwicklung von Simulationsanforderungen in
Normen
Ta slovenski standard je istoveten z: FprCEN/TR 17833
ICS:
01.120 Standardizacija. Splošna Standardization. General
pravila rules
45.020 Železniška tehnika na Railway engineering in
splošno general
kSIST-TP FprCEN/TR 17833:2022 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
kSIST-TP FprCEN/TR 17833:2022
---------------------- Page: 2 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FINAL DRAFT
TECHNICAL REPORT
FprCEN/TR 17833
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
January 2022
ICS
English Version
Railway applications - Guidance for the use of simulations
- Guidance for the use of simulations to demonstrate
compliance with technical and regulatory requirements
and on the introduction and development of simulation
requirements into standards
Bahnanwendungen - Leitfaden für den Einsatz von
Simulationen - Leitfaden für den Einsatz von
Simulationen zum Nachweis der Einhaltung
technischer und regulatorischer Anforderungen sowie
zur Einführung und Entwicklung von
Simulationsanforderungen in Normen
This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee CEN/TC
256.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are
aware and to provide supporting documentation.
Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without
notice and shall not be referred to as a Technical Report.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2022 CEN All rights of exploitation in any form and by any means reserved Ref. No. FprCEN/TR 17833:2022 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
Contents Page
European foreword . 3
1 Scope . 4
2 Normative references . 4
3 Terms and definitions . 4
4 Introduction . 6
4.1 Background . 6
4.2 Context . 7
5 Principles governing the use of simulation . 7
5.1 General . 7
5.2 Verification of simulation tools. 8
5.3 User capabilities/qualification . 9
5.4 Verification and validation of simulation models . 9
5.4.1 Verification of models . 9
5.4.2 Validation of models . 9
5.5 Specific additional conditions for Hardware- and Software in the Loop . 9
5.6 Documentation when using simulations . 10
6 Guidance for technical assessors (acceptance of simulation results) . 10
7 Guidance for WG Convenors . 11
Annex A (informative) Examples where simulations have been substituted for physical
tests on the real system . 15
Annex B (informative) Example of replacement of physical testing on the real system by
simulation – aerodynamic pressures at the trackside . 17
B.1 Introduction . 17
B.2 Analysis of uncertainty and impact on output parameter Δp2σ . 17
Bibliography . 19
2
---------------------- Page: 4 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
European foreword
This document (FprCEN/TS 17833:2022) has been prepared by Technical Committee CEN/TC 256
“Railway applications”, the secretariat of which is held by DIN.
This document is currently submitted to the Vote on TR.
3
---------------------- Page: 5 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
1 Scope
The aim of this document is to help CEN/CENELEC Working Group convenors and experts to
promote/develop simulation in their standards as an alternative to physical tests on the real system for
proving conformity. It can also provide useful guidance to assessors in the railway sector in approving
simulations where they are not yet specifically defined or where physical tests on the real system are not
defined in standards. Consequently, this document is also relevant to companies developing and applying
simulations with the intention to achieve their acceptance for the purpose of system validation. It is not
intended to provide technical guidance on applying simulations in general.
Where simulations are already introduced in existing standards, this guide is not intended to modify the
specified requirements. However, technical harmonisation between standards might benefit from this
guide for the introduction of additional alternative methods for simulations.
This document principally covers:
— numerical simulation, using complex methods or using simple spreadsheets methods;
— hardware and software in the loop;
— mathematical models solved using numerical methods or iteration, including spreadsheets.
It does not cover the following, although the general principles outlined can be applied to these methods:
— laboratory tests of components;
— fatigue rig tests;
— model scale tests;
— mathematical models solved analytically.
NOTE Due to the limited experience in the railway sector in the application of data-based (as opposed to model-
based) simulations, for example using artificial intelligence (AI), neural networks, big data, etc., this approach is not
further developed at this stage in this document.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
simulation (action and object)
(action) use of a similar or equivalent system to imitate a real system so that it behaves like or appears
to be the real system; (object) similar or equivalent system used to imitate a real system
Note 1 to entry: Simulation can be mathematical, analogue or scale modelling. Mathematical simulation includes
analytical and numerical calculation.
[SOURCE, ISO 16781:2013, 2.9, modified, Note 1 has been added.]
4
---------------------- Page: 6 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
3.2
numerical simulation
simulation based on numerical methods
3.3
test
technical operation that consists of applying to the object a set of environmental and operating conditions
under a specified procedure.
Note 1 to entry: A test can be carried out to determine one or more characteristics of a given object, process or
service according to a specified procedure. It can be used for different purposes (verifying requirements, calibration,
test cases and correct implementation of a model, etc.).
Note 2 to entry: A test can be conducted on the real system, or by entirely or partially using simulation
(simulation testing)
[SOURCE: IEV modified, ISO/IEC Guide 2 (13.1), notes 1 and 2 have been added]
3.4
simulation tool
in house, vendor or open source framework in which one may develop or embed models enabling the
execution of tests. It can be software and/or hardware and parts of the real system can be installed in the
tool.
3.5
model
mathematical and/or physical representation of a system or a process
3.6
numerical model
numerical representation of a mathematical model
3.7
verification (of simulation)
process of determining that a simulation in its tool environment produces expected results according to
the underlying model
3.8
validation (of simulation)
process of determining the degree to which a model is an accurate representation of the real system in
its environment
[ASME V&V 10 2006, modified] [ASME V&V 40 2018, modified]
3.9
system validation
process of proving conformity to system requirements, ensuring that the system is fit for its intended use
in its intended operational environment
5
---------------------- Page: 7 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
3.10
regression testing
testing required to determine that a change to a system (e.g. a model or a tool) has not adversely affected
functionality, reliability or performance and has not introduced additional defects
[SOURCE, ISO/IEC 27034-7:2018, 3.15, modified.]
3.11
environment
external aspects influencing the behaviour of a system
3.12
user
entity using the simulation
3.13
certification
third-party attestation related to components, sub-systems or systems
3.14
hardware in the loop simulation
type of simulation, in which some parts of the system or its environment are implemented or modelled by
real equipment
Note 1 to entry: Hardware in the loop simulations are characterised by a two-way coupling between the
simulated and the real components.
[SOURCE: IEC 16781:2013, 2.5, modified, note 1 added]
3.15
software in the loop simulation
type of simulation, in which a software that is executable on the real system is interfaced with simulation
models
4 Introduction
4.1 Background
The rationale for producing this document is the perception that physical testing on the real system for
train, infrastructure and command and control system certification leads to:
— excessive costs;
— delays bringing products to market.
The use of simulation is widespread in the automobile and the aerospace sectors, both for design and
validation. The challenge is for the European railway sector to examine its certification processes and
allow for using simulation methods as well as physical testing on the real system for system validation,
where it is possible and safe. In the majority of instances, the demonstration methods are defined in CEN
and CENELEC standards.
6
---------------------- Page: 8 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
At the JPC Rail (Sector Forum Rail) meeting in March 2018, the issue of promoting the use of simulation
within the Railway Sector was raised. In response, three steps were proposed by CEN:
a) to set up a survey group to identify and ideally respond to transversal questions and needs to support
WG Conveners and experts to introduce or to further define simulation requirements in their
standards. It was foreseen that the outcome of the Survey Group would be a preliminary issue of a
guide;
b) to urge current CEN and CENELEC Working Groups’ conveners and experts to consider either
introducing or further defining existing simulation requirements within standards under their
responsibilities;
c) to invite the ERA and the EC to promote simulation approaches throughout the regulatory framework
whenever possible.
4.2 Context
Historically, the demonstration of safety and conformity to standards for obtaining the certification of
rolling stock, fixed installations, control-command systems and infrastructure has been mainly based on
physical tests on the real system.
Although already widely used in the design and pre-validation of sub-systems, simulation is still relatively
rarely applied to improve and accelerate the system validation phase, where physical tests on the real
system are often required for compliance assessments.
There are several ways in which simulations can help improve the system validation phase. They can be
used to better understand certain phenomena, enabling experts to study/explore a wider range of cases
than those practicably covered by physical tests on the real system (which are limited by environment
parameters such as weather, geographical range and configuration, boundary conditions etc.), and hence
complement them. Another possibility is to use simulation in order to reduce the amount of physical
testing on the real system, and to reduce delays bringing products to market. Use of simulation should
not be limited to exploring system behaviour in fault free conditions; it may also be extended to consider
failures or degraded modes.
Driven by the increase in simulation quality and reliability, the trend of evolution in the railway sector is
towards the use of more simulation and to less physical testing on the real system. The regulations and
the standards set requirements which have to be fulfilled. These may stipulate compulsory physical
testing on the real system or leave significant room for simulation or fail to specify the method of
demonstration at all.
5 Principles governing the use of simulation
5.1 General
Simulation can be used fully or partially to prove conformity (see example 1 in Annex A). For full
simulation proofs, only results from simulation testing are used as the final means to prove conformity.
In partial simulation proofs, physical tests on the real system or parts of it are required for some test
cases.
It should be noted that, in most cases, the initial validation of the simulation model might require physical
tests on the real system or parts of it (see Subclause 5.4).
Especially for design evolutions, where physical tests on the real system have already been conducted in
a previous similar case, it may be feasible to fully prove conformity by simulation if changes to the system
subject to testing remain within certain limits. For changes exceeding these limits, a partial proof by
simulation can be feasible, (see example 2 in Annex A).
7
---------------------- Page: 9 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
NOTE It is also possible to fully prove conformity by simulation without starting from a pre existing design.
It should be ensured that the simulation tool is compatible with the intended purpose of the simulation
and that the models are representative of the real system in its environment. It is good practice to
evaluate qualitatively and quantitatively the uncertainty and the sensitivity of the models (more
information can be found in [1] and [2]). The user is responsible for critically interpreting the results
obtained.
Despite being separated in the following paragraphs, the concepts of tool verification, user capabilities
and model verification and validation are highly interconnected. Additional guidance can be found in
Clauses 7 and 8.
The simulation tool should be verified for the particular use to which it is being applied. Good practice
recommendations for the verification of simulation tools are given in Subclause 5.2.
Once the simulation tool is verified, it is necessary that measures are in place to ensure that the users of
the tools have the expertise and knowledge to apply them. Good practice recommendations for
demonstration of the skills necessary for users to use simulation tools and associated quality processes
are given in Subclause 5.3.
The primary purpose of validating a simulation model is to generate sufficient confidence in it, in order
to replace physical tests on the real system by simulations. This is further developed in Subclause 5.4.
Specific additional conditions for Hardware- and Software in the Loop are introduced in Subclause 5.5.
In Subclause 5.6, documentation requirements when using simulations are given.
5.2 Verification of simulation tools
Simulation tools consist of one or more components, which are interfaced in order to enable embedding
the model(s) of the system to be tested and possibly parts of the real system. Those components can be
software and/or hardware and their complexity may vary depending on the tool.
NOTE Software tools generally provide a library of tool-specific elementary models. They can range from
simple models, such as mathematical functions, to complex ones, such as a set of physics.
In order to determine that a simulation in its tool environment produces expected results according to
the underlying model, as a first step, the simulation tool should be verified for the particular use to which
it is being applied. For tools consisting of several components, each component should be verified
separately, and in combination, in order to verify their interfaces.
Verification should consider the accuracy, range of validity, boundary conditions and limitations of tool
components and their interfaces, which should be appropriate for the intended purpose of the simulation
tool.
The organisation performing tool component verification usually depends on its origin:
— for in-house simulation tools, it is the user organisation;
— for third-party simulation tools, it is the providing organisation;
— for open source simulation tools, it is, depending on the case, either an identified organisation or the
user organisation.
The verification process should also cover configuration and change management of tool components, for
example by performing regression testing during version changes and following updates to operating
systems and to host machines.
In every case, the user should verify the complete simulation tool by applying specific reference cases,
and checking the outputs against known results.
The part of the verification process performed in the user organisation should be traced and documented.
8
---------------------- Page: 10 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
It is good practice to maintain a log of user experiences of the tool.
5.3 User capabilities/qualification
As well as verifying the simulation tool for the intended use, measures should be in place to ensure that
the users of the simulations have the expertise and knowledge to set up and/or to apply them, and
critically analyse and interpret the results. This is the responsibility of the user organisation.
The user should ensure that the adopted simulation tool is fit for purpose.
NOTE The user in this context can refer to a range of different people.
It is good practice to verify the user skills for setting up and/or applying simulations, for instance, by
undertaking standard simulation test cases and recording skill levels.
Where relevant, it is good practice to maintain sufficient independence between designers and validation
experts, to ensure confidence in the results. The degree of independence should be based on the safety
impact of the intended use.
5.4 Verification and validation of simulation models
5.4.1 Verification of models
The second step of determining that a simulation in its tool environment produces expected results
involves verifying the consistency of the different choices of modelling and particularly checking:
— the individual sub models of the system and its environment;
— the full model of the system in its environment;
— the simulation method and its numerical convergence.
5.4.2 Validation of models
A validation process should demonstrate that the simulation model sufficiently represents a reference
system for different reference scenarios. This reference system might consist, preferably, of a physical
test on the real system, or of a reduced scale test or a generic case (e.g. analytical, benchmark, etc.). If a
model is an adaptation of, or similar to, a previously validated model, it may be possible to conduct a
reduced validation. In this case, the rationale for the reduced validation should be made and documented.
Generally, validation should be performed for the entire model, where necessary after validating
particular sub models individually.
The comparison between the simulation results and the reference system will give the level of confidence
in the simulation. The validation includes performing investigations concerning uncertainties (e.g.
accuracy, robustness and reproducibility). The domain of validity of the simulation should be given. (See
e.g. examples 3 and 4 in Annex A).
Best practice for validation involves proper consideration of the uncertainties in both the simulation and
in the reference system.
Validation of the simulation models should be documented as detailed in Subclause 5.6.
Particular attention should be paid to simulations of systems that have a safety function or a safety
impact.
5.5 Specific additional conditions for Hardware- and Software in the Loop
The concept of 'Hardware in the Loop' (HiL) simulations is based on using real components or pieces of
equipment directly interfaced with numerical simulation models (see example 5 in Annex A). The system
and the environment are partly real and partly modelled. The real part of the system can be mechanical,
electrical, electronic, etc.
9
---------------------- Page: 11 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
Interfaces between components are physical and virtual; numerical data being converted into electrical
signals and vice versa. The 'time step' of the simulation should be adapted to the reaction time of the
equipment being studied and should be consistent between the components; HiL simulation is
considered to be in 'real time'. This should be considered when performing verification and validation.
The concept of 'Software in the Loop' (SiL) simulations is based on running a software that is executable
on the real system interfaced with a simulated system and environment model. Interfaces between
components are virtual.
As the real components are neither part of the tool, nor of the model, it is not necessary to verify and
validate them as part of this process. However, special attention should be paid to the interfaces to SiL or
HiL.
5.6 Documentation when using simulations
The following elements should be included in the documentation:
— a summary of the verification and validation of the simulation (tool and models) including:
— the domain of validity of the simulation;
— proof of numerical convergence;
— a comparison of the simulation to a reference system for one or more reference scenarios as
defined in 5.4, including details such as the test conditions, the degree of precision/uncertainty
of the measurements and the representativeness of the boundary conditions considering the
uncertainties;
— other evidence supporting the validity of the simulation may be presented as an alternative;
— a statement confirming the validity of the simulation (tool and models) for the given application;
— a statement confirming the qualification and capabilities of the user;
— a simulation report containing detailed information on the simulated scenarios and analyses of their
results, including:
— a statement of conformance of the system to its requirements (e.g. as defined in standards);
— a statement that the simulation was executed in its domain of validity.
It is good practice to document in detail the different steps in performing the simulation.
6 Guidance for technical assessors (acceptance of simulation results)
If the simulation tool and the user's aptitude are verified and the simulation models have been validated,
the simulation results can be considered as representative for the behaviour of the real system within the
scope of application specified.
For simulation results to be acknowledged as proof of compliance, either the applicable reference
documents and/or standards define the conditions in which simulation can be used, or they do not
directly deal with the use of simulations. In the first case, if the standards and/or regulatory requirements
have been respected, then the simulation results can be accepted.
In the second case:
— there may be generally accepted principles in fields where the knowledge and tools available provide
a sufficient degree of reliability to dispense with physical tests on the real system, allowing the
simulation results to be accepted; such principles should be documented;
10
---------------------- Page: 12 ----------------------
kSIST-TP FprCEN/TR 17833:2022
FprCEN/TS 17833:2022 (E)
— in other cases, a risk analysis (e.g. as outlined in CSM-RA [3]) should be conducted with an emphasis
on the following considerations:
— scope of the simulation: can the simulation cover all of the conformity demonstrations required
in the physical test on the real system in question or is it to be considered in conjunction with
results of other physical tests on the real system in cases involving complex configurations (e.g.
fail-safe modes, excess speed, values measured, etc.)?
— criticality of the simulation: The safety considerations associated with each demonstration need
to be specified. While the evidence provided should still comply with the applicable reference
standards, the degree of reliability required from simulation performed to establish specific
parameters may differ depending on the safety level associated with the parameter considered.
EXAMPLE Reliability of simulations for noise levels may be different from that required of a simulation
of braking behaviour.
7 Guidance for WG Convenors
This Clause gives particular guidance for CEN/CENELEC Working Group (WG) convenors when they are
considering and introducing simulation as an alternative to physical tests on the real system in their
standards. The following are general considerations to take into account, but are not exhaustive.
1) Clearly identify the reasons for introducing simulation in the standard.
— e.g. reduction in time/cost, difficulty of physical testing on the real system, e.g. in failure
modes.
— It should be noted that insufficient maturity in simulation at the time should not impede the
WG from considering the introduction of simulation within the standard.
2) Assess if the WG could benefit from external expertise in the field of simulation. If necessary, WG
members who currently use simulation can perform a common benchmark simulation to
establish best practice, and to have a basis for comparing different simulation tools and different
users. Alternatively, known experts in the field can be invited to contribute as guests or as
members (e.g. by opening up to universities, other research institutions, etc.), thus broadening
the expertise in the WG.
3) Use the princ
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.