SIST-TP CEN/CLC/TR 17603-10-02:2021
(Main)Space engineering - Verification guidelines
Space engineering - Verification guidelines
This handbook provides additional information for the application of the verification standard EN 16603-10-02 to a space system product.
This handbook does not contain requirements and therefore cannot be made applicable. In case of conflict betw een the standard and this handbook, the standard prevails.
This handbook is relevant for both the customer and the supplier of the product during all project phases.
To facilitate the cross-reference, this handbook follow s as much as is practical, the structure of the standard and quotes the requirements, to make itself standing and easier to read (the text from the standard is in italic).
As the Standard applies to different products at different product levels from single equipment to the overall system (including space segment hardw are and softw are, launchers and Transportation Systems, ground segment, Verification tools, and GSE) several examples of tailoring, to match the specificity of each application, are proposed in Annex B.
Specific discipline related verification aspects are covered in other dedicated standards and handbooks. In particular the detailed aspects for Testing are covered in the EN 16603-10-03 and in its corresponding handbook.
The application of the requirements of the standard to a particular project is intended to result in effective product
verification and consequently to a high confidence in achieving successful product operations for the intended use, in this respect this handbook has the goal to help reaching these objectives.
Raumfahrttechnik - Leitfaden zur Verifikation
Ingénierie spatiale - Lignes directrices pour la vérification
Vesoljska tehnika - Smernice za preverjanje
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
01-november-2021
Vesoljska tehnika - Smernice za preverjanje
Space engineering - Verification guidelines
Raumfahrttechnik - Leitfaden zur Verifikation
Ingénierie spatiale - Lignes directrices pour la vérification
Ta slovenski standard je istoveten z: CEN/CLC/TR 17603-10-02:2021
ICS:
49.140 Vesoljski sistemi in operacije Space systems and
operations
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
TECHNICAL REPORT
CEN/CLC/TR 17603-10-
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
September 2021
ICS 49.140
English version
Space engineering - Verification guidelines
Ingénierie spatiale - Lignes directrices pour la Raumfahrttechnik - Leitfaden zur Verifikation
vérification
This Technical Report was approved by CEN on 19 March 2021. It has been drawn up by the Technical Committee CEN/CLC/JTC
5.
CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,
Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,
Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,
Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.
CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels
© 2021 CEN/CENELEC All rights of exploitation in any form and by any means Ref. No. CEN/CLC/TR 17603-10-02:2021 E
reserved worldwide for CEN national Members and for
CENELEC Members.
Table of contents
European Foreword . 5
1 Scope . 6
2 References . 7
3 Terms, definitions and abbreviated terms . 8
3.1 Terms from other documents . 8
3.2 Terms specific to the present handbook . 8
3.3 Abbreviated terms. 9
4 Verification principles . 12
4.1 Introduction . 12
4.2 Verification versus Validation . 12
4.3 Applicability to all engineering domains . 13
4.4 Development . 13
5 Verification guidelines . 14
5.1 Verification process . 14
5.2 Verification planning . 14
5.2.1 Verification approach . 14
5.2.2 Verification methods . 19
5.2.3 Verification levels . 23
5.2.4 Verification stages . 24
5.2.5 Models and Models Description . 27
5.2.6 Verification tools . 42
5.2.7 Verification process phasing . 44
5.3 Verification execution and reporting . 51
5.3.1 General . 51
5.3.2 Example of verification team responsibility and interfaces . 51
5.4 Verification control and close-out . 53
5.4.1 General . 53
5.4.2 Verification control board (VCB) . 54
5.4.3 Re-verification . 54
6 Verification documentation . 55
6.1 Introduction . 55
6.2 Verification planning documents . 57
6.2.1 Verification plan (VP) . 57
6.2.2 Verification control document (VCD) . 64
6.2.3 Other verification planning Documents . 67
6.3 Verification execution and reporting documentation . 68
6.3.1 Test report (TRPT) . 68
6.3.2 Analysis report (ARPT) . 70
6.3.3 Review-of-design report (RRPT) . 71
6.3.4 Inspection report (IRPT) . 73
6.3.5 Verification report (VRPT) . 75
6.3.6 VRPT DRD explanation . 76
6.3.7 Other verification execution and reporting Document . 77
6.3.8 Other close-out documents . 79
Annex A Verification documents delivery per review . 80
Annex B Verification Standard Tailoring . 81
Figures
Figure 5-1: Basic verification approach . 16
Figure 5-2: Parameters for Model Philosophy definition. 34
Figure 5-3: Example of Unmanned project model philosophy . 36
Figure 5-4: Example of Manned project model philosophy . 37
Figure 5-5: Example of Protoflight model philosophy . 38
Figure 5-6: Example of Hybrid model philosophy. 40
Figure 5-7: Example of verification process phasing with the project life cycle . 45
Figure 5-8: Verification activities flow (Phases A/B) . 48
Figure 5-9: Verification activities flow (Phases C/D) . 49
Figure 5-10: Verification activities flow (Phases E/F) . 50
Figure 6-1: Verification documentation . 56
Figure 6-2: Example of Verification Strategies per Group/level . 59
Figure 6-3: Example of verification strategy for a single Requirement Group . 60
Figure 6-4: Example of verification planning . 61
Figure 6-5: Example of activity sheet for analysis programme . 62
Figure 6-6: Example of Activity Sheet for Integration and Test Programme . 63
Figure 6-7: Example of the close-out status table . 66
Figure 6-8: Example of VCD sheet . 67
Figure 6-9: Example of test report sheet . 70
Figure 6-10: Example of an analysis report sheet . 71
Figure 6-11: Example of review-of-design report sheet. 73
Figure 6-12: Example of an inspection report sheet . 75
Figure 6-13: Example of verification report sheet. 77
Tables
Table 5-1: Product categories according to heritage . 25
Table 5-2 : Summary model definitions . 32
Table 5-3 : Example of a product matrix as viewed with a satellite perspective . 41
Table B-1 : Tailoring guidelines and some examples per product type . 82
European Foreword
This document (CEN/CLC/TR 17603-10-02:2021) has been prepared by Technical Committee
CEN/CLC/JTC 5 “Space”, the secretariat of which is held by DIN.
It is highlighted that this technical report does not contain any requirement but only collection of data
or descriptions and guidelines about how to organize and perform the work in support of EN 16603-
10-02.
This Technical report CEN/CLC/(TR 17603-10-02:2021) originates from ECSS-E-HB-10-02A.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN [and/or CENELEC] shall not be held responsible for identifying any or all such
patent rights.
This document has been prepared under a mandate given to CEN by the European Commission and
the European Free Trade Association.
This document has been developed to cover specifically space systems and has therefore precedence
over any TR covering the same scope but with a wider domain of applicability (e.g.: aerospace).
Scope
This handbook provides additional information for the application of the verification standard EN
16603-10-02 to a space system product.
This handbook does not contain requirements and therefore cannot be made applicable. In case of
conflict between the standard and this handbook, the standard prevails.
This handbook is relevant for both the customer and the supplier of the product during all project
phases.
To facilitate the cross-reference, this handbook follows as much as is practical, the structure of the
standard and quotes the requirements, to make it self standing and easier to read (the text from the
standard is in italic).
As the Standard applies to different products at different product levels from single equipment to the
overall system (including space segment hardware and software, launchers and Transportation
Systems, ground segment, Verification tools, and GSE) several examples of tailoring, to match the
specificity of each application, are proposed in Annex B.
Specific discipline related verification aspects are covered in other dedicated standards and
handbooks. In particular the detailed aspects for Testing are covered in the EN 16603-10-03 and in its
corresponding handbook TR 17603-10-03.
The application of the requirements of the standard to a particular project is intended to result in
effective product verification and consequently to a high confidence in achieving successful product
operations for the intended use, in this respect this handbook has the goal to help reaching these
objectives.
References
This document is the handbook corresponding to the Verification standard ECSS-E-ST-10-02C.
The following documents are referenced in this text or provide additional information useful for the
reader.
EN Reference Reference in text Title
EN 16601-00-01 ECSS-S-ST-00-01 ECSS system - Glossary of terms
EN 16603-10 ECSS-E-ST-10 Space engineering - System engineering
general requirements
EN 16603-10-02 ECSS-E-ST-10-02 Space engineering - Verification
EN 16603-10-03 ECSS-E-ST-10-03 Space engineering - Testing
EN 16603-40 ECSS-E-ST-40 Space engineering - Software
EN 16603-50 ECSS-E-ST-50 Space engineering - Communications
EN 16603-70 ECSS-E-ST-70 Space engineering - Ground systems and
operations
TR 16703-10-03 ECSS-E-HB-10-03 Space engineering - Testing guidelines
- ECSS-E-TM-10-21 Space engineering - System modelling and
simulation
EN 16601-10 ECSS-M-ST-10 Space project management - Project planning
and implementation.
EN 16602-10-09 ECSS-Q-ST-10-09 Space product assurance - Nonconformance
control system.
EN 16602-20 ECSS-Q-ST-20 Space product assurance - Quality assurance.
EN 16602-20-07 ECSS-Q-20-07 Space product assurance - Quality assurance
for test centres.
EN 16602-40 ECSS-Q-ST-40 Space product assurance - Safety.
EN 16602-60 ECSS-Q-ST-60 Space product assurance - Electrical,
electronic and electromechanical (EEE)
components.
EN 16602-70 ECSS-Q-ST-70 Space product assurance - Materials,
mechanical parts and processes.
Terms, definitions and abbreviated terms
3.1 Terms from other documents
For the purpose of this document, the terms and definitions from ECSS-ST-00-01 apply, in particular
for the following terms:
validation
verification
3.2 Terms specific to the present handbook
3.2.1 acceptance stage
verification stage with the objective of demonstrating that the product is free of workmanship defects,
is in accordance with the qualified design and is ready for its intended use
3.2.2 analysis
verification method performing a theoretical or empirical evaluation using techniques agreed with the
customer
NOTE The selected techniques can typically include statistics, qualitative
design analysis, modelling and computer simulation.
3.2.3 commissioning
verification and validation activities conducted after the launch and before the entry in operational
service either on the space elements only or on the overall system (including the ground elements)
3.2.4 in-orbit stage
verification stage valid for projects for which in-orbit verification is performed, including the
commissioning and verification activities which are delayed because the activation of a space element
is performed later during the mission (e.g. for Interplanetary mission, lander).
3.2.5 inspection
verification method by visual determination of physical characteristics
NOTE 1 Product characteristics include constructional features, hardware
conformance to document drawing or workmanship requirements,
physical conditions, software source code conformance with
coding standards
NOTE 2 See also ECSS-ST-00-01.
3.2.6 model philosophy
definition of the optimum number and the characteristics of physical models required to achieve
confidence in the product verification with the shortest planning and a suitable weighing of costs and
risks
3.2.7 post-landing stage
verification stage valid for projects for which post-landing verification is performed (e.g. for
Multimission projects)
3.2.8 pre-launch stage
verification stage with the objective to verify that the flight article is properly configured for launch
and capable of functioning as planned for launch
3.2.9 qualification stage
verification stage with the objective to demonstrate that the design fulfils the applicable requirements
including proper margins
3.2.10 review-of-design
verification method using approved records or evidence that unambiguously show that the
requirement is met (e.g. using design documents, design reports, technical descriptions, engineering
drawings)
3.2.11 test
verification method by measurement of product performance and functions under representative
simulated environments
NOTE See also ECSS-ST-00-01.
3.2.12 Verification Control Board (VCB)
a board composed of customer and supplier representatives that monitors the verification process and
formally assesses the requirements verification close-out.
3.2.13 verification level
product architectural level at which the relevant verification is performed
3.3 Abbreviated terms
The following abbreviated terms are used within this document:
Abbreviation Meaning
AIT assembly, integration and test
AITP assembly, integration and test plan
AIV assembly, integration and verification
AIVP assembly, integration and verification plan
AOCS attitude and orbit control system
Abbreviation Meaning
AR acceptance review
ARPT analysis report
BB Breadboard
CDR critical design review
CRR commissioning result review
CP commissioning plan
DM development model
DRD document requirements definition
ECSS European Cooperation for Space Standardization
EEE electronic electrical and electromechanical
EIDP end item data package
ELR End of Life Review
EM engineering model
EMC electromagnetic compatibility
EOL end-of-life
EQM engineering qualification model
FM flight model
FMECA failure mode effects and criticality analysis
FRR flight readiness review
FS flight spare
GPS global positioning system
GSE ground support equipment
H/W Hardware
HFE human factors engineering
I/F Interface
IM integration model
IRPT inspection report
ISO International Organisation for Standardisation
LRR launch readiness review
LTM Life Test Model
MU mock-up
NCR Non conformance report
NRB Non conformance review board
OBDH on-board data handling
Abbreviation Meaning
ORR Operations Readiness Review
P/L Payload
PDR preliminary design review
PFM protoflight model
PRR preliminary requirement review
PTR post test review
QA quality assurance
QM qualification model
QR qualification review
RCS reaction control system
RF radio frequency
RFW request for waiver
ROD review of design
RRPT review of design report
S/C spacecraft
S/W software
SM structural model
SRR system requirements review
SS subsystem
STM structural-thermal model
SVF software validation facility
TCL test configuration list
ThM thermal model
TPRO Test Procedure
TRR test readiness review
TRPT test report
TSPE Test Specification
TT&C telemetry, tracking and command
VCB verification control board
VCD verification control document
VP verification plan
VRPT verification report
Verification principles
4.1 Introduction
ECSS-E-ST-10 states that verification demonstrates, through a dedicated process, that the deliverable
system meets the specified requirements and is capable of sustaining its operational role during the
project life cycle.
ECSS-E-ST-10-02 establishes the requirements for the verification of a space system product. It
specifies the fundamental concepts of the verification process, the criteria for defining the verification
strategy and the requirements for the implementation of the verification programme. It is intended to
apply to different products at different levels, from single equipment to the overall system (including
space segment hardware and software, ground segment, launchers and transportation systems,
Verification tools and GSE).
Concerning the scope of the standard, it is useful to address at this point some frequently asked
questions posed by users, in order to emphasize certain concepts and definitions imposed by higher
level standards and by the accepted European practices enshrined within the standard.
4.2 Verification versus Validation
A question often posed is why, within European space projects, we mandate a “verification”
programme as opposed to a “verification and validation” programme, as practiced in other
engineering disciplines (e.g. software, ground segment).
In general terms verification addresses whether a product satisfies the requirements placed upon it,
whilst validation addresses whether a product will satisfy the needs of its users, or as is often more
simply said,
Verification proves the product is right.
Validation proves it is the right product.
The Verification Standard does not mandate the need for a separate programme of validation of space
products, since product verification is performed against a set of requirements that also address the
suitability of the product to fulfil the needs of its intended use. However, the standard does not
prevent the execution of a separate validation activity if this is considered appropriate, as is practiced
for example, in the operation or ground segment domains. Essentially the process to be followed is the
same, although it addresses mainly the use of the product.
4.3 Applicability to all engineering domains
The verification standard is applicable to all engineering domains where space products are
developed and as such it is viewed as an “umbrella” under which all domains are covered.
In order to use the standard in a specific engineering domain it is necessary to tailor the standard for
that domain and where necessary, to make applicable the standards that define the verification
requirements of that domain. A clear example is the verification of the ground segment and
operations, whereby its verification is addressed specifically in ECSS-E-ST-70 (Ground systems and
operations), by mandating specific verification (and validation) requirements and processes for the
ground segment. The fact that ECSS-E-ST-10-02C addresses in detail the space segment does not
preclude the use of the standard in other domains, subject to correct tailoring.
4.4 Development
The ECSS glossary defines development as the process by which the capability to adequately
implement a technology or design is established before manufacture and that this process can include
the building of various partial or complete models of the products in order to assess amongst other
things, their performance.
Whilst it is obvious that testing and analysis activities occur during the product development process,
they are not addressed by the standard because they are not formal requirement verification activities
in the sense of the customer-supplier relationship and consequently do not fall within the mandate of
ECSS verification standard.
Verification guidelines
5.1 Verification process
ECSS-E-ST-10-02C clause 5.1 specifies that:
a. The verification process shall demonstrate that the deliverable product meets the specified
customer requirements and is capable of sustaining its operational role through:
1. Verification planning;
2. Verification execution and reporting;
3. Verification control and close-out.
The detailed objectives of the Verification process are as follows:
a. to demonstrate the qualification of design and performance, as meeting the specified
requirements at the specified levels;
b. to ensure that the product is in agreement with the qualified design, is free from workmanship
defects and acceptable for use;
c. to confirm product integrity and performance at particular steps of the project life cycle (e.g.
launch, commissioning, mission events and landing).
While this process looks sequential in nature, it is in fact more complex because the verification
process of a multi level product is conducted in a top down approach for the planning, while the
execution and reporting is conducted bottom up. In addition, the verification control and close-out is
conducted in parallel to the entire process.
The verification process activities are incrementally performed at various levels and in different
stages, and utilizing a combination of the different verification methods as described in the following
clause 5.2.
5.2 Verification planning
5.2.1 Verification approach
5.2.1.1 General
ECSS-E-ST-10-02C clause 5.2.1 specifies that:
a. The customer shall define the project requirements, verification objectives and constraints
affecting the supplier verification process.
Note: For example, ground segment characteristics, launch service, envisaged
end to end tests involving several suppliers. The usual general objectives
are listed in clause 4.1.1 “Verification objectives”.
b. The requirements specified in 5.2.2.1a shall always include those of the technical
specification
c. The supplier shall define the verification approach by conducting the following steps:
1. Identify and agree with the customer the set of requirements to be subject of the
verification process;
2. Select the methods and levels of verification, associated model philosophy and
verification tools;
3. Identify the stages and events in which the verification is implemented.
d. The verification approach shall be defined by the supplier in the Verification Plan (VP) for
approval by the customer prior to implementation.
e. For each requirement to be verified, the verification strategy shall be defined in terms of the
combination of the selected verification methods for the different verification levels at the
applicable verification stages in the initial issue of the Verification Control Document
(VCD also called verification matrix (see Annex B), for approval by the customer.
To reach the verification objectives a verification approach is defined in phases A and B of the project
by analyzing the requirements to be verified, taking into account:
a. design peculiarities and constraints,
b. qualification status of candidate solutions (product category),
c. availability and maturity of verification tools,
d. verification (including test) methodologies,
e. programmatic constraints, and
f. cost and schedule.
The requirement criticality, in terms of technical and programmatic impacts on the verification
implementation, should be assessed by the involvement of the verification team in the requirement
definition process during phases A and B, since it drives the verification strategy.
The verification approach should allow:
a. To ensure the definition of correct verification criteria for each requirement by participating in
the preparation of product specifications.
b. To assess the impact that verification has on the design (e.g. modularity, testability, and
accessibility).
c. To ensure a coherent approach to verification implementation throughout the various levels
avoiding duplication of activities.
d. To ensure early verification of critical items to reduce the risks of late failure identification.
e. To ensure the coverage of the interface verification.
f. To optimize the design and use of ground support equipment, simulators, test tools and test
software (e.g. re-use between levels, stages and models).
g. To optimize the use of test facilities.
h. To plan for feedback to the verification activity from the commissioning results in case of multi-
mission projects or recurring products.
i. To consider innovative solutions that can reduce overall verification costs.
j. To provide visibility and objective evidence of verifications performed.
The use of requirement categories and requirement traceability helps to ensure consistency between
verification activities, and to avoid duplications or gaps in the entire verification.
In generating the verification approach, the supplier conducts the following verification steps (see
Figure 5-1) while ensuring coherency across all verification levels
a. Identify “What” are the products and requirements subject of the verification process;
b. Identify “How” to verify them by a selected method of verification at a certain level on the basis
of a determined model philosophy during the project implementation phases;
c. Identify “When” to implement the verification steps through selected stages and events all
along the project life cycle.
When ?
What? How ?
Identify the consistent set Select methods and Identify the phases and
of verifiable project levels of verification and events in which the
requirements to be associated model verification is
subject of the verification philosophy implemented
process
Reiterations for
technical - cost - schedule reasons
Figure 5-1: Basic verification approach
5.2.1.2 What
Requirements may be grouped in homogeneous category at this step if it facilitates subsequent work.
The requirements applicable to a particular product are usually all contained in technical
specifications (including interface requirements). A subset of some standards may be part of the set.
In order to facilitate the verification implementation, the project requirement should be:
a. traceable,
b. unique and associated to a proper identifier (preferably a unique requirement ID, in its absence
a document and sub clause number,
c. single and not a combination of several requirements,
d. verifiable using one or more approved verification methods,
e. unambiguous,
f. referenced as necessary to other requirements (with applicable document and sub clause
identification),
Some requirements are obvious in their verification execution (example: requirements to issue test
procedures/reports). To limit the documentation and tracking effort these requirements should be
identified and in agreement with the customer marked as "not to be tracked" in the VCD. This means
that these requirements are still applicable requirements but no formal verification report is issued
and tracked in the VCD.
In other words a successful verification starts with a satisfactory set of requirements. Each
requirement is seen as both the origin and the conclusion of the verification process, and treated as
fundamental technical information rather than a constituent of a verbose text.
5.2.1.3 How
5.2.1.3.1 Overview
Then the supplier defines the verification approach in steps, generally conducted in an iterative
process based on technical, cost and schedule considerations.
The verification planning activity should take into account the following elements:
a. the product and specification trees;
b. the applicable models;
c. the estimated duration of procurement, design and manufacturing of each model;
d. the utilization of models (in line with the model philosophy);
e. the estimated duration of the integration of models;
f. the selected test programme and sequences at different levels with estimated time and
resources;
g. the analysis, review-of-design and inspection activities combined on the basis of the verification
strategies and estimated time and resources;
h. the activities and time associated with the procurement of the verification tools to be used;
i. the project milestones and the verification output.
5.2.1.3.2 Step 1 – method, level, model philosophy and verification tool.
After identification of the requirements to be verified the potential verification methods and
alternatives in each particular case are assessed in the context of the overall flow, the testing
techniques, and the analytical tools available.
The verification levels are selected, taking into account as a minimum the programmatic aspects
specified below:
a. standardized product levels (e.g. service modules);
b. make or buy decisions (e.g. off-the-shelf products);
c. reduction of overhead costs (e.g. deletion of subsystem level);
d. responsibility.
Additionally to the aspects associated with a specific requirement (category), the following rules
apply to the selection of levels of verification:
a. In order to have an early feedback in the programme, verification of critical requirements (for
instance related to new technologies) should be achieved at the lowest level of integration of the
product to which the critical requirement applies.
b. All physical or functional requirements for a given product should be verified on that product
level.
c. Special attention should be given to the selection of levels of verification of external interfaces
due to the potential impacts on the verification program.
d. In the case of verification by test, the use of actual interfaces is preferable to the use of
simulators.
e. Duplication of verification activities on different levels should be avoided.
f. In addition to the combination of the verification activities between the different levels, the
verification of complex functional chains (e.g. operational performances and mechanisms
actuation) should include integrated end-to-end testing.
Models are selected by defining the optimum number and type of physical models to achieve a high
confidence in the product verification, with the shortest schedule and a suitable balance of costs and
risks. Model philosophy should be defined and frozen in project Phase A or B. Examples of model
philosophies are prototype, protoflight or a mix of both, called hybrid model philosophy. This is
further discussed in clause 5.2.5. The number of models is minimized and their reuse between
different levels maximized to reduce costs. Parallel models are utilized in order to suitably separate
the test activities from each other and consequently to reduce risks of schedule slippage.
The Design verification (qualification main objective) is carried out on the product which is
representative of the end product configuration (e.g. prototype models, flight models in case of
protoflight or hybrid approach).
The Workmanship verification (acceptance main objective) is carried out on the final products (e.g.
flight models, and spares).
Customer approval is necessary when design parameters, are verified using development models.
5.2.1.3.3 Step 2 - facilities
Selection of the facilities according to their technical characteristics, location, availability and cost.
5.2.1.3.4 Step 3 - resources
Identification of the resources required.
5.2.1.4 When (stage and event)
The verification stages are selected taking into account:
a. the project characteristics,
b. the associated life cycle
c. the adopted model philosophy
Additionally to the aspects associated with a specific requirement (category), the following rules
apply to the selection of stages of verification:
a. When due to the mission characteristics verification includes in-flight activities, the verification
does not rely only on in-flight activities. Requirements are verified prior to the flight to the
extent compatible with the acceptable risks and with the physical possibilities (e.g.
microgravity).
b. All verifications associated with any stage should be completed prior to the start of the next
stage. In particular, qualification is finished before the start of acceptance.
5.2.2 Verification methods
5.2.2.1 General
ECSS-E-ST-10-02C clause 5.2.2.1 specifies that:
a. Verification shall be accomplished by one or more of the followings verification methods:
1. test (including demonstration);
2. analysis (including similarity);
3. review-of-design;
4. inspection.
b. All safety critical functions shall be verified by test.
c. Verification of software shall include testing in the target hardware environment.
d. For each requirement verified only by analysis or review-of-design, a risk assessment (part
of the VP) shall be conducted to determine the level (major/minor) of the impact of this
requirement on the mission.
e. If the impact of the requirement is major, a risk mitigation plan (part of the VP) shall be
defined which includes a cross check based on two independent analyses (in terms of model
used and suppliers).
Test is the preferred verification method as it provides generally a higher confidence in the
verification of a product's performance and design. If a representative test cannot be conducted, then
an analysis substitutes or complements it.
In selecting the verification method/levels/stages the following guidelines are considered:
a. The selected method does not produce risks to personnel, flight hardware and facilities;
b. The selected method is feasible and the required facilities exist as far as possible.
c. The selection of the methods considers the level of confidence that can be achieved with a given
fidelity, accuracy, and validity;
d. Analysis is used when flight conditions cannot be accurately simulated on the ground and/or
when it is not economically feasible to test the entire spectrum of flight conditions. Analysis and
test methods are very often complementary.
e. If several verification methods are possible with the same level of confidence on the verification
results, the selection is oriented to minimize the required effort (impact on cost and schedule)
f. Verification of critical requirements (for instance related to new technologies) should be
achieved as soon as possible (e.g. at the lowest suitable level) to have early feedback on the
programme.
g. To bypass difficulties of testing (i.e. size of facilities, test representativeness) in presence of
complex systems or to minimize expensive system testing a combined approach could be
selected including test campaign at lower level and analysis at higher level.
h. In particular if environmental testing at higher level is impracticable or too costly, lower level
testing should be carried out, completed by analysis, to demonstrate compliance with higher
level requirements.
i. When mission characteristics require in-flight verification, the verification does not rely on on-
orbit activities only; requirements are verified with proper methods prior to the first flight.
j. Internal and external interface verification is carried-out at the appropriate responsibility level.
In case of testing the use of actual interfaces is preferable to the use of simulators.
k. The lower level verification should be completed prior to the associated higher level one.
l. Duplication of verification activities on different levels should be avoided.
m. The verification in any stage should be completed prior of the start of the next stage, in
particular qualification should be finished before acceptance start.
n. Final verification of software is performed by test in the target hardware environment.
o. the verification of complex functional chains (typically operational performances and
mechanisms actuation) should plan for an integrated end-to-end testing, in addition to the
verification activities performed at lower levels
p. All physical or functional requirements for a given product should be verified on that particular
product level to the maximum extent feasible and reasonable.
q. Attention should be paid to avoid over-testing, since parts and equipment are being tested at
parts, equipment, subsystem and system level.
r. Verification by similarity is an exceptional process, which can be applied only if all the
conditions mentioned in Clause 5.2.2.3.b are met. It cannot be considered as a standard
verification method.
The four verification methods are not necessarily applicable or efficient for all stages (and all the
stages are not applicable to all the products). The following table provides a synthesis of the following
requirements while adding some example.
Method Test Analysis Review of Design Inspection
Qualification stage X X X X
Acceptance stage X X
pre-launch stage X X
in-orbit stage X X(1) X(2) X (3)
(including commissioning)
post-landing stage X X
(1) For example budgets (e.g. link, power, mass).
(2) In this case limited to the software change process for in-flight software or ground segment
software.
(3) For manned spacecraft or for ground segment.
5.2.2.2 Test
ECSS-E-ST-10-02C clause 5.2.2.2 specifies that:
a. Verification by test shall consist of measuring product performance and functions
under representative simulated environments.
b. The analysis of data derived from testing shall be an integral part of the test and the
results included in the test report.
c. When the test objectives include the demonstration of qualitative operational
performance, the execution shall be observed and results recorded.
d. A test programme shall be prepared for each product in compliance with test
requirements specified in ECSS-E-ST-10-03.
e. The test programme shall be coordinated with the integration flow.
f. Tests performed as part of the integration flow to check quality and status of the
in-progress configuration (including interfaces), having a formal verification
purpose, shall be included in the test programme.
g. The test programme shall be defined in the Assembly, Integration and Test plan (AITP)).
Although the detailed requirements concerning testing are captured in the test standard ECSS-E-ST-
10-03, the verification standard provides high level requirements for all verification methods in order
to have a coherent verification approach
In defining the test programme the following guidelines are considered:
a. Critical products and interfaces are tested early in phase C/D of the programme.
b. Test flow and sequence are based on the possibility to detect potential failures and defects early
in the test programme.
c. The balance of costs and risks is taken into account.
d. Work around plans and contingencies are taken into account.
e. Re-integration testing (also called regression testing) should be avoided.
f. Test
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...