SIST-TP CEN/CLC/TR 17602-80-04:2021
(Main)Space product assurance - Software metrication programme definition and implementation
Space product assurance - Software metrication programme definition and implementation
The scope of this Handbook is the software metrication as part of a space project, i.e. a space system, a subsystem including hardware and software, or ultimately a software product. It is intended to complement the EN 16602-80 (equivalent to ECSS-Q-ST-80) with specific guidelines related to use of different software metrics including their collection, analysis and reporting. Tailoring guidelines for the software metrication process are also provided to help to meet specific project requirements.
This Handbook provides recommendations, methods and procedures that can be used for the selection and application of appropriate metrics, but it does not include new requirements w ith respect to those provided by EN 16602-80 (equivalent to ECSS-ST-Q-80).
The scope of this Handbook covers the following topics:
• Specification of the goals and objectives for a metrication programme.
• Identification of criteria for selection of metrics in a specific project / environment (goal driven).
• Planning of metrication in the development life cycle.
• Interface of metrication with engineering processes.
• Data collection aspects (including use of tools).
• Approach to the analysis of the collected data.
• Feedback into the process and product based on the analysis results.
• Continuous improvement of measurement process.
• Use of metrics for process and product improvement.
This Handbook is applicable to all types of software of all major parts of a space system, including the space segment, the launch service segment and the ground segment software.
Raumfahrtproduktsicherung - Softwaremetrikhandbuch
Assurance produit des projets spatiaux - Guide de métrologie du logiciel
Zagotavljanje kakovosti proizvodov v vesoljski tehniki - Definicija in izvajanje programa za merjenje programske opreme
General Information
Buy Standard
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-TP CEN/CLC/TR 17602-80-04:2021
01-december-2021
Zagotavljanje kakovosti proizvodov v vesoljski tehniki - Definicija in izvajanje
programa za merjenje programske opreme
Space product assurance - Software metrication programme definition and
implementation
Raumfahrtproduktsicherung - Softwaremetrikhandbuch
Assurance produit des projets spatiaux - Guide de métrologie du logiciel
Ta slovenski standard je istoveten z: CEN/CLC/TR 17602-80-04:2021
ICS:
03.120.99 Drugi standardi v zvezi s Other standards related to
kakovostjo quality
35.080 Programska oprema Software
49.140 Vesoljski sistemi in operacije Space systems and
operations
SIST-TP CEN/CLC/TR 17602-80-04:2021 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
---------------------- Page: 2 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
TECHNICAL REPORT
CEN/CLC/TR 17602-80-
04
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
October 2021
ICS 49.140; 35.240.99
English version
Space product assurance - Software metrication
programme definition and implementation
Assurance produit des projets spatiaux - Guide de Raumfahrtproduktsicherung -
métrologie du logiciel Softwaremetrikhandbuch
This Technical Report was approved by CEN on 13 September 2021. It has been drawn up by the Technical Committee
CEN/CLC/JTC 5.
CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,
Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,
Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,
Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.
CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels
© 2021 CEN/CENELEC All rights of exploitation in any form and by any means Ref. No. CEN/CLC/TR 17602-80-04:2021 E
reserved worldwide for CEN national Members and for
CENELEC Members.
---------------------- Page: 3 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
Table of contents
European Foreword . 5
Introduction . 6
1 Scope . 8
2 References . 9
3 Terms, definitions and abbreviated terms . 12
3.1 Terms from other documents . 12
3.2 Definitions in other clauses of the present HB . 12
3.3 Terms specific to the present document . 12
3.4 Abbreviated terms. 13
4 Overview of the Handbook . 14
4.1 Introduction . 14
4.2 Relation to other ECSS Standards . 15
4.2.1 General . 15
4.2.2 Software engineering . 15
4.2.3 Software product assurance . 15
4.2.4 Project management . 16
5 A reference software quality model . 17
5.1 Introduction . 17
5.2 Reference software quality model . 18
5.3 Tailoring the metrication programme . 23
5.4 Detailed tailoring guidelines . 26
6 Measurement process . 29
6.1 Introduction . 29
6.2 Planning of metrication in the development life cycle . 30
6.2.1 Characterize the project quality requirements and select the metrics
to be collected . 30
6.2.2 Define data collection and analysis procedures . 30
6.2.3 Define criteria for validating the metrics and the measurement
process . 31
2
---------------------- Page: 4 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
6.2.4 Define resources and infrastructure for measurement tasks . 32
6.2.5 Define how reporting will be performed . 32
6.2.6 Review and approve the measurement plan . 32
6.2.7 Provide resources and infrastructure for measurement tasks . 33
6.3 Data collection . 33
6.3.1 Integrate procedures . 33
6.3.2 Collect data . 34
6.4 Data validation . 34
6.5 Data analysis . 34
6.6 Data archiving . 35
6.7 Reporting . 35
6.8 Feedback to the measurement process . 36
6.8.1 Evaluate analysis results and the measurement process . 36
6.8.2 Identify potential improvements . 38
Annex A Definition of the quality model . 39
A.1 General introduction . 39
A.2 Characteristics and sub-characteristics definition. 39
A.2.1 Functionality . 39
A.2.2 Reliability . 40
A.2.3 Maintainability . 40
A.2.4 Reusability . 41
A.2.5 Suitability for safety . 41
A.2.6 Security . 42
A.2.7 Usability . 42
A.2.8 Software development effectiveness . 43
A.3 List of proposed metrics . 43
A.3.1 Introduction . 43
A.3.2 Standard metric template . 44
A.3.3 Detailed description of all metrics . 46
A.4 List of proposed OO metrics . 96
A.4.1 Introduction . 96
A.4.2 Detail description of the proposed OO metrics . 96
Figures
Figure 4-1:Organization of this document . 14
3
---------------------- Page: 5 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
Figure 5-1: Elements of the software quality model . 19
Figure 6-1: Metrication process activities . 29
Figure A-1 : Example of report format for requirements allocation . 47
Figure A-2 : Example of SPR/NCR trend analysis . 52
Figure A-3 : Example of cyclomatic complexity . 60
Figure A-4 : Example of nesting level . 62
Figure A-5 : Sample of requirements stability . 93
Figure A-6 : Sample of RID/action status . 94
Figure A-7 : Sample of V&V progress . 95
Tables
Table 5-1: Proposed reference quality model . 21
Table 5-2: Applicability of the metrics depending on the criticality category . 24
Table 5-3: Target value for metric depending on criticality category . 25
Table A-1 : Example of V&V coverage reporting. 50
Table A-2 : Example of summary of NCR-SPR recorded . 51
Table A-3 : Sample checklist for Suitability of development documentation . 54
Table A-4 : Checklist for process reliability adequacy . 70
Table A-5 : Test coverage requirements . 73
Table A-6 : Sample checklist for reusability checklist . 77
Table A-7 : Sample checklist for safety activities adequacy . 81
Table A-8 : Checklist for security checklist. 83
Table A-9 : Checklist for User manual suitability . 87
4
---------------------- Page: 6 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
European Foreword
This document (CEN/CLC/TR 17602-80-04:2021) has been prepared by Technical Committee
CEN/CLC/JTC 5 “Space”, the secretariat of which is held by DIN.
It is highlighted that this technical report does not contain any requirement but only collection of data
or descriptions and guidelines about how to organize and perform the work in support of EN 17602-
80.
This Technical report (CEN/CLC/TR 17602-80-04:2021) originates from ECSS-Q-HB-80-04A.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN [and/or CENELEC] shall not be held responsible for identifying any or all such
patent rights.
This document has been prepared under a mandate given to CEN by the European Commission and
the European Free Trade Association.
This document has been developed to cover specifically space systems and has therefore precedence
over any TR covering the same scope but with a wider domain of applicability (e.g.: aerospace).
5
---------------------- Page: 7 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
Introduction
This Handbook describes the approach to be taken for the definition and implementation of an
effective and efficient metrication programme for the development of software in a space project.
This Handbook provides guidelines and examples of software metrics that can be used in space
system developments, in line with the requirements defined by [ECSS-E-40] and [ECSS-Q-80], given
guidelines and examples to provide a coherent view of the software metrication programme definition
and implementation.
This Handbook is intended to help customers in formulating their quality requirements and suppliers
in preparing their response and implementing the work. This Handbook is not intended to replace
textbook material on computer science or technology and software metrics, so repeating such material
is avoided in this Handbook. The readers and users of this Handbook are assumed to posse’s general
knowledge of computer science and software engineering.
In space projects, the demand of high quality software to be developed within allocated budget and
time is a priority objective, particularly in presence of dependability requirements. Also the size of the
operational software has increased significantly due to the increase in functionality to allow new
challenging missions and to reply to increasingly sophisticated and demanding user requirements.
The space software development is therefore characterized by:
• High dependability of the software products, in both space and ground segments;
• Stringent schedule and cost estimates that are more and more accurate to enable reliable cost
estimates for the overall project;
• Demand of high quality software, and
• Increasing productivity requirements.
To improve, suppliers should know what can be done better, and also what to look at in order to
understand where lessons learnt can be applied to support the improvement.
Measurements are the only way to quantitatively assess the quality of a process or a product.
In presence of complex software projects reliable measures of both processes and products provide a
powerful tool to software management for keeping the project in track and preserve the intended
quality of the software product. Improvement of both the space software product and its development
processes depends upon improved ability to identify, measure, and control essential parameters that
affect software product and its development processes. This is the goal of any software metrication
programme.
The main reasons for measuring software processes and products are:
To characterize the existing processes and the status of products under development or
operations, in order to gain a better understanding and support the overall verification of
the software, as well as to acquire data/information for future assessments of similar
processes/products.
6
---------------------- Page: 8 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
To evaluate the status of the project to determine its status and possible deviation from
the established plans, and to support identification of actions to bring it back under
control; the evaluation includes an assessment of an achievement of quality goals and an
assessment of the impacts of technology and process improvements on products and
processes.
To predict, with the aim to improve the ability to plan. Measuring for prediction involves
gaining understanding of relationships among processes and products and building
models of these relationships, so that the observed values for some attributes can be used
to predict others. The reason for that is to establish achievable goals for cost, schedule,
and quality—so that appropriate resources can be applied. Predictive measures are also
the basis for extrapolating trends, so estimates for cost, time, and quality can be updated
based on current evidence. Projections and estimates based on historical data also help to
analyse risks and make design/cost tradeoffs.
To improve. Gathering quantitative information helps to identify roadblocks, root causes,
inefficiencies, and other opportunities for improving product quality and process
performance. Measures also help to plan and track improvement efforts.
Measures of current performance give baselines to compare against, so that it can be possible to judge
whether or not the improvement actions are working as intended and what the side effects can be.
Good measures also help to communicate goals and convey reasons for improving. This helps engage
and focus the support of those who work within the processes to make them successful.
7
---------------------- Page: 9 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
1
Scope
The scope of this Handbook is the software metrication as part of a space project, i.e. a space system, a
subsystem including hardware and software, or ultimately a software product. It is intended to
complement the [ECSS-Q-80] with specific guidelines related to use of different software metrics
including their collection, analysis and reporting. Tailoring guidelines for the software metrication
process are also provided to help to meet specific project requirements.
This Handbook provides recommendations, methods and procedures that can be used for the
selection and application of appropriate metrics, but it does not include new requirements with
respect to those provided by ECSS-ST-Q-80C Standard.
The scope of this Handbook covers the following topics:
• Specification of the goals and objectives for a metrication programme.
• Identification of criteria for selection of metrics in a specific project / environment (goal driven).
• Planning of metrication in the development life cycle.
• Interface of metrication with engineering processes.
• Data collection aspects (including use of tools).
• Approach to the analysis of the collected data.
• Feedback into the process and product based on the analysis results.
• Continuous improvement of measurement process.
• Use of metrics for process and product improvement.
This Handbook is applicable to all types of software of all major parts of a space system, including the
space segment, the launch service segment and the ground segment software.
8
---------------------- Page: 10 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
2
References
For each document or Standard listed, a mnemonic (used to refer to that source throughout this
document) is proposed in the left side, and then the complete reference is provided in the right one.
ECSS Standards
EN Reference Reference in text Title
EN 16601-00-01 [ECSS-S-ST-00-01] ECSS-S-ST-00-01C, ECSS - Glossary of terms
EN 16602-80 [ECSS-Q-80] ECSS-Q-ST-80C, Space Product Assurance – Software
Product Assurance
EN 16603-40 [ECSS-E-40] ECSS-E-ST-40C, Space Engineering – Software
ISO/IEC Standards
ISO 9126 ISO/IEC 9126 Software engineering - Product quality, Parts 1 to 4 (complete
series)
ISO 9126-1 ISO/IEC 9126-1:2001 Part 1: Quality model
ISO 9126-2 ISO/IEC TR 9126-2:2003 Part 2: External metrics
ISO 9126-3 ISO/IEC TR 9126-3:2003 Part 3: Internal metrics
ISO 9126-4 ISO/IEC TR 9126-4:2004 Part 4: Quality in use metrics
ISO 12207 ISO/IEC 12207:2008 Information Technology - Software life cycle processes.
ISO 14143 ISO/IEC 14143 Information technology - Software measurement, Parts 1 to 6
(complete series)
ISO 14598 ISO/IEC 14598 Software engineering - Product evaluation, Parts 1 to 6 (complete
series)
ISO 14598-1 ISO/IEC 14598-1:1999 Part 1: General overview
ISO 24765 ISO/IEC 24765, Systems and Software Engineering Vocabulary
ISO 15939 ISO/IEC 15939:2007 Software Engineering - Software Measurement Process
ISO 17799 ISO/IEC 17799:2005 Information technology - Code of practice for information
security management
ISO 25000 ISO/IEC 25000:2005 Ed. 1 Software Engineering - Software product Quality
Requirements and Evaluation (SQuaRE) - Guide to SQuaRE
Other ESA documents
SPEC ESTEC Contract No. 12650/97/NL/NB(SC) - SPEC – Software Product Evaluation
and Certification (complete series)
9
---------------------- Page: 11 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
SPEC-I ESTEC Contract No. 20421/06/NL/PA - SPEC Method Improvement
SPEC-I-QM SPEC/QM SPEC Method improvement - Quality Model – Issue 1.F
SPEC-I-TN1 SPEC-I/TN1 SPEC Analysis results - Issue 1.F
SPEC-I -TN2 SPEC-I/TN2 Concept for Space Software Product Evaluation for Conformity
WP4 - part 2- Issue 3.B
SPEC-I –TN3 SPEC-I/TN3 Concept for Space Software Product Evaluation for Conformity
WP4 - part 3- Issue 3.2
SPEC-TN3 SPEC/TN3 Space Domain Specific Software Product Quality Models,
Requirements and Related Evaluation Methods - Issue 3.4
SPEC-TN4.1 SPEC/TN4 part 1 Overview of existing software certification schemes - Issue 2
SPEC-TN4.2 SPEC/TN4 part 2 Concept for Space Software Product Evaluation and
Certification - Issue 2
SPEC-TN4.3 SPEC/TN4 part 3 Concept for Space Software Product Evaluation and
Certification - Issue 2
Reports and articles
CHALMERS Chalmers University of Technology Presentation - Department of Computer
Engineering - Dependability and Security Modelling and Metrics
EADS-ST Astrium-ST- Astrium ST internal documents
FENTON “Software Metrics - A Rigorous & Practical Approach” (second edition)
Norman E. Fenton, Shari Lawrence Pfleeger
NASA-1740 NSS 1740.13 “NASA Software Safety Standard” February 1996
NASA-8719 NASA-STD-8719.13A Software Safety, 15 September 1997
NIAC Common Vulnerability Scoring System - Final Report and Recommendations by
the Council, 12 October 2004
NIST-1 NIST and Federal Computer Security Program Managers Forum IT
Security Metrics Workshop - A Practical Approach to Measuring Information
Security: Measuring Security at the System Level, 21 May 2002
NIST-2 NIST Special Publication 800-55
Security Metrics Guide for Information Technology Systems, July 2003
NIST-3 NIST Special Publication 800-35
Guide to Information Technology Security Services, October 2003
SANS-1 SANS Institute - A Guide to Security Metrics - SANS Security Essentials GSEC
Practical Assignment, Version 1.2, 11 July 2001
SANS-2 SANS Institute - Systems Maintenance Programs - The Forgotten Foundation
and Support of the CIA Triad, GSEC v1.3, 10 January 2002
SEC-FIN VTT Technical Research Centre of Finland Publications 544
Process Approach to Information Security Metrics in Finnish Industry and State
Institutions, 2004
SUN-JAVA SUN Java Coding Conventions
10
---------------------- Page: 12 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
Websites
ISECOM The Institute for Security and Open Methodologies (security)
http://www.isecom.org/securitymetrics.shtml
EDUCAUSE http://www.educause.edu
SEC-METRICS Community website
http://www.securitymetrics.org
SW-METRICS International Software Metrics Organization
http://www.swmetrics.org
SEC-DOCS Security white papers and documents
http://www.securitydocs.com
IEEE http://www.ieee.com
IEEE-CS http://www.computer.org
NIST NIST Computer Security Resource Centre
http://csrc.nist.gov/ispab/
COSMICON The Common Software Measurement International Consortium
http://www.cosmicon.com/
UKSMA The UK Software Metrics Association
http://www.uksma.co.uk/?action=0&what=90
ARMY-METRICS Army Software Metrics Office
http;//www.armysoftwaremetrics.org
PSSM Practical Software & Systems Measurement
http://www.psmsc.com
11
---------------------- Page: 13 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
3
Terms, definitions and abbreviated terms
3.1 Terms from other documents
For the purpose of this document, the terms and definitions from ECSS-S-ST-00-01 and ECSS-Q-ST-80
apply.
3.2 Definitions in other clauses of the present HB
Subclause A.2 of Annex A includes the definitions for all characteristics/sub-characteristics contained
in the quality model used in this Handbook
3.3 Terms specific to the present document
3.3.1 base measure
measure defined in terms of an attribute and the method for quantifying it.
[ISO 24765]
3.3.2 measure (noun)
variable to which a value is assigned as the result of measurement.
[ISO 24765]
3.3.3 measure (verb)
Make a measurement.
[ISO 24765]
3.3.4 measurement
act or process of assigning a number or category to an entity to describe an attribute of that entity.
NOTE "Category" is used to denote qualitative measures of attributes. For
example, some important attributes of software products, e.g. the
language of a source program (such as ADA, C, COBOL) are
qualitative.
[ISO 24765]
12
---------------------- Page: 14 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
3.3.5 metric
a quantitative measure of the degree to which a system, component, or process possesses a given
attribute.
[ISO 24765]
3.3.6 quality model
defined set of characteristics, and of relationships between them, which provides a framework for
specifying quality requirements and evaluating quality
[ISO 24765]
3.4 Abbreviated terms
For the purpose of this document, the abbreviated terms from [ECSS-S-ST-00-01] and the following
apply:
Abbreviation Meaning
CBO coupling between objects
CM configuration management
DDR detailed design review
DIT depth of inheritance tree
IEC International Electrotechnical Commission
ISO International Organization for Standardization
HOOD Hierarchical Object Oriented Design
LOC lines of code
LCOM lack of cohesion method
MIPS millions of instructions per second
MMI man-machine interface
NASA National Aeronautics and Space Administration
NOC number of children
PPF parametric polymorphic factor
RB requirements baseline
RFD request for deviation
RFW request for waiver
SW PA software product assurance
SPR software problem report
SRR system requirements review
UML Unified Modelling Language
V&V verification and validation
VG cyclomatic complexity (McCabe)
13
---------------------- Page: 15 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
4
Overview of the Handbook
4.1 Introduction
This subclause contains an introduction of the content of this Handbook, the intended audience and
how to use it.
It introduces the rationale of the need of a metrication programme for space software projects based
on [ECSS-E-40] and [ECSS-Q-80] requirements (especially this one).
The organization of this Handbook is reflected in detail in Figure 4-1. This Handbook is organized in
seven main parts:
• Clause 1. Scope
• Clause 2: Normative references
• Clause 3: Terms, definitions and abbreviated terms.
• Clause 4: Overview of the Handbook
• Clause 5: A reference software quality model
• Clause 6: Measurement process
• Annex A: Definition of the quality model
The annex is provided for information only.
Clause 3
Clause 2
Clause 1
Terms, definitions
Normative references
Scope
and abbreviated
terms
Clause 5
Clause 4 Clause 6
A reference
Measurement
Overview of the
software quality
process
Handbook
model
Annex A
Definition of the
quality model
Figure 4-1:Organization of this document
14
---------------------- Page: 16 ----------------------
SIST-TP CEN/CLC/TR 17602-80-04:2021
CEN/CLC/TR 17602-80-04:2021 (E)
4.2 Relation to other ECSS Standards
4.2.1 General
This subclause discusses how this Handbook interfaces with other ECSS series, namely the ECSS-Q
series of standards (product assurance), ECSS-E series of standards (engineering) and the ECSS-M
series of standards (management).
4.2.2 Software engineering
The interface of this Handbook to the ECSS-E branch is via [ECSS-E-40]; and in turn, the interface of
[ECSS-E-40] to this Handbook is via the [ECSS-Q-80].
[ECSS-E-40] covers all aspects of space software engineering from requirements definition to
retirement. It defines the scope of the space software engineering processes, including details of the
verification and validation processes, and their interfaces with management and product assurance,
which are addressed in the management (-M) an
...
SLOVENSKI STANDARD
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
01-julij-2021
Zagotavljanje kakovosti proizvodov v vesoljski tehniki - Definicija in izvajanje
programa za merjenje programske opreme
Space product assurance - Software metrication programme definition and
implementation
Produktsicherung in der Raumfahrt - Definition und Implementierung von Software-
Metrifizierungsprogrammen
Assurance produit des projets spatiaux - Définition et mise en œuvre d’un programme de
métriques logiciel
Ta slovenski standard je istoveten z: FprCEN/CLC/TR 17602-80-04
ICS:
35.240.99 Uporabniške rešitve IT na IT applications in other fields
drugih področjih
49.140 Vesoljski sistemi in operacije Space systems and
operations
kSIST-TP FprCEN/CLC/TR 17602-80- en,fr,de
04:2021
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
---------------------- Page: 2 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
TECHNICAL REPORT
FINAL DRAFT
FprCEN/CLC/TR 17602-
RAPPORT TECHNIQUE
80-04
TECHNISCHER BERICHT
May 2021
ICS 49.140; 35.240.99
English version
Space product assurance - Software metrication
programme definition and implementation
Assurance produit des projets spatiaux - Définition et Produktsicherung in der Raumfahrt - Definition und
mise en œuvre d'un programme de métriques logiciel Implementierung von Software-
Metrifizierungsprogrammen
This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee
CEN/CLC/JTC 5.
CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,
Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,
Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,
Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are
aware and to provide supporting documentation.
Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without
notice and shall not be referred to as a Technical Report.
CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels
© 2021 CEN/CENELEC All rights of exploitation in any form and by any means Ref. No. FprCEN/CLC/TR 17602-80-04:2021 E
reserved worldwide for CEN national Members and for
CENELEC Members.
---------------------- Page: 3 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
Table of contents
European Foreword . 5
Introduction . 6
1 Scope . 8
2 References . 9
3 Terms, definitions and abbreviated terms . 12
3.1 Terms from other documents . 12
3.2 Definitions in other clauses of the present HB . 12
3.3 Terms specific to the present document . 12
3.4 Abbreviated terms. 13
4 Overview of the Handbook . 14
4.1 Introduction . 14
4.2 Relation to other ECSS Standards . 15
4.2.1 General . 15
4.2.2 Software engineering . 15
4.2.3 Software product assurance . 15
4.2.4 Project management . 16
5 A reference software quality model . 17
5.1 Introduction . 17
5.2 Reference software quality model . 18
5.3 Tailoring the metrication programme . 23
5.4 Detailed tailoring guidelines . 26
6 Measurement process . 29
6.1 Introduction . 29
6.2 Planning of metrication in the development life cycle . 30
6.2.1 Characterize the project quality requirements and select the metrics
to be collected . 30
6.2.2 Define data collection and analysis procedures . 30
6.2.3 Define criteria for validating the metrics and the measurement
process . 31
2
---------------------- Page: 4 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
6.2.4 Define resources and infrastructure for measurement tasks . 32
6.2.5 Define how reporting will be performed . 32
6.2.6 Review and approve the measurement plan . 32
6.2.7 Provide resources and infrastructure for measurement tasks . 33
6.3 Data collection . 33
6.3.1 Integrate procedures . 33
6.3.2 Collect data . 34
6.4 Data validation . 34
6.5 Data analysis . 34
6.6 Data archiving . 35
6.7 Reporting . 35
6.8 Feedback to the measurement process . 36
6.8.1 Evaluate analysis results and the measurement process . 36
6.8.2 Identify potential improvements . 38
Annex A Definition of the quality model . 39
A.1 General introduction . 39
A.2 Characteristics and sub-characteristics definition. 39
A.2.1 Functionality . 39
A.2.2 Reliability . 40
A.2.3 Maintainability . 40
A.2.4 Reusability . 41
A.2.5 Suitability for safety . 41
A.2.6 Security . 42
A.2.7 Usability . 42
A.2.8 Software development effectiveness . 43
A.3 List of proposed metrics . 43
A.3.1 Introduction . 43
A.3.2 Standard metric template . 44
A.3.3 Detailed description of all metrics . 46
A.4 List of proposed OO metrics . 96
A.4.1 Introduction . 96
A.4.2 Detail description of the proposed OO metrics . 96
Figures
Figure 4-1:Organization of this document . 14
3
---------------------- Page: 5 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
Figure 5-1: Elements of the software quality model . 19
Figure 6-1: Metrication process activities . 29
Figure A-1 : Example of report format for requirements allocation . 47
Figure A-2 : Example of SPR/NCR trend analysis . 52
Figure A-3 : Example of cyclomatic complexity . 60
Figure A-4 : Example of nesting level . 62
Figure A-5 : Sample of requirements stability . 93
Figure A-6 : Sample of RID/action status . 94
Figure A-7 : Sample of V&V progress . 95
Tables
Table 5-1: Proposed reference quality model . 21
Table 5-2: Applicability of the metrics depending on the criticality category . 24
Table 5-3: Target value for metric depending on criticality category . 25
Table A-1 : Example of V&V coverage reporting. 50
Table A-2 : Example of summary of NCR-SPR recorded . 51
Table A-3 : Sample checklist for Suitability of development documentation . 54
Table A-4 : Checklist for process reliability adequacy . 70
Table A-5 : Test coverage requirements . 73
Table A-6 : Sample checklist for reusability checklist . 77
Table A-7 : Sample checklist for safety activities adequacy . 81
Table A-8 : Checklist for security checklist. 83
Table A-9 : Checklist for User manual suitability . 87
4
---------------------- Page: 6 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
European Foreword
This document (FprCEN/CLC/TR 17602-80-04:2021) has been prepared by Technical Committee
CEN/CLC/JTC 5 “Space”, the secretariat of which is held by DIN.
It is highlighted that this technical report does not contain any requirement but only collection of data
or descriptions and guidelines about how to organize and perform the work in support of EN 17602-
80.
This Technical report (FprCEN/CLC/TR 17602-80-04:2021) originates from ECSS-Q-HB-80-04A.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN [and/or CENELEC] shall not be held responsible for identifying any or all such
patent rights.
This document has been prepared under a mandate given to CEN by the European Commission and
the European Free Trade Association.
This document has been developed to cover specifically space systems and has therefore precedence
over any TR covering the same scope but with a wider domain of applicability (e.g.: aerospace).
This document is currently submitted to the CEN Technical Report.
5
---------------------- Page: 7 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
Introduction
This Handbook describes the approach to be taken for the definition and implementation of an
effective and efficient metrication programme for the development of software in a space project.
This Handbook provides guidelines and examples of software metrics that can be used in space
system developments, in line with the requirements defined by [ECSS-E-40] and [ECSS-Q-80], given
guidelines and examples to provide a coherent view of the software metrication programme definition
and implementation.
This Handbook is intended to help customers in formulating their quality requirements and suppliers
in preparing their response and implementing the work. This Handbook is not intended to replace
textbook material on computer science or technology and software metrics, so repeating such material
is avoided in this Handbook. The readers and users of this Handbook are assumed to posse’s general
knowledge of computer science and software engineering.
In space projects, the demand of high quality software to be developed within allocated budget and
time is a priority objective, particularly in presence of dependability requirements. Also the size of the
operational software has increased significantly due to the increase in functionality to allow new
challenging missions and to reply to increasingly sophisticated and demanding user requirements.
The space software development is therefore characterized by:
High dependability of the software products, in both space and ground segments;
Stringent schedule and cost estimates that are more and more accurate to enable reliable cost
estimates for the overall project;
Demand of high quality software, and
Increasing productivity requirements.
To improve, suppliers should know what can be done better, and also what to look at in order to
understand where lessons learnt can be applied to support the improvement.
Measurements are the only way to quantitatively assess the quality of a process or a product.
In presence of complex software projects reliable measures of both processes and products provide a
powerful tool to software management for keeping the project in track and preserve the intended
quality of the software product. Improvement of both the space software product and its development
processes depends upon improved ability to identify, measure, and control essential parameters that
affect software product and its development processes. This is the goal of any software metrication
programme.
The main reasons for measuring software processes and products are:
To characterize the existing processes and the status of products under development or
operations, in order to gain a better understanding and support the overall verification of
the software, as well as to acquire data/information for future assessments of similar
processes/products.
6
---------------------- Page: 8 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
To evaluate the status of the project to determine its status and possible deviation from
the established plans, and to support identification of actions to bring it back under
control; the evaluation includes an assessment of an achievement of quality goals and an
assessment of the impacts of technology and process improvements on products and
processes.
To predict, with the aim to improve the ability to plan. Measuring for prediction involves
gaining understanding of relationships among processes and products and building
models of these relationships, so that the observed values for some attributes can be used
to predict others. The reason for that is to establish achievable goals for cost, schedule,
and quality—so that appropriate resources can be applied. Predictive measures are also
the basis for extrapolating trends, so estimates for cost, time, and quality can be updated
based on current evidence. Projections and estimates based on historical data also help to
analyse risks and make design/cost tradeoffs.
To improve. Gathering quantitative information helps to identify roadblocks, root causes,
inefficiencies, and other opportunities for improving product quality and process
performance. Measures also help to plan and track improvement efforts.
Measures of current performance give baselines to compare against, so that it can be possible to judge
whether or not the improvement actions are working as intended and what the side effects can be.
Good measures also help to communicate goals and convey reasons for improving. This helps engage
and focus the support of those who work within the processes to make them successful.
7
---------------------- Page: 9 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
1
Scope
The scope of this Handbook is the software metrication as part of a space project, i.e. a space system, a
subsystem including hardware and software, or ultimately a software product. It is intended to
complement the [ECSS-Q-80] with specific guidelines related to use of different software metrics
including their collection, analysis and reporting. Tailoring guidelines for the software metrication
process are also provided to help to meet specific project requirements.
This Handbook provides recommendations, methods and procedures that can be used for the
selection and application of appropriate metrics, but it does not include new requirements with
respect to those provided by ECSS-ST-Q-80C Standard.
The scope of this Handbook covers the following topics:
Specification of the goals and objectives for a metrication programme.
Identification of criteria for selection of metrics in a specific project / environment (goal driven).
Planning of metrication in the development life cycle.
Interface of metrication with engineering processes.
Data collection aspects (including use of tools).
Approach to the analysis of the collected data.
Feedback into the process and product based on the analysis results.
Continuous improvement of measurement process.
Use of metrics for process and product improvement.
This Handbook is applicable to all types of software of all major parts of a space system, including the
space segment, the launch service segment and the ground segment software.
8
---------------------- Page: 10 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
2
References
For each document or Standard listed, a mnemonic (used to refer to that source throughout this
document) is proposed in the left side, and then the complete reference is provided in the right one.
ECSS Standards
EN Reference Reference in text Title
EN 16601-00-01 [ECSS-S-ST-00-01] ECSS-S-ST-00-01C, ECSS - Glossary of terms
EN 16602-80 [ECSS-Q-80] ECSS-Q-ST-80C, Space Product Assurance – Software
Product Assurance
EN 16603-40 [ECSS-E-40] ECSS-E-ST-40C, Space Engineering – Software
ISO/IEC Standards
ISO 9126 ISO/IEC 9126 Software engineering - Product quality, Parts 1 to 4 (complete
series)
ISO 9126-1 ISO/IEC 9126-1:2001 Part 1: Quality model
ISO 9126-2 ISO/IEC TR 9126-2:2003 Part 2: External metrics
ISO 9126-3 ISO/IEC TR 9126-3:2003 Part 3: Internal metrics
ISO 9126-4 ISO/IEC TR 9126-4:2004 Part 4: Quality in use metrics
ISO 12207 ISO/IEC 12207:2008 Information Technology - Software life cycle processes.
ISO 14143 ISO/IEC 14143 Information technology - Software measurement, Parts 1 to 6
(complete series)
ISO 14598 ISO/IEC 14598 Software engineering - Product evaluation, Parts 1 to 6 (complete
series)
ISO 14598-1 ISO/IEC 14598-1:1999 Part 1: General overview
ISO 24765 ISO/IEC 24765, Systems and Software Engineering Vocabulary
ISO 15939 ISO/IEC 15939:2007 Software Engineering - Software Measurement Process
ISO 17799 ISO/IEC 17799:2005 Information technology - Code of practice for information
security management
ISO 25000 ISO/IEC 25000:2005 Ed. 1 Software Engineering - Software product Quality
Requirements and Evaluation (SQuaRE) - Guide to SQuaRE
Other ESA documents
SPEC ESTEC Contract No. 12650/97/NL/NB(SC) - SPEC – Software Product Evaluation
and Certification (complete series)
9
---------------------- Page: 11 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
SPEC-I ESTEC Contract No. 20421/06/NL/PA - SPEC Method Improvement
SPEC-I-QM SPEC/QM SPEC Method improvement - Quality Model – Issue 1.F
SPEC-I-TN1 SPEC-I/TN1 SPEC Analysis results - Issue 1.F
SPEC-I -TN2 SPEC-I/TN2 Concept for Space Software Product Evaluation for Conformity
WP4 - part 2- Issue 3.B
SPEC-I –TN3 SPEC-I/TN3 Concept for Space Software Product Evaluation for Conformity
WP4 - part 3- Issue 3.2
SPEC-TN3 SPEC/TN3 Space Domain Specific Software Product Quality Models,
Requirements and Related Evaluation Methods - Issue 3.4
SPEC-TN4.1 SPEC/TN4 part 1 Overview of existing software certification schemes - Issue 2
SPEC-TN4.2 SPEC/TN4 part 2 Concept for Space Software Product Evaluation and
Certification - Issue 2
SPEC-TN4.3 SPEC/TN4 part 3 Concept for Space Software Product Evaluation and
Certification - Issue 2
Reports and articles
CHALMERS Chalmers University of Technology Presentation - Department of Computer
Engineering - Dependability and Security Modelling and Metrics
EADS-ST Astrium-ST- Astrium ST internal documents
FENTON “Software Metrics - A Rigorous & Practical Approach” (second edition)
Norman E. Fenton, Shari Lawrence Pfleeger
NASA-1740 NSS 1740.13 “NASA Software Safety Standard” February 1996
NASA-8719 NASA-STD-8719.13A Software Safety, 15 September 1997
NIAC Common Vulnerability Scoring System - Final Report and Recommendations by
the Council, 12 October 2004
NIST-1 NIST and Federal Computer Security Program Managers Forum IT
Security Metrics Workshop - A Practical Approach to Measuring Information
Security: Measuring Security at the System Level, 21 May 2002
NIST-2 NIST Special Publication 800-55
Security Metrics Guide for Information Technology Systems, July 2003
NIST-3 NIST Special Publication 800-35
Guide to Information Technology Security Services, October 2003
SANS-1 SANS Institute - A Guide to Security Metrics - SANS Security Essentials GSEC
Practical Assignment, Version 1.2, 11 July 2001
SANS-2 SANS Institute - Systems Maintenance Programs - The Forgotten Foundation
and Support of the CIA Triad, GSEC v1.3, 10 January 2002
SEC-FIN VTT Technical Research Centre of Finland Publications 544
Process Approach to Information Security Metrics in Finnish Industry and State
Institutions, 2004
SUN-JAVA SUN Java Coding Conventions
10
---------------------- Page: 12 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
Websites
ISECOM The Institute for Security and Open Methodologies (security)
http://www.isecom.org/securitymetrics.shtml
EDUCAUSE http://www.educause.edu
SEC-METRICS Community website
http://www.securitymetrics.org
SW-METRICS International Software Metrics Organization
http://www.swmetrics.org
SEC-DOCS Security white papers and documents
http://www.securitydocs.com
IEEE http://www.ieee.com
IEEE-CS http://www.computer.org
NIST NIST Computer Security Resource Centre
http://csrc.nist.gov/ispab/
COSMICON The Common Software Measurement International Consortium
http://www.cosmicon.com/
UKSMA The UK Software Metrics Association
http://www.uksma.co.uk/?action=0&what=90
ARMY-METRICS Army Software Metrics Office
http;//www.armysoftwaremetrics.org
PSSM Practical Software & Systems Measurement
http://www.psmsc.com
11
---------------------- Page: 13 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
3
Terms, definitions and abbreviated terms
3.1 Terms from other documents
For the purpose of this document, the terms and definitions from ECSS-S-ST-00-01 and ECSS-Q-ST-80
apply.
3.2 Definitions in other clauses of the present HB
Subclause A.2 of Annex A includes the definitions for all characteristics/sub-characteristics contained
in the quality model used in this Handbook
3.3 Terms specific to the present document
3.3.1 base measure
measure defined in terms of an attribute and the method for quantifying it.
[ISO 24765]
3.3.2 measure (noun)
variable to which a value is assigned as the result of measurement.
[ISO 24765]
3.3.3 measure (verb)
Make a measurement.
[ISO 24765]
3.3.4 measurement
act or process of assigning a number or category to an entity to describe an attribute of that entity.
NOTE "Category" is used to denote qualitative measures of attributes. For
example, some important attributes of software products, e.g. the
language of a source program (such as ADA, C, COBOL) are
qualitative.
[ISO 24765]
12
---------------------- Page: 14 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
3.3.5 metric
a quantitative measure of the degree to which a system, component, or process possesses a given
attribute.
[ISO 24765]
3.3.6 quality model
defined set of characteristics, and of relationships between them, which provides a framework for
specifying quality requirements and evaluating quality
[ISO 24765]
3.4 Abbreviated terms
For the purpose of this document, the abbreviated terms from [ECSS-S-ST-00-01] and the following
apply:
Abbreviation Meaning
CBO coupling between objects
CM
configuration management
DDR detailed design review
DIT depth of inheritance tree
IEC International Electrotechnical Commission
ISO
International Organization for Standardization
HOOD Hierarchical Object Oriented Design
LOC lines of code
LCOM lack of cohesion method
MIPS millions of instructions per second
MMI man-machine interface
NASA National Aeronautics and Space Administration
NOC number of children
PPF parametric polymorphic factor
RB requirements baseline
RFD request for deviation
RFW request for waiver
SW PA software product assurance
SPR software problem report
SRR system requirements review
UML Unified Modelling Language
V&V verification and validation
VG cyclomatic complexity (McCabe)
13
---------------------- Page: 15 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
4
Overview of the Handbook
4.1 Introduction
This subclause contains an introduction of the content of this Handbook, the intended audience and
how to use it.
It introduces the rationale of the need of a metrication programme for space software projects based
on [ECSS-E-40] and [ECSS-Q-80] requirements (especially this one).
The organization of this Handbook is reflected in detail in Figure 4-1. This Handbook is organized in
seven main parts:
Clause 1. Scope
Clause 2: Normative references
Clause 3: Terms, definitions and abbreviated terms.
Clause 4: Overview of the Handbook
Clause 5: A reference software quality model
Clause 6: Measurement process
Annex A: Definition of the quality model
The annex is provided for information only.
Clause 3
Clause 2
Clause 1
Terms, definitions
Normative references
Scope
and abbreviated
terms
Clause 5
Clause 4 Clause 6
A reference
Overview of the Measurement
software quality
process
Handbook
model
Annex A
Definition of the
quality model
Figure 4-1:Organization of this document
14
---------------------- Page: 16 ----------------------
kSIST-TP FprCEN/CLC/TR 17602-80-04:2021
FprCEN/CLC/TR 17602-80-04:2021 (E)
4.2 Relation to other ECSS Standards
4.2.1 General
This subclause discusses how this Handbook interfaces wi
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.