ISO/IEC/IEEE 42030:2019
(Main)Software, systems and enterprise — Architecture evaluation framework
Software, systems and enterprise — Architecture evaluation framework
This document specifies the means to organize and record architecture evaluations for enterprise, systems and software fields of application. The aim of this document is to enable architecture evaluations that are used to: a) validate that architectures address the concerns of stakeholders; b) assess the quality of architectures with respect to their intended purpose; c) assess the value of architectures to their stakeholders; d) determine whether architecture entities address their intended purpose; e) provide knowledge and information about architecture entities; f) assess progress towards achieving architecture objectives; g) clarify understanding of problem space and of stakeholder needs and expectations; h) identify risks and opportunities associated with architectures; and i) support decision making where architectures are involved. NOTE This document addresses the evaluation of an architecture and not an evaluation of the architecture description's suitability. Matters concerning the evaluation of the architecture description fall within the scope of the architecture conceptualization and architecture elaboration processes as defined in ISO/IEC/IEEE 42020. However, it is sometimes the case that the architecture description is evaluated concurrently with the evaluation of the architecture itself. The entity being evaluated can be of several kinds, as illustrated in the following examples: enterprise, organization, solution, system, subsystem, business, data (as a data element or data structure), application, information technology (as a collection), mission, product, service, software item, hardware item, etc. The kind of entity can also be a product line, family of systems, system of systems, etc. It also spans the variety of applications that utilize digital technology such as mobile, cloud, big data, robotics, Internet of Things (IoT), web, desktop, embedded systems, and so on. The generic Architecture Evaluation (AE) framework specified in this document can be used in support of the Architecture Evaluation process defined in ISO/IEC/IEEE 42020. Specific frameworks can be derived from this generic framework, which can provide a mapping to the system life cycle processes in ISO/IEC/IEEE 15288 or to the software life cycle processes in ISO/IEC/IEEE 12207.
Logiciel, systèmes et entreprise — Cadre d'évaluation de l'architecture
General Information
Standards Content (Sample)
INTERNATIONAL ISO/IEC/
STANDARD IEEE
First edition
2019-07
Software, systems and enterprise —
Architecture evaluation framework
Logiciel, systèmes et entreprise — Cadre d'évaluation de
l'architecture
Reference number
©
ISO/IEC 2019
©
IEEE 2019
© ISO/IEC 2019
© IEEE 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org stds.ipr@ieee.org
www.iso.org www.ieee.org
Published in Switzerland
© ISO/IEC 2019 – All rights reserved
ii © IEEE 2019 – All rights reserved
Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Conceptual foundation . 5
4.1 General . 5
4.2 Architecture evaluation context . 6
4.3 Architecture evaluation tiers . 7
4.3.1 Evaluation synthesis . 7
4.3.2 Value assessment . 9
4.3.3 Architectural analysis .11
4.4 Architecture evaluation conceptual model .12
4.5 Comparison between assessment and analysis.13
4.6 Architecture evaluation factors .14
4.7 Customized architecture evaluation frameworks .14
4.8 Tailoring .15
5 Conformance .16
5.1 General .16
5.2 Creating AE artifacts.16
5.3 Using generic AE framework to conduct AE efforts .16
5.4 Verbal forms for the expression of provisions .17
6 Architecture evaluation framework elements .17
6.1 Evaluation synthesis .17
6.1.1 General requirements .17
6.1.2 Architecture evaluation objectives .18
6.1.3 Architecture evaluation approaches .19
6.1.4 Architecture evaluation factors .19
6.1.5 Architecture evaluation results .20
6.2 Value assessment .20
6.2.1 General requirements .20
6.2.2 Value assessment objectives .21
6.2.3 Value assessment methods .21
6.2.4 Value assessment factors .22
6.2.5 Value assessment results .22
6.3 Architectural analysis .23
6.3.1 General requirements .23
6.3.2 Architectural analysis objectives .23
6.3.3 Architectural analysis methods .24
6.3.4 Architectural analysis factors .25
6.3.5 Architectural analysis results.25
7 Customized architecture evaluation frameworks .26
7.1 General requirements .26
7.2 Framework requirements for architecture evaluation .27
7.3 Framework requirements for value assessment .27
7.4 Framework requirements for architectural analysis .27
7.5 Framework requirements for architecture evaluation work products .27
8 Architecture evaluation work products .27
8.1 General requirements .27
8.2 Architecture evaluation plan.28
8.2.1 AE plan requirements .28
© ISO/IEC 2019 – All rights reserved
© IEEE 2019 – All rights reserved iii
8.2.2 AE plan recommendations .28
8.2.3 AE plan permissions .29
8.3 Architecture evaluation report .30
8.3.1 AE report requirements .30
8.3.2 AE report recommendations .31
8.3.3 AE report permissions .32
Annex A (informative) Value and quality concepts .34
Annex B (informative) Relationship to other standards .46
Annex C (informative) Architecture evaluation examples .48
Annex D (informative) Example architecture evaluation frameworks .64
Bibliography .74
IEEE notices and abstract .78
© ISO/IEC 2019 – All rights reserved
iv © IEEE 2019 – All rights reserved
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
rules given in the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating
Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its
standards through a consensus development process, approved by the American National Standards
Institute, which brings together volunteers representing varied viewpoints and interests to achieve the
final product. Volunteers are not necessarily members of the Institute and serve without compensation.
While the IEEE administers the process and establishes rules to promote fairness in the consensus
development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of
the information contained in its standards.
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso
.org/iso/foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering, in cooperation with the Systems and Software
Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards
Development Organization cooperation agreement between ISO and IEEE.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
© ISO/IEC 2019 – All rights reserved
© IEEE 2019 – All rights reserved v
Introduction
The complexity of human-made systems has grown to an unprecedented level. This complexity leads
to new opportunities and greater challenges for organizations that conceive, develop, industrialize,
produce, maintain, utilize, recycle and dismantle enterprises, systems and software, and for various
stakeholders that are impacted by these things. To address these opportunities and challenges,
organizations increasingly apply concepts, principles, procedures and tools to drive better architecture
strategies, make better architecture-related decisions, create more useful and effective architectures
and improve architecture maturity. Architecture-related activities are not only strategic in nature; they
are tactical and operational as well. Furthermore, the use of architecture frameworks, architecture
description languages and generalist modeling languages have become common practice in commercial,
public service, government, civil and military domains.
The concept of architecture used in this document goes beyond the case where the architecture entity
is a system. Architecture is increasingly being applied to things not normally thought of as systems,
including entities with system-like structure and behavior such as enterprises, services, data, business
functions, mission areas, product lines, families of systems, software items, etc. This allows for a
more generalized usage of the concept of architecture when the evaluation elements specified in this
document are applied.
Architecture evaluations are performed for many reasons, such as:
a) determining if an entity of interest has been or is being architected in such a way that it fulfils its
intended purpose (or can be changed in a way that suits a new purpose);
b) evaluating the effectiveness and suitability of an architecture towards addressing stakeholder
needs and expectations;
c) identifying risks for mitigation;
d) identifying opportunities for the improvement of an entity or its architecture;
e) clarifying the problem space and stakeholder needs; and
f) assessing progress towards meeting architecture objectives.
Architecture evaluations can be performed on any kind of architecture, including a reference
architecture, an architecture for a family of systems or an architecture for a product line where there
are multiple kinds of architecture entities for a single architecture.
This document provides a generic, conceptual guiding framework that can be used for the planning,
execution and documentation of architecture evaluations. Execution is addressed by specification
of evaluation elements that can be used during performance of an evaluation effort. Planning
and documentation are addressed by specification of work products for the evaluation effort. An
organization using this document can establish specific frameworks for the work products and the
evaluation elements that can be used as the basis for multiple, recurring architecture evaluation
efforts. An organization can also establish tools, methods, best practices, capabilities and resources
based on the generic framework provided in this document. The generic framework makes it easier
to compare evaluations and evaluation frameworks used in specific cases. Implementation of the
proposed architecture framework will in time result in improvement of architecture maturity of the
organization.
© ISO/IEC 2019 – All rights reserved
vi © IEEE 2019 – All rights reserved
INTERNATIONAL STANDARD ISO/IEC/IEEE 42030:2019(E)
Software, systems and enterprise — Architecture
evaluation framework
1 Scope
This document specifies the means to organize and record architecture evaluations for enterprise,
systems and software fields of application.
The aim of this document is to enable architecture evaluations that are used to:
a) validate that architectures address the concerns of stakeholders;
b) assess the quality of architectures with respect to their intended purpose;
c) assess the value of architectures to their stakeholders;
d) determine whether architecture entities address their intended purpose;
e) provide knowledge and information about architecture entities;
f) assess progress towards achieving architecture objectives;
g) clarify understanding of problem space and of stakeholder needs and expectations;
h) identify risks and opportunities associated with architectures; and
i) support decision making where architectures are involved.
NOTE This document addresses the evaluation of an architecture and not an evaluation of the architecture
description’s suitability. Matters concerning the evaluation of the architecture description fall within the scope
of the architecture conceptualization and architecture elaboration processes as defined in ISO/IEC/IEEE 42020.
However, it is sometimes the case that the architecture description is evaluated concurrently with the evaluation
of the architecture itself.
The entity being evaluated can be of several kinds, as illustrated in the following examples: enterprise,
organization, solution, system, subsystem, business, data (as a data element or data structure),
application, information technology (as a collection), mission, product, service, software item, hardware
item, etc. The kind of entity can also be a product line, family of systems, system of systems, etc. It also
spans the variety of applications that utilize digital technology such as mobile, cloud, big data, robotics,
Internet of Things (IoT), web, desktop, embedded systems, and so on.
The generic Architecture Evaluation (AE) framework specified in this document can be used in support
of the Architecture Evaluation process defined in ISO/IEC/IEEE 42020. Specific frameworks can be
derived from this generic framework, which can provide a mapping to the system life cycle processes in
ISO/IEC/IEEE 15288 or to the software life cycle processes in ISO/IEC/IEEE 12207.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
© ISO/IEC 2019 – All rights reserved
© IEEE 2019 – All rights reserved 1
ISO, IEC and IEEE maintain terminological databases for use in standardization at the following
addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
— IEEE Standards Dictionary Online: available at: http: //ieeexplore .ieee .org/xpls/dictionary .jsp
1)
NOTE Definitions for other terms typically can be found in ISO/IEC/IEEE 24765 .
3.1
architecture
fundamental concepts or properties of an entity in its environment (3.7) and governing principles for
the realization and evolution of this entity and its related life cycle processes
Note 1 to entry: Architecture entity (3.3) is the term used in this document when referring to the entity being
architected or the entity subject to architecture processes. The fundamental concepts or properties of the
architecture entity are usually intended to be embodied in the entity’s components, the relationships between
components, and the relationships between the entity and its environment.
Note 2 to entry: The concept of architecture used in this document applies broadly to the entity being architected
or evaluated. This allows for a more generalized usage when the elements in this document are applied.
Note 3 to entry: The entity to be architected can be of several kinds, as illustrated in the following examples:
enterprise, organization, solution, system, subsystem, business, data (as a data element or data structure),
application, information technology (as a collection), mission, product, service, software item, hardware item,
product line, family of systems, system of systems, etc. It also spans the variety of applications that utilize digital
technology such as mobile, cloud, big data, robotics, Internet of Things (IoT), web, desktop, embedded systems,
and so on.
Note 4 to entry: Representation of the concepts or properties of an entity and governing principles is captured in
architecture models.
Note 5 to entry: Architectures can address a wide range of concerns (3.6) expressed, for example, through
architecture views and models, as illustrated in the following examples associated with particular kinds
of architectures such as: security architecture, functional architecture, physical architecture, resilience
architecture, etc.
[SOURCE: ISO/IEC/IEEE 42020:2019, 3.3]
3.2
architecture description
work product used to express an architecture (3.1)
Note 1 to entry: This document does not require the existence or use of an architecture description when
performing an architecture evaluation (3.4). Some value (3.10) assessment methods do not demand existence
of documented architecture models or views. Examples are customer focus group, expert panels and quality
workshops where sufficient knowledge of the architecture is in the people participating in use of these methods.
The same is true for architectural analysis in that not all methods applied here necessarily need an explicit
description of the architecture.
[SOURCE: ISO/IEC/IEEE 42010:2011, 3.3, modified — The abbreviated term “AD” has been removed;
Note 1 to entry has been added.]
1) System and software engineering — Vocabulary, available at www .computer .org/sevocab.
© ISO/IEC 2019 – All rights reserved
2 © IEEE 2019 – All rights reserved
3.3
architecture entity
thing being characterized by an architecture (3.1)
EXAMPLE The following are kinds of architecture entities that can be dealt with by the architecture
processes: enterprise, organization, solution, system (including software systems), subsystem, business, data
(as a data element or data structure), application, information technology (as a collection), mission, product,
service, software item, hardware item, product line, family of systems, system of systems, collection of systems,
collection of applications, etc.
Note 1 to entry: When referring to the architecture itself of these architecture entities, it is common practice to
place the name of the kind of entity in front of the word architecture. For example, the phrase system architecture
is used when the thing being dealt with during the architecting effort is a system. Likewise, for the other kinds of
entities that are being dealt with during the architecting effort.
[SOURCE: ISO/IEC/IEEE 42020:2019, 3.6, modified — The words “considered, described, discussed,
studied, or otherwise addressed during the architecting effort” have been replaced with “characterized
by an architecture”.]
3.4
architecture evaluation
AE
judgment about one or more architectures (3.1) with respect to the specified evaluation objectives
EXAMPLE 1 Various kinds of judgments could be made during an architecture evaluation, such as validating
that architectures address the concerns (3.6) of stakeholders (3.9), assessing the quality of architectures
with respect to their intended purpose, assessing the value (3.10) of architectures or architecture entities to
their stakeholders, determining whether architecture entities address their intended purpose, providing
knowledge and information about architecture entities and identifying risks and opportunities associated with
architectures.
EXAMPLE 2 Examples of architecture evaluations are provided in Annex C.
Note 1 to entry: A decision regarding disposition of the architecture is usually outside the scope of an AE effort,
although it could be done in conjunction with the AE effort. The AE results are often reported to a decision maker
who makes the actual determination of disposition based on those results and sometimes also on other factors
(3.8) not considered by the AE effort. Sometimes this determination is called an “evaluation” but for the purpose
of this document, the evaluation is limited to just the judgment with respect to relevant evaluation objectives.
3.5
architecture evaluation framework
conventions, principles and practices for evaluating architectures (3.1) in a consistent and
repeatable manner
EXAMPLE Examples of AE frameworks are provided in Annex D for the following cases: Architecture Tradeoff
Analysis Method (ATAM), the Method Framework and QUASAR method and Analysis of Alternatives (AoA).
Note 1 to entry: This framework can be generic in nature or specific to a domain of application, a collection of
concerns (3.6) to be examined or a methodology. This document defines a generic AE framework and a specific
AE framework can be derived from the generic framework.
Note 2 to entry: An AE framework can enable AE efforts to be performed in a more consistent and repeatable
manner.
Note 3 to entry: The evaluation framework can consist of different sub-architecture frameworks for an entity
with many layers or levels. These could be defined and consolidated as part of the comprehensive architecture
framework package.
© ISO/IEC 2019 – All rights reserved
© IEEE 2019 – All rights reserved 3
3.6
concern
matter of interest or importance to a stakeholder (3.9)
EXAMPLE Affordability, agility, availability, dependability, flexibility, maintainability, reliability, resilience,
usability and viability are examples of concerns. Survivability, depletion, degradation, loss, obsolescence are
examples of concerns. The PESTEL mnemonic is a reminder of other possible areas of concern: political, economic,
social, technological, environmental, and legal. A longer list of examples is provided in 4.2.
Note 1 to entry: The concept of concern is similar to “quality attributes” as used in the ATAM. See Annex D for an
overview of the ATAM approach. In ATAM, quality attributes are typically decomposed into concerns.
Note 2 to entry: The concept of concern is similar to the concept of quality. See A.4 for an overview of the quality
concept.
[SOURCE: ISO/IEC/IEEE 42020:2019, 3.8, modified — In EXAMPLE, reference to 4.2 has been added;
Notes 1 and 2 to entry have been added.]
3.7
environment
context determining the setting and circumstances of influences upon an architecture entity (3.3) or
upon which the architecture entity can have an influence
Note 1 to entry: There can be things beyond the environment that have an indirect impact on the architecture
entity. It could be important to account for these indirect effects by incorporating these causative agents in the
environment even though they are not usually considered to be within the immediate context. Value (3.10) chain
analysis is an example of where this is done.
3.8
factor
circumstance, fact or influence that contributes to a result or outcome
Note 1 to entry: A factor is something that contributes causally to a result. Factors identification can sometimes
be driven by knowledge of desired effects.
3.9
stakeholder
role, position, individual or organization having a right, share, claim or other interest in an architecture
entity (3.3) or its architecture (3.1) that reflects their needs and expectations
[SOURCE: ISO/IEC/IEEE 42020:2019, 3.20]
3.10
value
regard that something is held to deserve; the importance, worth, or usefulness of something to
somebody
Note 1 to entry: Architecture evaluation (3.4) is focused primarily on the value of an architecture (3.1) with respect
to stakeholder (3.9) concerns (3.6) or architecture objectives for that thing. However, sometimes the purpose of
the evaluation effort is, by inference, to determine the impact of the architecture on the value of the architecture
entity (3.3) when the entity is developed or evolved to align with the architecture concepts and properties.
Note 2 to entry: The determination of architecture value can take various aspects into account, such as worth,
significance, importance, usefulness, benefit, and quality. These words have similar but not identical meaning.
Worth is usually what one is willing to pay for something. Significance is about being worthy of attention.
Importance is about the state or fact of being of great significance or value. Usefulness is about serving some
purpose, or about being advantageous, helpful or of good effect. Benefit is about an advantage or profit gained
from something. Quality is about the degree of excellence of something. Throughout this document, the term
value is used to mean one or more of these other concepts, as appropriate.
© ISO/IEC 2019 – All rights reserved
4 © IEEE 2019 – All rights reserved
Note 3 to entry: Even though a new architecture could be found to be of greater value with respect to the current
situation, this needs to be balanced against the costs and risks of adopting the new architecture. So, it is not
necessarily the case that when examining architecture alternatives, the one with the maximum value is proposed
as the preferred choice since the extra cost or risk of this architecture might not be worth the extra burden. This
is sometimes referred to as the benefit-cost ratio or some other term with similar meaning.
Note 4 to entry: Value is determined primarily in the Value Assessment Tier of the evaluation framework
illustrated in Figure 1. Requirements on value assessment are specified in 6.2.
4 Conceptual foundation
4.1 General
This clause introduces key concepts used in this document with respect to architecture evaluation. The
terms and the concepts presented in this clause are used in Clauses 6 through 8 to express requirements.
The conceptual model of architecture evaluation is presented in parts throughout this clause.
The generic AE framework work products and elements, illustrated in Figure 1 and specified in this
document, can be used in support of the Architecture Evaluation process defined in ISO/IEC/IEEE 42020.
Specific frameworks can be derived from this generic framework.
Figure 1 — Generic architecture evaluation framework
These specific frameworks may range from those targeting industry segments, such as automotive
or refinery operations, to those targeting common business processes, such as portfolio management
and program planning, and to those targeting common business architectures, such as banking, retail,
insurance, telecom, travel and hospitality, etc. Specific frameworks allow the user to capture and reuse
concepts common to many enterprises and thereby increase the efficiency with which architectures
can be evaluated.
Architecture evaluation makes a judgment with respect to how well architecture objectives have been
or will be achieved. It can provide answers to an identified set of questions to, for example, provide
inputs to strategic decision making (such as whether it would be cheaper in the long run to modify
an existing architecture to close value gaps), or to produce a new architecture that better addresses
current and future stakeholder needs. An architecture evaluation can also provide inputs to decisions
made at the operational and tactical levels. For example, the evaluation may provide useful information
regarding capability limitations of the entity in question.
© ISO/IEC 2019 – All rights reserved
© IEEE 2019 – All rights reserved 5
The subclauses below describe the elements used in each tier of the generic framework and describe
the different kinds of specific frameworks that utilize these elements.
4.2 Architecture evaluation context
Figure 2 depicts the context of architecture evaluation in terms of key concepts and the relations
between them.
NOTE 1 The graphical notation used in this document is a simplified version of entity-relationship modeling.
NOTE 2 Only the key associations are shown in the diagrams.
Figure 2 — Context of architecture evaluation
The AE effort can be performed at many stages of entity development, including during conceptual
design through to during the operation and maintenance. The evaluation may need to be updated to
reflect changes as the entity design progresses through its lifecycle.
An AE effort is often performed to determine the potential or actual value of the associated architecture
entity. However, the primary focus of the AE effort is on the value of the architecture itself even though
the ultimate aim could be a determination of the value of the architecture entity, or the impact on
the environment, or the impact on the business that uses the architecture entity, etc. But the value of
the architecture entity or other benefits might be determined by some other effort, such as system
analysis, requirements analysis, business needs analysis, portfolio management, program assessment
and evaluation, environmental impact assessment, etc.
Architecture evaluation makes a judgment regarding the extent to which architecture objectives have
been or will be achieved. Because of this, it is dealing with the degree to which the architecture provides
things such as needs satisfaction, feasibility, understandability, usability desired qualities.
The environment within which the architecture entity is situated could provide strategic context for
determining the ways in which the architecture evaluation is conducted. The environment could also
be a key factor in understanding the nature of stakeholder concerns. States and modes of operation of
the architecture entity are often from the usage perspective of entities in the environment.
Stakeholders have interests in the architecture or associated architecture entities. These interests
(called concerns) are usually the primary focus of architecture evaluation. This judgment provided by
the AE effort represents the extent to which stakeholder concerns have been or will be satisfied by
decisions that affect the associated architecture entities or their environments. This judgment can also
© ISO/IEC 2019 – All rights reserved
6 © IEEE 2019 – All rights reserved
represent the extent to which the architecture fulfills its intended purpose. Ways to measure this will
be identified or specified during the AE effort along with the means by which these measures will be
ascertained.
NOTE 3 Determination of the extent to which concerns are satisfied could entail either measurement of the
degree to which something is done or a determination of whether something is true or not. Pass/fail criteria
could need to be established prior to performing the evaluation. These criteria could be defined in the AE plan or
could be established by the organization as a matter of policy or directive.
EXAMPLE 1 Stakeholders include people and organizations such as: users, operators, acquirers, owners,
suppliers, developers, builders and maintainers. It also includes authorities engaged in certifying the architecture
entity for a variety of purposes such as its readiness for use, conformance to legal provisions and compliance
with regulations and policies with respect to safety, security, privacy, environmental impact, etc., as well as
evaluators such as funding agencies, integration authorities, governance boards, management boards, client
representatives and regulatory authorities. Stakeholders can go beyond individuals and organizations to also
include things like governmental bodies, supply chains, value chains, institutions, and social groups.
EXAMPLE 2 Concerns include such things as: affordability, agility, alignment with business goals and
strategies, autonomy, availability, behavior, business impact, capability, complexity, compliance to regulation,
concurrency, control, cost, customer experience, data accessibility, deadlock, disposability, environment impact,
error and exception handling, extensibility, evolvability, feasibility, flexibility, functionality, graceful degradation,
information assurance, interoperability, inter-process communication, known limitations, maintainability,
misuse, mission impact, modifiability, modularity, openness, performance, portability, privacy, quality of service,
recoverability, reliability, resilience, resource utilization, schedule, security, shortcomings, state transitions
through lifecycle, scalability, software and systems assurance (ISO/IEC 15026-1), structure, subsystem
integration, architecture entity features, architecture entity properties, architecture entity purposes, usability,
usage, viability, etc.
EXAMPLE 3 The PESTEL mnemonic is a reminder of other possible areas of concern: political, economic,
social, technological, environmental, and legal. Survivability, depletion, degradation, loss and obsolescence are
other examples of areas of concern. Other mnemonics that could be useful include STEEPLED that adds ethics
and demographic factors, SPELIT that adds intercultural factors, STEER that adds regulatory factors and STEP
that adds ecological factors.
EXAMPLE 4 Examples of value include such things as: physiological well-being, safety from harm, feelings,
aesthetics, price, savings, sense of belonging, self-esteem and self-actualization.
Architecture principles, although not shown in the diagram, will shape the architecture and can perform
a key role in the architecture evaluation. An understanding of these principles can help guide proper
evaluation of an architecture. These principles will influence selection of AE factors used throughout
the evaluation effort and help in the identification of relevant concerns. Architectural features and
functions need to be consistent with the architecture principles. See Reference [17] and [18].
4.3 Architecture evaluation tiers
4.3.1 Evaluation synthesis
Synthesis involves the combination of results from multiple value assessments to determine to what
extent the evaluation objectives will be achieved. Stakeholders who have concerns about the subject
of the evaluation could have specific goals that should be addressed in the evaluation. (These concerns
could be about the architecture, the architecture entity or both.) These goals should be considered when
establishing the factors and objectives to be used in the evaluation. These goals might not correspond
to the original goals for the architecture when it was initially conceived.
NOTE 1 The experts involved in the evaluation are also stakeholders and can bring important evaluation
objectives that are not a known concern for traditional stakeholders (such as acquirer, user, service provider),
but are concerns that the profession defines to be important (and where the evaluators could be the best placed
stakeholders to represent the profession).
Architecture trade-offs are identified and characterized during architecture development. However,
they can be revisited during the evaluation synthesis. Trade-offs among stakeholder concerns and
feasibility limitations will be identified. Typical trade-offs to consider are the following: cost vs
© ISO/IEC 2019 – All rights reserved
© IEEE 2019 – All rights reserved 7
performance, cost vs schedule, weight vs speed, accuracy vs timeliness, acquisition cost vs operating
cost, ease of use vs security, flexibility vs predictability, agility vs robustness, risk vs reward, etc.
Trade-offs could be with respect to the various factors within a single architecture or across alternative
architectures under examination.
An AE effort examines one or more architectures with respect to potential stakeholder concerns about
the associated architecture entities. Figure 3 depicts AE elements that can be used in an evaluation
synthesis effort in terms of the key concepts and the relations between them. Most of these concepts
are also used in related standards described in Annex B.
NOTE 2 Value is determined primarily in the Value Assessment Tier of the evaluation framework illustrated in
Figure 1. Requirements on value assessment are specified in 6.2.
The evaluation synthesis effort is the result of applying the concepts in the document during the
evaluation of one or more architectures to determine their value to stakeholders or the extent to which
the architecture objectives are satisfied.
NOTE 3 The evaluation synthesis eff
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...