ISO/IEC 14102:1995
(Main)Information technology — Guideline for the evaluation and selection of CASE tools
Information technology — Guideline for the evaluation and selection of CASE tools
Provides guidance on identifying organizational requirements for CASE tools, on mapping those requirements to CASE tool characteristics to be evaluated and describes a process for selecting the most appropriate CASE tool from several tools, based on measurements of the defined characteristics.
Technologies de l'information — Lignes directrices pour l'évaluation et la sélection d'outils CASE
General Information
Relations
Standards Content (Sample)
INTERNATIONAL
lSO/IEC
STANDARD 14102
First edition
1995-1 l-l 5
Information technology - Guideline for
the evaluation and selection of CASE tools
Lignes directrices pour I’baluation et Ia
Technologies de I’information -
sklection d’outils CASE
Reference number
ISO/IEC 14102:1995(E)
---------------------- Page: 1 ----------------------
ISOLCEC 14102:1995(E)
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Introduction.vi
1
Scope .
Normative references . 3
Definitions and acronyms . 4
4
31 0 Definitions .
5
32 l Acronyms .
..... 6
Overview of evaluation and selection of CASE tools
41 Initiation process . 6
l
................................. 7
42 0 Structuring process
7
43 l Evaluation process .
44 Selection process . 8
.
8
45 . General process considerations .
Sequencing of processes . 8
4.5.1
9
4.5.2 Reducing tost and risk .
10
Initiation process .
51 l Goal setting . 10
........................ 11
52 l Establishing selection criteria
53 Project planning and control . 12
l
13
Structuring process .
............................. 13
61 l Requirements definition
6.1.1 Organizational information gathering . 13
Requirements identification . 14
6.1.2
15
6.1.3 Requirements structuring .
..................... 16
62 l CASE tool information gathering
16
63 . Identifying final candidate CASE tools .
0 ISO/IEC 1995
All rights reverved. Unless otherwise specified, no part of this publication may be reproduced
or utilized in any form or by any means, electronie or mechanical, including photocopying and
microfilm, without Permission in writing from the publisher.
ISO/IEC Copyright Office l Case Postale 56 l CH-121 1 Geneve 20 l Switzerland
Printed in Switzerland
ii
---------------------- Page: 2 ----------------------
ISO/IEC 14102:1995(E)
01s0/lEc
7 Evaluation process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
71 l Preparing for evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
72 l Evaluating CASE tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.2.1
7.2.2 Rating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.2.3 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
73 l Evaluation reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
8 Selection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
81 Preparing for selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
0
82 l Applying the selection algorithm . . . . . . . . . . . . . . . . . . . . . . 23
83 l Recommending a selection decision . . . . . . . . . . . . . . . . . . . . 24
84 Validating the selection decision . . . . . . . . . . . . . . . . . . . . . . 24
l
9 CASE tool characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
91 l Functionality - characteristics related to life-cycle processes. 25
............. 26
9.1.1 Characteristic: Management Process
26
9.1.2 Characteristic: Development Process .
9ele2el Subcharacteristic: Modeling . 27
..... 29
9ele2e2 Subcharacteristic: Construction
Characteristic: Maintenance Process . 30
9.1.3
........... 3 1
9.1.4 Characteristic: Documentation Process
9.1.5 Characteristic: Configuration Management Process 3 1
........ 32
9.1.6 Characteristic: Quality Assurance Process
33
9.1.7 Characteristic: Verification Process .
............... 34
9.1.8 Characteristic: Validation Process
Functionality - characteristics related to CASE tool usage, . 36
92 l
9.2.1 Characteristic: Environment in which the CASE tool
operatese . 36
Characteristic: CASE tool integrability. . 37
9.2.2
9e2e3 Characteristic: Aspects of the CASE tool’s application39
40
93 General quality characteristics .
l
................... 40
9.3.1 Characteristic: Functionality
4 1
9.3e2 Characteristic: Reliability .
4 1
Characteristic: Usability .
9.3.3
...................... 43
9e3e4 Characteristic: Effkiency
................. 44
9.3.5 Characteristic: Maintainability
44
9.3.6 Characteristic: Portability .
............ 45
94 l General characteristics not related to quality
45
9.4.1 Characteristic: Acquisition .
...
111
---------------------- Page: 3 ----------------------
01s0/lEc
ISOAJX 14102:1995(E)
................. 46
9.4.2 Characteristic: Implementation
.............. 46
9.4.3 Characteristic: Support Indicators
9.4.4 Characteristic: Evaluation or Certification . 47
Annex A Considerations on the use of this International
Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Annex B Examples of selection algorithms . . . . . . . . . . . . . . . . . . . 50
Annex C Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
iv
---------------------- Page: 4 ----------------------
ISOLIEC 14102:1995(E)
0 1s0/IEc
Foreword
ISO (the International Organization for Standardization) and IEC (the Inter-
national Electrotechnical Commission) form the specialized System for worldwide
standardization. National bodies that are members of ISO or IEC participate in the
development of International Standards through technical committees established
by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with
ISO and IEC, also take part in the work.
In the field of information technology, ISO and IEC have established a joint
technical committee, ISO/IEC JTC 1. Draft International Standards adopted by the
joint technical committee are circulated to national bodies for voting. Publication
as an International Standard requires approval by at least 75 % of the national
bodies casting a vote.
International Standard ISOLIEC 14102 was prepared by Joint Technical
Committee ISOLIEC JTC 1, Information technology, Subcommittee SC 7,
Sofnyare engineering.
Annexes A to C of this International Standard are for information only.
---------------------- Page: 5 ----------------------
ISOLIEC 14102: 1995(E) OISO/IEC
Introduction
Within Software engineering, Computer-Aided Software Engineering (CASE)
tools represent a major part of the supporting technologies used to develop and
maintain Software Systems. Their selection must be carried out with careful
consideration of both the technical and management requirements.
This International Standard defines both a sequence of processes and a structured
set of CASE tool characteristics for use in the technical evaluation and the ultimate
selection of a CASE tool. It follows the Software product evaluation model
described in ISOKEC 9 126: 199 1, Information technology - Software product
evaluation - Quality characteristics and guidelines for their use. This
International Standard adopts the general model of Software product quality
characteristics and subcharacteristics described in ISO/IEC 9 126, and extends
these when the Software product is a CASE tool; it provides product
characteristics unique to CASE tools. This larger set of characteristics is then
organized into five groups. This grouping provides a more manageable approach
to the Overall evaluation and selection process.
While the technical evaluation may answer how well a CASE tool meets its user’s
stated requirements, it may also answer the question of how well the tool meets its
claimed functionality.
The objective of the technical evaluation process is to provide quantitative results
on which the final selection tan be based. Measurement assigns numbers (or other
ratings) to attributes of entities; a major activity of evaluation is to obtain these
measurements for use in selection. The final selection results should aim to
achieve objectivity, repeatability and impartiality. These objectives and the
confidence in the outcomes will in part depend on the resources allocated to the
Overall evaluation and selection process. The user of this International Standard is
asked to deal with these issues at an early Stage.
To be widely acceptable, these CASE tool evaluation and selection processes must
be of value to the users of CASE tools, to the suppliers of CASE to
community at large. The information outlined in this International Standard should
lead to more tost effective selections of CASE tools and to a greater uniformity in
how CASE tool functions and features are described.
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD 0 ISO/lEC ISOLIEC 14102: 1995(E)
Information technology - Guideline for the
evaluation and selection of CASE tools
1 Scope
This International Standard deals with the evaluation and selection of CASE tools,
covering a partial or fi~ll Portion of the Software engineering life cycle. It
establishes processes and activities to be applied for the evaluation of CASE tools
and selecting the most appropriate CASE tools from several candidates. These
processes are generic, and organizations must tailor them to meet organizational
needs. The CASE tool evaluation and selection processes should be viewed in the
larger context of the organization’s technology adoption process.
This International Standard provides:
a. Guidance on identifling organizational requirements for CASE tools.
b . Guidance on mapping those requirements to CASE tool characteristics to
be evaluated.
C. A process for selecting the most appropriate CASE tool from several tools,
based on measurements of the defined characteristics.
Primary users of this International Standard are organizations that intend to adopt
CASE tools to support their Software life cycle processes. CASE tool suppliers
may also use this International Standard to describe characteristics of their CASE
tools.
This International Standard is not intended to apply to:
a. Software engineering frameworks whose purpose is to provide mechanisms
for data, control and presentation integration.
1
---------------------- Page: 7 ----------------------
ISO/IEC 14102:1995(E) 01s0/IEc
b . General purpose tools (e.g., word processors, Spreadsheets) which may be
used in Software engineering activities, nor CASE tools of very narrow
scope or specific purpose (e.g., a Compiler).
C. Planning for the implementation of CASE tools within an organization
(even though it is recognized that this is an important subject).
NOTE - A user of this International Standard may make the best possible selection of a CASE
tool and have no guarantee of a successful implementation. ISO/IEC JTC 1 SC7 WG4 is working on a
drafi technical report, Adoption of CASE Tools, which addresses this subject.
This International Standard contains a set of processes, activities, and tasks
designed to be tailored. The tailoring process is the selection of applicable
processes, activities and tasks.
Compliance with this International Standard is defmed as the Performance of the
processes, activities, and tasks selected from this International Standard for the
evaluation and selection project. Any organization imposing this International
Standard as a condition of trade is responsible for specifling the minimum set of
required processes, activities, and tasks which constitute compliance for a given
application of this International Standard. Defming and documenting that
specification forms part of the initiation process (clause 5).
---------------------- Page: 8 ----------------------
ISO/IEC 14102:1995(E)
0 ISO/IEC
2 Normative references
The following Standards contain provisions which, through reference in this text,
At the time of publication, the
constitute provisions of this International Standard.
editions indicated were valid. All Standards are subject to revision, and Parties to
agreement based upon this International Standard are encouraged to investigate
the possibility of applying the most recent editions of the Standards indicated
below. Members of IEC and ISO maintain registers of currently valid International
Standards.
ISO 5807: 1985, Informationprocessing - Documentation symbols and
conventions for data, program and system flowcharts, program network Charts
and system resources Charts.
ISOIIEC 12 119: 1994, Information technology - Software packages - Quality
requirements and testing.
ISOIIEC 12207: 1995, Information technology - Software ltfe cycle processes.
ISOIIEC 9 126: 199 1, Information technology - Software product evaluation -
Quality characteristics and guidelines for their use.
---------------------- Page: 9 ----------------------
OISO/IEC
ISO/IEC 14102:1995(E)
3 Definitions and acronyms
31 Definitions
0
For the purposes of this International Standard, the following definitions apply.
3.1.1 assessment: An action of applying specific documented criteria to a
specific Software module, package or product for the purpose of determining
acceptance or release of the Software module, package or product. (ISO/IEC
9126: 1991)
3.1.2 atomic subcharacteristic: The highest level evaluation categories are
called characteristics. Characteristics are usually subdivided into
subcharacteristics. Many subcharacteristics may be further subdivided into lower
level subcharacteristics. At the lowest-level, when no further subdivision is
appropriate, the subcharacteristics are referred to as atomic subcharacteristics.
3.1.3 CASE tool: A Software product that tan assist Software engineers by
providing automated support for Software life-cycle activities as defined in
ISO/IEC 12207:1995.
NOTES
1 - A CASE tool may provide support in only selected functional areas or in a wide variety of functional
2 - CASE tools may be used in several modes:
0
As stand alone tools; in this case, only compatibility with environment elements should be
addressed.
0
In small groups which communicate directly with one another; it may be supposed that
integration is predefined, perhaps proprietorily.
0
In the presence of a larger framework of the SEE; in this case the ability of the tool to use the
relevant Services of the framework should be addressed.
3.1.4 characteristic: An aspect of a product by which it tan be described and
evaluated. A characteristic may be refined into multiple levels of subcharacteristics
that bear on its ability to satisfy stated or implied needs.
3.1.5 measurement: The action of applying a Software quality metric to a
specific Software product. (ISO/IEC 9 126: 199 1)
---------------------- Page: 10 ----------------------
0 1s0mx ISOLIEC 14102:1995(E)
NOTES
1 - Measurement tan apply to metrics other than Software quality metrics.
measured indirectly by the application of metrics
2 - An Object may be measured directly, or may be
or representations of the Object.
information about
3.1.6 metric: A quantitative scale and method which tan be used to determine the
value a subcharacteristic takes for a specific Software product.
The action of mapping the measured value to the appropriate rating
3.1.7 rating:
level. Used to determine the rating level associated with the Software for a specific
quality characteristic. (ISO/IEC 9 126: 199 1)
NOTE - Rating and rating levels tan be applied to characteristics other than quality characteristics.
A range of values on a scale to allow Software to be
3.1.8 rating level:
classified (rated) in accordance with the stated or implied needs. Appropriate
rating levels may be associated with the different views of quality, i.e., users,
managers or developers. These levels are called rating levels. (ISO/IEC
9126:1991)
3.1.9 Software Engineering Environment: The Software engineering
environment (SEE) is that Portion of the System which provides automated
support for the engineering of Software Systems and the management of the
Software process. It includes platform, System Software, Utilities, and CASE tools
installed.
NOTE - The SEE architecture has two aspects:
0
the CASE tools which provide facilities for supporting life-cycle processes, and
0
framework which provides a set of capabilities that offer common Services used by
a general
the tools.
32 l Acronyms
BMT Bench Mark Test
Computer Aided Software Engineering
CASE
GUI Graphical User Interface
SEE Software Engineering Environment
Structured Query Language
SQL
---------------------- Page: 11 ----------------------
01s0AEc
ISO/IEC 14102:1995(E)
4 Overview of evaluation and selection of CASE tools
This section provides an overview of the evaluation and selection of CASE tools
discussed in this International Standard as shown in Figure 1. Evaluation and
selection of CASE tools includes four major processes:
Initiation Process
Structuring Process
Evaluation Process
Selection Process
1 Structuring 1
GT /.--.“’ \
I Selection 4 \
IRecommendation 1 T-*--. . . . . . . . . . . . . [r]
1 DataFbw
Figure 1 - Overview of evaluation and
selection of CASE tools
A key process is the structuring of a set of requirements against which candidate
CASE tools are to be evaluated, and upon which selection decisions will be based.
The CASE tool characteristics defined in clause 9 form the basis for requirements
structuring, and play a central role in the Overall process.
41 . Initiation process
The purpose of the initiation process is to defme the general objectives and
requirements of the intended evaluation and selection of CASE tools, to establish the
high level direction, and to define the management aspects of the effort (e.g.,
schedule, resources, tost).
The initiation process, discussed in detail in clause 5, is composed of three activities:
0
goal setting: provides the rationale and general policy for evaluation and
6
---------------------- Page: 12 ----------------------
ISO/IEC 14102:1995(E)
0 1s0/IEc
selection.
0
establishing selection criteria: provides criteria to be used in the subsequent
selection process.
0
project planning: results in a plan which includes generic planning
information and also information which defines the structure of the evaluation and
selection effort.
42 0 Structuring process
The purpose of the structuring process is to elaborate a set of structured
requirements, based upon the CASE tool characteristics of clause 9 against which
CASE tools should be evaluated, and to obtain the necessary information on CASE
It is assumed that a set of general organizational
tools to permit evaluation.
information and guidelines is available to be used as inputs.
The structuring process, discussed in detail in clause 6, is composed of three
activities:
0
requirements analysis: transforms organizational needs into measurable
structures.
0
CASE tool information gathering: captures a snapshot of the current state-
of-the-art in CASE tools.
0
identifying final candidate CASE tools: candidate CASE tools are identified
for evaluation using the results of the last two activities.
requirements may require revision. If this occurs, some repetition of
NOTE - During the evaluation,
activities of this, and subsequent processes may be necessary.
43
0 Evaluation process
The purpose of the evaluation process is to produce technical evaluation reports that
will be the major input for the selection process. Esch evaluation process results in
a Profile of the quality and other characteristics of the tool which was evaluated.
Comparisons between tools are not made as part of this process.
The evaluation process, discussed in detail in clause 7, is composed of three activities:
0
preparation for evaluation: finalization of the various details of the
7
---------------------- Page: 13 ----------------------
ISO/IEC 14102:1995(E) OISO/IEC
evaluation (e.g., scenario, subcharacteristics, metrics, tool characteristics) in an
evaluation plan.
0
evaluating CASE tools: measurement, rating and assessment.
0
evaluation reporting: an evaluation report is prepared which provides the
results of the evaluation for each CASE tool considered.
44 0 Selection process
The purpose of the selection process is to identify the most suitable CASE tool(s)
among the candidate tools, and to ensure that the recommended tool(s) meets the
original goals. The selection process compares the results of the evaluations of the
candidate tools to determine which is the most appropriate for selection.
The selection process, discussed in clause 8, is composed of four activities:
0
preparing for selection: the selection criteria are finalized and the selection
algorithm is defined.
0
assessing the evaluation results: the selection algorithm is applied to the
evaluation results.
0
recommending a selection decision: the best of the candidates is determined.
l
validating the selection decision: the recommended selection is validated
against the original goals.
45 General process considerations
l
There are several considerations that apply to the processes described in this
International Standard on a global basis. The intent is for the user of this International
Standard to tailor its application in such a way as to maximize the probability of a
successful evaluation and selection process, and minimize its tost and risk.
4.5.1 Sequencing of processes
This International Standard does not impose the sequence of process activities
described above and in the following sections. It is up to the organization to select
the relevant processes and activities needed to meet its evaluation and selection goals.
The organization will decide which to employ, in what sequence, and with what
8
---------------------- Page: 14 ----------------------
0 ISO/IEC ISO/IEC 14102:1995(E)
degree of parallelism. The sequencing of the processes’ activities is then documented
in the project plan developed during the initiation process.
4.5.2 Reducing tost and risk
In general, organizations which apply this International Standard will want to
minimize the tost ofthe entire evaluation and selection process to the extent possible,
while maintaining the level of effort necessary to select the most appropriate CASE
tool(s) for their use. These objectives may be addressed by minimizing the number of
tools evaluated, minimizin g the tost of evaluating specific tools, and ensuring that the
formality of the process is appropriate to the organization.
The activities of CASE tool information gathering and identifying final
candidates for selection (see clause 6) effectively allow the user of this International
Standard to Screen the available tools against the organization’s needs, and eliminate
fi-om consideration tools which do not, or are not likely to, substantially address the
organization’s needs.
NOTE 1 - It may be that the organization is unable to find any tool which appear likely to
sufficiently meet its needs. In such a case, the stated needs themselves should be re-examined, and if they
are found to accurately reflect the organization’s actual requirements for technology improvement, the
Overall evaluation and selection process may be abandoned. Similarly, if the final candidate tools appear
to be marginal in addressing the organization’s needs, the level of detail and formal@ of the subsequent
activities should be made to reflect the risk factor, and the organization should be prepared to not select a
tool if the evaluation process so indicates, as the typical tost of bringing a new tool into operational use is
substantial.
Evaluations of candidate tools may have already been performed and be
available to the organization. Such information may be used to reduce the tost of
candidate tool evaluation.
NOTE 2 - Previous evaluations which have been performed on a different Version of the candidate
tool may still yield useful information. Similarly, evaluations which addressed a different set of
organizational needs may still provide useM information.
This International Standard calls for the development of several Plans and
reports, and implicitly, for their review by various Personne1 within the organization.
In addition, activities are required to perform the four processes outlined. The format
and level of detail of the data products is left to the discretion of the organization, as
is the level of effort necessary to perform the activities.
NOTE 3 - Some organizations may need to limit the scope, detail and formal@ of the processes
to apply this International Standard within existing resource constraints.
9
---------------------- Page: 15 ----------------------
ISO/IEC 14102:1995(E) OISO/IEC
5 Initiation process
The evaluation and selection processes require the agreement of management. In line
with this agreement, a set of goals for the introduction (or enhancement) of CASE
technology will be established. A set of CASE tool selection guidelines will be
identified and a project plan developed. The process is shown in Figure 2.
Goal
Setting
l-J\
. . . . . .**.
4
Dah Fbw
I
Figure 2 - Overview of initiation process
51 Goal setting
0
In developing
The development of a set of realistic goals is a necessary first activity.
goals, both a rationale for acquisition (why acquire a CASE tool) and a general policy
for acquisition (what type of tool to acquire and how to do it) should be developed.
NOTE - Goal setting activities, including possibly the identification of selection criteria, may have already
been perfomxd as a patt of other efforts Prior to formally entering the initiation process of evaluation and
selection of CASE tools.
The following tasks should be performed:
Develon rationale for acauisition:
Review the organization’s current softwar-e development process, determining
its maturity and areas of concern.
Review the current state of CASE technology and observe trends for
consideration as future reference technology.
Compare the organization’s current practices to possible future practices if
10
---------------------- Page: 16 ----------------------
ISWIEC 14102:1995(E)
0 ISO/IEC
CASE tools are adopted and identifl areas of potential benefit.
Identify probable impacts of CASE tools on the organization; e.g., areas
where training and education, procedure guides, and technical support are needed to
effectively deploy CASE technology.
Define goals and expectations:
Set Overall goals (e.g., productivity improvement, quality improvement,
enhanced process manageability).
Define evaluation and selection constraints (e.g.,cost, schedule, resources).
Quantify and classify expectations (based upon goals).
t general nolicy for acquisition:
IdentifSI constraints on tool acquisition (e.g., implementation tost, schedule,
other resources).
Develop alternate approaches to introducing/augmenting CASE technology
(e.g., buy a tool, modify an existing tool, develop a new tool).
Assess the feasibility of the various alternatives in light of organizational
readiness, technical considerations, Performance specifications, and resources.
The goals and expectations established here will be used to guide subsequent activities
in the Overall process and, finally, to validate the selection decision.
52 l Establishing selection criteria
Based upon the goals and expec ztations d leveloped above, selection criteria should be
established:
Decompose the high 1 evel goal s into a set of selection criteria to make the
(go/no go) selection decision.
NOTE 1 - The selection criteria should be objective and quantitative. Esch selection criterion
should include some defined threshold specified on which the major go/no go decision will be made during
selection.
Define the relative importante of the selection criteria.
11
---------------------- Page: 17 ----------------------
OISO/IEC
ISOLIEC 14102:1995(E)
NOTE 2 - The relative importante of the selection criteria will be used to determine the weights
assigned to tool characteristics and subcharacteristics for evaluation.
Define the level of detail and the nature of the evaluation activities to be
performed.
NOTE 3 - The nature of the evaluation activities covers the methods used in collecting the data.
Reference, for example, how the data are measured, collected with predefmed criteria, or based upon
subjective Observation.
Define the evaluation/selection scenario to be performed (see Annex A).
53 Project planning and control
0
Based upon the goals and selection criteria which have been established for the Overall
evaluation and selection process, a project plan should be created and a control
mechanism implemented. The plan and control mechanism should be developed in
accordance with the organization’s normal planning and control process, and it should
contain the following:
A project team organization with assigned responsibilities.
NOTE - The skill of
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.