ISO/IEC 14102:1995

Information technology - Guideline for the evaluation and selection of CASE tools

Provides guidance on identifying organizational requirements for CASE tools, on mapping those requirements to CASE tool characteristics to be evaluated and describes a process for selecting the most appropriate CASE tool from several tools, based on measurements of the defined characteristics.

Technologies de l'information — Lignes directrices pour l'évaluation et la sélection d'outils CASE

General Information

• Status: Withdrawn
• Publication Date: 08-Nov-1995
• Withdrawal Date: 08-Nov-1995
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 4 - Tools and environment
• Current Stage: 9599 - Withdrawal of International Standard
• Start Date: 23-Oct-2008
• Completion Date: 30-Oct-2025

Relations

• Revised: ISO/IEC 14102:2008 - Information technology - Guideline for the evaluation and selection of CASE tools
• Effective Date: 15-Apr-2008

Standards Content (Sample)

INTERNATIONAL
lSO/IEC
STANDARD 14102
First edition
1995-1 l-l 5
Information technology - Guideline for
the evaluation and selection of CASE tools
Lignes directrices pour I’baluation et Ia
Technologies de I’information -
sklection d’outils CASE
Reference number
ISOLCEC 14102:1995(E)
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Introduction.vi
Scope .
Normative references . 3
Definitions and acronyms . 4
31 0 Definitions .
32 l Acronyms .
..... 6
Overview of evaluation and selection of CASE tools
41 Initiation process . 6
l
................................. 7
42 0 Structuring process
43 l Evaluation process .
44 Selection process . 8
.
45 . General process considerations .
Sequencing of processes . 8
4.5.1
4.5.2 Reducing tost and risk .
Initiation process .
51 l Goal setting . 10
........................ 11
52 l Establishing selection criteria
53 Project planning and control . 12
l
Structuring process .
............................. 13
61 l Requirements definition
6.1.1 Organizational information gathering . 13
Requirements identification . 14
6.1.2
6.1.3 Requirements structuring .
..................... 16
62 l CASE tool information gathering
63 . Identifying final candidate CASE tools .
0 ISO/IEC 1995
All rights reverved. Unless otherwise specified, no part of this publication may be reproduced
or utilized in any form or by any means, electronie or mechanical, including photocopying and
microfilm, without Permission in writing from the publisher.
ISO/IEC Copyright Office l Case Postale 56 l CH-121 1 Geneve 20 l Switzerland
Printed in Switzerland
ii
01s0/lEc
7 Evaluation process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
71 l Preparing for evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
72 l Evaluating CASE tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.2.1
7.2.2 Rating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.2.3 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
73 l Evaluation reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
8 Selection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
81 Preparing for selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
82 l Applying the selection algorithm . . . . . . . . . . . . . . . . . . . . . . 23
83 l Recommending a selection decision . . . . . . . . . . . . . . . . . . . . 24
84 Validating the selection decision . . . . . . . . . . . . . . . . . . . . . . 24
l
9 CASE tool characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
91 l Functionality - characteristics related to life-cycle processes. 25
............. 26
9.1.1 Characteristic: Management Process
9.1.2 Characteristic: Development Process .
9ele2el Subcharacteristic: Modeling . 27
..... 29
9ele2e2 Subcharacteristic: Construction
Characteristic: Maintenance Process . 30
9.1.3
........... 3 1
9.1.4 Characteristic: Documentation Process
9.1.5 Characteristic: Configuration Management Process 3 1
........ 32
9.1.6 Characteristic: Quality Assurance Process
9.1.7 Characteristic: Verification Process .
............... 34
9.1.8 Characteristic: Validation Process
Functionality - characteristics related to CASE tool usage, . 36
92 l
9.2.1 Characteristic: Environment in which the CASE tool
operatese . 36
Characteristic: CASE tool integrability. . 37
9.2.2
9e2e3 Characteristic: Aspects of the CASE tool’s application39
93 General quality characteristics .
l
................... 40
9.3.1 Characteristic: Functionality
4 1
9.3e2 Characteristic: Reliability .
4 1
Characteristic: Usability .
9.3.3
...................... 43
9e3e4 Characteristic: Effkiency
................. 44
9.3.5 Characteristic: Maintainability
9.3.6 Characteristic: Portability .
............ 45
94 l General characteristics not related to quality
9.4.1 Characteristic: Acquisition .
...
01s0/lEc
ISOAJX 14102:1995(E)
................. 46
9.4.2 Characteristic: Implementation
.............. 46
9.4.3 Characteristic: Support Indicators
9.4.4 Characteristic: Evaluation or Certification . 47
Annex A Considerations on the use of this International
Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Annex B Examples of selection algorithms . . . . . . . . . . . . . . . . . . . 50
Annex C Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
iv
ISOLIEC 14102:1995(E)
0 1s0/IEc
Foreword
ISO (the International Organization for Standardization) and IEC (the Inter-
national Electrotechnical Commission) form the specialized System for worldwide
standardization. National bodies that are members of ISO or IEC participate in the
development of International Standards through technical committees established
by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with
ISO and IEC, also take part in the work.
In the field of information technology, ISO and IEC have established a joint
technical committee, ISO/IEC JTC 1. Draft International Standards adopted by the
joint technical committee are circulated to national bodies for voting. Publication
as an International Standard requires approval by at least 75 % of the national
bodies casting a vote.
International Standard ISOLIEC 14102 was prepared by Joint Technical
Committee ISOLIEC JTC 1, Information technology, Subcommittee SC 7,
Sofnyare engineering.
Annexes A to C of this International Standard are for information only.

ISOLIEC 14102: 1995(E) OISO/IEC
Introduction
Within Software engineering, Computer-Aided Software Engineering (CASE)
tools represent a major part of the supporting technologies used to develop and
maintain Software Systems. Their selection must be carried out with careful
consideration of both the technical and management requirements.
This International Standard defines both a sequence of processes and a structured
set of CASE tool characteristics for use in the technical evaluation and the ultimate
selection of a CASE tool. It follows the Software product evaluation model
described in ISOKEC 9 126: 199 1, Information technology - Software product
evaluation - Quality characteristics and guidelines for their use. This
International Standard adopts the general model of Software product quality
characteristics and subcharacteristics described in ISO/IEC 9 126, and extends
these when the Software product is a CASE tool; it provides product
characteristics unique to CASE tools. This larger set of characteristics is then
organized into five groups. This grouping provides a more manageable approach
to the Overall evaluation and selection process.
While the technical evaluation may answer how well a CASE tool meets its user’s
stated requirements, it may also answer the question of how well the tool meets its
claimed functionality.
The objective of the technical evaluation process is to provide quantitative results
on which the final selection tan be based. Measurement assigns numbers (or other
ratings) to attributes of entities; a major activity of evaluation is to obtain these
measurements for use in selection. The final selection results should aim to
achieve objectivity, repeatability and impartiality. These objectives and the
confidence in the outcomes will in part depend on the resources allocated to the
Overall evaluation and selection process. The user of this International Standard is
asked to deal with these issues at an early Stage.
To be widely acceptable, these CASE tool evaluation and selection processes must
be of value to the users of CASE tools, to the suppliers of CASE to
community at large. The information outlined in this International Standard should
lead to more tost effective selections of CASE tools and to a greater uniformity in
how CASE tool functions and features are described.

INTERNATIONAL STANDARD 0 ISO/lEC ISOLIEC 14102: 1995(E)
Information technology - Guideline for the
evaluation and selection of CASE tools
1 Scope
This International Standard deals with the evaluation and selection of CASE tools,
covering a partial or fi~ll Portion of the Software engineering life cycle. It
establishes processes and activities to be applied for the evaluation of CASE tools
and selecting the most appropriate CASE tools from several candidates. These
processes are generic, and organizations must tailor them to meet organizational
needs. The CASE tool evaluation and selection processes should be viewed in the
larger context of the organization’s technology adoption process.
This International Standard provides:
a. Guidance on identifling organizational requirements for CASE tools.
b . Guidance on mapping those requirements to CASE tool characteristics to
be evaluated.
C. A process for selecting the most appropriate CASE tool from several tools,
based on measurements of the defined characteristics.
Primary users of this International Standard are organizations that intend to adopt
CASE tools to support their Software life cycle processes. CASE tool suppliers
may also use this International Standard to describe characteristics of their CASE
tools.
This International Standard is not intended to apply to:
a. Software engineering frameworks whose purpose is to provide mechanisms
for data, control and presentation integration.
b . General purpose tools (e.g., word processors, Spreadsheets) which may be
used in Software engineering activities, nor CASE tools of very narrow
scope or specific purpose (e.g., a Compiler).
C. Planning for the implementation of CASE tools within an organization
(even though it is recognized that this is an important subject).
NOTE - A user of this International Standard may make the best possible selection of a CASE
tool and have no guarantee of a successful implementation. ISO/IEC JTC 1 SC7 WG4 is working on a
drafi technical report, Adoption of CASE Tools, which addresses this subject.
This International Standard contains a set of processes, activities, and tasks
designed to be tailored. The tailoring process is the selection of applicable
processes, activities and tasks.
Compliance with this International Standard is defmed as the Performance of the
processes, activities, and tasks selected from this International Standard for the
evaluation and selection project. Any organization imposing this International
Standard as a condition of trade is responsible for specifling the minimum set of
required processes, activities, and tasks which constitute compliance for a given
application of this International Standard. Defming and documenting that
specification forms part of the initiation process (clause 5).

0 ISO/IEC
2 Normative references
The following Standards contain provisions which, through reference in this text,
At the time of publication, the
constitute provisions of this International Standard.
editions indicated were valid. All Standards are subject to revision, and Parties to
agreement based upon this International Standard are encouraged to investigate
the possibility of applying the most recent editions of the Standards indicated
below. Members of IEC and ISO maintain registers of currently valid International
Standards.
ISO 5807: 1985, Informationprocessing - Documentation symbols and
conventions for data, program and system flowcharts, program network Charts
and system resources Charts.
ISOIIEC 12 119: 1994, Information technology - Software packages - Quality
requirements and testing.
ISOIIEC 12207: 1995, Information technology - Software ltfe cycle processes.
ISOIIEC 9 126: 199 1, Information technology - Software product evaluation -
Quality characteristics and guidelines for their use.

OISO/IEC
3 Definitions and acronyms
31 Definitions
For the purposes of this International Standard, the following definitions apply.
3.1.1 assessment: An action of applying specific documented criteria to a
specific Software module, package or product for the purpose of determining
acceptance or release of the Software module, package or product. (ISO/IEC
9126: 1991)
3.1.2 atomic subcharacteristic: The highest level evaluation categories are
called characteristics. Characteristics are usually subdivided into
subcharacteristics. Many subcharacteristics may be further subdivided into lower
level subcharacteristics. At the lowest-level, when no further subdivision is
appropriate, the subcharacteristics are referred to as atomic subcharacteristics.
3.1.3 CASE tool: A Software product that tan assist Software engineers by
providing automated support for Software life-cycle activities as defined in
ISO/IEC 12207:1995.
NOTES
1 - A CASE tool may provide support in only selected functional areas or in a wide variety of functional
2 - CASE tools may be used in several modes:
As stand alone tools; in this case, only compatibility with environment elements should be
addressed.
In small groups which communicate directly with one another; it may be supposed that
integration is predefined, perhaps proprietorily.
In the presence of a larger framework of the SEE; in this case the ability of the tool to use the
relevant Services of the framework should be addressed.
3.1.4 characteristic: An aspect of a product by which it tan be described and
evaluated. A characteristic may be refined into multiple levels of subcharacteristics
that bear on its ability to satisfy stated or implied needs.
3.1.5 measurement: The action of applying a Software quality metric to a
specific Software product. (ISO/IEC 9 126: 199 1)

0 1s0mx ISOLIEC 14102:1995(E)
NOTES
1 - Measurement tan apply to metrics other than Software quality metrics.
measured indirectly by the application of metrics
2 - An Object may be measured directly, or may be
or representations of the Object.
information about
3.1.6 metric: A quantitative scale and method which tan be used to determine the
value a subcharacteristic takes for a specific Software product.
The action of mapping the measured value to the appropriate rating
3.1.7 rating:
level. Used to determine the rating level associated with the Software for a specific
quality characteristic. (ISO/IEC 9 126: 199 1)
NOTE - Rating and rating levels tan be applied to characteristics other than quality characteristics.
A range of values on a scale to allow Software to be
3.1.8 rating level:
classified (rated) in accordance with the stated or implied needs. Appropriate
rating levels may be associated with the different views of quality, i.e., users,
managers or developers. These levels are called rating levels. (ISO/IEC
9126:1991)
3.1.9 Software Engineering Environment: The Software engineering
environment (SEE) is that Portion of the System which provides automated
support for the engineering of Software Systems and the management of the
Software process. It includes platform, System Software, Utilities, and CASE tools
installed.
NOTE - The SEE architecture has two aspects:
the CASE tools which provide facilities for supporting life-cycle processes, and
framework which provides a set of capabilities that offer common Services used by
a general
the tools.
32 l Acronyms
BMT Bench Mark Test
Computer Aided Software Engineering
CASE
GUI Graphical User Interface
SEE Software Engineering Environment
Structured Query Language
SQL
01s0AEc
4 Overview of evaluation and selection of CASE tools
This section provides an overview of the evaluation and selection of CASE tools
discussed in this International Standard as shown in Figure 1. Evaluation and
selection of CASE tools includes four major processes:
Initiation Process
Structuring Process
Evaluation Process
Selection Process
1 Structuring 1
GT /.--.“’ \
I Selection 4 \
IRecommendation 1 T-*--. . . . . . . . . . . . . [r]
1 DataFbw
Figure 1 - Overview of evaluation and
selection of CASE tools
A key process is the structuring of a set of requirements against which candidate
CASE tools are to be evaluated, and upon which selection decisions will be based.
The CASE tool characteristics defined in clause 9 form the basis for requirements
structuring, and play a central role in the Overall process.
41 . Initiation process
The purpose of the initiation process is to defme the general objectives and
requirements of the intended evaluation and selection of CASE tools, to establish the
high level direction, and to define the management aspects of the effort (e.g.,
schedule, resources, tost).
The initiation process, discussed in detail in clause 5, is composed of three activities:
goal setting: provides the rationale and general policy for evaluation and
0 1s0/IEc
selection.
establishing selection criteria: provides criteria to be used in the subsequent
selection process.
project planning: results in a plan which includes generic planning
information and also information which defines the structure of the evaluation and
selection effort.
42 0 Structuring process
The purpose of the structuring process is to elaborate a set of structured
requirements, based upon the CASE tool characteristics of clause 9 against which
CASE tools should be evaluated, and to obtain the necessary information on CASE
It is assumed that a set of general organizational
tools to permit evaluation.
information and guidelines is available to be used as inputs.
The structuring process, discussed in detail in clause 6, is composed of three
activities:
requirements analysis: transforms organizational needs into measurable
structures.
CASE tool information gathering: captures a snapshot of the current state-
of-the-art in CASE tools.
identifying final candidate CASE tools: candidate CASE tools are identified
for evaluation using the results of the last two activities.
requirements may require revision. If this occurs, some repetition of
NOTE - During the evaluation,
activities of this, and subsequent processes may be necessary.
0 Evaluation process
The purpose of the evaluation process is to produce technical evaluation reports that
will be the major input for the selection process. Esch evaluation process results in
a Profile of the quality and other characteristics of the tool which was evaluated.
Comparisons between tools are not made as part of this process.
The evaluation process, discussed in detail in clause 7, is composed of three activities:
preparation for evaluation: finalization of the various details of the
evaluation (e.g., scenario, subcharacteristics, metrics, tool characteristics) in an
evaluation plan.
evaluating CASE tools: measurement, rating and assessment.
evaluation reporting: an evaluation report is prepared which provides the
results of the evaluation for each CASE tool considered.
44 0 Selection process
The purpose of the selection process is to identify the most suitable CASE tool(s)
among the candidate tools, and to ensure that the recommended tool(s) meets the
original goals. The selection process compares the results of the evaluations of the
candidate tools to determine which is the most appropriate for selection.
The selection process, discussed in clause 8, is composed of four activities:
preparing for selection: the selection criteria are finalized and the selection
algorithm is defined.
assessing the evaluation results: the selection algorithm is applied to the
evaluation results.
recommending a selection decision: the best of the candidates is determined.
l
validating the selection decision: the recommended selection is validated
against the original goals.
45 General process considerations
l
There are several considerations that apply to the processes described in this
International Standard on a global basis. The intent is for the user of this International
Standard to tailor its application in such a way as to maximize the probability of a
successful evaluation and selection process, and minimize its tost and risk.
4.5.1 Sequencing of processes
This International Standard does not impose the sequence of process activities
described above and in the following sections. It is up to the organization to select
the relevant processes and activities needed to meet its evaluation and selection goals.
The organization will decide which to employ, in what sequence, and with what
0 ISO/IEC ISO/IEC 14102:1995(E)
degree of parallelism. The sequencing of the processes’ activities is then documented
in the project plan developed during the initiation process.
4.5.2 Reducing tost and risk
In general, organizations which apply this International Standard will want to
minimize the tost ofthe entire evaluation and selection process to the extent possible,
while maintaining the level of effort necessary to select the most appropriate CASE
tool(s) for their use. These objectives may be addressed by minimizing the number of
tools evaluated, minimizin g the tost of evaluating specific tools, and ensuring that the
formality of the process is appropriate to the organization.
The activities of CASE tool information gathering and identifying final
candidates for selection (see clause 6) effectively allow the user of this International
Standard to Screen the available tools against the organization’s needs, and eliminate
fi-om consideration tools which do not, or are not likely to, substantially address the
organization’s needs.
NOTE 1 - It may be that the organization is unable to find any tool which appear likely to
sufficiently meet its needs. In such a case, the stated needs themselves should be re-examined, and if they
are found to accurately reflect the organization’s actual requirements for technology improvement, the
Overall evaluation and selection process may be abandoned. Similarly, if the final candidate tools appear
to be marginal in addressing the organization’s needs, the level of detail and formal@ of the subsequent
activities should be made to reflect the risk factor, and the organization should be prepared to not select a
tool if the evaluation process so indicates, as the typical tost of bringing a new tool into operational use is
substantial.
Evaluations of candidate tools may have already been performed and be
available to the organization. Such information may be used to reduce the tost of
candidate tool evaluation.
NOTE 2 - Previous evaluations which have been performed on a different Version of the candidate
tool may still yield useful information. Similarly, evaluations which addressed a different set of
organizational needs may still provide useM information.
This International Standard calls for the development of several Plans and
reports, and implicitly, for their review by various Personne1 within the organization.
In addition, activities are required to perform the four processes outlined. The format
and level of detail of the data products is left to the discretion of the organization, as
is the level of effort necessary to perform the activities.
NOTE 3 - Some organizations may need to limit the scope, detail and formal@ of the processes
to apply this International Standard within existing resource constraints.
5 Initiation process
The evaluation and selection processes require the agreement of management. In line
with this agreement, a set of goals for the introduction (or enhancement) of CASE
technology will be established. A set of CASE tool selection guidelines will be
identified and a project plan developed. The process is shown in Figure 2.
Goal
Setting
l-J\
. . . . . .**.
Dah Fbw
I
Figure 2 - Overview of initiation process
51 Goal setting
In developing
The development of a set of realistic goals is a necessary first activity.
goals, both a rationale for acquisition (why acquire a CASE tool) and a general policy
for acquisition (what type of tool to acquire and how to do it) should be developed.
NOTE - Goal setting activities, including possibly the identification of selection criteria, may have already
been perfomxd as a patt of other efforts Prior to formally entering the initiation process of evaluation and
selection of CASE tools.
The following tasks should be performed:
Develon rationale for acauisition:
Review the organization’s current softwar-e development process, determining
its maturity and areas of concern.
Review the current state of CASE technology and observe trends for
consideration as future reference technology.
Compare the organization’s current practices to possible future practices if
ISWIEC 14102:1995(E)
0 ISO/IEC
CASE tools are adopted and identifl areas of potential benefit.
Identify probable impacts of CASE tools on the organization; e.g., areas
where training and education, procedure guides, and technical support are needed to
effectively deploy CASE technology.
Define goals and expectations:
Set Overall goals (e.g., productivity improvement, quality improvement,
enhanced process manageability).
Define evaluation and selection constraints (e.g.,cost, schedule, resources).
Quantify and classify expectations (based upon goals).
t general nolicy for acquisition:
IdentifSI constraints on tool acquisition (e.g., implementation tost, schedule,
other resources).
Develop alternate approaches to introducing/augmenting CASE technology
(e.g., buy a tool, modify an existing tool, develop a new tool).
Assess the feasibility of the various alternatives in light of organizational
readiness, technical considerations, Performance specifications, and resources.
The goals and expectations established here will be used to guide subsequent activities
in the Overall process and, finally, to validate the selection decision.
52 l Establishing selection criteria
Based upon the goals and expec ztations d leveloped above, selection criteria should be
established:
Decompose the high 1 evel goal s into a set of selection criteria to make the
(go/no go) selection decision.
NOTE 1 - The selection criteria should be objective and quantitative. Esch selection criterion
should include some defined threshold specified on which the major go/no go decision will be made during
selection.
Define the relative importante of the selection criteria.
OISO/IEC
ISOLIEC 14102:1995(E)
NOTE 2 - The relative importante of the selection criteria will be used to determine the weights
assigned to tool characteristics and subcharacteristics for evaluation.
Define the level of detail and the nature of the evaluation activities to be
performed.
NOTE 3 - The nature of the evaluation activities covers the methods used in collecting the data.
Reference, for example, how the data are measured, collected with predefmed criteria, or based upon
subjective Observation.
Define the evaluation/selection scenario to be performed (see Annex A).
53 Project planning and control
Based upon the goals and selection criteria which have been established for the Overall
evaluation and selection process, a project plan should be created and a control
mechanism implemented. The plan and control mechanism should be developed in
accordance with the organization’s normal planning and control process, and it should
contain the following:
A project team organization with assigned responsibilities.
NOTE - The skill of the evaluators will have an impact on the results of the evaluation and its
applicability to the organization. The evaluation personnel should be selected with this in mind, and the
skill level of evaluators should be a factor in assessing evaluation results. The evaluation team should be
representative of the intended tool user group.
A set of operational goals obtained by decomposing the Overall goals
previously established.
A set of selection guidelines: weighted selection criteria, definition of level of
detail and nature, and an evaluation and selection scenario (see Annex A).
A schedule of activities and their tasks, along with an estimate of resource
requirements and a tost estimate.
A means of monitoring and controlling the execution of the plan.
If developed, the project plan and control mechanism should be updated as the project
evolves.
0 ISOLEC ISOBEC 14102:1995(E)
6 Structuring process
The structuring of the evaluation and selection activities tan begin when a set of high
level goals, selection guidelines, and a project plan are in place. The structuring
process begins with a requirements definition activity which is followed by two
parallel activities: the gathering of information on existing CASE tools, and the
preparation of a list of candidate CASE tools to be evaluated.
The Organkation of CASE tool requirements will follow the four groups of CASE
tool characteristics as outlined in clause 9. The major activities are shown in Figure
3 .
~.z.ziz)
*.
. .
-.
. .
*.
‘. i
. . . . . . . . . . . .
Data Flow
I
Figure 3 - Overview of structuring process
61 0 Requirements definition
During requirements definition, the requirements for the CASE tool are collected and
organized into the CASE tool characteristics as noted in clause 9. 9.1 and 9.2 identifl
the major CASE specific characteristics; 9.3 identifies general Software quality
characteristics, and 9.4 identifies a set of characteristics not related to quality. A
comprehensive set of requirements is necessary to select the most appropriate CASE
tool, and the structuring process provides for greater ease and repeatability in the
evaluation process. Three activities are required.
6.1.1 Organizational information gathering
To be able to define a set of detailed requirements to be satisfied by the CASE tool,
information about the organization should be gathered, including:
Willingness of the organization to fully fund and implement CASE tool use.
Current Software engineering environment within the organization, including
data describing current hardware, operating Software, and tool use.
Types of Software development projects undertaken by the organization
include size, domain of application.
Characteristics and constraints of the target Systems for which Software is
developed.
Specific expected impacts and improvements of CASE technology on the
organization.
Requirements from potential tool users and end users.
Current organizational procurement policies.
This information is necessary to ensure the tool or tools are appropriate for use within
the organization, they address organizational needs, and needs perceived by their
Uure users.
NOTE - This information tan be gathered in a number of ways, including Surveys and focus groups.
6.1.2 Requirements identification
The tool user’s requirements should deal with the question of what the CASE tool
should do as well as its impact on the existing environment. The following tasks
should be performed in building the list of requirements:
Analyze the requirements and adjust the level of detail to which requirements
are defined and measured.
Evaluate the current need for CASE tools while taking into consideration
those projects where the CASE tool may initially be used.
Identifl desired methodology (e.g., process-oriented, data-oriented, object-
oriented).
IdentifL portions of the life-cycle to be supported (e.g., planning, analysis,
design).
0 ISO/IEC ISO/lEC 14102:1995(E)
Identify required functions of the CASE tool.
Identify required quality characteristics of the CASE tool.
Check the consistency of the requirements with the previously established
goals.
NOTE - These requirements represent the total set of organizational requirements. It is possible that no
Single CASE tool may satisfy all of the requirements, but that individual CASE tools may satisfl a
sufficient number to just@ their use by the organization, which may continue to search for tools to support
remaining requirements.
6.1.3 Requirements structuring
The applicability of the user needs identified in clause 9, and any others which the
The purpose of this structuring is
organization may wish to add, should be defined.
to organize the requirements in such a way that the evaluation tan proceed more
effectively. The tasks include:
Categorize the user requirements in terms of the organization of clause 9, and
decompose them into detailed specifications.
Select characteristics and specific subcharacteristics from clause 9 which tan
be evaluated to determine the extent to which the CASE tool meets the detailed
specifications.
NOTE 1 - The extent to which a CASE tool supports or implements a specific methodology may
be a critical factor, and should be seriously considered when selecting characteristics and subcharacteristics
and weighting those subcharacteristics.
Identify weights for the characteristics and subcharacteristics.
NOTES
2 - The weights are applied to the ratings determined during the evaluation as part of the selection
process, and reflect the relative importante of the related selection criterion as determined during the
initiation process.
3 - The assignment of weights is a subjective task which has a fundamental impact on the outcome
of the entire evaluation and selection process. The assignment of weights should reflect both the
organization’s actual requirements and the ability of the organization to evaluate the characteristic. See
Annex B for further discussion.
4 - ISOLIEC 12 119: 1994 addresses quality requirements applicable to CASE tools when
considered as Software packages, and should be consulted as part of the requirements structuring task. It
provides additional guidance on a subset of the quality requirements of ISOLlEC 9 126: 199 1.
OISO/IEC
62 0 CASE tool information gathering
A general search of potential CASE tools to be evaluated is undertaken based upon
the requirements and selection criteria established. The activities of gathering
information and identifling the candidate CASE tools may require several iterations
For
to quickly and efficiently identify the most promising tools for further evaluation.
the CASE tools which appear most promising for further evaluation, additional and
more detailed data that deal with their potential acquisition are obtained. This
additional information may help to quickly eliminate many tools, allowing attention
to be focused on the remaining candidates. Information to be obtained includes:
Vendor general information (e.g., business history, available support, Plans &
strategies).
Vendor’s specific product development strategy.
The tool’s tost (e.g., price, maintenance, modifications, training).
The hardware and Software required to support tool use.
The hardware and Software required to support final application/product use.
The training required for efficient tool use.
The tool’s functional capabilities.
The tool’s methodology and life-cycle support.
How the tool interfaces to external Systems.
The number of users, existente of a user’s group, the users’ response to the
tool.
The tool’s license mechanism (e.g., floating license, multi-user licenses, Cross
platform licenses).
63 0 Identifying final candidate CASE tools
When the set of potential candidate tools has been identified, the final candidates for
selection (those to be evaluated) may be Chosen. This is accomplished through the
following tasks:
0 ISO/IEC ISODEC 14102:1995(E)
Establish a set of high-priority or critical, requirements to be met by CASE
tools.
Compare the user’s functional requirements with the CASE tool’s fbnctional
capabilities, supporting methodology, System environment .
Compare the managerial requirements with the CASE tool’s tost, available
training and support.
Analyze the tool vendors’ user base, user response, support and business
history.
Identify tools satisfling a sufficient number of high-priority or critical
requirements which then become the final candidates for formal evaluation. The
results of the previous tasks provide the justification for the list of candidates.
NOTE - The tasks described in this Paragraph represent a “screening” of possible candidates to allow the
organization to identify the candidates most likely to be acceptable, given the organization’s requirements
or suppliers abilities. The identification of final candidates tan be performed in parallel with CASE tool
information gathering, or the two activities may be iterated. The goal is to reduce the tost of tool evaluation
by only considering a screened set of final candidates during the evaluation process.
OISO/IEC
ISOLIEC 14102:1995(E)
7 Evaluation process
Evaluation tan begin when the structured requirements have been defined and a
screened set of final candidates for selection have been Chosen. Final preparations will
be made for evaluating the candidate CASE tools, including the development of an
The evaluation activities are then performed and documented,
evaluation plan.
resulting in a Profile of how each CASE tool measures up to the structured
requirements. The objective is to produce the technical evaluation reports necessary
for the selection txocess, as shown in Figure 4.
Preparing
for
E”alnution 4 . . . . . . . .-*-**
l I
. . . . . . . . . . . .
Data Fbw
Figure 4 - Overview of evaluation process
71 l Preparing for evaluation
To define the necessary level of detail Prior to beginning evaluation activities, final
Based upon the list of candidate CASE tools and the
preparations are necessary.
structured requirements, the following tasks should be accomplished:
For each atomic subcharacteristic, define or select one or more metrics and
define the details of their use.
NOTE 1 - ISOLEC JTC 1 SC7 Working Group 6 has technical reports relating to metrics
under development which may help the user of this International Standard select some of the necessary
metrics.
Set the rating levels and identify the means by which the levels will be
generated or computed.
0 ISO/IEC
NOTE 2 - A measured metric value (e.g., average lines of Code per module = 274) must then
be assigned a rating value (e.g., 1.3 on a scale of 0 to 4). The means by which rating levels are obtained
Flom measurements must be identified.
Define the assessment characteristics for evaluation, establishing what is
acceptable, taking into consideration the rating levels previously defined and the
context of use of the product.
IdentifL and schedule all activities which must be performed as part of the
evaluation process.
NOTE 3 - Activities include preparing any data sets necessary for the evaluation, obtaining
tool documentation and an instance of the tool to be evaluated, providing evaluators any necessary
tmining in tool use, hands-on tool use, recording of tool Outputs, and analysis of results.
In some cases, a Bench Mark Test (BMT) may be a part of the evaluation
process. The recommended approach for a BMT includes:
Identify the required critical tool functions.
Identifjr a test project or Sample program to be the basis for the BMT.
w
Develop a BMT scenario, defining inputs and expected Outputs.
To focus evaluation activities and provide for traceability of the evaluation
process, develop an evaluation plan which includes the information above.
72 0 Evaluating CASE tools
The Software is evaluated in comparison with each of the Chosen characteristics.
Evaluation is a process of measurement, rating and assessment.
7.2.1 Measurement
Measurements tan be made based upon information obtained by examining the
CASE tool itself, or information about it, through the following types of tasks:
Examining the vendor-supplied documentation.
Examining the Source code and other intermediate products, if available.
Interviewing actual users of the Software.
ISOLIEC 14102:1995(E) 01s0/IEc
Viewing demonstrations and interviewing demonstrators.
Executing test cases.
Applying to test projects.
Examining results of previous evaluations (whether in-house, third Party,
or other evaluations).
Performing a BMT on the candidate tools and analyzing the results.
Measurement values may be binar-y, based on a continuous scale (quantifiable), or
textual. There are both objective and subjective characteristics.
NOTE - Objective characteristics are those which permit independent and repeatable test or metric.
Subjective characteristics are those for which no independent and repeatable test or metric exists (e.g.,
fitness of the user interface to the culture of the User).
For objective characteristics, the evaluation should be made by a repeatable
procedure such that another evaluator would be able to produce the same results.
During evaluation, if test cases are used, a uniform, predefined, and documented
set of cases should be used.
For subjective characteristics, the evaluation should be performed repeatedly by
more than one person or group, who will discuss and agree upon results.
The evaluation results should be recorded in a quantified manner, where possible,
together with textual justification, where applicable.
7.2.2 Rating
In the rating task, each measured value is rated against the scale of values defined
in the evaluation plan. Rating levels are either directly generated or computed
according to previously defined algorithms.
NOTE - requirements may be revised evaluation, and this may require
It is possible that during the
revision of rating scales.
7.2.3 Assessment
Based upon the resulting ratings and the previously defined assessment criteria,
assess the subcharacteristics and characteristics. In accordance with the selection
0 ISO/IEC
guidelines and the evaluation plan, ratings should be aggr
...