IEC TR 62248:2002

Approaches to conformance and certification testing for automation standards

Provides an approach to conformance and certification methodology for TC 93 that takes into account the diversity of organizations contributing to the work of this committee.

General Information

• Status: Withdrawn
• Publication Date: 04-Dec-2002
• Withdrawal Date: 31-May-2011
• ICS: 25.040.01 - Industrial automation systems in general
• Technical Committee: TC 91 - Electronics assembly technology
• Drafting Committee: WG 6 - TC 91/WG 6
• Current Stage: WPUB - Publication withdrawn
• Completion Date: 01-Jun-2011

Standards Content (Sample)

TECHNICAL IEC
REPORT
TR 62248
First edition
2002-12
Approaches to conformance and certification
testing for design automation standards
Reference number
IEC/TR 62248:2002(E)
Publication numbering
As from 1 January 1997 all IEC publications are issued with a designation in the
60000 series. For example, IEC 34-1 is now referred to as IEC 60034-1.
Consolidated editions
The IEC is now publishing consolidated versions of its publications. For example,
edition numbers 1.0, 1.1 and 1.2 refer, respectively, to the base publication, the
base publication incorporating amendment 1 and the base publication incorporating
amendments 1 and 2.
Further information on IEC publications
The technical content of IEC publications is kept under constant review by the IEC,
thus ensuring that the content reflects current technology. Information relating to
this publication, including its validity, is available in the IEC Catalogue of
publications (see below) in addition to new editions, amendments and corrigenda.
Information on the subjects under consideration and work in progress undertaken
by the technical committee which has prepared this publication, as well as the list
of publications issued, is also available from the following:
• IEC Web Site (www.iec.ch)
• Catalogue of IEC publications
The on-line catalogue on the IEC web site (http://www.iec.ch/searchpub/cur_fut.htm)
enables you to search by a variety of criteria including text searches, technical
committees and date of publication. On-line information is also available on
recently issued publications, withdrawn and replaced publications, as well as
corrigenda.
• IEC Just Published
This summary of recently issued publications (http://www.iec.ch/online_news/
justpub/jp_entry.htm) is also available by email. Please contact the Customer
Service Centre (see below) for further information.
• Customer Service Centre
If you have any questions regarding this publication or need further assistance,
please contact the Customer Service Centre:
Email: [e-mail]
Tel: [phone]
Fax: [phone]
TECHNICAL IEC
REPORT
TR 62248
First edition
2002-12
Approaches to conformance and certification
testing for design automation standards
 IEC 2002  Copyright - all rights reserved
No part of this publication may be reproduced or utilized in any form or by any means, electronic or
mechanical, including photocopying and microfilm, without permission in writing from the publisher.
International Electrotechnical Commission, 3, rue de Varembé, PO Box 131, CH-1211 Geneva 20, Switzerland
Telephone: [phone] Telefax: [phone] E-mail: [e-mail]  Web: www.iec.ch
PRICE CODE
Commission Electrotechnique Internationale
L
International Electrotechnical Commission
Международная Электротехническая Комиссия
For price, see current catalogue

– 2 – TR 62248  IEC:2002(E)
INTERNATIONAL ELECTROTECHNICAL COMMISSION
____________
APPROACHES TO CONFORMANCE AND CERTIFICATION TESTING
FOR DESIGN AUTOMATION STANDARDS
FOREWORD
1) The IEC (International Electrotechnical Commission) is a worldwide organization for standardization comprising
all national electrotechnical committees (IEC National Committees). The object of the IEC is to promote
international co-operation on all questions concerning standardization in the electrical and electronic fields. To
this end and in addition to other activities, the IEC publishes International Standards. Their preparation is
entrusted to technical committees; any IEC National Committee interested in the subject dealt with may
participate in this preparatory work. International, governmental and non-governmental organizations liaising
with the IEC also participate in this preparation. The IEC collaborates closely with the International
Organization for Standardization (ISO) in accordance with conditions determined by agreement between the
two organizations.
2) The formal decisions or agreements of the IEC on technical matters express, as nearly as possible, an
international consensus of opinion on the relevant subjects since each technical committee has representation
from all interested National Committees.
3) The documents produced have the form of recommendations for international use and are published in the form
of standards, technical specifications, technical reports or guides and they are accepted by the National
Committees in that sense.
4) In order to promote international unification, IEC National Committees undertake to apply IEC International
Standards transparently to the maximum extent possible in their national and regional standards. Any
divergence between the IEC Standard and the corresponding national or regional standard shall be clearly
indicated in the latter.
5) The IEC provides no marking procedure to indicate its approval and cannot be rendered responsible for any
equipment declared to be in conformity with one of its standards.
6) Attention is drawn to the possibility that some of the elements of this technical report may be the subject of
patent rights. The IEC shall not be held responsible for identifying any or all such patent rights.
The main task of IEC technical committees is to prepare International Standards. However, a
technical committee may propose the publication of a technical report when it has collected
data of a different kind from that which is normally published as an International Standard, for
example “state of the art”.
IEC 62248, which is a technical report, has been prepared by IEC technical committee 93:
Design automation.
This technical report focuses primarily on the work undertaken by IEC TC 93.
The text of this technical report is based on the following documents:
Enquiry draft Report on voting
93/152/DTR 93/156/RVC
Full information on the voting for the approval of this technical report can be found in the
report on voting indicated in the above table.
This publication has been drafted in accordance with the ISO/IEC Directives, Part 2.
The committee has decided that the contents of this publication will remain unchanged until 2005.
At this date, the publication will be
• transformed into an International standard;
• reconfirmed;
• withdrawn;
• replaced by a revised edition, or
• amended.
A bilingual version of this publication may be issued at a later date.

TR 62248  IEC:2002(E) – 3 –
INTRODUCTION
The attention of the reader is drawn to the fact that this Technical Report provides an
approach to conformance and certification methodology for TC 93 that takes into account
the diversity of organizations contributing to the work of this committee.

– 4 – TR 62248  IEC:2002(E)
APPROACHES TO CONFORMANCE AND CERTIFICATION TESTING
FOR DESIGN AUTOMATION STANDARDS
1 Scope
This Technical Report provides guidelines for test suite specifications and benefits of
conformance and certification testing.
2 Normative references
The following referenced documents are indispensable for the application of this document.
For dated references, only the edition cited applies. For undated references, the latest edition
of the referenced document (including any amendments) applies.
ISO 10303, Industrial automation systems and integration -- Product data representation and
exchange
IEEE 1003, IEEE standard for information technology
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1 Application Program Interface (API)
program intended to provide a high level of interoperability between different supplier EDA
platforms
3.2 Backus-Naur Format (BNF)
format used to describe computer actions through the use of keywords and attributes
3.3 certification
procedure by which a third party gives written assurance that a product, process, or service
conforms to specific requirements
3.4 conformance
to be in accordance with some specified standard or specification
3.5 Core Model of the Electronics Domain (CMED)
includes the semantics definitions for various categories of information related to electronic
circuit designs
3.6 Electronic Design Automation (EDA)
general term for automation standards related to electronic design
3.7 Electronic Design Interchange Format (EDIF)
format used to exchange design data between different CAD systems, and between CAD
systems and printed circuit fabrication and assembly manufacturing facilities
3.8 I/O Buffer Information Specification (IBIS)
modeling data silicon template intended to specify a consistent format that can be parsed by
software, allowing each simulation vendor to derive models compatible with their own product

TR 62248  IEC:2002(E) – 5 –
3.9 POSIX
part of IEEE 1003 is part of the POSIX series of standards. It defines security interfaces
to open systems for access control lists, audit, separation of privilege (capabilities),
mandatory access control, and information label mechanisms. The standard is stated in
terms of its C binding
3.10 Standard for Product Data Exchange (STEP)
group of standards that comply with ISO 10303 STEP consisting of a great variety of products
that use EXPRESS information models
3.11 Very High Speed Integrated Circuit Hardware Description Language (VHDL)
programming language that has been designed and optimized for describing the behaviour of
digital systems
3.12 VHDL - Language Reference Manual (VHDL-LRM)
manual that describes the foundation for most VHDL simulators and synthesis tools currently
on the market
4 Diversity of feeder organizations
The approach to conformance and certification methodology for TC 93 must take into account
the diversity of the international organizations that feed standards into TC 93. This diversity is
a strength of TC 93 within the international Electronic Design Automation community; however
it does add some complexity; such as with regard to conformance and certification
methodologies. Due to the diversity of feeder organizations that input standards to TC 93 it
would be very difficult to enforce a strict formal methodology, the goal of TC 93 should be to
encourage the implementation of an effective methodology for conformance and certification
that puts a minimal burden on the feeder organizations. Because of the diverse and
international nature of TC 93,and the difficulty in identifying resources to apply to the task of
standards development, including the related activity of conformance and certification, a
generalized set of guidelines is needed. This report is intended as an initial definition of these
guidelines. This report is also intended to identify the benefits of conformance and
certification for TC 93 members and their constituents, in an effort to help them justify the
necessary resources to support conformance and certification. This will be an ongoing and
iterative process, whereby lessons learned are incorporated into the report. There are for
example many concepts from the STEP approach that can be applied to the TC 93 domain.
The emphasis will be on test suite specifications, because TC 93 does not wish to become
a developer or maintainer of the executable test suites. It is, however, intended that
TC 93/WG 5: Test validation, conformance and qualification technologies, should provide
guidelines (which will later become requirements) for development of test suite specifications
which should be incorporated with or referenced by any standard submitted to TC 93.
5 Benefits of conformance and certification testing
One of the most critical aspects of a certification program is having it be accepted by the
industry, and primarily the suppliers since they are most directly affected by the program. The
suppliers (EDA vendors in this case) should be involved as early as possible in the definition
of a certification program in order to ensure it's success.
It is reasonable to ask the question “what is the point of certification?” It is not just assurance
of some level of quality. Usually certification conjures up notions of compatibility, inter-
operability, and portability (see MALLIS in clause 12). In industry today, interoperability tests
often refer to the testing, via pair wise matching, of specific applications. This is a very
expensive proposition especially as the number of applications to be certified increases.
In some cases, the certification of the application program interfaces (APIs) themselves
provides a high level of interoperability. POSIX is a case in point. There is no explicit
interoperability certification involved in the POSIX certification. However, one of the results

– 6 – TR 62248  IEC:2002(E)
of POSIX certification is the ability of different Unix implementations to interoperate at certain
levels. This is due to the fact that the POSIX standard itself provides good coverage of the
domain for which it is intended.
Interoperability or compatibility are loose terms that suggest some kind of cooperation or
harmony among unlike components of a system. These terms have been applied to features
ranging from “is written in the same language” to “can read ASCII” to “plug-and-play”.
Portability is often mentioned when defining interoperability goals, and it usually means the
ability to move a program or piece of data around among different environments and still be
able to use it with a minimum of effort, even though the program may be very unlike other
components in design or function. Conformance and certification programs can also stimulate
the marketplace, because they give customers a level of assurance that the products they
purchase will work with their existing infrastructure. If for example a company currently has
installed tools from company “A” but is interested in acquiring a tool from company “B”, they
would be less apprehensive about such a purchase if there were some formal mechanism in
place by which they could gain assurance that the tools from these different companies would
interoperate. Conformance and certification testing can provide such assurance if the program
is well implemented. Another benefit to having conformance and certification programs in
place (which assumes that test suites are available) is that the developers of conformant
applications greatly benefit from the use of the test suites during their development process.
Finding conformance problems early in the development process is another benefit. In
addition, running the test suites often helps improve the overall quality of the product by
identifying issues that might have been overlooked if the test suite had not been run against
the application.
One of the key goals of WG 5's conformance and certification plan is to support the WG 1:
Electronical data harmon
...