ISO/IEC 40180:2017
(Main)Information technology — Quality for learning, education and training — Fundamentals and reference framework
Information technology — Quality for learning, education and training — Fundamentals and reference framework
ISO/IEC 40180 provides the fundamentals and the reference framework for quality assurance, quality management and quality improvement in IT-enhanced learning, education and training (called E-Learning). It consists mainly of the Quality Reference Framework (QRF) for E-Learning, which is a common and generic framework to describe, specify and understand critical properties, characteristics and metrics of quality. The QRF combines an elaborated and extensive process model with a descriptive model for the processes. ISO/IEC 40180 harmonizes existing approaches, concepts, specifications, terms and definitions related to quality for E-Learning, education and training.
Technologies de l'information — Qualité pour l'apprentissage, l'éducation et la formation — Principes fondamentaux et cadre de référence
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 40180
First edition
2017-10
Information technology — Quality
for learning, education and training
— Fundamentals and reference
framework
Technologies de l'information — Qualité pour l'apprentissage,
l'éducation et la formation — Principes fondamentaux et cadre de
référence
Reference number
ISO/IEC 40180:2017(E)
©
ISO/IEC 2017
---------------------- Page: 1 ----------------------
ISO/IEC 40180:2017(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2017 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 40180:2017(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 4
5 Conformance . 4
6 The Quality Reference Framework (QRF) for E-Learning . 4
6.1 General . 4
6.2 Descriptive model of the QRF . 5
6.3 Process model of the QRF . 7
6.3.1 Overview . 7
6.3.2 Description . . . 8
Annex A (informative) Introduction and use of the QRF.16
Annex B (informative) DIN process model (DIN PAS 1032-1) .20
Annex C (informative) French code of practice in E-Learning (AFNOR Z76-001) .47
Annex D (informative) Harmonized model for quality management for learning, education,
and training: mapping CELTSC and QRF .59
Annex E (informative) Specific quality objectives .63
Bibliography .67
© ISO/IEC 2017 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 40180:2017(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology, SC 36,
Information technology for learning, education and training.
ISO/IEC 40180 cancels and replaces ISO/IEC 19796-1:2005 (initially published under the general title
Information technology — Learning, education and training — Quality management, assurance and
metrics) which has been technically revised.
iv © ISO/IEC 2017 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 40180:2017(E)
Introduction
This document provides the fundamentals and the reference framework for quality assurance,
quality management and quality improvement in IT-enhanced learning, education and training
(called E-Learning). This document primarily consists of the Quality Reference Framework (QRF) for
E-Learning. It is a framework to describe, compare and analyse approaches for quality management
and quality assurance. These approaches can be mapped to QRF. Therefore, the framework is not a
quality management model or guiding management system – it is a framework for the description of
quality approaches. It serves to compare different existing standards and to harmonize these towards
a common quality model. For a better understanding of the QRF, several annexes show samples of its
usage – the annexes are based on the German DIN PAS 1032-1 (Annex B), the French code of practice in
E-Learning (Annex C); and an example of how it can be mapped to other quality systems, such as the
Chinese E-Learning Technology Standards(CELTS) (Annex D).
This document provides the fundamentals for the implementation and adaptation of the general quality
standards within the fields of E-Learning, education and training. This includes specific vocabulary
as well as the Quality Reference Framework (QRF) with its Process Model and Descriptive Model. It
provides sector specific information that may be used to support the better integration of mandatory
and voluntary quality approaches at the local, regional, national and transnational levels. Through the
application of this document, organizations may continue to improve the quality of their processes,
products, services and solutions within the fields of learning, education and training.
This document was carefully developed and revised over a period of more than 10 years to ensure
compatibility with ISO 9001 and ISO 14001 standards and facilitate their use by organizations that
wish to introduce and improve quality assurance, quality management and quality improvement in
learning, education and training together with the implementation of both environmental and quality
management systems to benefit themselves and their customers and stakeholders.
Overview of ISO/IEC 40180:
— Clause 6 describes the process-oriented Quality Reference Framework (QRF) for the comparison
and analysis of quality approaches.
— Annex A (informative) provides an introduction and description of the QRF and illustrates its
relation to the PDCA Cycle and the IDEAL Reference Model. It further provides answers on key
questions in order to provide a clearer understanding of the QRF concept and its use.”
— Annex B (informative) shows the full German process model (DIN PAS 1032-1) as an example of how
the basic model can be extended.
— Annex C (informative) describes the use of the model describing the “French Code of Practice in
E-Learning” (AFNOR Z 76-001) as a second sample of the use of this document.
— Annex D (informative) describes how other quality approaches can be mapped to QRF. Specifically,
the Chinese Model CELTSC is used as an example of the mapping procedure.
— Annex E (informative) describes the use of the model for specific quality objectives such as metadata
quality.
© ISO/IEC 2017 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 40180:2017(E)
Information technology — Quality for learning, education
and training — Fundamentals and reference framework
1 Scope
This document provides the fundamentals and the reference framework for quality assurance, quality
management and quality improvement in IT-enhanced learning, education and training (called
E-Learning). It consists mainly of the Quality Reference Framework (QRF) for E-Learning, which is a
common and generic framework to describe, specify and understand critical properties, characteristics
and metrics of quality. The QRF combines an elaborated and extensive process model with a descriptive
model for the processes. This document harmonizes existing approaches, concepts, specifications,
terms and definitions related to quality for E-Learning, education and training.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
organization
person or group of people that has its own functions with responsibilities, authorities and relationships
to achieve its objectives
[SOURCE: ISO 9000:2015, 3.2.1]
3.2
customer
person or organization who consumes the product (studying and training) directly or indirectly
EXAMPLE Learner, learner’s parents, education institutions and potential employer.
3.3
interested party
person or organization that can affect, be affected by, or perceive itself to be affected by a decision or
activity of ITLET
EXAMPLE Learner, employee, provider, partner, investor, owner, society.
[SOURCE: ISO 9000:2015, 3.2.3, modified, adapted within ITLET context]
3.4
process
set of interrelated or interacting activities which transforms inputs into outputs
Note 1 to entry: Inputs to a process are generally outputs of other processes.
© ISO/IEC 2017 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 40180:2017(E)
Note 2 to entry: Processes in an organization are generally planned and carried out under controlled conditions
to add value.
Note 3 to entry: A process where the conformity of the resulting product cannot be readily or economically
verified is frequently referred to as a “special process”.
3.5
product
output of an organization that can be produced without any transaction taking place between the
organization and the customer
Note 1 to entry: Production of a product is achieved without any transaction necessarily taking place between
provider and customer, but can often involve this service (3.6) element upon its delivery to the customer.
Note 2 to entry: The dominant element of a product is that it is generally tangible.
Note 3 to entry: Hardware is tangible and its amount is a countable characteristic (e.g. tyres). Processed materials
are tangible and their amount is a continuous characteristic (e.g. fuel and soft drinks). Hardware and processed
materials are often referred to as goods. Software consists of information regardless of delivery medium (e.g.
computer programme, mobile phone app, instruction manual, dictionary content, musical composition copyright,
driver's licence).
[SOURCE: ISO 9000:2015, 3.7.6]
3.6
service
output of an organization with at least one activity necessarily performed between the organization
and the customer
Note 1 to entry: The dominant elements of a service are generally intangible.
Note 2 to entry: Service often involves activities at the interface with the customer to establish customer
requirements as well as upon delivery of the service and can involve a continuing relationship such as banks,
accountancies or public organizations, e.g. schools or hospitals.
Note 3 to entry: Provision of a service can involve, for example, the following: — an activity performed on a
customer-supplied tangible product (e.g. a car to be repaired); — an activity performed on a customer-supplied
intangible product (e.g. the income statement needed to prepare a tax return); — the delivery of an intangible
product (e.g. the delivery of information in the context of knowledge transmission); — the creation of ambience
for the customer (e.g. in hotels and restaurants);
Note 4 to entry: A service is generally experienced by the customer.
[SOURCE: ISO 9001:2015, 3.7.7]
3.7
quality
degree to which a set of inherent characteristics of an object fulfils requirements
Note 1 to entry: The term “quality” can be used with adjectives such as poor, good or excellent.
Note 2 to entry: “Inherent”, as opposed to “assigned”, means existing in an object.
[SOURCE: ISO 9001:2015, 3.6.2, modified]
3.8
data quality
set of features that concern the collection, analysis, persistence, dissemination, and usage of data
EXAMPLE Relevance, accuracy, timeliness, punctuality, accessibility, clarity, comparability, coherence.
2 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 40180:2017(E)
3.9
quality assessment
measures carried out consistently and systematically in order to insure conformity with the
requirements of a defined specification
3.10
quality assurance
part of quality management focused on providing confidence that quality requirements will be fulfilled
[SOURCE: ISO 9000:2015, 3.3.6]
3.11
quality control
part of quality management focused on fulfilling quality requirements
[SOURCE: ISO 9000:2015, 3.3.7]
3.12
quality improvement
part of quality management focused on increasing the ability to fulfil quality requirements
Note 1 to entry: The quality requirements can be related to any aspect such as effectiveness, efficiency or
traceability.
[SOURCE: ISO 9000:2015, 3.3.8]
3.13
quality management
management with regard to quality
Note 1 to entry: Quality management can include establishing quality policies and quality objectives , and
processes to achieve these quality objectives through quality planning, quality assurance , quality control , and
quality improvement.
Note 2 to entry: [SOURCE ISO 9000:2015, 3.3.4]
3.14
quality objective
objective related to quality
Note 1 to entry: Quality objectives are generally based on the organization's quality policy.
Note 2 to entry: Quality objectives are generally specified for relevant functions, levels and processes in the
organization
Note 3 to entry: [SOURCE ISO 9000:2015, 3.7.2]
3.15
quality planning
part of quality management focused on setting quality objectives and specifying necessary operational
processes and related resources to achieve the quality objectives
Note 1 to entry: Establishing quality plans can be part of quality planning.
[SOURCE: ISO 9000:2015, 3.3.5]
3.16
quality policy
policy related to quality
Note 1 to entry: Generally the quality policy is consistent with the overall policy of the organization, can be
aligned with the organization’s vision and mission and provides a framework for the setting of quality objectives.
© ISO/IEC 2017 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 40180:2017(E)
Note 2 to entry: Quality management principles presented in this International Standard can form a basis for the
establishment of a quality policy.
[SOURCE: ISO 9000:2015, 3.5.9]
3.17
total quality management
management approach of an organization, centred on quality, based on the participation of all its
members and aiming at long-term success through customer satisfaction, and benefits to all members
of the organization and to society
4 Abbreviated terms
QRF Quality Reference Framework
RQC Reference Quality Criteria
SCORM Sharable Content Object Reference Model
W3C World Wide Web Consortium
IDEAL Initiating, Diagnosing, Establishing, Acting and Learning model
PDCA Plan-Do-Check-Act cycle
5 Conformance
A quality description conforms to this document if each process included in a QRF instance corresponds
to the appropriate specification in this document and includes all sub-processes. A conforming
description may contain descriptions of processes not included in this document. The description may
contain additional data elements.
This document can assist users who are implementing ISO 9000, ISO 9001 and can be used with other
ISO quality standards, such as ISO 14001.
This document is informed by the concept and philosophy of Total Quality Management and by ISO 9000
and ISO 9001. However, it is not a Management System Standard and does not provide a model to
follow when setting up and operating a quality management system like ISO 9001. Instead, it provides
guidance and support for the introduction of quality management and improving learning quality.
This document is informed by the concept and philosophy of Environmental Management and by
ISO 14001. However, it is not an Environmental Management System Standard and does not provide a
model to follow when setting up and operating an environmental management system like ISO 14001.
Instead, it provides guidance and support for introducing and improving quality processes in learning,
education and training.
6 The Quality Reference Framework (QRF) for E-Learning
6.1 General
The QRF provides a process model and a descriptive model for the description, comparison and
analysis of process-oriented quality approaches. The QRF can be used as a meta-model for approaches
of quality management and quality assurance in learning, education and training. This means that no
assumptions and prescriptive requirements of the quality approaches are made.
4 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 40180:2017(E)
The use scenarios of the QRF can be described as follows:
Scenario 1: Description of a quality approach
— Select a specific quality management or quality assurance approach Q1.
— Identify the processes that are covered within Q1.
— Describe Q1 according to the framework using the description categories.
Scenario 2: Comparison of quality approaches
— Select quality management or quality assurance approaches [Q1.Qn].
— Identify the processes that are covered within [Q1.Qn].
— Describe [Q1.Qn] according to the QRF using the description categories.
— Define a metric to compare [Q1.Qn].
— Perform analysis and comparison.
Scenario 3: Harmonization of quality approaches
— Select quality management or quality assurance approaches [Q1.Qn].
— Identify the processes that are covered within [Q1.Qn].
— Describe [Q1.Qn] according to the QRF using the description categories.
— Define a metric to compare [Q1.Qn].
— Perform analysis and comparison.
— Combine [Q1.Qn] towards a consensus model.
It is important to mention that the QRF process model and its descriptive model shall be used as a
general, descriptive reference framework. In a second step, good practice approaches and profiles
can be generated – these profiles could contain specific recommendations, guidelines, procedures, or
criteria.
NOTE In addition, it is important to underline that the QRF process model and its descriptive model
are not prescriptive and do not require or determine a specific sequence of the processes: In fact, in
practice, many processes are realized in parallel.
Finally, the QRF process model and its descriptive model can be extended and modified to be adapted to
the specific situation, organization, target group and requirements.
6.2 Descriptive model of the QRF
The QRF descriptive model shows the classification and documentation scheme for quality processes
(see Table 1). It is based on CEN CWA 14644. Each process will be described by this scheme.
© ISO/IEC 2017 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 40180:2017(E)
Table 1 — QRF descriptive model
Attribute Description Example
ID Unique Identifier ID1234
Category Main Process Course development
Process Name Process name Method selection
Description Description of the process Within this process the didactic concept and
methods are evaluated and selected
Relations Relation to other processes Before the method selection a target group anal-
ysis must be performed; FA.6
Sub-processes/ Sub-processes/sub-aspects/tasks Method identification, method alternatives,
sub-aspects method prioritization
Objective Objective of a Process Adequate selection of one or more didactic
concepts
Method Methodology for this process Method selection shall be based on the target
group
Reference to guideline/
documents Methods are selected based on the teachers’
experience
See Method Guidelines Handbook
Result Expected result of a process Method specification
Documents
Actors Responsible/participating actors Team Didactical Design
Metrics/Criteria Evaluation and Metrics for this process Criteria catalogue 3.2.2–3.2.6
Standards Standards used ISO 9241, IEEE 1484.12.1:2003 Learning Object
Metadata
Annotation/ Further Information, Examples of usage
Example
The overview of the descriptive model of the QRF is presented in Figure 1.
Please note that the QRF process model and its descriptive model are not prescriptive and are not
determining a specific sequence of the processes: In fact and in practice, many processes are realized
in parallel. In addition, the QRF process model and its descriptive model can be extended and modified
and has to be adapted to the specific situation, organization, target group and requirements.
6 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 40180:2017(E)
ID Category Process Description Relation
Sub-processes/
Sub-aspects
Objective
Method
Result
Actors
Metrics/Criteria
Standards
Annotation/Example
Figure 1 — Overview of the QRF descriptive model
6.3 Process model of the QRF
The QRF process model includes the relevant processes within the whole life-cycle of learning,
education and training (Table 2). The QRF process model is divided into seven parts called process
categories. Sub-processes are included with reference to a classification of processes – examples of the
use of classifications are shown in Annexes B and C.
Table 2 — QRF process model
ID Process Category Sub-Processes
NA Needs Analysis Classification
FA Framework Analysis Classification
CD Conception/Design Classification
DP Development/Production Classification
IM Implementation Classification
LP Learning Process Classification
EO Evaluation/Optimization Classification
In each part, the essential processes are described. For selected processes, examples how the reference
model can be used are included; these examples are written in italics.
The use of the process model is shown using the classification of processes from DIN PAS 1032-1, listed
fully in Annex B.
6.3.1 Overview
The overview of the process model of the QRF is presented in Figure 2.
Please note that the QRF process model and its descriptive model are not prescriptive and are not
determining a specific sequence of the processes: In fact and in practice, many processes are realized
in parallel. In addition, the QRF process model and its descriptive model can be extended and modified
and has to be adapted to the specific situation, organization, target group and requirements.
© ISO/IEC 2017 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC 40180:2017(E)
Figure 2 — Overview of the QRF process model
6.3.2 Description
In this section, the process model of the QRF is described including examples for the use of each category
of the description scheme (see Tables 3 to 9).
8 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/IEC 40180:2017(E)
6.3.2.1 Needs analysis
Table 3 — QRF process model: Needs analysis
ID Category Process Description Relation
Identification and description of
requirements, demands,
NA Needs Analysis
and constraints of an education-
al project
Sub-processes/ NA.1 Initiation
Sub-aspects
NA.2 Stakeholder identification
NA.3 Definition of objectives
NA.4 Demand analysis
Objective To describe the needs and demands leading to an educational project
Method Quality Function Deployment
Result Documentation of goals, objectives, needs, and requirements of an educational project
Actors Project manager; specialists, learners, sponsors
Metrics/Criteria Indicators
Standards ISO 9000
Annotation/Example
© ISO/IEC 2017 – All rights reserved 9
---------------------- Page: 14 ----------------------
ISO/IEC 40180:2017(E)
6.3.2.2 Framework analysis
Table 4 — QRF process model: Framework analysis
ID Category Process Description Relation
Identification of the framework and
FA Framework Analysis NA, CD
the context of an educational process
FA.1 Analysis of the external context
FA.2 Analysis of staff resources
FA.3 Analysis of target groups
Sub-processes/
Sub-aspects
FA.4 Analysis of the institutional and organizational context
FA.5 Time and budget planning
FA.6 Environment analysis
Objective To describe relevant factors for an educational project
Method Methods of empirical social research; methods in legal and economic research and analysis
Result Documentation and validation of relevant parameters
Actors Project manager; specialists
Metrics/Criteria Check for plausibility; consultation of additional experts
Standards
Annotation/Example
10 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 15 ----------------------
ISO/IEC 40180:2017(E)
6.3.2.3 Conception/Design
Table 5 — QRF process model: Conception/Design
ID Category Process Name Description Relations
CD Conception/Design Conception and Design
of an educational process
CD.1 Learning objectives
CD.2 Concept for contents
CD.3 Didactical concept/methods
CD.4 Roles and activities
CD.5 Organizational concept
Sub-processes /
CD.6 Technical concept
Sub-aspects
CD.7 Concept for media and interaction design
CD.8 Media concept
CD.9 Communication concept
CD.10 Concept for tests and evaluation
CD.11 Concept for maintenance
Objective To plan and design the concepts for an educational process
Method Use of Design Guidelines
Result To provide a conception and design for educational processes
Actors Consultant, Media Designers
Metrics/Criteria
Standards
Annotation/Example
© ISO/IEC 2017 – All rights reserved 11
---------------------- Page: 16 ----------------------
ISO/IEC 40180:2017(E)
6.3.2.4 Development/Production
Table 6 — QRF process model: Development/Production
ID Category Process Description Relation
Development/
DP Realization of concepts CD
Production
DP.1 Content realization
DP.2 Design realization
Sub-processes/
DP.3 Media realization
Sub-aspects
DP.4 Technical realization
DP.5 Maintenance
Objective To realize the conceptions
Method Implementation manual
Result Educatio
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.