ISO/IEC TR 18121:2015

Information technology — Learning, education and training — Virtual experiment framework

ISO/IEC TR 18121:2015 defines the framework for IT standards and specifications on virtual experiments supporting IT-enhanced learning, education and training. It is based on implementations of standards and specifications that are used to support virtual experiment, development, evaluation and management that rely on ITLET. This ISO/IEC TR 18121:2015 - provides a framework that can be used for virtual experiment systems that rely on ITLET, - determines the categories of different virtual experiment standards and specifications and their relationships to facilitate their integration, - promotes the appropriate design and application of virtual experiment components so that IT systems that are being used are reusable, low cost, and more broadly applicable, - indicates considerations to be taken into account when developing ITLET systems that are being developed or used to support virtual experiment systems, - provides sample architecture of a virtual experiment system framework that can be used to support further development of virtual experiments, - illustrates various roles that different actors can have at various points within the virtual experiment system, - demonstrates how standards and specifications that support virtual experiments can be combined to form the basis for future work and implementations. The following aspects are not addressed in this ISO/IEC TR 18121:2015: - specifications of the implementation details of virtual experiment systems; - accessibility; - privacy; - security.

Technologies de l'information — Apprentissage, éducation et formation — Cadre d'expérience virtuel

General Information

• Status: Published
• Publication Date: 02-Dec-2015
• ICS: 35.240.90 - IT applications in education
• ICS: 35.240.99 - IT applications in other fields
• Technical Committee: ISO/IEC JTC 1/SC 36 - Information technology for learning, education and training
• Drafting Committee: ISO/IEC JTC 1/SC 36 - Information technology for learning, education and training
• Current Stage: 9093 - International Standard confirmed
• Completion Date: 28-Sep-2021

Standards Content (Sample)

TECHNICAL ISO/IEC TR
REPORT 18121
First edition
2015-12-15
Information technology — Learning,
education and training — Virtual
experiment framework
Technologies de l’information — Apprentissage, éducation et
formation — Cadre d’expérience virtuel
Reference number
ISO/IEC TR 18121:2015(E)
©
ISO/IEC 2015

---------------------- Page: 1 ----------------------
ISO/IEC TR 18121:2015(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2015, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2015 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC TR 18121:2015(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Conformance . 1
3 Terms and definitions . 2
4 Symbols and abbreviated terms . 2
5 Framework . 3
5.1 General . 3
5.1.1 Overview of virtual experiment framework . 3
5.1.2 Core feature of virtual experiment framework . 4
5.1.3 Basic specifications . 4
5.1.4 Relationship . 4
5.2 Basic framework standards . 5
5.3 Application service standards . 6
5.4 Data processing standards . 6
5.5 Management and evaluation standards . 7
5.6 Relationship among specifications . 7
5.7 Considerations for virtual experiment systems . 8
Annex A (informative) Major parts of virtual experiment framework . 9
Annex B (informative) Application examples of the virtual experiment framework .10
Annex C (informative) Abstract actors for virtual experiment framework .16
Bibliography .18
© ISO/IEC 2015 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC TR 18121:2015(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/IEC JTC 1 Information technology, Subcommittee
SC 36, Information technology for learning, education and training.
iv © ISO/IEC 2015 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC TR 18121:2015(E)

Introduction
New and emerging technologies are allowing learners to test and develop new knowledge for
the creation of virtual experiments. By employing a collection of different technologies, virtual
experiments are widely used within K-12 and higher education institutions (e.g. scientific laboratory
experiments, technical training in medicine and biomedicine) and also within corporate training (e.g.
flight simulators, manufacturing process control). Virtual experiments allow access to a wide range
of different experiments, decrease the timeframe to complete experiments, eliminate hazardous
situations, and reduce the costs of materials.
To facilitate the development of virtual experiment technology and specification integration, this
Technical Report defines a virtual experiment framework that indicates how various standards and
specifications can be combined to support the design, implementation, analysis and evaluation of
virtual experiment systems. This Technical Report denotes the components and categories of related
standards and specifications that can be used to support virtual experiment systems and learners.
It illustrates how the components and categories can be structured to support implementations to
improve reusability, reduce costs, and broaden applicability. It defines the framework of related virtual
experiment standards and determines the mutual relations between different standards for designing,
analysing and comparing different virtual experiment systems in order to promote the design and
application of the virtual experimental components and systems. The framework of virtual experiment
standards also helps to illustrate the categories of related virtual experiment systems and their
relations in order to guide the development of other standard-setting work and normatively describe
virtual experiment system.
Virtual experiments are typically accessed from an IT system, such as a learning management
system (LMS), which provides the environment for learning. In addition to tools specifically tailored for
learning, IT systems (such as LMSs) can also provide access to external tools. In a common scenario, an
identity federation mechanism will automatically log into the external service but it is up to the learner
whether to provide data inputs to the external tool. At the end of the activity, a score is returned to the
IT system (e.g. LMS). This type of approach can be used to support activities, such as those involved
in the scientific experimental approach, where there are requirements to repeat an experiment with
a variety of parameters in order to assess their individual role in a process. There is a need to bridge
the switch from the IT system (e.g. LMS) to an external virtual experiment tool, providing data inputs
and recording outputs for further analysis. This Technical Report can be used to support these types
of activities. In addition, this Technical Report provides specific considerations for the development of
ITLET supportive technology and specification integration.
© ISO/IEC 2015 – All rights reserved v

---------------------- Page: 5 ----------------------
TECHNICAL REPORT ISO/IEC TR 18121:2015(E)
Information technology — Learning, education and
training — Virtual experiment framework
1 Scope
This Technical Report defines the framework for IT standards and specifications on virtual
experiments supporting IT-enhanced learning, education and training. It is based on implementations
of standards and specifications that are used to support virtual experiment, development, evaluation
and management that rely on ITLET.
This Technical Report
— provides a framework that can be used for virtual experiment systems that rely on ITLET,
— determines the categories of different virtual experiment standards and specifications and their
relationships to facilitate their integration,
— promotes the appropriate design and application of virtual experiment components so that IT
systems that are being used are reusable, low cost, and more broadly applicable,
— indicates considerations to be taken into account when developing ITLET systems that are being
developed or used to support virtual experiment systems,
— provides sample architecture of a virtual experiment system framework that can be used to support
further development of virtual experiments,
— illustrates various roles that different actors can have at various points within the virtual
experiment system,
— demonstrates how standards and specifications that support virtual experiments can be combined
to form the basis for future work and implementations.
The following aspects are not addressed in this Technical Report:
— specifications of the implementation details of virtual experiment systems;
— accessibility;
— privacy;
— security.
2 Conformance
It is noted that it is possible for a virtual experimental instruction system to support more than one
standard or specification by utilizing different combinations that include base standards, specifications,
etc. The standards or specifications coordinate with circumstances or the virtual experiment instruction
system that is configured separately to support specific standard or specification combinations.
This Technical Report provides the framework and objective to assist in the development of virtual
experiments for information technology supporting learning, education, and training. This Technical
Report also provides a proposed architecture of a system framework that is used to support the further
development of virtual experiments. This Technical Report provides specific considerations for the
development of ITLET supportive technology and for the integration of IT standards and specifications.
© ISO/IEC 2015 – All rights reserved 1

---------------------- Page: 6 ----------------------
ISO/IEC TR 18121:2015(E)

3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
component cartridge
assembly of all the junior units based on specific rules to form a bigger component cell
3.2
remote access laboratory
laboratory in the physical world which can be accessed remotely
Note 1 to entry: Access to simulated equipment and virtual laboratories are excluded.
Note 2 to entry: A remote access laboratory enables learners to perform experiments at their own pace, time and
location and allows teaching staff and learners to access laboratory facilities beyond their institutions.
3.3
virtual experiment
experiment based on multimedia, simulation and virtual reality (3.6), etc. which can assist or replace
the operating segments of traditional, face-to-face experiments
3.4
virtual experiment component
smallest unit in a virtual experiment (3.3) which is composed of controllable and operable virtual
experiment objects that may be reused
3.5
virtual laboratory
open networked instructional system based on web and virtual reality (3.6) technologies, which
consists of virtual benches
3.6
virtual reality
artificial environment presented in the computer
4 Symbols and abbreviated terms
CP Content Packaging
CC Common Cartridge
LOM Learning Object Metadata
LIS Learning Information System
LMS Learning Management System
MLR Metadata for Learning Resources
QTI Question and Test Interoperability
SCORM Sharable Course Object Reference Model
SOA Service-Oriented Architecture
XML Extensible Markup Language
2 © ISO/IEC 2015 – All rights reserved

---------------------- Page: 7 ----------------------
ISO/IEC TR 18121:2015(E)

5 Framework
5.1 General
The framework for IT standards and specifications on virtual experiments supports IT-enhanced
learning, education and training. For assistance with the development of profiles, the related standards
and specifications for virtual experiment instruction is shown in Figure 1.
Figure 1 — Related standards of a virtual experiment instruction system
The virtual experiment standard system establishes the whole virtual experiment standards
framework system and identifies the various links to standards of the virtual experiment instruction.
The virtual experiment standards system also defines the relationship of standards and specifications
and combines them with each other properly.
5.1.1 Overview of virtual experiment framework
The virtual experiment framework standard consists of the following four major parts.
— Basic framework standards and specifications
Basic framework standards focus on various compatible specifications of the virtual experiment
platform and tools. They are the basis of the whole virtual experiment standard system.
© ISO/IEC 2015 – All rights reserved 3

---------------------- Page: 8 ----------------------
ISO/IEC TR 18121:2015(E)

— Application service standards and specifications
Application service standards focus on the normative description of service interface provided by
virtual experiment teaching systems which can help to improve the management of services.
— Data processing standards and specifications
Data processing standards are the hard core of the whole virtual experiment standard system,
providing data elements of virtual experiment component metadata and a conceptual data mode,
which are used to define an instance of metadata structure.
— Management and evaluation standards and specifications
Management and evaluation standards concentrate on how to control and manage the virtual
experiment procedures in a standardized way for the purpose of helping learner complete the
experiment in the experimental processes.
5.1.2 Core feature of virtual experiment framework
The virtual experiment framework mainly illustrates the categories of related virtual experiment
standards and describes the relations among the standards in order to promote the virtual experiment
resources-sharing and interoperability of virtual experiment systems.
5.1.3 Basic specifications
The virtual experiment framework is established on some related specification such as CP, CC, SCORM,
etc. All the specifications are described in detail in their own documents.
5.1.4 Relationship
The examples below provide the standard structure of a typical virtual experiment instruction system
which normatively describes the standardized environment of the virtual experiment instruction system.
As shown in Figure 1, this virtual experiment instruction system conforms to some related virtual
experiment standards which are divided into four categories: basic framework standards, application
service standards, data processing standards, and management and evaluation standards. The specific
explanations of virtual experiment instruction system are provided below.
— Basic framework standards and specifications
The virtual experiment instruction system may conform to the virtual experiment terminology
specification to regulate the definition of key terms and the explanation of symbols which
are used in the system. The system unify the terms in the development process and help users
to translate and use the standard correctly; meanwhile, it may also be beneficial to manage the
virtual experiments system. Basic framework standards are in reference to framework standards
(ISO/IEC/TR 24725-1:2011), platform and media taxonomy standards (ISO/IEC/TR 24725-3:2010),
which help to manage the platform effectively and efficiently.
— Application service standards and specifications
In the process of virtual experiment instruction, the system has interoperability between
different platforms. LIS is the definition of how to realize the exchange of information related to
learning domains. Interoperability is best defined through the use of a domain profile. Common
Cartridge is another successful example to establish a standardized way to package course so as
to export and import content quickly and easily in learning management system. To some extent,
the application service standards of virtual experiment are established on the basis of these two
standards. However, it has its unique characters according to actual needs. Personal information
and operation procedure, such as saving, resetting, configuring, are all recorded in the application
service programs which can provide personalized service. Simultaneously, real-time feedback is
offered to encourage correct operation or revise errors timely in the course of the experiment.
4 © ISO/IEC 2015 – All rights reserved

---------------------- Page: 9 ----------------------
ISO/IEC TR 18121:2015(E)

— Data processing standards and specifications
While using the virtual experiment instruction system, the data processing is a core part. The
example of virtual experiment instruction system conforms to component description specification
and component packaging specification. The function of component description specification is
similar to LOM and includes exclusive environmental elements of virtual experiment beyond LOM;
additionally, it helps to describe and classify the components of virtual experiments of the system
normatively to promote the interchange and sharing of virtual experiment resources. The content
packaging specification can be a method to assist packaging the components of multi-domain
modelling standardized and define detailed interface to facilitate the retrieval and management
for learners and instructors in the future.
— Management and evaluation standards and specifications
The virtual experiment instruction system also conforms to the procedure of management
specification and the evaluation of test specification, which helps to standardize the procedure
controlling and management of virtual experiment on the purpose of helping learners complete
the experiment in learning process, tracking the learners’ behaviour and providing accurate
feedback. QTI specification and Simple Sequencing Specification can offer the references to deal
with procedure, evaluation, and feedback.
5.2 Basic framework standards
Figure 2 — Basic framework standards
Basic framework standards include virtual experiment terminology standard, virtual experiment
Extensible Markup Language (XML) data binding, virtual experiment platform and tools specification
and virtual experiment extensible specification as shown in Figure 2.
Virtual experiment platform and tool specification focuses on various compatible specifications of
the virtual experiment platform and tools. Virtual experiment extensible specification studies the
extensible rules and implementation method aiming at the development process of virtual resources
and experiments. Virtual experiment terminology standard provides the definition of key terms and
the explanation of symbols which are used in virtual experiment standard system to unify the terms in
the development process and help users translate and use the standard correctly. Virtual experiment
XML data binding uses the common rules of the data format of XML binding for the processing of data
streams and the integration of the system.
© ISO/IEC 2015 – All rights reserved 5

---------------------- Page: 10 ----------------------
ISO/IEC TR 18121:2015(E)

5.3 Application service standards
Figure 3 — Application service standards
Application service standards/specification include teaching guide specification, service description
specification, service registry specification and service index specification as shown in Figure 3.
The teaching guide specification formulates guiding model for virtual experiment evaluation and defines
the rules, data formats, and interface of virtual experiment platform. Service description specification
focuses on the normative description of services provided in virtual experiment teaching systems for
the management of services. The service registry specification deals with the registration of virtual
experiment component and interface specification. Besides, it defines rules in the implementation
of web services for Service-Oriented Architecture (SOA) integration. The service index specification
provides advices and guidelines on the retrieval of the virtual experiment service which conform to
virtual experiment standards.
Application service standards and specifications, such as the common cartridge (CC) specification
can be used to package content with more interactive application type functionality (e.g. discussion
board, tests, web links, etc.). The CC specification allows the content and application functionality to be
bundled into a common structured format that can be shared across different systems.
5.4 Data processing standards
Figure 4 — Data processing standards
Data processing standards include the following sub-categories: component description specification,
component packing specification, data extension specification, and data transfer specification, as
shown in Figure 4.
Component description specification studies how to describe the multi-domain modelling components
in order to retrieve and manage the components in the future. Component packaging specification
6 © ISO/IEC 2015 – All rights reserved

---------------------- Page: 11 ----------------------
ISO/IEC TR 18121:2015(E)

specifies how to package the multi-domain modelling components and defines communication interface
between components. Thus, it can provide a standardized way to package resources and make different
virtual experiment systems sharable. Data extension specification and data transfer specification are
involved in the transfer of resources and components among different systems and they build a bridge
between different virtual experiment systems.
Some examples of different standards and specifications that may be used within the data processing
category are provided below. A data processing standard or specification, such as Learning Object
Metadata specification (LOM) can be used to describe resources (sometimes referred to as assets) that
are being used in the virtual experiment system. The Content packaging (CP) standard (ISO/IEC 12785)
can be used to exchange virtual experiment content between different virtual experiment systems
without converting the material into new formats. ISO/IEC 12785 allows the original resources
to be packaged into a standard structure so that they can be imported, exported, aggregated and
disaggregated as needed.
5.5 Management and evaluation standards
Management and evaluation standards include procedure management specification, evaluation
specification and test model specification as shown in Figure 5. Procedure management specification
studies how to control and manage the virtual experiment procedure scientifically for the purpose of
helping a learner complete the experiment within the process. Evaluation specification studies how
to evaluate and test learners’ behaviour in a standardized way in virtual experiments, and provides
accurate feedback.
Test model specification provides assistance in choosing the correct test approaches and constructing
proper test models which help to build a systematic, comprehensive and effective test in the defined
standard. In addition, test model specification provides proper index for effective system performance
and rational guidelines for selecting the appropriate methods to test, such as black box testing, white
box testing, automated testing, regression testing, integration testing, etc.
Figure 5 — Management and evaluation standards
Similar to Simple Sequence Specification which defines a method for representing the intended
behaviour, management and evaluation standards describe behaviour model to sequence and evaluate
discrete experimental activities in a consistent way.
5.6 Relationship among specifications
Virtual experiment development, evaluation and management specifications and test standards are
technical standards which meet the demands of virtual experiment instruction. They solve the problem
of resource sharing and system interoperability in the field of virtual experiment research and provide
the conforming framework and interface for the virtual experiment resource and system.
Basic framework standards are the foundation of the whole virtual experiment standard system.
As peripheral standards, basic framework standards are the precondition and requisition of the
© ISO/IEC 2015 – All rights reserved 7

---------------------- Page: 12 ----------------------
ISO/IEC TR 18121:2015(E)

development of other standards. All the sub-standards should be developed referring to basic
framework standards.
Application service standards are standards related to service management in the virtual experiment
standard system. They are the general process specifications which should be complied with in the
practical use and service provision of the virtual experiment application system and they help to lower
the threshold for standard use.
Data processing standards are the hard core of the whole virtual experiment standard system,
providing data elements of virtual experimental component metadata and a conceptual data model
which is used to define the instance of metadata structure.
Management and evaluation standards are frequently used during the development of virtual
experiment system which includes many functions, such as evaluating and testing the standardized
effect and the standardization degree of experimental components and procedures. As to data
processing standards, management and evaluation standards also test the standardization degree of
component description and component cartridge specification.
5.7 Considerations for virtual experiment systems
When developing or using a virtual experiment system, it is important to fully understand the
underlying specifications and standards that are being used. More detailed information regarding the
component standards and specifications can be obtained from the following documents.
a) Specification text
The specification or standard text describes the relevant specification or standard and includes
information such as purpose, scope, terms and definitions, elements and interrelation, metadata
definition, data exchange format, etc.
b) Practice guideline
Practice guidelines usually include detailed explanation on the key points about the standards or
specifications and provide practical examples of
...