ISO/TR 20983:2003
(Main)Information and documentation - Performance indicators for electronic library services
Information and documentation - Performance indicators for electronic library services
ISO/TR 20983:2003 is applicable to all types of libraries in all countries. Limitations on the applicability of individual performance indicators are listed in the scope clause of the description of each indicator. Indicators may be used for comparison over time for the services provided by the same library. Comparisons between libraries and services should only be made with caution, taking into account differences in the constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the data. ISO/TR 20983:2003 does not include indicators for the evaluation of the impact of libraries either on individuals or on society.
Information et documentation — Indicateurs de performance pour les services électroniques des bibliothèques
L'ISO/TR 20983:2003 s'applique à tous les types de bibliothèques dans tous les pays. Les restrictions apportées à l'application d'indicateurs de performance particuliers sont énumérées dans la rubrique Domaine d'application qui figure dans la description de chaque indicateur. Les indicateurs peuvent être utilisés pour des comparaisons au cours du temps pour les services fournis par la même bibliothèque. On peut aussi, sous réserve de prudence, comparer des bibliothèques entre elles, à condition de prendre en compte leurs différences d'organisation, de faire preuve d'une bonne compréhension des indicateurs utilisés, et d'interpréter les données avec précaution. L'ISO/TR 20983:2003 ne comporte pas d'indicateur pour l'évaluation de l'impact des bibliothèques sur les individus ou sur la société.
Informatika in dokumentacija – Kazalci uspešnosti elektronskih knjižničnih storitev
General Information
Relations
Frequently Asked Questions
ISO/TR 20983:2003 is a technical report published by the International Organization for Standardization (ISO). Its full title is "Information and documentation - Performance indicators for electronic library services". This standard covers: ISO/TR 20983:2003 is applicable to all types of libraries in all countries. Limitations on the applicability of individual performance indicators are listed in the scope clause of the description of each indicator. Indicators may be used for comparison over time for the services provided by the same library. Comparisons between libraries and services should only be made with caution, taking into account differences in the constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the data. ISO/TR 20983:2003 does not include indicators for the evaluation of the impact of libraries either on individuals or on society.
ISO/TR 20983:2003 is applicable to all types of libraries in all countries. Limitations on the applicability of individual performance indicators are listed in the scope clause of the description of each indicator. Indicators may be used for comparison over time for the services provided by the same library. Comparisons between libraries and services should only be made with caution, taking into account differences in the constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the data. ISO/TR 20983:2003 does not include indicators for the evaluation of the impact of libraries either on individuals or on society.
ISO/TR 20983:2003 is classified under the following ICS (International Classification for Standards) categories: 01.140.20 - Information sciences. The ICS classification helps identify the subject area and facilitates finding related standards.
ISO/TR 20983:2003 has the following relationships with other standards: It is inter standard links to ISO 13132:2011, ISO 11620:2008. Understanding these relationships helps ensure you are using the most current and applicable version of the standard.
You can purchase ISO/TR 20983:2003 directly from iTeh Standards. The document is available in PDF format and is delivered instantly after payment. Add the standard to your cart and complete the secure checkout process. iTeh Standards is an authorized distributor of ISO standards.
Standards Content (Sample)
TECHNICAL ISO/TR
REPORT 20983
First edition
2003-11-01
Information and documentation —
Performance indicators for electronic
library services
Information et documentation — Indicateurs de performance pour les
services électroniques des bibliothèques
Reference number
©
ISO 2003
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO 2003
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2003 — All rights reserved
Contents Page
Foreword. iv
Introduction . v
1 Scope. 1
2 Normative references . 1
3 Terms and definitions. 1
4 Relationships with other indicators . 4
4.1 The networked environment . 4
4.2 Categories of performance indicators. 5
4.3 The relevance of ISO 11620 to electronic library services . 5
4.3.1 Indicators that apply equally to traditional and electronic library services . 5
4.3.2 Indicators that apply only to traditional library services. 6
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services. 7
5 Use of performance indicators . 7
5.1 Purposes. 7
5.2 Selection of performance indicators. 8
5.3 Limitations . 8
5.3.1 Measuring the impact of electronic library services. 8
5.3.2 Degree of accuracy. 8
5.3.3 Comparability of performance indicator data . 9
5.3.4 Indicators of temporary importance . 9
Annex A List of performance indicators for electronic library services . 10
Annex B Descriptions of performance indicators . 12
Bibliography . 31
Alphabetical index . 32
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 20983 was prepared by Technical Committee ISO/TC 46, Information and documentation,
Subcommittee SC 8, Quality — Statistics and performance evaluation.
iv © ISO 2003 — All rights reserved
Introduction
This Technical Report is concerned with the evaluation of electronic library services.
The successful provision of these services depends on close collaboration between publishers, network
providers, and librarians. Their needs for data about various aspects of performance are different, and the
ability to collect data may not rest with the person most interested in that data. For example, the publisher may
be best placed to collect data on the usage of a particular service; whereas the librarian may have most
interest in using that data in developing and evaluating the service.
The main purpose of this Technical Report is to spread knowledge about the practice of evaluating electronic
library services. Although electronic library services have been developed over more than thirty years, the
technical environment has changed significantly in the last ten years. The need for performance indicators for
electronic services was acknowledged in ISO 11620: 1998 Information and documentation - Library
performance indicators. It was also acknowledged that at the time of publication of that International Standard,
there were no indicators that had been tested and documented, or that were in widespread use. Although
there has been significant progress in the last few years on the definition, testing and documentation of
indicators, it is still too soon to determine which indicators will be most generally useful, or become widely
used. This Technical Report provides a standardized terminology and concise definitions and descriptions of a
selection of performance indicators, in the same format as that used in ISO 11620. Publications listed in the
Bibliography [2,3,5] provide more detailed information on data collection and analysis.
Only a small number of the indicators presented here are indicators of service quality: work on this area is not
yet mature enough to be included. The indicators that are included have been tested in libraries, or are simple
adaptations of similar indicators. In each case they have been judged against the criteria established in
ISO 11620. That is, each indicator is judged to have informative content, to be reliable, valid, appropriate,
practical, and in some circumstances may be used for comparative purposes.
The indicators presented in this Report are a representative selection of those which have been tested. A
further selection would have taken longer to prepare, and to publish. Developments in the measurement and
evaluation of electronic library services will be monitored by a Working Group, which will propose additional
indicators, and modifications or adaptations to the indicators here as they are tested and validated. This
Technical Report should be considered as work in progress towards the development of an International
Standard. It is intended that, where appropriate, the contents of this Technical Report will be incorporated in a
future revision of ISO 11620.
TECHNICAL REPORT ISO/TR 20983:2003(E)
Information and documentation — Performance indicators for
electronic library services
1 Scope
This Technical Report is applicable to all types of libraries in all countries. Limitations on the applicability of
individual performance indicators are listed in the scope clause of the description of each indicator (see
Annex B).
Indicators may be used for comparison over time for the services provided by the same library. Comparisons
between libraries and services should only be made with caution, taking into account differences in the
constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the
data.
This Technical Report does not include indicators for the evaluation of the impact of libraries either on
individuals or on society.
Throughout the text, the names of indicators are printed with initial capitals for significant words, e.g.
Workstation Use Rate. This helps to distinguish the names from supporting text.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 2789:2003, Information and documentation — International library statistics
ISO 5127:2001, Information and documentation — Vocabulary
ISO 11620:1998, Information and documentation — Library performance indicators
ISO 11620:1998/Amd. 1:2003, Information and documentation — Library performance indicators —
Amendment 1: Additional performance indicators for libraries
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
database
collection of electronically stored data or unit records (facts, bibliographic data, texts) with a common user
interface and software for the retrieval and manipulation of the data [ISO 2789]
NOTE 1 The data or records are usually collected with a particular intent and are related to a defined topic. A database
may be issued on CD-ROM, diskette, or other direct access method, or as a computer file accessed via dial-up methods
or via the Internet.
NOTE 2 Licensed databases are counted separately even if access to several licensed database products is effected
through the same interface.
3.2
digital document
information unit with a defined content that has been digitized by the library or acquired in digital form as part
of the library collection [ISO 2789]
NOTE 1 This includes eBooks, electronic patents, networked audiovisual documents and other digital documents, e.g.
reports, cartographic and music documents, pre-prints etc. Databases and electronic serials are excluded.
NOTE 2 Items incorporated in databases are covered by 3.1.
NOTE 3 A digital document can be structured into one or more files.
3.3
document
recorded information or material object which can be treated as a unit in a documentation process [ISO 5127]
NOTE Documents may differ in their physical form or characteristics.
3.4
document downloaded
full text of a document or part of a document in the electronic collection that is delivered to a user [ISO 2789]
3.5
electronic collection
all resources in electronic form in the library collection [ISO 2789]
NOTE The electronic collection includes databases, electronic serials, and digital documents. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.6
electronic resource
a document in electronic form which forms a separate item with a distinctive title, whether issued in one or
several units
NOTE This includes electronic books, serials, maps, but not databases.
3.7
electronic service
electronic library service which is either supplied from local servers or accessible via networks [ISO 2789]
NOTE Electronic library services comprise the OPAC, the library web site, the electronic collection, electronic
document delivery (mediated), electronic reference service, user training on electronic services and Internet access
offered via the library.
3.8
entry
a unit of bibliographic or factual information or multimedia objects included in a searchable database
3.9
information request
information contact that involves the knowledge or use of one or more information sources (such as printed
and non-printed materials, machine-readable databases, the library’s own and other institutions’ catalogues)
by library staff [ISO 2789]
NOTE 1 May also involve recommendations, interpretation or instruction in the use of such sources.
NOTE 2 The request can be delivered personally or by means of telephone, regular mail, fax or electronic media (via e
mail, the library web site, instant messaging or other networked communications mechanisms).
2 © ISO 2003 — All rights reserved
NOTE 3 It is essential that libraries do not include directional and administrative inquiries, eg for locating staff or
facilities, regarding opening times or about handling equipment such as reader printers or computer terminals.
NOTE 4 Inquiries are also excluded if library staff simply help the user to locate items of stock that have already been
identified bibliographically.
3.10
library collection
all documents provided by a library for its users [ISO 2789]
NOTE 1 Comprises documents held locally and remote resources for which permanent or temporary access rights
have been acquired.
NOTE 2 Access rights may be acquired by the library itself, by a consortium and/or through external funding.
NOTE 3 Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in
the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4 Does not include links to Internet resources for which the library has not secured access rights by legal
agreements (eg legal deposit right), license or other contractual and/or co-operative agreement. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.11
population to be served
number of individuals for whom the library is set up to provide its services and materials [ISO 11620]
NOTE For public libraries this will normally be the population of the legal service area (authority); for libraries of an
institution of higher education this will normally be the total of academic and professional staff plus students.
3.12
rejected session (turnaway)
unsuccessful request of a database or the OPAC because of requests exceeding simultaneous user limit
[ISO 2789]
NOTE Rejection through entry of wrong passwords is excluded.
3.13
remote session
a successful request of a database or the OPAC established from outside the library building (adapted from
[1]
EQUINOX )
3.14
session
successful request of a database or the OPAC [ISO 2789]
NOTE 1 A session is one cycle of user activities that typically starts when a user connects to a database or the OPAC
and ends with explicit (by leaving the database through log-out or exit) or implicit (timeout due to user inactivity)
termination of activities in the database. The average timeout period would be 30 minutes. If another time period is used
this should be reported.
NOTE 2 Sessions on the library web site are counted as virtual visits.
NOTE 3 Requests of a general entrance or gateway page should be excluded.
NOTE 4 If possible, requests by search engine should be excluded.
3.15
user training
training programme set up with a specified lesson plan, which aims at specific learning outcomes for the use
of library services [ISO 2789]
NOTE 1 User training can be offered as a tour of the library, as library tuition, or as a web based service for users.
NOTE 2 The duration of a lesson is irrelevant.
3.16
virtual visit
a user's request of the library web site from outside the library premises regardless of the number of pages or
elements viewed [ISO 2789]
3.17
web site
electronic service that has a unique domain on the Internet and consists of a collection of digital documents
[ISO 2789]
NOTE 1 The pages of a web site are usually interconnected by the use of hypertext links.
NOTE 2 Excludes the documents that fit the definitions of electronic collection and external Internet resources that may
be linked from the library web site.
3.18
workstation
computer that may stand alone or be networked, or a dumb terminal [ISO 2789]
4 Relationships with other indicators
4.1 The networked environment
The networked environment provides a different frame of reference for the provision of services: the
boundaries between internal and external provision are different. For example, consider document delivery. In
traditional library services this is represented by the interlending of physical documents between libraries. In
the electronic world, document delivery is accomplished using a network. The network may be provided by a
variety of external providers. Special equipment has to be installed in the libraries at both ends of document
delivery process and staff have to be trained to use it. The suppliers of documents may be publishers or
libraries; the end users may be directly in communication with the suppliers.
Again, the provision of equipment within the organisation of which the library forms a part, may be outside the
library's own control: however, the library’s performance may be judged on its service delivery even though it
does not control some of the infrastructure essential to the delivery of the service.
It is because the boundaries within which electronic services operate differ from those in which traditional
library services operate that comparisons between traditional and electronic library services are difficult, if not
impossible. Since the extent to which different libraries have adopted electronic library services will differ then
comparisons between libraries are increasingly difficult. The position is further complicated by the changes in
expectations, experience and behaviour of users and of publishers. New generations of users will increasingly,
at least in the developed world, be familiar with the use of computers and Internet applications generally and
will expect electronic services as a matter of course. There will, however be for some considerable time
significant numbers of older users who are less familiar with the concepts and practice of computer usage and
will need careful support if they are not to be disadvantaged.
On the other hand, publishers are operating in a variety of ways to deal with the possibilities of electronic
publishing. Long established practices in the supply of printed materials do not operate in the same way when
original works are often generated in electronic format. The economics of publication in electronic formats are
significantly different from those in printed formats and different charging models are being tried. The
preferred format varies for electronic publications between types of publication, and may also vary from year
to year; and the financial arrangements between library and publisher also vary between publications and
from year to year.
4 © ISO 2003 — All rights reserved
Furthermore, the ability to measure the usage of publications in electronic format differs radically from that
associated with print formats. It may be possible, although expensive, for a library to account for every use
made of a printed document within a library: it may well be impossible for the library to count every use made
of electronic documents. The publisher may be able count use made of the electronic format, while having no
idea of the use made of printed documents. Effective performance measurement thus depends on new
arrangements and collaboration between the copyright owners, publishers, agents, librarians and users.
A final point relates to ‘Per capita’ indicators. Some electronic library services (web-sites, OPACs, ‘free’
information services) can be accessed by anyone, and are not limited to the population to be served. In these
cases, usage per capita ratios would be artificially inflated, if the data were collected automatically. If a library
is interested in knowing what proportion of its population to be served is using its services remotely, those
data could be collected by a conventional survey technique.
4.2 Categories of performance indicators
In defining performance indicators for electronic library services it is natural to start by seeking analogies with
performance indicators for traditional library services. Reference to ISO 11620:1998 and its amendment
ISO 11620/Amd. 1 show a list of 34 indicators. These are grouped in a number of categories according to
Table 1.
Table 1
Service, activity or aspect measured Sub-category
User perception General
Public services General
Providing documents
Retrieving documents
Lending documents
Document delivery from external sources
Enquiry and reference services
Information searching
User Education
Facilities
Technical services Acquiring documents
Processing documents
Cataloguing
Promotion of services
User services Deployment of staff
In drawing up this technical report it has proved possible to use the same categories for the proposed
performance indicators with only minor modifications (see Annex A). The applicability of the indicators in
ISO 11620 is discussed in the following subclause.
4.3 The relevance of ISO 11620 to electronic library services
4.3.1 Indicators that apply equally to traditional and electronic library services
Some indicators are the same for both traditional and electronic library services. An obvious example is User
Satisfaction. Others need only very slight changes to existing definitions and descriptions of methods to make
it clear that they are relevant to both types of service: an example is Cost per Title Catalogued. Similarly, the
group of indicators relating to providing documents: here the definitions and descriptions would need to be
modified to include documents in electronic formats. There are other indicators that have analogues in the
networked environment (see 4.3.3), but more substantial revision would be needed to accommodate different
formats, and different bases for measurement. See Table 2.
Table 2
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
User perception: General User Satisfaction B.1.1.1
Public services: Providing Titles Availability B.2.2.1
documents
Required Titles Availability B.2.2.2
B.2.2.3
Percentage of Required Titles in the
Collection
Required Titles Extended Availability B.2.2.4
Public services: Enquiry and Correct Answer Fill Rate B.2.6.1
reference services
Title Catalogue Search Success Rate B.2.7.1
Public services: Information
searching
Subject Catalogue Search Success Rate B.2.7.2
Public services: Facilities Facilities Availability B.2.9.1
Facilities Use Rate B.2.9.2
Seat Occupancy Rate B.2.9.3
Automated Systems Availability B.2.9.4
Median Time of Document Acquisition B.3.1.1
Technical services: Acquiring
documents
Technical services: Cataloguing Cost per Title Catalogued B.3.3.1
User services; Deployment of staff User Services Staff per Capita B.5.1.1
User Services Staff as a Percentage of B.5.1.2
Total Staff
4.3.2 Indicators that apply only to traditional library services
The indicators in this group are so closely related to the physical format of documents that they have no
precise equivalent in the electronic world. See Table 3.
Table 3
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: Providing In-library Use per Capita B.2.2.5
documents
Document Use Rate B.2.2.6
Proportion of Stock not Used B.2.2.7
Shelving Accuracy B.2.2.8
Public services: Retrieving Median Time of Document Retrieval from B.2.3.1
documents Closed Stacks
Median Time of Document Retrieval from B.2.3.2
Open Access Areas
Public services: Document delivery Speed of Interlibrary Lending B.2.5.1
from external sources
Technical services: Processing Median Time of Document Processing B.3.2.1
documents
6 © ISO 2003 — All rights reserved
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services
In this group, the indicators for electronic services are based on the same concepts as those for traditional
services, but the basis for measurement is different. For example, Library Visits per Capita would be replaced
by an indicator relating virtual visits to the population to be served. The various indicators for lending
documents have analogues now that digital lending is possible.
The fact that analogues can be found suggests that, in a future revision of ISO 11620, it may be necessary to
devise composite indicators that integrate traditional and networked services. For example, the indicator
Library Visits per Capita could be redefined to include both physical and virtual visits. Two composite
indicators are included in this Technical Report: Percentage of Virtual Visits to Total Visits, and Percentage of
Information Requests Submitted Electronically. Librarians may wish to create other composite measures and
indicators if that proves helpful in their own circumstances. See Table 4.
Table 4
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: General Percentage of Target Population Reached B.2.1.1
Cost per User B.2.1.2
Public services: General Library Visits per Capita B.2.1.3
Cost per Library Visit B.2.1.4
Public services: Lending Collection Turnover B.2.4.1
documents
Loans per Capita B.2.4.2
Documents on Loan per Capita B.2.4.3
Cost per Loan B.2.4.4
Loans per Employee B.2.4.5
Proportion of Stock on Loan B.2.4.6
5 Use of performance indicators
5.1 Purposes
The performance indicators described in this Technical Report are used as tools to compare the effectiveness,
efficiency and quality of the library's services and products to the library's mission and goals. They can be
used for evaluation purposes in the following areas:
comparing a single library's performance over years
support for management decisions, e. g. reallocating resources, introducing new services, reducing or
deleting existing services
demonstrating the library's performance and its costs to the funders, the population, and the public
comparing performance between libraries of similar structure and mission indicators should be able to
suggest
whether the library's performance or the use of its services has changed over years
how far performance or use in one library differs from that in other libraries.
5.2 Selection of performance indicators
The indicators included in this Technical Report have been developed and tested by libraries in several
projects worldwide. They were chosen for this report as most helpful for libraries in general at the present time,
and as complying with the criteria for performance indicators named in ISO 11620.
Not all of these indicators might be useful for every type of library or every single library. Libraries will have to
decide what indicators are most appropriate for evaluating their special services and products. The decision
should be taken in consideration of the library's mission, goals, and objectives, and the population it is set up
to serve. There should be an agreement with the institution and relevant authorities, as well as with users, as
to what indicators would be most informative for evaluation of the library's electronic services.
Collecting data for performance indicators can be time-consuming and expensive. The decision to use certain
indicators may therefore also depend on their practicality, or possibilities of automated data collection, or on
the urgent need for the evaluation of certain services. This might be the case when an authority requires data
for a particular service, or when it is apparent that a service does not operate satisfactorily. It should be
ensured that the data of indicators will be used subsequently for operational management and for the planning
process.
The indicators named in this Technical Report form a limited set that is applicable to the main electronic library
services and that has been already tested and applied by libraries. They might not prove sufficient for all sorts
of special services, and libraries will probably develop and test more and specialised indicators for their
special needs.
5.3 Limitations
5.3.1 Measuring the impact of electronic library services
The indicators in this Technical Report evaluate the library's performance by quantitative statements about the
supply, use, costs, or market penetration of electronic library services. Libraries are most concerned about the
impact of their services, the question whether users benefit by using the services. Such qualitative data should
be collected in addition to the quantitative data named in this Technical Report. They can be collected by
using methods like user satisfaction surveys, focus groups, or interviews.
The method chosen for collecting qualitative data will depend very much on local circumstances, e. g. the
library's target groups. The general method for assessing user satisfaction is described in ISO 11620.
5.3.2 Degree of accuracy
In order to establish scores for the indicators in this Technical Report, it may be necessary to collect data from
different sources (suppliers, the library's own servers, consortial servers etc.). Thus, data may be varying or
incomplete. Such inconsistencies should be stated explicitly when libraries publish or compare scores.
Other reasons for inaccuracy might be:
search engines can affect the number of sessions
several users, one after the other, might use an electronic service in a way that different sessions could
not be identified
it might be difficult to differentiate use generated by the library's population from other use.
Libraries should define exactly what they regard as their own electronic services, especially as to their
electronic collection (see: definitions). This Technical Report tries to give help in defining the contents of data,
e. g. as to staff employed in developing and maintaining IT services (see: indicator B.2.2.1).
Comment: Such possible sources of inaccuracy should not prevent data collection, but care should be taken
with the interpretation of results.
8 © ISO 2003 — All rights reserved
5.3.3 Comparability of performance indicator data
Libraries will use performance indicators primarily for evaluating their own performance, and for comparing
trends and developments over years. A secondary purpose is to compare results with those of other libraries.
This Technical Report, by describing the indicators and methods, contributes to uniformity of evaluation and
thus to make comparison possible.
In comparing results of indicators, libraries should consider:
the mission, goals, and objectives of each library
the structure of the population to be served
the different services and products of each library
the general conditions under which a library is set up to work.
5.3.4 Indicators of temporary importance
Some of the performance indicators named in this Technical Report have an informative value restricted to a
certain time and goal. They are valid as indicators if the library sees one of its main goals in offering its
services - as far as possible and reasonable - in electronic form.
Such indicators (e.g. B.1.2.1, B.1.4.1, B.2.2.1) measure the amount of resources allocated to electronic
services in comparison to traditional services. The indicators show the library's engagement in electronic
services and the development over years in the direction of a digital library. They will be valid until a
satisfactory state has been reached.
Other performance indicators — indicators of use, costs, or market penetration of electronic services — will
have long-term relevance for libraries.
Annex A
List of performance indicators for electronic library services
Table A.1 lists electronic library services and related activities grouped according to the classification used in
ISO 11620:1998 [1]. Reference is given to the descriptions provided in annex B.
NOTE There are a number of other indicators that have been tested in libraries that are not described. The basic
data elements used for the indicators described here may be combined in various ways to calculate additional indicators of
interest to individual libraries.
Table A.1
Service, activity or Description in
Performance indicator
aspect measured Annex B
Public services B.1
General B.1.1
Percentage of Population Reached by Electronic B.1.1.1
Services
Providing electronic library B.1.2
services
Percentage of Expenditure on Information Provision B.1.2.1
Spent on the Electronic Collection
Retrieving documents B.1.3
Number of Documents Downloaded Per Session B.1.3.1
Cost Per Database Session B.1.3.2
Cost Per Document Downloaded B.1.3.3
Percentage of Rejected Sessions B.1.3.4
Percentage of Remote OPAC Sessions B.1.3.5
Virtual Visits as a Percentage of Total Visits B.1.3.6
Enquiry and reference B.1.4
services
Percentage of Information Requests Submitted B.1.4.1
Electronically
User education B.1.5
Number of User Attendances at Electronic Service B.1.5.1
Training Lessons Per Capita
Facilities B.1.6
Workstation Hours Available Per Capita B.1.6.1
Population Per Public Access Workstation B.1.6.2
Workstation Use Rate B.1.6.3
Availability and use of B.2
human resources
10 © ISO 2003 — All rights reserved
Table A.1 (continued)
Service, activity or Description in
Performance indicator
aspect measured Annex B
Staff training B.2.1
Number of Attendances at Formal IT and Related B.2.1.1
Training Lessons Per Staff Member
Deployment of staff B.2.2
Percentage of Library Staff Providing and Developing B.2.2.1
Electronic Services
Annex B
Descriptions of performance indicators
B.1 Public services
B.1.1 General
B.1.1.1 Percentage of Population Reached by Electronic Services
B.1.1.1.1 Objective
To establish the success of the library in reaching its population to be served.
B.1.1.1.2 Scope
All libraries with a defined population to be served. Libraries may make separate calculations for users of
OPACs, subscription databases, electronic journals etc.
B.1.1.1.3 Definition of the indicator
The percentage of the population to be served who have used any of the electronic services provided by the
library during a specified time period.
B.1.1.1.4 Method
Draw a random sample from the population to be served and ask each person in the sample if they have used
the electronic service during a specified time period. The sample chosen should be representative of the
complete population to be served. Questions on the use of electronic services can usefully be included within
general surveys of library use.
The Percentage of the Population Reached by Electronic Services is
A
×100%
B
where
A is the number of persons in the sample who have used electronic library services during the specified
time period
B is the total number in the sample.
Round off to the nearest integer.
NOTE Alternative sampling and survey methods may also be used. Where a library wishes to make comparisons
over time, it is important that the sampling and survey methods used should be consistent between successive time
periods.
12 © ISO 2003 — All rights reserved
B.1.1.1.5 Interpretation and factors affecting the indicator
The indicator is an integer in the range of 0 to 100. A high score is generally considered better than a low
score. A score that increases from year to year is generally regarded as good.
The score may be affected by several factors, some outside the control of the library service. Examples are:
demographic composition of the population to be served; level of education; general social and economic
conditions; provision of networked access terminals in the library or in the institution; level of network access
available to the population to be served in their homes. It will also be affected by fees charged for services, or
by the rules for access to electronic services.
B.1.1.1.6 Source (see Bibliography)
[1] Adapted from Equinox PI 1
B.1.1.1.7 Related indicators
Percentage of Target Population Reached [ISO 11620:1998, B.2.1.1]; Number of Remote Sessions on
Electronic Services per Capita
B.1.2 Providing electronic services
B.1.2.1 Percentage of Expenditure on Information Provision Spent on the Electronic Collection
B.1.2.1.1 Objective
To assess the extent to which the library is committed to building an electronic collection.
B.1.2.1.2 Scope
All libraries.
May be used for specified parts of a library collection (e. g. journals, subject areas) or individual branches of a
library. Within each of these categories the resulting indicators may be compared to see whether the
percentage differs significantly.
Comparisons between libraries may be possible if differences in subjects, collection policies, and socio-
economic factors in the population are taken into consideration.
B.1.2.1.3 Definition of the indicator
The percentage of the library's total expenditure on information provision spent on the electronic collection.
The electronic collection includes digital documents, electronic serials, and databases.
Expenditure on the electronic collection, for the purpose of this indicator, includes the library's acquisition,
subscription and licensing costs. As an alternative, a library might decide to include pay-per-view and
electronic document delivery costs with the costs of collection building. This should be stated clearly when
publishing or comparing scores.
Total acquisitions expenditure would exclude expenditure on binding.
Expenditure on infrastructure, such as hardware, software or networking, and on digitisation of documents
should not be included.
Value added taxes, sales and service taxes or other local taxes are included. Their inclusion may affect
international comparisons.
B.1.2.1.4 Method
For a given budget period, determine the library's acquisition, subscription and licensing expenditure (and
including, if desired, pay-per-view and electronic document delivery charges) for the electronic collection. If
the library joins in consortia or other over-all contracts, only the library's own share in the contractual
expenses should be counted. Where electronic versions of documents are acquired in a package with print
versions, only the surplus payment for the electronic version should be counted.
The Percentage of Expenditure on Information Provision Spent on the Electronic Collection is
A
×100%
B
where
A is expenditure on the electronic collection
B is total expenditure on information provision.
Round off to the nearest integer.
B.1.2.1.5 Interpretation and factors affecting the indicator
The indicator is an integer between 0 and 100.
Comparison over time shows the extent to which a library shifts its focus to electronic information. However,
differences in pricing structures between printed and electronic resources have considerable effects on
comparisons over time.
The indicator must be judged against mission and objectives of the library. Collection policies, the structure of
the population and especially the subjects collected by the library can greatly influence the score.
The indicator should therefore not be used by itself, but in conjunction with indicators of collection use and
user satisfaction.
B.1.2.1.6 Source (see Bibliography)
[1] Equinox PI 11
B.1.2.1.7 Related indicators
Percentage of Staff Devoted to IT Services; Percentage of the Population Reached by Electronic Services
B.1.3 Retrieving documents
B.1.3.1 Number of Documents Downloaded Per Session
B.1.3.1.1 Objective
To assess whether users find items of interest in an electronic resource.
B.1.3.1.2 Scope
All libraries.
14 © ISO 2003 — All rights reserved
B.1.3.1.3 Definition of the indicator
The number of documents and entries downloaded in part or in whole from each electronic resource, divided
by the number of sessions on each service during a specified time period.
Use of electronic resources by library staff and for user training is included in the number of sessions as well
as in the number of documents and entries downloaded.
B.1.3.1.4 Method
Count the number
...
TECHNICAL ISO/TR
REPORT 20983
First edition
2003-11-01
Information and documentation —
Performance indicators for electronic
library services
Information et documentation — Indicateurs de performance pour les
services électroniques des bibliothèques
Reference number
©
ISO 2003
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO 2003
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2003 — All rights reserved
Contents Page
Foreword. iv
Introduction . v
1 Scope. 1
2 Normative references . 1
3 Terms and definitions. 1
4 Relationships with other indicators . 4
4.1 The networked environment . 4
4.2 Categories of performance indicators. 5
4.3 The relevance of ISO 11620 to electronic library services . 5
4.3.1 Indicators that apply equally to traditional and electronic library services . 5
4.3.2 Indicators that apply only to traditional library services. 6
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services. 7
5 Use of performance indicators . 7
5.1 Purposes. 7
5.2 Selection of performance indicators. 8
5.3 Limitations . 8
5.3.1 Measuring the impact of electronic library services. 8
5.3.2 Degree of accuracy. 8
5.3.3 Comparability of performance indicator data . 9
5.3.4 Indicators of temporary importance . 9
Annex A List of performance indicators for electronic library services . 10
Annex B Descriptions of performance indicators . 12
Bibliography . 31
Alphabetical index . 32
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 20983 was prepared by Technical Committee ISO/TC 46, Information and documentation,
Subcommittee SC 8, Quality — Statistics and performance evaluation.
iv © ISO 2003 — All rights reserved
Introduction
This Technical Report is concerned with the evaluation of electronic library services.
The successful provision of these services depends on close collaboration between publishers, network
providers, and librarians. Their needs for data about various aspects of performance are different, and the
ability to collect data may not rest with the person most interested in that data. For example, the publisher may
be best placed to collect data on the usage of a particular service; whereas the librarian may have most
interest in using that data in developing and evaluating the service.
The main purpose of this Technical Report is to spread knowledge about the practice of evaluating electronic
library services. Although electronic library services have been developed over more than thirty years, the
technical environment has changed significantly in the last ten years. The need for performance indicators for
electronic services was acknowledged in ISO 11620: 1998 Information and documentation - Library
performance indicators. It was also acknowledged that at the time of publication of that International Standard,
there were no indicators that had been tested and documented, or that were in widespread use. Although
there has been significant progress in the last few years on the definition, testing and documentation of
indicators, it is still too soon to determine which indicators will be most generally useful, or become widely
used. This Technical Report provides a standardized terminology and concise definitions and descriptions of a
selection of performance indicators, in the same format as that used in ISO 11620. Publications listed in the
Bibliography [2,3,5] provide more detailed information on data collection and analysis.
Only a small number of the indicators presented here are indicators of service quality: work on this area is not
yet mature enough to be included. The indicators that are included have been tested in libraries, or are simple
adaptations of similar indicators. In each case they have been judged against the criteria established in
ISO 11620. That is, each indicator is judged to have informative content, to be reliable, valid, appropriate,
practical, and in some circumstances may be used for comparative purposes.
The indicators presented in this Report are a representative selection of those which have been tested. A
further selection would have taken longer to prepare, and to publish. Developments in the measurement and
evaluation of electronic library services will be monitored by a Working Group, which will propose additional
indicators, and modifications or adaptations to the indicators here as they are tested and validated. This
Technical Report should be considered as work in progress towards the development of an International
Standard. It is intended that, where appropriate, the contents of this Technical Report will be incorporated in a
future revision of ISO 11620.
TECHNICAL REPORT ISO/TR 20983:2003(E)
Information and documentation — Performance indicators for
electronic library services
1 Scope
This Technical Report is applicable to all types of libraries in all countries. Limitations on the applicability of
individual performance indicators are listed in the scope clause of the description of each indicator (see
Annex B).
Indicators may be used for comparison over time for the services provided by the same library. Comparisons
between libraries and services should only be made with caution, taking into account differences in the
constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the
data.
This Technical Report does not include indicators for the evaluation of the impact of libraries either on
individuals or on society.
Throughout the text, the names of indicators are printed with initial capitals for significant words, e.g.
Workstation Use Rate. This helps to distinguish the names from supporting text.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 2789:2003, Information and documentation — International library statistics
ISO 5127:2001, Information and documentation — Vocabulary
ISO 11620:1998, Information and documentation — Library performance indicators
ISO 11620:1998/Amd. 1:2003, Information and documentation — Library performance indicators —
Amendment 1: Additional performance indicators for libraries
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
database
collection of electronically stored data or unit records (facts, bibliographic data, texts) with a common user
interface and software for the retrieval and manipulation of the data [ISO 2789]
NOTE 1 The data or records are usually collected with a particular intent and are related to a defined topic. A database
may be issued on CD-ROM, diskette, or other direct access method, or as a computer file accessed via dial-up methods
or via the Internet.
NOTE 2 Licensed databases are counted separately even if access to several licensed database products is effected
through the same interface.
3.2
digital document
information unit with a defined content that has been digitized by the library or acquired in digital form as part
of the library collection [ISO 2789]
NOTE 1 This includes eBooks, electronic patents, networked audiovisual documents and other digital documents, e.g.
reports, cartographic and music documents, pre-prints etc. Databases and electronic serials are excluded.
NOTE 2 Items incorporated in databases are covered by 3.1.
NOTE 3 A digital document can be structured into one or more files.
3.3
document
recorded information or material object which can be treated as a unit in a documentation process [ISO 5127]
NOTE Documents may differ in their physical form or characteristics.
3.4
document downloaded
full text of a document or part of a document in the electronic collection that is delivered to a user [ISO 2789]
3.5
electronic collection
all resources in electronic form in the library collection [ISO 2789]
NOTE The electronic collection includes databases, electronic serials, and digital documents. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.6
electronic resource
a document in electronic form which forms a separate item with a distinctive title, whether issued in one or
several units
NOTE This includes electronic books, serials, maps, but not databases.
3.7
electronic service
electronic library service which is either supplied from local servers or accessible via networks [ISO 2789]
NOTE Electronic library services comprise the OPAC, the library web site, the electronic collection, electronic
document delivery (mediated), electronic reference service, user training on electronic services and Internet access
offered via the library.
3.8
entry
a unit of bibliographic or factual information or multimedia objects included in a searchable database
3.9
information request
information contact that involves the knowledge or use of one or more information sources (such as printed
and non-printed materials, machine-readable databases, the library’s own and other institutions’ catalogues)
by library staff [ISO 2789]
NOTE 1 May also involve recommendations, interpretation or instruction in the use of such sources.
NOTE 2 The request can be delivered personally or by means of telephone, regular mail, fax or electronic media (via e
mail, the library web site, instant messaging or other networked communications mechanisms).
2 © ISO 2003 — All rights reserved
NOTE 3 It is essential that libraries do not include directional and administrative inquiries, eg for locating staff or
facilities, regarding opening times or about handling equipment such as reader printers or computer terminals.
NOTE 4 Inquiries are also excluded if library staff simply help the user to locate items of stock that have already been
identified bibliographically.
3.10
library collection
all documents provided by a library for its users [ISO 2789]
NOTE 1 Comprises documents held locally and remote resources for which permanent or temporary access rights
have been acquired.
NOTE 2 Access rights may be acquired by the library itself, by a consortium and/or through external funding.
NOTE 3 Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in
the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4 Does not include links to Internet resources for which the library has not secured access rights by legal
agreements (eg legal deposit right), license or other contractual and/or co-operative agreement. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.11
population to be served
number of individuals for whom the library is set up to provide its services and materials [ISO 11620]
NOTE For public libraries this will normally be the population of the legal service area (authority); for libraries of an
institution of higher education this will normally be the total of academic and professional staff plus students.
3.12
rejected session (turnaway)
unsuccessful request of a database or the OPAC because of requests exceeding simultaneous user limit
[ISO 2789]
NOTE Rejection through entry of wrong passwords is excluded.
3.13
remote session
a successful request of a database or the OPAC established from outside the library building (adapted from
[1]
EQUINOX )
3.14
session
successful request of a database or the OPAC [ISO 2789]
NOTE 1 A session is one cycle of user activities that typically starts when a user connects to a database or the OPAC
and ends with explicit (by leaving the database through log-out or exit) or implicit (timeout due to user inactivity)
termination of activities in the database. The average timeout period would be 30 minutes. If another time period is used
this should be reported.
NOTE 2 Sessions on the library web site are counted as virtual visits.
NOTE 3 Requests of a general entrance or gateway page should be excluded.
NOTE 4 If possible, requests by search engine should be excluded.
3.15
user training
training programme set up with a specified lesson plan, which aims at specific learning outcomes for the use
of library services [ISO 2789]
NOTE 1 User training can be offered as a tour of the library, as library tuition, or as a web based service for users.
NOTE 2 The duration of a lesson is irrelevant.
3.16
virtual visit
a user's request of the library web site from outside the library premises regardless of the number of pages or
elements viewed [ISO 2789]
3.17
web site
electronic service that has a unique domain on the Internet and consists of a collection of digital documents
[ISO 2789]
NOTE 1 The pages of a web site are usually interconnected by the use of hypertext links.
NOTE 2 Excludes the documents that fit the definitions of electronic collection and external Internet resources that may
be linked from the library web site.
3.18
workstation
computer that may stand alone or be networked, or a dumb terminal [ISO 2789]
4 Relationships with other indicators
4.1 The networked environment
The networked environment provides a different frame of reference for the provision of services: the
boundaries between internal and external provision are different. For example, consider document delivery. In
traditional library services this is represented by the interlending of physical documents between libraries. In
the electronic world, document delivery is accomplished using a network. The network may be provided by a
variety of external providers. Special equipment has to be installed in the libraries at both ends of document
delivery process and staff have to be trained to use it. The suppliers of documents may be publishers or
libraries; the end users may be directly in communication with the suppliers.
Again, the provision of equipment within the organisation of which the library forms a part, may be outside the
library's own control: however, the library’s performance may be judged on its service delivery even though it
does not control some of the infrastructure essential to the delivery of the service.
It is because the boundaries within which electronic services operate differ from those in which traditional
library services operate that comparisons between traditional and electronic library services are difficult, if not
impossible. Since the extent to which different libraries have adopted electronic library services will differ then
comparisons between libraries are increasingly difficult. The position is further complicated by the changes in
expectations, experience and behaviour of users and of publishers. New generations of users will increasingly,
at least in the developed world, be familiar with the use of computers and Internet applications generally and
will expect electronic services as a matter of course. There will, however be for some considerable time
significant numbers of older users who are less familiar with the concepts and practice of computer usage and
will need careful support if they are not to be disadvantaged.
On the other hand, publishers are operating in a variety of ways to deal with the possibilities of electronic
publishing. Long established practices in the supply of printed materials do not operate in the same way when
original works are often generated in electronic format. The economics of publication in electronic formats are
significantly different from those in printed formats and different charging models are being tried. The
preferred format varies for electronic publications between types of publication, and may also vary from year
to year; and the financial arrangements between library and publisher also vary between publications and
from year to year.
4 © ISO 2003 — All rights reserved
Furthermore, the ability to measure the usage of publications in electronic format differs radically from that
associated with print formats. It may be possible, although expensive, for a library to account for every use
made of a printed document within a library: it may well be impossible for the library to count every use made
of electronic documents. The publisher may be able count use made of the electronic format, while having no
idea of the use made of printed documents. Effective performance measurement thus depends on new
arrangements and collaboration between the copyright owners, publishers, agents, librarians and users.
A final point relates to ‘Per capita’ indicators. Some electronic library services (web-sites, OPACs, ‘free’
information services) can be accessed by anyone, and are not limited to the population to be served. In these
cases, usage per capita ratios would be artificially inflated, if the data were collected automatically. If a library
is interested in knowing what proportion of its population to be served is using its services remotely, those
data could be collected by a conventional survey technique.
4.2 Categories of performance indicators
In defining performance indicators for electronic library services it is natural to start by seeking analogies with
performance indicators for traditional library services. Reference to ISO 11620:1998 and its amendment
ISO 11620/Amd. 1 show a list of 34 indicators. These are grouped in a number of categories according to
Table 1.
Table 1
Service, activity or aspect measured Sub-category
User perception General
Public services General
Providing documents
Retrieving documents
Lending documents
Document delivery from external sources
Enquiry and reference services
Information searching
User Education
Facilities
Technical services Acquiring documents
Processing documents
Cataloguing
Promotion of services
User services Deployment of staff
In drawing up this technical report it has proved possible to use the same categories for the proposed
performance indicators with only minor modifications (see Annex A). The applicability of the indicators in
ISO 11620 is discussed in the following subclause.
4.3 The relevance of ISO 11620 to electronic library services
4.3.1 Indicators that apply equally to traditional and electronic library services
Some indicators are the same for both traditional and electronic library services. An obvious example is User
Satisfaction. Others need only very slight changes to existing definitions and descriptions of methods to make
it clear that they are relevant to both types of service: an example is Cost per Title Catalogued. Similarly, the
group of indicators relating to providing documents: here the definitions and descriptions would need to be
modified to include documents in electronic formats. There are other indicators that have analogues in the
networked environment (see 4.3.3), but more substantial revision would be needed to accommodate different
formats, and different bases for measurement. See Table 2.
Table 2
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
User perception: General User Satisfaction B.1.1.1
Public services: Providing Titles Availability B.2.2.1
documents
Required Titles Availability B.2.2.2
B.2.2.3
Percentage of Required Titles in the
Collection
Required Titles Extended Availability B.2.2.4
Public services: Enquiry and Correct Answer Fill Rate B.2.6.1
reference services
Title Catalogue Search Success Rate B.2.7.1
Public services: Information
searching
Subject Catalogue Search Success Rate B.2.7.2
Public services: Facilities Facilities Availability B.2.9.1
Facilities Use Rate B.2.9.2
Seat Occupancy Rate B.2.9.3
Automated Systems Availability B.2.9.4
Median Time of Document Acquisition B.3.1.1
Technical services: Acquiring
documents
Technical services: Cataloguing Cost per Title Catalogued B.3.3.1
User services; Deployment of staff User Services Staff per Capita B.5.1.1
User Services Staff as a Percentage of B.5.1.2
Total Staff
4.3.2 Indicators that apply only to traditional library services
The indicators in this group are so closely related to the physical format of documents that they have no
precise equivalent in the electronic world. See Table 3.
Table 3
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: Providing In-library Use per Capita B.2.2.5
documents
Document Use Rate B.2.2.6
Proportion of Stock not Used B.2.2.7
Shelving Accuracy B.2.2.8
Public services: Retrieving Median Time of Document Retrieval from B.2.3.1
documents Closed Stacks
Median Time of Document Retrieval from B.2.3.2
Open Access Areas
Public services: Document delivery Speed of Interlibrary Lending B.2.5.1
from external sources
Technical services: Processing Median Time of Document Processing B.3.2.1
documents
6 © ISO 2003 — All rights reserved
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services
In this group, the indicators for electronic services are based on the same concepts as those for traditional
services, but the basis for measurement is different. For example, Library Visits per Capita would be replaced
by an indicator relating virtual visits to the population to be served. The various indicators for lending
documents have analogues now that digital lending is possible.
The fact that analogues can be found suggests that, in a future revision of ISO 11620, it may be necessary to
devise composite indicators that integrate traditional and networked services. For example, the indicator
Library Visits per Capita could be redefined to include both physical and virtual visits. Two composite
indicators are included in this Technical Report: Percentage of Virtual Visits to Total Visits, and Percentage of
Information Requests Submitted Electronically. Librarians may wish to create other composite measures and
indicators if that proves helpful in their own circumstances. See Table 4.
Table 4
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: General Percentage of Target Population Reached B.2.1.1
Cost per User B.2.1.2
Public services: General Library Visits per Capita B.2.1.3
Cost per Library Visit B.2.1.4
Public services: Lending Collection Turnover B.2.4.1
documents
Loans per Capita B.2.4.2
Documents on Loan per Capita B.2.4.3
Cost per Loan B.2.4.4
Loans per Employee B.2.4.5
Proportion of Stock on Loan B.2.4.6
5 Use of performance indicators
5.1 Purposes
The performance indicators described in this Technical Report are used as tools to compare the effectiveness,
efficiency and quality of the library's services and products to the library's mission and goals. They can be
used for evaluation purposes in the following areas:
comparing a single library's performance over years
support for management decisions, e. g. reallocating resources, introducing new services, reducing or
deleting existing services
demonstrating the library's performance and its costs to the funders, the population, and the public
comparing performance between libraries of similar structure and mission indicators should be able to
suggest
whether the library's performance or the use of its services has changed over years
how far performance or use in one library differs from that in other libraries.
5.2 Selection of performance indicators
The indicators included in this Technical Report have been developed and tested by libraries in several
projects worldwide. They were chosen for this report as most helpful for libraries in general at the present time,
and as complying with the criteria for performance indicators named in ISO 11620.
Not all of these indicators might be useful for every type of library or every single library. Libraries will have to
decide what indicators are most appropriate for evaluating their special services and products. The decision
should be taken in consideration of the library's mission, goals, and objectives, and the population it is set up
to serve. There should be an agreement with the institution and relevant authorities, as well as with users, as
to what indicators would be most informative for evaluation of the library's electronic services.
Collecting data for performance indicators can be time-consuming and expensive. The decision to use certain
indicators may therefore also depend on their practicality, or possibilities of automated data collection, or on
the urgent need for the evaluation of certain services. This might be the case when an authority requires data
for a particular service, or when it is apparent that a service does not operate satisfactorily. It should be
ensured that the data of indicators will be used subsequently for operational management and for the planning
process.
The indicators named in this Technical Report form a limited set that is applicable to the main electronic library
services and that has been already tested and applied by libraries. They might not prove sufficient for all sorts
of special services, and libraries will probably develop and test more and specialised indicators for their
special needs.
5.3 Limitations
5.3.1 Measuring the impact of electronic library services
The indicators in this Technical Report evaluate the library's performance by quantitative statements about the
supply, use, costs, or market penetration of electronic library services. Libraries are most concerned about the
impact of their services, the question whether users benefit by using the services. Such qualitative data should
be collected in addition to the quantitative data named in this Technical Report. They can be collected by
using methods like user satisfaction surveys, focus groups, or interviews.
The method chosen for collecting qualitative data will depend very much on local circumstances, e. g. the
library's target groups. The general method for assessing user satisfaction is described in ISO 11620.
5.3.2 Degree of accuracy
In order to establish scores for the indicators in this Technical Report, it may be necessary to collect data from
different sources (suppliers, the library's own servers, consortial servers etc.). Thus, data may be varying or
incomplete. Such inconsistencies should be stated explicitly when libraries publish or compare scores.
Other reasons for inaccuracy might be:
search engines can affect the number of sessions
several users, one after the other, might use an electronic service in a way that different sessions could
not be identified
it might be difficult to differentiate use generated by the library's population from other use.
Libraries should define exactly what they regard as their own electronic services, especially as to their
electronic collection (see: definitions). This Technical Report tries to give help in defining the contents of data,
e. g. as to staff employed in developing and maintaining IT services (see: indicator B.2.2.1).
Comment: Such possible sources of inaccuracy should not prevent data collection, but care should be taken
with the interpretation of results.
8 © ISO 2003 — All rights reserved
5.3.3 Comparability of performance indicator data
Libraries will use performance indicators primarily for evaluating their own performance, and for comparing
trends and developments over years. A secondary purpose is to compare results with those of other libraries.
This Technical Report, by describing the indicators and methods, contributes to uniformity of evaluation and
thus to make comparison possible.
In comparing results of indicators, libraries should consider:
the mission, goals, and objectives of each library
the structure of the population to be served
the different services and products of each library
the general conditions under which a library is set up to work.
5.3.4 Indicators of temporary importance
Some of the performance indicators named in this Technical Report have an informative value restricted to a
certain time and goal. They are valid as indicators if the library sees one of its main goals in offering its
services - as far as possible and reasonable - in electronic form.
Such indicators (e.g. B.1.2.1, B.1.4.1, B.2.2.1) measure the amount of resources allocated to electronic
services in comparison to traditional services. The indicators show the library's engagement in electronic
services and the development over years in the direction of a digital library. They will be valid until a
satisfactory state has been reached.
Other performance indicators — indicators of use, costs, or market penetration of electronic services — will
have long-term relevance for libraries.
Annex A
List of performance indicators for electronic library services
Table A.1 lists electronic library services and related activities grouped according to the classification used in
ISO 11620:1998 [1]. Reference is given to the descriptions provided in annex B.
NOTE There are a number of other indicators that have been tested in libraries that are not described. The basic
data elements used for the indicators described here may be combined in various ways to calculate additional indicators of
interest to individual libraries.
Table A.1
Service, activity or Description in
Performance indicator
aspect measured Annex B
Public services B.1
General B.1.1
Percentage of Population Reached by Electronic B.1.1.1
Services
Providing electronic library B.1.2
services
Percentage of Expenditure on Information Provision B.1.2.1
Spent on the Electronic Collection
Retrieving documents B.1.3
Number of Documents Downloaded Per Session B.1.3.1
Cost Per Database Session B.1.3.2
Cost Per Document Downloaded B.1.3.3
Percentage of Rejected Sessions B.1.3.4
Percentage of Remote OPAC Sessions B.1.3.5
Virtual Visits as a Percentage of Total Visits B.1.3.6
Enquiry and reference B.1.4
services
Percentage of Information Requests Submitted B.1.4.1
Electronically
User education B.1.5
Number of User Attendances at Electronic Service B.1.5.1
Training Lessons Per Capita
Facilities B.1.6
Workstation Hours Available Per Capita B.1.6.1
Population Per Public Access Workstation B.1.6.2
Workstation Use Rate B.1.6.3
Availability and use of B.2
human resources
10 © ISO 2003 — All rights reserved
Table A.1 (continued)
Service, activity or Description in
Performance indicator
aspect measured Annex B
Staff training B.2.1
Number of Attendances at Formal IT and Related B.2.1.1
Training Lessons Per Staff Member
Deployment of staff B.2.2
Percentage of Library Staff Providing and Developing B.2.2.1
Electronic Services
Annex B
Descriptions of performance indicators
B.1 Public services
B.1.1 General
B.1.1.1 Percentage of Population Reached by Electronic Services
B.1.1.1.1 Objective
To establish the success of the library in reaching its population to be served.
B.1.1.1.2 Scope
All libraries with a defined population to be served. Libraries may make separate calculations for users of
OPACs, subscription databases, electronic journals etc.
B.1.1.1.3 Definition of the indicator
The percentage of the population to be served who have used any of the electronic services provided by the
library during a specified time period.
B.1.1.1.4 Method
Draw a random sample from the population to be served and ask each person in the sample if they have used
the electronic service during a specified time period. The sample chosen should be representative of the
complete population to be served. Questions on the use of electronic services can usefully be included within
general surveys of library use.
The Percentage of the Population Reached by Electronic Services is
A
×100%
B
where
A is the number of persons in the sample who have used electronic library services during the specified
time period
B is the total number in the sample.
Round off to the nearest integer.
NOTE Alternative sampling and survey methods may also be used. Where a library wishes to make comparisons
over time, it is important that the sampling and survey methods used should be consistent between successive time
periods.
12 © ISO 2003 — All rights reserved
B.1.1.1.5 Interpretation and factors affecting the indicator
The indicator is an integer in the range of 0 to 100. A high score is generally considered better than a low
score. A score that increases from year to year is generally regarded as good.
The score may be affected by several factors, some outside the control of the library service. Examples are:
demographic composition of the population to be served; level of education; general social and economic
conditions; provision of networked access terminals in the library or in the institution; level of network access
available to the population to be served in their homes. It will also be affected by fees charged for services, or
by the rules for access to electronic services.
B.1.1.1.6 Source (see Bibliography)
[1] Adapted from Equinox PI 1
B.1.1.1.7 Related indicators
Percentage of Target Population Reached [ISO 11620:1998, B.2.1.1]; Number of Remote Sessions on
Electronic Services per Capita
B.1.2 Providing electronic services
B.1.2.1 Percentage of Expenditure on Information Provision Spent on the Electronic Collection
B.1.2.1.1 Objective
To assess the extent to which the library is committed to building an electronic collection.
B.1.2.1.2 Scope
All libraries.
May be used for specified parts of a library collection (e. g. journals, subject areas) or individual branches of a
library. Within each of these categories the resulting indicators may be compared to see whether the
percentage differs significantly.
Comparisons between libraries may be possible if differences in subjects, collection policies, and socio-
economic factors in the population are taken into consideration.
B.1.2.1.3 Definition of the indicator
The percentage of the library's total expenditure on information provision spent on the electronic collection.
The electronic collection includes digital documents, electronic serials, and databases.
Expenditure on the electronic collection, for the purpose of this indicator, includes the library's acquisition,
subscription and licensing costs. As an alternative, a library might decide to include pay-per-view and
electronic document delivery costs with the costs of collection building. This should be stated clearly when
publishing or comparing scores.
Total acquisitions expenditure would exclude expenditure on binding.
Expenditure on infrastructure, such as hardware, software or networking, and on digitisation of documents
should not be included.
Value added taxes, sales and service taxes or other local taxes are included. Their inclusion may affect
international comparisons.
B.1.2.1.4 Method
For a given budget period, determine the library's acquisition, subscription and licensing expenditure (and
including, if desired, pay-per-view and electronic document delivery charges) for the electronic collection. If
the library joins in consortia or other over-all contracts, only the library's own share in the contractual
expenses should be counted. Where electronic versions of documents are acquired in a package with print
versions, only the surplus payment for the electronic version should be counted.
The Percentage of Expenditure on Information Provision Spent on the Electronic Collection is
A
×100%
B
where
A is expenditure on the electronic collection
B is total expenditure on information provision.
Round off to the nearest integer.
B.1.2.1.5 Interpretation and factors affecting the indicator
The indicator is an integer between 0 and 100.
Comparison over time shows the extent to which a library shifts its focus to electronic information. However,
differences in pricing structures between printed and electronic resources have considerable effects on
comparisons over time.
The indicator must be judged against mission and objectives of the library. Collection policies, the structure of
the population and especially the subjects collected by the library can greatly influence the score.
The indicator should therefore not be used by itself, but in conjunction with indicators of collection use and
user satisfaction.
B.1.2.1.6 Source (see Bibliography)
[1] Equinox PI 11
B.1.2.1.7 Related indicators
Percentage of Staff Devoted to IT Services; Percentage of the Population Reached by Electronic Services
B.1.3 Retrieving documents
B.1.3.1 Number of Documents Downloaded Per Session
B.1.3.1.1 Objective
To assess whether users find items of interest in an electronic resource.
B.1.3.1.2 Scope
All libraries.
14 © ISO 2003 — All rights reserved
B.1.3.1.3 Definition of the indicator
The number of documents and entries downloaded in part or in whole from each electronic resource, divided
by the number of sessions on each service during a specified time period.
Use of electronic resources by library staff and for user training is included in the number of sessions as well
as in the number of documents and entries downloaded.
B.1.3.1.4 Method
Count the number of documents and entries downloaded fr
...
SLOVENSKI STANDARD
01-november-2005
,QIRUPDWLNDLQGRNXPHQWDFLMD±.D]DOFLXVSHãQRVWLHOHNWURQVNLKNQMLåQLþQLK
VWRULWHY
Information and documentation -- Performance indicators for electronic library services
Information et documentation -- Indicateurs de performance pour les services
électroniques des bibliothèques
Ta slovenski standard je istoveten z: ISO/TR 20983:2003
ICS:
01.140.20 Informacijske vede Information sciences
35.240.30 Uporabniške rešitve IT v IT applications in information,
informatiki, dokumentiranju in documentation and
založništvu publishing
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
TECHNICAL ISO/TR
REPORT 20983
First edition
2003-11-01
Information and documentation —
Performance indicators for electronic
library services
Information et documentation — Indicateurs de performance pour les
services électroniques des bibliothèques
Reference number
©
ISO 2003
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO 2003
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2003 — All rights reserved
Contents Page
Foreword. iv
Introduction . v
1 Scope. 1
2 Normative references . 1
3 Terms and definitions. 1
4 Relationships with other indicators . 4
4.1 The networked environment . 4
4.2 Categories of performance indicators. 5
4.3 The relevance of ISO 11620 to electronic library services . 5
4.3.1 Indicators that apply equally to traditional and electronic library services . 5
4.3.2 Indicators that apply only to traditional library services. 6
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services. 7
5 Use of performance indicators . 7
5.1 Purposes. 7
5.2 Selection of performance indicators. 8
5.3 Limitations . 8
5.3.1 Measuring the impact of electronic library services. 8
5.3.2 Degree of accuracy. 8
5.3.3 Comparability of performance indicator data . 9
5.3.4 Indicators of temporary importance . 9
Annex A List of performance indicators for electronic library services . 10
Annex B Descriptions of performance indicators . 12
Bibliography . 31
Alphabetical index . 32
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 20983 was prepared by Technical Committee ISO/TC 46, Information and documentation,
Subcommittee SC 8, Quality — Statistics and performance evaluation.
iv © ISO 2003 — All rights reserved
Introduction
This Technical Report is concerned with the evaluation of electronic library services.
The successful provision of these services depends on close collaboration between publishers, network
providers, and librarians. Their needs for data about various aspects of performance are different, and the
ability to collect data may not rest with the person most interested in that data. For example, the publisher may
be best placed to collect data on the usage of a particular service; whereas the librarian may have most
interest in using that data in developing and evaluating the service.
The main purpose of this Technical Report is to spread knowledge about the practice of evaluating electronic
library services. Although electronic library services have been developed over more than thirty years, the
technical environment has changed significantly in the last ten years. The need for performance indicators for
electronic services was acknowledged in ISO 11620: 1998 Information and documentation - Library
performance indicators. It was also acknowledged that at the time of publication of that International Standard,
there were no indicators that had been tested and documented, or that were in widespread use. Although
there has been significant progress in the last few years on the definition, testing and documentation of
indicators, it is still too soon to determine which indicators will be most generally useful, or become widely
used. This Technical Report provides a standardized terminology and concise definitions and descriptions of a
selection of performance indicators, in the same format as that used in ISO 11620. Publications listed in the
Bibliography [2,3,5] provide more detailed information on data collection and analysis.
Only a small number of the indicators presented here are indicators of service quality: work on this area is not
yet mature enough to be included. The indicators that are included have been tested in libraries, or are simple
adaptations of similar indicators. In each case they have been judged against the criteria established in
ISO 11620. That is, each indicator is judged to have informative content, to be reliable, valid, appropriate,
practical, and in some circumstances may be used for comparative purposes.
The indicators presented in this Report are a representative selection of those which have been tested. A
further selection would have taken longer to prepare, and to publish. Developments in the measurement and
evaluation of electronic library services will be monitored by a Working Group, which will propose additional
indicators, and modifications or adaptations to the indicators here as they are tested and validated. This
Technical Report should be considered as work in progress towards the development of an International
Standard. It is intended that, where appropriate, the contents of this Technical Report will be incorporated in a
future revision of ISO 11620.
TECHNICAL REPORT ISO/TR 20983:2003(E)
Information and documentation — Performance indicators for
electronic library services
1 Scope
This Technical Report is applicable to all types of libraries in all countries. Limitations on the applicability of
individual performance indicators are listed in the scope clause of the description of each indicator (see
Annex B).
Indicators may be used for comparison over time for the services provided by the same library. Comparisons
between libraries and services should only be made with caution, taking into account differences in the
constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the
data.
This Technical Report does not include indicators for the evaluation of the impact of libraries either on
individuals or on society.
Throughout the text, the names of indicators are printed with initial capitals for significant words, e.g.
Workstation Use Rate. This helps to distinguish the names from supporting text.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 2789:2003, Information and documentation — International library statistics
ISO 5127:2001, Information and documentation — Vocabulary
ISO 11620:1998, Information and documentation — Library performance indicators
ISO 11620:1998/Amd. 1:2003, Information and documentation — Library performance indicators —
Amendment 1: Additional performance indicators for libraries
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
database
collection of electronically stored data or unit records (facts, bibliographic data, texts) with a common user
interface and software for the retrieval and manipulation of the data [ISO 2789]
NOTE 1 The data or records are usually collected with a particular intent and are related to a defined topic. A database
may be issued on CD-ROM, diskette, or other direct access method, or as a computer file accessed via dial-up methods
or via the Internet.
NOTE 2 Licensed databases are counted separately even if access to several licensed database products is effected
through the same interface.
3.2
digital document
information unit with a defined content that has been digitized by the library or acquired in digital form as part
of the library collection [ISO 2789]
NOTE 1 This includes eBooks, electronic patents, networked audiovisual documents and other digital documents, e.g.
reports, cartographic and music documents, pre-prints etc. Databases and electronic serials are excluded.
NOTE 2 Items incorporated in databases are covered by 3.1.
NOTE 3 A digital document can be structured into one or more files.
3.3
document
recorded information or material object which can be treated as a unit in a documentation process [ISO 5127]
NOTE Documents may differ in their physical form or characteristics.
3.4
document downloaded
full text of a document or part of a document in the electronic collection that is delivered to a user [ISO 2789]
3.5
electronic collection
all resources in electronic form in the library collection [ISO 2789]
NOTE The electronic collection includes databases, electronic serials, and digital documents. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.6
electronic resource
a document in electronic form which forms a separate item with a distinctive title, whether issued in one or
several units
NOTE This includes electronic books, serials, maps, but not databases.
3.7
electronic service
electronic library service which is either supplied from local servers or accessible via networks [ISO 2789]
NOTE Electronic library services comprise the OPAC, the library web site, the electronic collection, electronic
document delivery (mediated), electronic reference service, user training on electronic services and Internet access
offered via the library.
3.8
entry
a unit of bibliographic or factual information or multimedia objects included in a searchable database
3.9
information request
information contact that involves the knowledge or use of one or more information sources (such as printed
and non-printed materials, machine-readable databases, the library’s own and other institutions’ catalogues)
by library staff [ISO 2789]
NOTE 1 May also involve recommendations, interpretation or instruction in the use of such sources.
NOTE 2 The request can be delivered personally or by means of telephone, regular mail, fax or electronic media (via e
mail, the library web site, instant messaging or other networked communications mechanisms).
2 © ISO 2003 — All rights reserved
NOTE 3 It is essential that libraries do not include directional and administrative inquiries, eg for locating staff or
facilities, regarding opening times or about handling equipment such as reader printers or computer terminals.
NOTE 4 Inquiries are also excluded if library staff simply help the user to locate items of stock that have already been
identified bibliographically.
3.10
library collection
all documents provided by a library for its users [ISO 2789]
NOTE 1 Comprises documents held locally and remote resources for which permanent or temporary access rights
have been acquired.
NOTE 2 Access rights may be acquired by the library itself, by a consortium and/or through external funding.
NOTE 3 Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in
the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4 Does not include links to Internet resources for which the library has not secured access rights by legal
agreements (eg legal deposit right), license or other contractual and/or co-operative agreement. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.11
population to be served
number of individuals for whom the library is set up to provide its services and materials [ISO 11620]
NOTE For public libraries this will normally be the population of the legal service area (authority); for libraries of an
institution of higher education this will normally be the total of academic and professional staff plus students.
3.12
rejected session (turnaway)
unsuccessful request of a database or the OPAC because of requests exceeding simultaneous user limit
[ISO 2789]
NOTE Rejection through entry of wrong passwords is excluded.
3.13
remote session
a successful request of a database or the OPAC established from outside the library building (adapted from
[1]
EQUINOX )
3.14
session
successful request of a database or the OPAC [ISO 2789]
NOTE 1 A session is one cycle of user activities that typically starts when a user connects to a database or the OPAC
and ends with explicit (by leaving the database through log-out or exit) or implicit (timeout due to user inactivity)
termination of activities in the database. The average timeout period would be 30 minutes. If another time period is used
this should be reported.
NOTE 2 Sessions on the library web site are counted as virtual visits.
NOTE 3 Requests of a general entrance or gateway page should be excluded.
NOTE 4 If possible, requests by search engine should be excluded.
3.15
user training
training programme set up with a specified lesson plan, which aims at specific learning outcomes for the use
of library services [ISO 2789]
NOTE 1 User training can be offered as a tour of the library, as library tuition, or as a web based service for users.
NOTE 2 The duration of a lesson is irrelevant.
3.16
virtual visit
a user's request of the library web site from outside the library premises regardless of the number of pages or
elements viewed [ISO 2789]
3.17
web site
electronic service that has a unique domain on the Internet and consists of a collection of digital documents
[ISO 2789]
NOTE 1 The pages of a web site are usually interconnected by the use of hypertext links.
NOTE 2 Excludes the documents that fit the definitions of electronic collection and external Internet resources that may
be linked from the library web site.
3.18
workstation
computer that may stand alone or be networked, or a dumb terminal [ISO 2789]
4 Relationships with other indicators
4.1 The networked environment
The networked environment provides a different frame of reference for the provision of services: the
boundaries between internal and external provision are different. For example, consider document delivery. In
traditional library services this is represented by the interlending of physical documents between libraries. In
the electronic world, document delivery is accomplished using a network. The network may be provided by a
variety of external providers. Special equipment has to be installed in the libraries at both ends of document
delivery process and staff have to be trained to use it. The suppliers of documents may be publishers or
libraries; the end users may be directly in communication with the suppliers.
Again, the provision of equipment within the organisation of which the library forms a part, may be outside the
library's own control: however, the library’s performance may be judged on its service delivery even though it
does not control some of the infrastructure essential to the delivery of the service.
It is because the boundaries within which electronic services operate differ from those in which traditional
library services operate that comparisons between traditional and electronic library services are difficult, if not
impossible. Since the extent to which different libraries have adopted electronic library services will differ then
comparisons between libraries are increasingly difficult. The position is further complicated by the changes in
expectations, experience and behaviour of users and of publishers. New generations of users will increasingly,
at least in the developed world, be familiar with the use of computers and Internet applications generally and
will expect electronic services as a matter of course. There will, however be for some considerable time
significant numbers of older users who are less familiar with the concepts and practice of computer usage and
will need careful support if they are not to be disadvantaged.
On the other hand, publishers are operating in a variety of ways to deal with the possibilities of electronic
publishing. Long established practices in the supply of printed materials do not operate in the same way when
original works are often generated in electronic format. The economics of publication in electronic formats are
significantly different from those in printed formats and different charging models are being tried. The
preferred format varies for electronic publications between types of publication, and may also vary from year
to year; and the financial arrangements between library and publisher also vary between publications and
from year to year.
4 © ISO 2003 — All rights reserved
Furthermore, the ability to measure the usage of publications in electronic format differs radically from that
associated with print formats. It may be possible, although expensive, for a library to account for every use
made of a printed document within a library: it may well be impossible for the library to count every use made
of electronic documents. The publisher may be able count use made of the electronic format, while having no
idea of the use made of printed documents. Effective performance measurement thus depends on new
arrangements and collaboration between the copyright owners, publishers, agents, librarians and users.
A final point relates to ‘Per capita’ indicators. Some electronic library services (web-sites, OPACs, ‘free’
information services) can be accessed by anyone, and are not limited to the population to be served. In these
cases, usage per capita ratios would be artificially inflated, if the data were collected automatically. If a library
is interested in knowing what proportion of its population to be served is using its services remotely, those
data could be collected by a conventional survey technique.
4.2 Categories of performance indicators
In defining performance indicators for electronic library services it is natural to start by seeking analogies with
performance indicators for traditional library services. Reference to ISO 11620:1998 and its amendment
ISO 11620/Amd. 1 show a list of 34 indicators. These are grouped in a number of categories according to
Table 1.
Table 1
Service, activity or aspect measured Sub-category
User perception General
Public services General
Providing documents
Retrieving documents
Lending documents
Document delivery from external sources
Enquiry and reference services
Information searching
User Education
Facilities
Technical services Acquiring documents
Processing documents
Cataloguing
Promotion of services
User services Deployment of staff
In drawing up this technical report it has proved possible to use the same categories for the proposed
performance indicators with only minor modifications (see Annex A). The applicability of the indicators in
ISO 11620 is discussed in the following subclause.
4.3 The relevance of ISO 11620 to electronic library services
4.3.1 Indicators that apply equally to traditional and electronic library services
Some indicators are the same for both traditional and electronic library services. An obvious example is User
Satisfaction. Others need only very slight changes to existing definitions and descriptions of methods to make
it clear that they are relevant to both types of service: an example is Cost per Title Catalogued. Similarly, the
group of indicators relating to providing documents: here the definitions and descriptions would need to be
modified to include documents in electronic formats. There are other indicators that have analogues in the
networked environment (see 4.3.3), but more substantial revision would be needed to accommodate different
formats, and different bases for measurement. See Table 2.
Table 2
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
User perception: General User Satisfaction B.1.1.1
Public services: Providing Titles Availability B.2.2.1
documents
Required Titles Availability B.2.2.2
B.2.2.3
Percentage of Required Titles in the
Collection
Required Titles Extended Availability B.2.2.4
Public services: Enquiry and Correct Answer Fill Rate B.2.6.1
reference services
Title Catalogue Search Success Rate B.2.7.1
Public services: Information
searching
Subject Catalogue Search Success Rate B.2.7.2
Public services: Facilities Facilities Availability B.2.9.1
Facilities Use Rate B.2.9.2
Seat Occupancy Rate B.2.9.3
Automated Systems Availability B.2.9.4
Median Time of Document Acquisition B.3.1.1
Technical services: Acquiring
documents
Technical services: Cataloguing Cost per Title Catalogued B.3.3.1
User services; Deployment of staff User Services Staff per Capita B.5.1.1
User Services Staff as a Percentage of B.5.1.2
Total Staff
4.3.2 Indicators that apply only to traditional library services
The indicators in this group are so closely related to the physical format of documents that they have no
precise equivalent in the electronic world. See Table 3.
Table 3
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: Providing In-library Use per Capita B.2.2.5
documents
Document Use Rate B.2.2.6
Proportion of Stock not Used B.2.2.7
Shelving Accuracy B.2.2.8
Public services: Retrieving Median Time of Document Retrieval from B.2.3.1
documents Closed Stacks
Median Time of Document Retrieval from B.2.3.2
Open Access Areas
Public services: Document delivery Speed of Interlibrary Lending B.2.5.1
from external sources
Technical services: Processing Median Time of Document Processing B.3.2.1
documents
6 © ISO 2003 — All rights reserved
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services
In this group, the indicators for electronic services are based on the same concepts as those for traditional
services, but the basis for measurement is different. For example, Library Visits per Capita would be replaced
by an indicator relating virtual visits to the population to be served. The various indicators for lending
documents have analogues now that digital lending is possible.
The fact that analogues can be found suggests that, in a future revision of ISO 11620, it may be necessary to
devise composite indicators that integrate traditional and networked services. For example, the indicator
Library Visits per Capita could be redefined to include both physical and virtual visits. Two composite
indicators are included in this Technical Report: Percentage of Virtual Visits to Total Visits, and Percentage of
Information Requests Submitted Electronically. Librarians may wish to create other composite measures and
indicators if that proves helpful in their own circumstances. See Table 4.
Table 4
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: General Percentage of Target Population Reached B.2.1.1
Cost per User B.2.1.2
Public services: General Library Visits per Capita B.2.1.3
Cost per Library Visit B.2.1.4
Public services: Lending Collection Turnover B.2.4.1
documents
Loans per Capita B.2.4.2
Documents on Loan per Capita B.2.4.3
Cost per Loan B.2.4.4
Loans per Employee B.2.4.5
Proportion of Stock on Loan B.2.4.6
5 Use of performance indicators
5.1 Purposes
The performance indicators described in this Technical Report are used as tools to compare the effectiveness,
efficiency and quality of the library's services and products to the library's mission and goals. They can be
used for evaluation purposes in the following areas:
comparing a single library's performance over years
support for management decisions, e. g. reallocating resources, introducing new services, reducing or
deleting existing services
demonstrating the library's performance and its costs to the funders, the population, and the public
comparing performance between libraries of similar structure and mission indicators should be able to
suggest
whether the library's performance or the use of its services has changed over years
how far performance or use in one library differs from that in other libraries.
5.2 Selection of performance indicators
The indicators included in this Technical Report have been developed and tested by libraries in several
projects worldwide. They were chosen for this report as most helpful for libraries in general at the present time,
and as complying with the criteria for performance indicators named in ISO 11620.
Not all of these indicators might be useful for every type of library or every single library. Libraries will have to
decide what indicators are most appropriate for evaluating their special services and products. The decision
should be taken in consideration of the library's mission, goals, and objectives, and the population it is set up
to serve. There should be an agreement with the institution and relevant authorities, as well as with users, as
to what indicators would be most informative for evaluation of the library's electronic services.
Collecting data for performance indicators can be time-consuming and expensive. The decision to use certain
indicators may therefore also depend on their practicality, or possibilities of automated data collection, or on
the urgent need for the evaluation of certain services. This might be the case when an authority requires data
for a particular service, or when it is apparent that a service does not operate satisfactorily. It should be
ensured that the data of indicators will be used subsequently for operational management and for the planning
process.
The indicators named in this Technical Report form a limited set that is applicable to the main electronic library
services and that has been already tested and applied by libraries. They might not prove sufficient for all sorts
of special services, and libraries will probably develop and test more and specialised indicators for their
special needs.
5.3 Limitations
5.3.1 Measuring the impact of electronic library services
The indicators in this Technical Report evaluate the library's performance by quantitative statements about the
supply, use, costs, or market penetration of electronic library services. Libraries are most concerned about the
impact of their services, the question whether users benefit by using the services. Such qualitative data should
be collected in addition to the quantitative data named in this Technical Report. They can be collected by
using methods like user satisfaction surveys, focus groups, or interviews.
The method chosen for collecting qualitative data will depend very much on local circumstances, e. g. the
library's target groups. The general method for assessing user satisfaction is described in ISO 11620.
5.3.2 Degree of accuracy
In order to establish scores for the indicators in this Technical Report, it may be necessary to collect data from
different sources (suppliers, the library's own servers, consortial servers etc.). Thus, data may be varying or
incomplete. Such inconsistencies should be stated explicitly when libraries publish or compare scores.
Other reasons for inaccuracy might be:
search engines can affect the number of sessions
several users, one after the other, might use an electronic service in a way that different sessions could
not be identified
it might be difficult to differentiate use generated by the library's population from other use.
Libraries should define exactly what they regard as their own electronic services, especially as to their
electronic collection (see: definitions). This Technical Report tries to give help in defining the contents of data,
e. g. as to staff employed in developing and maintaining IT services (see: indicator B.2.2.1).
Comment: Such possible sources of inaccuracy should not prevent data collection, but care should be taken
with the interpretation of results.
8 © ISO 2003 — All rights reserved
5.3.3 Comparability of performance indicator data
Libraries will use performance indicators primarily for evaluating their own performance, and for comparing
trends and developments over years. A secondary purpose is to compare results with those of other libraries.
This Technical Report, by describing the indicators and methods, contributes to uniformity of evaluation and
thus to make comparison possible.
In comparing results of indicators, libraries should consider:
the mission, goals, and objectives of each library
the structure of the population to be served
the different services and products of each library
the general conditions under which a library is set up to work.
5.3.4 Indicators of temporary importance
Some of the performance indicators named in this Technical Report have an informative value restricted to a
certain time and goal. They are valid as indicators if the library sees one of its main goals in offering its
services - as far as possible and reasonable - in electronic form.
Such indicators (e.g. B.1.2.1, B.1.4.1, B.2.2.1) measure the amount of resources allocated to electronic
services in comparison to traditional services. The indicators show the library's engagement in electronic
services and the development over years in the direction of a digital library. They will be valid until a
satisfactory state has been reached.
Other performance indicators — indicators of use, costs, or market penetration of electronic services — will
have long-term relevance for libraries.
Annex A
List of performance indicators for electronic library services
Table A.1 lists electronic library services and related activities grouped according to the classification used in
ISO 11620:1998 [1]. Reference is given to the descriptions provided in annex B.
NOTE There are a number of other indicators that have been tested in libraries that are not described. The basic
data elements used for the indicators described here may be combined in various ways to calculate additional indicators of
interest to individual libraries.
Table A.1
Service, activity or Description in
Performance indicator
aspect measured Annex B
Public services B.1
General B.1.1
Percentage of Population Reached by Electronic B.1.1.1
Services
Providing electronic library B.1.2
services
Percentage of Expenditure on Information Provision B.1.2.1
Spent on the Electronic Collection
Retrieving documents B.1.3
Number of Documents Downloaded Per Session B.1.3.1
Cost Per Database Session B.1.3.2
Cost Per Document Downloaded B.1.3.3
Percentage of Rejected Sessions B.1.3.4
Percentage of Remote OPAC Sessions B.1.3.5
Virtual Visits as a Percentage of Total Visits B.1.3.6
Enquiry and reference B.1.4
services
Percentage of Information Requests Submitted B.1.4.1
Electronically
User education B.1.5
Number of User Attendances at Electronic Service B.1.5.1
Training Lessons Per Capita
Facilities B.1.6
Workstation Hours Available Per Capita B.1.6.1
Population Per Public Access Workstation B.1.6.2
Workstation Use Rate B.1.6.3
Availability and use of B.2
human resources
10 © ISO 2003 — All rights reserved
Table A.1 (continued)
Service, activity or Description in
Performance indicator
aspect measured Annex B
Staff training B.2.1
Number of Attendances at Formal IT and Related B.2.1.1
Training Lessons Per Staff Member
Deployment of staff B.2.2
Percentage of Library Staff Providing and Developing B.2.2.1
Electronic Services
Annex B
Descriptions of performance indicators
B.1 Public services
B.1.1 General
B.1.1.1 Percentage of Population Reached by Electronic Services
B.1.1.1.1 Objective
To establish the success of the library in reaching its population to be served.
B.1.1.1.2 Scope
All libraries with a defined population to be served. Libraries may make separate calculations for users of
OPACs, subscription databases, electronic journals etc.
B.1.1.1.3 Definition of the indicator
The percentage of the population to be served who have used any of the electronic services provided by the
library during a specified time period.
B.1.1.1.4 Method
Draw a random sample from the population to be served and ask each person in the sample if they have used
the electronic service during a specified time period. The sample chosen should be representative of the
complete population to be served. Questions on the use of electronic services can usefully be included within
general surveys of library use.
The Percentage of the Population Reached by Electronic Services is
A
×100%
B
where
A is the number of persons in the sample who have used electronic library services during the specified
time period
B is the total number in the sample.
Round off to the nearest integer.
NOTE Alternative sampling and survey methods may also be used. Where a library wishes to make comparisons
over time, it is important that the sampling and survey methods used should be consistent between successive time
periods.
12 © ISO 2003 — All rights reserved
B.1.1.1.5 Interpretation and factors affecting the indicator
The indicator is an integer in the range of 0 to 100. A high score is generally considered better than a low
score. A score that increases from year to year is generally regarded as good.
The score may be affected by several factors, some outside the control of the library service. Examples are:
demographic composition of the population to be served; level of education; general social and economic
conditions; provision of networked access terminals in the library or in the institution; level of network access
available to the population to be served in their homes. It will also be affected by fees charged for services, or
by the rules for access to electronic services.
B.1.1.1.6 Source (see Bibliography)
[1] Adapted from Equinox PI 1
B.1.1.1.7 Related indicators
Percentage of Target Population Reached [ISO 11620:1998, B.2.1.1]; Number of Remote Sessions on
Electronic Services per Capita
B.1.2 Providing electronic services
B.1.2.1 Percentage of Expenditure on Information Provision Spent on the Electronic Collection
B.1.2.1.1 Objective
To assess the extent to which the library is committed to building an electronic collection.
B.1.2.1.2 Scope
All libraries.
May be used for specified parts of a library collection (e. g. journals, subject areas) or individual branches of a
library. Within each of these categories the resulting indicators may be compared to see whether the
percentage differs significantly.
Comparisons between libraries may be possible if differences in subjects, collection policies, and socio-
economic factors in the population are taken into consideration.
B.1.2.1.3 Definition of the indicator
The percentage of the library's total expenditure on information provision spent on the electronic collection.
The electronic collection includes digital documents, electronic serials, and databases.
Expenditure on the electronic collection, for the purpose of this indicator, includes the library's acquisition,
subscription and licensing costs. As an alternative, a library might decide to include pay-per-view and
electronic document delivery costs with the costs of collection building. This should be stated clearly when
publishing or comparing scores.
Total acquisitions expenditure would exclude expenditure on binding.
Expenditure on infrastructure, such as hardware, software or networking, and on digitisation of documents
should not be included.
Value added taxes, sales and service taxes or other local taxes are included. Their inclusion may affect
international comparisons.
B.1.2.1.4 Method
For a given budget period, determine the library's acquisition, subscription and licensing expenditure (and
including, if desired, pay-per-view and electronic document delivery charges) for the electronic collection. If
the library joins in consortia or other over-all contracts, only the library's own share in the contractual
expenses should be counted. Where electronic versions of documents are acquired in a package with print
versions, only the surplus payment for the electronic version should be counted.
The Percentage of Expenditure on Information Provision Spent on the Electronic Collection is
A
×100%
B
where
A is expenditure on the electronic collection
B is total expenditure on information provision.
Round off to the nearest integer.
B.1.2.1.5 Interpretation and factors affecting the indicator
The indicator is an integer between 0 and 100.
Comparison over time shows the extent to which a library shifts its
...
RAPPORT ISO/TR
TECHNIQUE 20983
Première édition
2003-11-01
Information et documentation —
Indicateurs de performance pour les
services électroniques des bibliothèques
Information and documentation — Performance indicators for electronic
library services
Numéro de référence
©
ISO 2003
PDF – Exonération de responsabilité
Le présent fichier PDF peut contenir des polices de caractères intégrées. Conformément aux conditions de licence d'Adobe, ce fichier
peut être imprimé ou visualisé, mais ne doit pas être modifié à moins que l'ordinateur employé à cet effet ne bénéficie d'une licence
autorisant l'utilisation de ces polices et que celles-ci y soient installées. Lors du téléchargement de ce fichier, les parties concernées
acceptent de fait la responsabilité de ne pas enfreindre les conditions de licence d'Adobe. Le Secrétariat central de l'ISO décline toute
responsabilité en la matière.
Adobe est une marque déposée d'Adobe Systems Incorporated.
Les détails relatifs aux produits logiciels utilisés pour la création du présent fichier PDF sont disponibles dans la rubrique General Info
du fichier; les paramètres de création PDF ont été optimisés pour l'impression. Toutes les mesures ont été prises pour garantir
l'exploitation de ce fichier par les comités membres de l'ISO. Dans le cas peu probable où surviendrait un problème d'utilisation,
veuillez en informer le Secrétariat central à l'adresse donnée ci-dessous.
© ISO 2003
Droits de reproduction réservés. Sauf prescription différente, aucune partie de cette publication ne peut être reproduite ni utilisée sous
quelque forme que ce soit et par aucun procédé, électronique ou mécanique, y compris la photocopie et les microfilms, sans l'accord écrit
de l'ISO à l'adresse ci-après ou du comité membre de l'ISO dans le pays du demandeur.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax. + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Publié en Suisse
ii © ISO 2003 — Tous droits réservés
Sommaire Page
Avant-propos. iv
Introduction . v
1 Domaine d'application. 1
2 Références normatives. 1
3 Termes et définitions . 1
4 Liens avec d'autres indicateurs. 5
4.1 Environnement en réseau . 5
4.2 Catégories d'indicateurs de performance . 6
4.3 Pertinence de l'ISO 11620 pour les services électroniques . 6
4.3.1 Indicateurs communs aux services traditionnels et électroniques. 6
4.3.2 Indicateurs qui ne s'appliquent qu'aux services traditionnels des bibliothèques. 7
4.3.3 Indicateurs s'appliquant aux services traditionnels des bibliothèques pour lesquels des
indicateurs analogues peuvent être trouvés pour les services électroniques . 8
5 Utilisation des indicateurs de performance . 8
5.1 Objectifs. 8
5.2 Choix d'indicateurs de performance. 9
5.3 Restrictions . 9
5.3.1 Mesure de l'impact des services électroniques des bibliothèques. 9
5.3.2 Degré de précision. 9
5.3.3 Comparabilité des données des indicateurs de performance . 10
5.3.4 Les indicateurs à usage provisoire. 10
Annexe A Liste des indicateurs de performance pour les services électroniques des
bibliothèques. 11
Annexe B Description des indicateurs de performance . 13
Bibliographie . 33
Index alphabétique. 34
Avant-propos
L'ISO (Organisation internationale de normalisation) est une fédération mondiale d'organismes nationaux de
normalisation (comités membres de l'ISO). L'élaboration des Normes internationales est en général confiée
aux comités techniques de l'ISO. Chaque comité membre intéressé par une étude a le droit de faire partie du
comité technique créé à cet effet. Les organisations internationales, gouvernementales et non
gouvernementales, en liaison avec l'ISO participent également aux travaux. L'ISO collabore étroitement avec
la Commission électrotechnique internationale (CEI) en ce qui concerne la normalisation électrotechnique.
Les Normes internationales sont rédigées conformément aux règles données dans les Directives ISO/CEI,
Partie 2.
La tâche principale des comités techniques est d'élaborer les Normes internationales. Les projets de Normes
internationales adoptés par les comités techniques sont soumis aux comités membres pour vote. Leur
publication comme Normes internationales requiert l'approbation de 75 % au moins des comités membres
votants.
Exceptionnellement, lorsqu'un comité technique a réuni des données de nature différente de celles qui sont
normalement publiées comme Normes internationales (ceci pouvant comprendre des informations sur l'état
de la technique par exemple), il peut décider, à la majorité simple de ses membres, de publier un Rapport
technique. Les Rapports techniques sont de nature purement informative et ne doivent pas nécessairement
être révisés avant que les données fournies ne soient plus jugées valables ou utiles.
L'attention est appelée sur le fait que certains des éléments du présent document peuvent faire l'objet de
droits de propriété intellectuelle ou de droits analogues. L'ISO ne saurait être tenue pour responsable de ne
pas avoir identifié de tels droits de propriété et averti de leur existence.
L'ISO/TR 20983 a été élaboré par le comité technique ISO/TC 46, Information et documentation, sous-comité
SC 8, Qualité — Statistiques et évaluation de la performance.
iv © ISO 2003 — Tous droits réservés
Introduction
Le présent Rapport technique a pour objet l'évaluation des services électroniques des bibliothèques.
La fourniture réussie de ces services dépend d'une collaboration étroite entre les éditeurs, les fournisseurs
d'accès et les bibliothécaires. Leurs besoins de collecte de données sur les divers aspects de la performance
sont différents, et les destinataires de ces données n'ont pas forcément la maîtrise de leur collecte. Par
exemple, l'éditeur peut être mieux placé pour collecter les données sur l'utilisation d'un service, tandis que
c'est sans doute le bibliothécaire qui sera le plus intéressé par l'emploi de ces données pour développer et
évaluer ce service.
L'objectif principal du présent Rapport technique est de diffuser les connaissances sur l'évaluation pratique
des services électroniques des bibliothèques. Bien que ces services aient été développés depuis plus de
trente ans, l'environnement technique a changé de façon significative au cours de la dernière décennie.
L'ISO 11620:1998, Information et documentation — Indicateurs de performance des bibliothèques, a reconnu
la nécessité de disposer d'indicateurs de performance pour les services électroniques. Il était en même temps
reconnu qu'au moment de la publication de cette Norme internationale, on ne disposait pas de tels indicateurs
qui aient été testés et publiés, ou qui aient été utilisés largement. Bien que l'on ait beaucoup progressé ces
dernières années dans la définition, dans les tests et la publication d'indicateurs, il est encore trop tôt pour
déterminer quels sont ceux qui s'avéreront les plus généralement utiles, ou qui seront d'un usage de plus en
plus répandu. Le présent Rapport technique fournit une terminologie normalisée ainsi que des définitions
concises et des descriptions d'une sélection d'indicateurs de performance dans une présentation identique à
celle de l'ISO 11620. Les publications référencées dans la bibliographie [2, 3, 5] fournissent une information
plus détaillée sur la collecte et l'analyse des données.
Seuls quelques indicateurs présentés ici sont des indicateurs de qualité du service: la réflexion sur ce sujet
n'est pas assez avancée pour être intégrée ici. Les indicateurs qui ont été retenus ont été testés dans des
bibliothèques, ou bien sont de simples adaptations d'indicateurs semblables. Dans tous les cas, ils ont été
retenus selon les critères établis dans l'ISO 11620. A savoir que chaque indicateur dispose d'un contenu
informatif, est fiable, valide, approprié, applicable, et peut dans certaines circonstances être employé pour des
comparaisons.
Les indicateurs figurant dans le présent Rapport constituent une sélection représentative de ceux qui ont été
testés. Une sélection plus élaborée aurait demandé trop de temps pour leur préparation et leur publication.
Les avancées dans la mesure et l'évaluation des services électroniques des bibliothèques seront suivies par
un groupe de travail, qui proposera des indicateurs supplémentaires, ainsi que des modifications ou des
adaptations des indicateurs présentés ici au fur et à mesure qu'ils seront testés et validés. Il convient de
considérer le présent Rapport technique comme un travail préparatoire en vue de la rédaction d'une Norme
internationale. Il est entendu que, le moment venu, son contenu sera intégré dans une révision ultérieure de
l'ISO 11620.
RAPPORT TECHNIQUE ISO/TR 20983:2003(F)
Information et documentation — Indicateurs de performance
pour les services électroniques des bibliothèques
1 Domaine d'application
Le présent Rapport technique s'applique à tous les types de bibliothèques dans tous les pays. Les restrictions
apportées à l'application d'indicateurs de performance particuliers sont énumérées dans la rubrique Domaine
d'application qui figure dans la description de chaque indicateur (voir Annexe B).
Les indicateurs peuvent être utilisés pour des comparaisons au cours du temps pour les services fournis par
la même bibliothèque. On peut aussi, sous réserve de prudence, comparer des bibliothèques entre elles, à
condition de prendre en compte leurs différences d'organisation, de faire preuve d'une bonne compréhension
des indicateurs utilisés, et d'interpréter les données avec précaution.
Le présent Rapport technique ne comporte pas d'indicateur pour l'évaluation de l'impact des bibliothèques sur
les individus ou sur la société.
2 Références normatives
Les documents de référence suivants sont indispensables pour l'application du présent document. Pour les
références datées, seule l'édition citée s'applique. Pour les références non datées, la dernière édition du
document de référence s'applique (y compris les éventuels amendements).
ISO 2789:2003, Information et documentation — Statistiques internationales de bibliothèques
ISO 5127:2001, Information et documentation — Vocabulaire
ISO 11620:1998, Information et documentation — Indicateurs de performance des bibliothèques
ISO 11620:1998/Amd. 1:2003, Information et documentation — Indicateurs de performance des bibliothèques,
Amendement 1: Indicateurs complémentaires de performance des bibliothèques
3 Termes et définitions
Pour les besoins du présent document, les termes et définitions suivants s'appliquent.
3.1
base de données
ensemble de données ou enregistrements informatiques (données factuelles ou bibliographiques, textes),
dotés d'une interface commune et d'un logiciel pour récupérer ou exploiter les données [ISO 2789]
(Terme anglais: database)
NOTE 1 Les données ou les références sont généralement classées avec un objectif précis et concernent un sujet
défini. Une base de données peut être publiée sur cédérom, disquette ou sur d'autres supports, ou en tant que fichier
informatique accessible en ligne ou par Internet.
NOTE 2 Les bases de données soumises à licence sont comptabilisées à l'unité, même si l'accès à plusieurs bases de
données soumises à licences est proposé par la même interface.
3.2
document numérique
unité documentaire au contenu identifié qui a été numérisée par la bibliothèque ou acquise sous forme
numérique comme élément de la collection de la bibliothèque [ISO 2789]
(Terme anglais: digital document)
NOTE 1 Cela comprend les livres électroniques, les brevets électroniques, les documents audiovisuels en réseau et
les autres documents numériques, par exemple les rapports, les documents cartographiques et musicaux, les pré-
publications, etc. Les bases de données et les périodiques électroniques sont exclus.
NOTE 2 Les éléments faisant partie des bases de données sont décrits en 3.1.
NOTE 3 Un document numérique peut être structuré en un ou plusieurs fichiers.
3.3
document
information enregistrée ou objet matériel qui peut être traité comme une unité dans une chaîne documentaire
[ISO 5127]
(Terme anglais: document)
NOTE Les documents peuvent varier dans leur forme et leurs caractéristiques.
3.4
document téléchargé
texte intégral ou extrait d'un document de la collection électronique qui est fourni à un usager [ISO 2789]
(Terme anglais: document downloaded)
3.5
collection électronique
toutes les ressources de la bibliothèque qui sont sous forme électronique [ISO 2789]
(Terme anglais: electronic collection)
NOTE La collection électronique comprend les bases de données, les périodiques électroniques et les documents
numériques. Les ressources Internet gratuites qui ont été cataloguées par la bibliothèque et figurent dans le catalogue
interrogeable en ligne par le public ou dans une base de données doivent être comptabilisées séparément.
3.6
ressource électronique
document sous forme électronique qui constitue un tout séparé avec un titre distinct, qu'il soit publié en une
ou plusieurs unités
(Terme anglais: electronic resource)
NOTE Cela comprend les livres, publications en série, et cartes électroniques, à l'exclusion des bases de données.
3.7
service électronique
service en ligne fourni par des serveurs locaux ou accessible à distance par les réseaux [ISO 2789]
(Terme anglais: electronic service)
NOTE Les services électroniques de bibliothèque comprennent le catalogue interrogeable en ligne par le public, le
site web de la bibliothèque, la collection électronique, la fourniture de documents électroniques par l'intermédiaire de la
bibliothèque, un service de référence électronique, la formation des usagers aux services électroniques et l'accès à
Internet proposé par la bibliothèque.
2 © ISO 2003 — Tous droits réservés
3.8
enregistrement
unité d'information bibliographique ou factuelle ou objet multimédia inclus dans une base de données
interrogeable
(Terme anglais: entry)
3.9
demande d'information
opération de médiation auprès du public par le personnel de la bibliothèque à partir d'une ou de plusieurs
sources d'informations (tels que les documents imprimés et non imprimés, les bases de données
électroniques, les catalogues de la bibliothèque et d'autres institutions) [ISO 2789]
(Terme anglais: information request)
NOTE 1 Concerne également des recommandations, interprétations et instructions pour l'utilisation de ces ressources.
NOTE 2 La demande peut être adressée de vive voix, par téléphone, par courrier, par télécopie ou par voie
électronique (messagerie électronique, site web de la bibliothèque, message instantané ou tout autre moyen de
communication en réseau).
NOTE 3 Il est essentiel que les bibliothèques ne comptabilisent pas les demandes de renseignement, d'orientation et
les questions administratives telles que trouver un agent de la bibliothèque ou repérer des équipements, préciser les
horaires d'ouverture ou expliquer le fonctionnement des équipements (imprimantes ou ordinateurs par exemple).
NOTE 4 Les questions relatives à la simple localisation de documents du fonds déjà identifiés dans le catalogue ne
sont pas comptées.
3.10
collection
tous les documents mis à disposition des usagers par une bibliothèque [ISO 2789]
(Terme anglais: library collection)
NOTE 1 Cela comprend les documents détenus localement et les ressources accessibles à distance pour lesquelles
des droits permanents ou temporaires ont été acquis.
NOTE 2 Les droits d'accès peuvent être acquis par la bibliothèque elle-même, par un consortium et/ou par un
financement extérieur.
NOTE 3 Les acquisitions doivent être considérées comme le résultat d'une sélection de documents, de droits d'accès
négociés, et être incluses dans le catalogue interrogeable en ligne par le public ou dans d'autres bases de données de la
bibliothèque. Le prêt entre bibliothèques et la fourniture de documents à distance sont exclus.
NOTE 4 Cela n'inclut pas les liens aux ressources Internet pour lesquels la bibliothèque n'a pas négocié de droits
d'accès par des autorisations légales (par exemple, droit lié au dépôt légal), des licences ou des accords soit contractuels
soit de coopération. Il convient de comptabiliser séparément les liens avec des ressources Internet gratuites qui ont été
cataloguées par la bibliothèque et qui sont présentées sur le catalogue interrogeable en ligne par le public ou dans une
base de données.
3.11
population à desservir
ensemble des individus auxquels la bibliothèque est destinée à fournir ses services et ses documents
[ISO 11620]
(Terme anglais: population to be served)
NOTE En règle générale, pour les bibliothèques publiques, il s'agit de la population relevant de la même collectivité
territoriale; pour les bibliothèques d'un établissement d'enseignement supérieur, cela comprend l'ensemble des
personnels enseignants et administratifs et des étudiants.
3.12
session rejetée
tentative infructueuse de connexion à une base de donnée ou à un catalogue interrogeable en ligne par le
public lorsque le nombre de requêtes dépasse le nombre autorisé d'utilisateurs simultanés [ISO 2789]
[Terme anglais: rejected session (turnaway)]
NOTE Les rejets dus à des mots de passe non conformes sont exclus.
3.13
session à distance
[1]
interrogation d'une base de données depuis l'extérieur des locaux de la bibliothèque (adapté d'EQUINOX )
(Terme anglais: remote session)
3.14
session
connexion effective à une base de données ou à un catalogue interrogeable en ligne par le public [ISO 2789]
(Terme anglais: session)
NOTE 1 Une session est un cycle qui débute lorsque l'usager se connecte à une base de données ou à un catalogue
interrogeable en ligne, et qui se termine de façon explicite (par déconnexion ou fonction quitter) ou implicite (mise en veille
après une période déterminée de non utilisation). Le délai moyen de déconnexion spontanée est de 30 minutes. Si un
autre délai est utilisé, il convient de le mentionner.
NOTE 2 Les sessions sur le site web de la bibliothèque sont comptabilisées comme visites virtuelles.
NOTE 3 Les connexions à une page d'accueil sont exclues.
NOTE 4 Si possible, il convient d'exclure les recherches effectuées par les moteurs de recherche.
3.15
formation des usagers
plan de formation mis en place selon un programme spécifique et qui a pour but de développer l'utilisation des
services de la bibliothèque [ISO 2789]
(Terme anglais: user training)
NOTE 1 La formation des usagers peut être proposée sous la forme d'une visite de la bibliothèque, de présentation de
la bibliothèque ou comme service sur le web à destination des usagers.
NOTE 2 La durée des formations ne doit pas être prise en compte.
3.16
visite virtuelle
recherche faite par un usager sur le site web de la bibliothèque depuis l'extérieur de la bibliothèque quel que
soit le nombre de pages ou d'informations consultées [ISO 2789]
(Terme anglais: virtual visit)
3.17
site web
service électronique ayant un nom de domaine unique sur Internet et se composant d'une ensemble de
documents numériques [ISO 2789]
(Terme anglais: web site)
NOTE 1 Les pages du site web sont généralement reliées grâce à des liens hypertextes.
4 © ISO 2003 — Tous droits réservés
NOTE 2 Sont exclus les documents qui répondent à la définition de collection électronique et les ressources externes
d'Internet pour lesquelles un lien a été établi à partir du site web de la bibliothèque.
3.18
poste de travail
ordinateur qui fonctionne seul ou en réseau, ou en terminal passif [ISO 2789]
(Terme anglais: workstation)
4 Liens avec d'autres indicateurs
4.1 Environnement en réseau
Le fonctionnement en réseau définit un cadre de référence différent pour la fourniture de services: les
frontières varient selon l'origine de l'approvisionnement, fourniture interne ou externe. Considérons par
exemple la fourniture de documents. Dans les services traditionnels de la bibliothèque, le prêt de documents
physiques entre bibliothèques en est une des formes. Dans le monde électronique, on procure le document
grâce à un réseau. L'accès à ce dernier peut être fourni par divers prestataires extérieurs. Un équipement
spécifique doit être installé dans les bibliothèques aux deux bouts de la chaîne de fourniture du document, et
le personnel doit être formé à son utilisation. Les fournisseurs de documents peuvent être des éditeurs ou des
bibliothèques; les utilisateurs finals peuvent entrer en liaison directe avec les fournisseurs.
Encore une fois, la mise en œuvre d'un équipement à l'intérieur de l'établissement dont la bibliothèque fait
partie peut être réalisée en dehors du contrôle de la bibliothèque: pourtant la performance de celle-ci sera
évaluée sur sa capacité à fournir un service, même si elle ne contrôle pas certains aspects essentiels de
l'infrastructure.
Du fait que les frontières diffèrent entre les services électroniques et les services traditionnels des
bibliothèques, les comparaisons entre ces services sont difficiles, sinon impossibles. Puisque le degré
d'intégration des services électroniques varie selon les bibliothèques, les comparaisons sont de plus en plus
ardues. La situation est rendue encore plus complexe par les évolutions dans les attentes, dans l'expérience
et le comportement des usagers et des éditeurs. Les nouvelles générations d'usagers vont être, au moins
dans les pays développés, de plus en plus familiarisées avec l'informatique et l'utilisation d'Internet en
général; ils considéreront les services électroniques comme une évidence. Toutefois, pendant une période
encore assez longue, de nombreux usagers plus âgés resteront moins familiers avec les concepts et la
pratique de l'informatique, et auront besoin d'un soutien attentif si l'on ne veut pas qu'ils soient désavantagés.
D'un autre côté, les éditeurs travaillent dans de nombreuses directions pour prendre en compte les
possibilités de l'édition électronique. Les pratiques de fourniture de documents imprimés, établies de longue
date, n'ont plus le même caractère opérationnel maintenant que les originaux sont souvent produits
directement en format électronique. L'économie de l'édition électronique diffère de manière significative de
celle de l'imprimé, et différents modes de tarification sont testés. Le format privilégié pour les publications
électroniques varie selon le type de publication mais peut aussi évoluer d'une année sur l'autre; et les
modalités financières des contrats entre la bibliothèque et l'éditeur peuvent aussi changer selon la publication
et d'une année sur l'autre.
De plus, la capacité à mesurer l'utilisation des publications diffère radicalement entre documents
électroniques et imprimés. S'il est possible pour une bibliothèque, bien que cela soit onéreux, de décompter
chaque utilisation faite d'un document imprimé dans ses locaux, il pourra bien lui être impossible de compter
chaque utilisation d'un document électronique. D'un autre côté, l'éditeur pourra être capable de compter
l'utilisation au format électronique, tout en n'ayant aucune idée de l'utilisation faite des documents imprimés.
Une mesure effective de la performance repose donc sur de nouveaux accords et sur la collaboration entre
les titulaires de droits d'auteurs, les éditeurs, les agents, les bibliothécaires et les usagers.
Un dernier point concerne les indicateurs «par personne de la population à desservir». Certains services
électroniques des bibliothèques (sites web, CIEL, services d'information libres) peuvent être utilisés par
quiconque, et ne sont pas réservés à la population à desservir. Dans ce cas, les ratios d'utilisation «par
personne de la population à desservir» seraient artificiellement élevés si les données étaient collectées
automatiquement. Si une bibliothèque souhaite connaître la proportion de sa population à desservir qui utilise
ses services à distance, une méthode d'enquête classique sera plus appropriée afin de collecter de telles
données.
4.2 Catégories d'indicateurs de performance
En définissant des indicateurs de performance pour les services électroniques des bibliothèques, il est naturel
de commencer par rechercher les analogies avec les indicateurs de performance pour les services
traditionnels. L'ISO 11620:1998 et son amendement ISO 11620/DAM 1 déterminent une liste de 34
indicateurs. Ils sont regroupés en un certain nombre de catégories, selon le Tableau 1.
Tableau 1
Service, activité ou aspect mesuré Sous-catégorie
Opinion des usagers Généralités
Services offerts au public Généralités
Fourniture de documents
Recherche de documents
Prêt de documents
Fourniture de documents provenant de sources externes
Service de renseignement et de référence
Recherche d'information
Formation des usagers
Installations
Services techniques Acquisition de documents
Traitement des documents
Catalogage
Promotion des services
Services aux usagers Disponibilité et emploi des ressources humaines
Durant l'élaboration du présent Rapport technique, il a été démontré qu'il était possible d'utiliser les mêmes
catégories pour les indicateurs de performance proposés, sous réserve de modifications mineures (voir
Annexe A). L'applicabilité des indicateurs de l'ISO 11620 est discutée dans le paragraphe suivant.
4.3 Pertinence de l'ISO 11620 pour les services électroniques
4.3.1 Indicateurs communs aux services traditionnels et électroniques
Certains indicateurs sont les mêmes à la fois pour les services traditionnels et les services électroniques des
bibliothèques. L'indicateur Satisfaction des usagers en est un exemple évident. Pour d'autres, de très légers
changements dans la définition et la description des méthodes permettent à l'évidence de les rendre
pertinents pour les deux types de service: par exemple, le Coût par titre catalogué. De même pour le groupe
des indicateurs relatifs à la fourniture de documents: là les définitions et les descriptions devront être
modifiées pour inclure les documents électroniques. D'autres indicateurs ont des équivalents dans
l'environnement en réseau (voir 4.3.3), mais une révision plus substantielle serait nécessaire pour harmoniser
les différents formats et les différentes unités de mesure et méthodes d'estimation. Voir Tableau 2.
6 © ISO 2003 — Tous droits réservés
Tableau 2
Service, activité ou aspect Référence dans
Indicateur de performance
mesuré l'ISO 11620
Opinion des usagers: Satisfaction des usagers B.1.1.1
généralités
Services offerts au public: Disponibilité des documents B.2.2.1
fourniture de documents
Disponibilité des titres demandés B.2.2.2
Proportion des titres demandés figurant dans la collection B.2.2.3
Mise à disposition des titres demandés B.2.2.4
Services offerts au public: Taux de réponses correctes B.2.6.1
services de renseignement et
de référence
Services offerts au public: Taux de succès de la recherche par titre dans le catalogue B.2.7.1
recherche d'information
Taux de succès de la recherche par sujet dans le catalogue B.2.7.2
Services offerts au public: Disponibilité des installations B.2.9.1
installations
Taux d'utilisation des installations B.2.9.2
Taux d'occupation des places assises B.2.9.3
Disponibilité des systèmes informatisés B.2.9.4
Services internes: acquisition Délai d'acquisition médian des documents B.2.1.1
de documents
Services internes: catalogage Coût par titre catalogué B.3.3.1
Services aux usagers Personnel affecté au service public par personne de la population B.5.1.1
à desservir
Pourcentage du personnel affecté au service public par rapport à B.5.1.2
l'effectif total
4.3.2 Indicateurs qui ne s'appliquent qu'aux services traditionnels des bibliothèques
Les indicateurs de ce groupe sont si étroitement liés à la forme physique des documents qu'ils n'ont pas
d'équivalent précis dans le monde électronique. Voir Tableau 3.
Tableau 3
Service, activité ou Référence dans
Indicateur de performance
aspect mesuré l'ISO 11620
Services offerts au Consultation sur place par personne de la population à desservir B.2.2.5
public: fourniture de
documents
Taux d'utilisation des documents B.2.2.6
Pourcentage du fonds non utilisé B.2.2.7
Exactitude du rangement B.2.2.8
Services offerts au Délai de recherche médian des documents en magasin B.2.3.1
public: recherche de
documents
Délai de recherche médian des documents en accès direct B.2.3.2
Services offerts au Délai du prêt entre bibliothèques B.2.5.1
public: fourniture de
documents provenant
de sources externes
Services internes: Délai de traitement médian des documents B.3.2.1
traitement des
documents
4.3.3 Indicateurs s'appliquant aux services traditionnels des bibliothèques pour lesquels des
indicateurs analogues peuvent être trouvés pour les services électroniques
Dans ce groupe, les indicateurs pour les services électroniques sont fondés sur les mêmes concepts que pour
les services traditionnels, mais l'objet de la mesure est différent. Par exemple, l'indicateur sur les Entrées à la
bibliothèque par personne de la population à desservir sera remplacé par un indicateur reliant le nombre de
visites virtuelles à la population à desservir. Les différents indicateurs relatifs au prêt de documents ont des
équivalents maintenant que le prêt sous forme numérique est possible.
Le fait que des équivalents puissent être trouvés suggère qu'il sera peut-être nécessaire, lors d'une prochaine
révision de l'ISO 11620, de définir des indicateurs composites qui intègrent services traditionnels et en réseau.
Par exemple l'indicateur Entrées à la bibliothèque par personne de la population à desservir pourrait être
redéfini pour inclure à la fois les visites réelles et virtuelles. Le présent Rapport technique comporte deux
indicateurs composites: Pourcentage de visites virtuelles par rapport au total des entrées, et Pourcentage de
demandes d'information transmises sous forme électronique. Les bibliothécaires pourront créer d'autres
indicateurs et éléments de mesures composites si cela s'avère utile dans leur contexte d'activité. Voir
Tableau 4.
Tableau 4
Service, activité ou Référence dans
Indicateur de performance
aspect mesuré l'ISO 11620
Services offerts au public: Pourcentage de fréquentation de la population-cible B.2.1.1
généralités
Coût par usager B.2.1.2
Entrées à la bibliothèque par personne de la population à desservir B.2.1.3
Coût par entrée à la bibliothèque B.2.1.4
Services offerts au public: Rotation des collections B.2.4.1
prêt de documents
Prêts par personne de la population à desservir B.2.4.2
Documents en prêt par personne de la population à desservir B.2.4.3
Coût par prêt B.2.4.4
Prêts par agent B.2.4.5
Pourcentage du fonds en prêt B.2.4.6
5 Utilisation des indicateurs de performance
5.1 Objectifs
Les indicateurs de performance décrits dans le présent Rapport technique sont utilisés en tant qu'outils de
comparaison de l'efficience, de l'efficacité et de la qualité des produits et services offerts par la bibliothèque
par rapport à ses missions et à ses objectifs. On peut les utiliser à des fins d'évaluation dans les secteurs
suivants.
Comparaison de la performance d'une seule bibliothèque sur plusieurs années.
Aide à la décision en matière de gestion, par exemple réaffectation de ressources, introduction de
nouveaux services, réduction ou fermeture de services existants.
Diffusion de l'information sur les performances et les coûts de la bibliothèque à ses baileurs de fonds, à la
population et au public.
Comparaison des performances entre bibliothèques à la structure et aux missions comparables.
8 © ISO 2003 — Tous droits réservés
Les indicateurs devraient pouvoir montrer
si la performance de la bibliothèque ou l'utilisation de ses services a changé au fil des années,
quel est l'écart en termes de performance ou d'usage d'une bibliothèque à l'autre.
5.2 Choix d'indicateurs de performance
Les indicateurs figurant dans le présent Rapport technique ont été mis au point et testés par des bibliothèques
dans plusieurs projets au niveau mondial. Ils ont été sélectionnés pour le présent Rapport comme étant les
plus utiles pour les bibliothèques en général à l'heure actuelle, et comme se conformant aux critères des
indicateurs de performance mentionnés dans l'ISO 11620.
Il se peut que tous ces indicateurs ne soient pas utiles pour chaque type de bibliothèque, ou pour chaque
bibliothèque prise isolément. Il appartiendra aux bibliothèques de décider quels sont les indicateurs les plus
appropriés pour évaluer leurs offres et services spécifiques. Il convient de prendre cette décision en tenant
compte de la mission, des buts et objectifs de la bibliothèque, et de la population qu'elle doit desservir. Il
convient de trouver un accord entre la bibliothèque et les autorités concernées, ainsi qu'avec ses usagers,
pour déterminer les indicateurs qui donnent les informations les plus pertinentes pour évaluer les services
électroniques de la bibliothèque.
La collecte des données pour établir des indicateurs de performance peut être coûteuse en temps et en
ressources financières. C'est pourquoi il se peut que la décision d'utiliser certains indicateurs dépende de leur
caractère pratique, ou des possibilités de collecter des données automatiquement, ou du besoin urgent
d'évaluer certains services. Ce peut être le cas lorsque des décideurs ou des autorités compétentes en la
matière demandent des données sur un service spécifique, ou lorsqu'il s'avère qu'un service ne fonctionne
pas de manière satisfaisante. Il convient alors de s'assurer que les données des indicateurs seront bien
utilisées par la suite dans le cadre de la gestion opérationnelle et des opérations de programmation.
Les indicateurs mentionnés dans le présent Rapport technique constituent un ensemble restreint applicable
aux principaux services électroniques de bibliothèques, qui a déjà été testé et mis en application par des
bibliothèques. Cela peut ne pas s'avérer suffisant pour certaines catégories de services, et les bibliothèques
vont probablement développer et tester d'autres indicateurs spécialisés en fonction de leurs besoins
spécifiques.
5.3 Restrictions
5.3.1 Mesure de l'impact des services électroniques des bibliothèques
Les indicateurs du présent Rapport technique évaluent les performances de la bibliothèque par des relevés
quantitatifs concernant la fourniture, l'utilisation, les coûts ou la pénétration du marché des services
électroniques des bibliothèques. Les bibliothèques sont surtout concernées par l'impact de leurs services, et
en particulier par le bénéfice qu'en retirent les usagers. On s'efforcera de collecter des données qualitatives
de ce type en plus des données quantitatives mentionnées dans le présent Rapport technique. On pourra les
collecter en utilisant des méthodes telles que des enquêtes de satisfaction des usagers, des groupes témoins,
ou des entretiens.
La méthode choisie pour collecter des données qualitatives dépendra beaucoup des conditions locales,
comme par exemple les groupes cible de la bibliothèque. La méthode courante pour évaluer la satisfaction
des usagers est décrite dans l'ISO 11620.
5.3.2 Degré de précision
Afin de disposer de résultats pour les indicateurs dans le présent Rapport technique, il se peut qu'il soit
nécessaire de collecter des données à partir de sources différentes (fournisseurs, système informatique de la
bibliothèque, serveurs de consortiums etc.). Ainsi, il se peut que les données soient variables ou incomplètes.
Il convient de signaler de telles incohérences de manière explicite lorsque les bibliothèques publient ou
comparent des résultats.
Les autres sources d'erreur peuvent être
un nombre de sessions surestimé à cause des moteurs de recherche,
une connexion unique à un service électronique utilisée par plusieurs usagers successivement,
empêchant ainsi de distinguer les différentes sessions,
la difficulté de différencier l'utilisation faite par la population à desservir d'une autre utilisation.
Il convient que les bibliothèques définissent avec exactitude ce qu'elles considèrent comme leurs propres
services électroniques, spécialement en ce qui concerne leur collection électronique (voir: définitions). Le
présent Rapport technique vise à fournir une aide pour définir le contenu des données, comme par exemple le
personnel affecté à la fourniture et au développement des services électroniques (voir indicateur B.2.2.1).
Commentaire: De telles causes éventuelles d'inexactitude ne devraient pas empêcher la collecte des données,
mais une attention particulière sera portée à l'interprétation des résultats.
5.3.3 Comparabilité des données des indicateurs de performance
Les bibliothèques utiliseront les indicateurs de performance en premier lieu pour évaluer leur propre
performance, et pour comparer les tendances et les développements au fil des ans. Un objectif secondaire
porte sur la comparaison des résultats avec ceux des autres bibliothèques. Le présent Rapport technique, en
décrivant les indicateurs et les méthodes, contribue à uniformiser l'évaluation et à rendre ainsi les
comparaisons possibles.
En comparant les résultats des indicateurs, il convient que les bibliothèques prennent en compte
les mission, buts et objectifs de chaque bibliothèque,
la structure de la population à desservir,
les différents services et produits de chaque bibliothèque,
le cadre général qui régit le travail de la bibliothèque.
5.3.4 Les indicateurs à usage provisoire
Certains indicateurs de performance mentionnés dans le présent Rapport technique ont une valeur
d'information limitée à une certaine période et à un certain objectif. Ce sont des indicateurs valides seulement
si la bibliothèque considère parmi ses objectifs principaux d'offrir ses services sous forme électronique, autant
que possible et de manière raisonnable.
De tels indicateurs (par exemple B.1.2.1, B.1.4.1, B.2.2.1) mesurent le montant des ressources allouées aux
services électroniques par rapport aux services traditionnels. Ces indicateurs montrent l'implication de la
bibliothèque dans les services électroniques et son évolution dans le temps vers une bibliothèque numérique.
Ils seront valables jusqu'à ce qu'un état satisfaisant ait été atteint.
D'autres indicateurs de performance (indicateurs d'usage, de coût, ou de pénétration de marché des services
électroniques) auront une pertinence sur le long terme pour les bibliothèques.
10 © ISO 2003 — Tous droits réservés
Annexe A
Liste des indicateurs de performance pour les services électroniques
des bibliothèques
Le tableau A.1 présente les services électroniques des bibliothèques et les activités qui s'y rapportent selon la
classification utilisée dans l'ISO 11620:1998 [1]. Il indique la référence aux descriptions figurant en Annexe B.
NOTE Il existe d'autres indicateurs qui ont été testés dans des bibliothèques et qui ne sont pas décrits ici. Les
données de base utilisées pour les indicateurs ici décrits peuvent être combinées de diverses façons pour calculer des
indicateurs supplémentaires qui présentent un intérêt pour une bibliothèque en particulier.
Tableau A.1
Service, activité ou aspect Description dans
Indicateur de performance
mesuré l'Annexe B
Services publics B.1
Généralités B.1.1
Pourcentage de la population touchée par les services B.1.1.1
électroniques
Fourniture de services B.1.2
électroniques par la
bibliothèque
Pourcentage des dépenses en fourniture d'information consacrées B.1.2.1
à la collection électronique
Recherche de documents B.1.3
Nombre de documents téléchargés par session B.1.3.1
Coût des bases de données par session B.1.3.2
Coût par document téléchargé B.1.3.3
Pourcentage de sessions rejetées B.1.3.4
Pourcentage de sessions de consultation à distance du catalogue B.1.3.5
interrogeable en ligne
Pourcentage de visites virtuelles par rapport au total des entrées B.1.3.6
Services de renseignement et B.1.4
de référence
Pourcentage des demandes d'information transmises sous forme B.1.4.1
électronique
Formation des usagers B.1.5
Nombre de séances de formation d'usagers à des services B.1.5.1
électroniques par personne de la population à desservir
Installations B.1.6
Nombre d'heures d'accès à des postes de travail par personne de B.1.6.1
la population à desservir
Population par poste de travail en libre accès B.1.6.2
Taux d'utilisation des postes de travail B.1.6.3
Tableau A.1 (suite)
Service, activité ou aspect Description dans
Indicateur de performance
mesuré l'Annexe B
Disponibilité et emploi des B.2
ressources humaines
Formation du personnel B.2.1
Nombre de sessions de formation aux nouvelles technologies B.2.1.1
suivies par agent
Affectation du
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...