ETSI TR 103 305-2 V1.1.1 (2016-08)

CYBER; Critical Security Controls for Effective Cyber Defence; Part 2: Measurement and auditing

DTR/CYBER-0012-2

General Information

• Status: Published
• Publication Date: 31-Jul-2016
• Technical Committee: CYBER - Cyber Security
• Current Stage: 12 - Completion
• Due Date: 01-Aug-2016
• Completion Date: 01-Aug-2016

Standards Content (Sample)

ETSI TR 103 305-2 V1.1.1 (2016-08)






TECHNICAL REPORT
CYBER;
Critical Security Controls for Effective Cyber Defence;
Part 2: Measurement and auditing

---------------------- Page: 1 ----------------------
2 ETSI TR 103 305-2 V1.1.1 (2016-08)



Reference
DTR/CYBER-0012-2
Keywords
Cyber Security, Cyber-defence, information
assurance

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2016.
All rights reserved.

TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI TR 103 305-2 V1.1.1 (2016-08)
Contents
Intellectual Property Rights . 4
Foreword . 4
Modal verbs terminology . 4
Executive summary . 4
Introduction . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definitions and abbreviations . 5
3.1 Definitions . 5
3.2 Abbreviations . 6
4 Critical Security Controls: Measures Metrics, and Thresholds . 6
4.0 Control measures, metrics, and thresholds . 6
5 Critical Security Controls: Effectiveness Tests . 12
History . 18


ETSI

---------------------- Page: 3 ----------------------
4 ETSI TR 103 305-2 V1.1.1 (2016-08)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Cyber Security (CYBER).
The present document is part 2 of a multi-part deliverable. Full details of the entire series can be found in part 1 [i.3].
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Executive summary
The present document is intended as an evolving repository for guidelines on measurement and auditing of Critical
Security Control implementations. Measurement is an essential component of any successful security program. To
support good decision-making, the current state of a protected IT system or network should assessed. Means should
exist to measure and report on progress. The records kept constitute an audit.
Introduction
The Critical Security Controls ("the Controls") have always included a set of Metrics for every Control in order to help
adopters manage implementation projects. Adopters can use the sample Metrics as a starting point to identify key
information to help track progress, and to encourage the use of automation.
However, there is considerable security "fog" around the use of the terms. For example, there are lots of things that can
be measured, but it is very unclear which of them are in fact worth measuring (in terms of adding value to security
decisions). And since there are very few "absolutes" in security, there is always the challenge of making a judgment
about the measurement value that is "good enough" in terms of managing risk.
The problem of inconsistent terminology across the industry cannot be solved, but consistency within the Critical
Security Controls can be enhanced. The definitions found in a NIST article, Cyber Security Metrics and Measures are a
useful point of departure. [i.1] This approach separates the attribute being measured (the "Measure") from a value
judgment of what is "good" or "good enough".
ETSI

---------------------- Page: 4 ----------------------
5 ETSI TR 103 305-2 V1.1.1 (2016-08)
1 Scope
The present document is an evolving repository for measurement and effectiveness tests of Critical Security Control
implementations. The CSC are a specific set of technical measures available to detect, prevent, respond, and mitigate
damage from the most common to the most advanced of cyber attacks.
The present document is also technically equivalent and compatible with the 6.0 version of the "CIS Controls
Measurement Companion Guide" October 2015, which can be found at the website http://www.cisecurity.org/critical-
controls/ [i.1].
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
reference document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] The Center for Internet Cybersecurity: "A Measurement Companion to the CIS Critical Security
Controls" version 6, October 15, 2015.
NOTE: Available at https://www.cisecurity.org/critical-controls.cfm.
[i.2] Paul E. Black, Karen Scarfone and Murugiah Souppaya, Cyber Security Metrics and Measures, in
Handbook of Science and Technology for Homeland Security, Vol. 5, Edited by John G. Voeller.
NOTE: Available at https://hissa.nist.gov/~black/Papers/cyberSecurityMetrics2007proof.pdf.
[i.3] ETSI TR 103 305-1: "CYBER; Critical Security Controls for Effective Cyber Defence; Part 1: The
Critical Security Controls".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
Critical Security Control (CSC): specified capabilities that reflect the combined knowledge of actual attacks and
effective defences of experts that are maintained by the Center for Internet Security and found at the website
http://www.cisecurity.org/critical-controls/
measure: concrete, objective attribute, such as the percentage of systems within an organization that are fully patched,
the length of time between the release of a patch and its installation on a system, or the level of access to a system that a
vulnerability in the system could provide [i.2]
ETSI

---------------------- Page: 5 ----------------------
6 ETSI TR 103 305-2 V1.1.1 (2016-08)
metric: abstract, somewhat subjective attribute, such as how well an organization's systems are secured against external
threats or how effective the organization's incident response team is [i.2]
NOTE: An analyst can approximate the value of a metric by collecting and analyzing groups of measures, as is
explained later 3 and CSC 12.
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CIS Center for Internet Security
CSC Critical Security Control or Capability
DLP Data Loss Prevention
DMZ DeMilitarized Zone
EICAR European Expert Group for IT-Security
ID Identifier
IDS Intrusion Detection System
IPS Intrusion prevention system
IPv6 Internet Protocol version 6
IT Information Technology
LAN local area network
NIST National Institute of Standards and Technology
NLA Network Level Authentication
SCAP Security Content Automation Protocol
URL Uniform Resource Locator
USB Universal Serial Bus
VLAN Virtual Local Area Network
4 Critical Security Controls: Measures Metrics, and
Thresholds
4.0 Control measures, metrics, and thresholds
For each Control, a list of Measures is presented in the table below. Each Measure is given a unique ID number to allow
tracking.
NOTE: These numbers do not correspond to the individual sub-controls in the Critical Security Controls
document.
These Measures are similar to what "Metrics" in previous versions of the Controls.
For each Measure, Metrics are presented, which consist of three "Risk Threshold" values. These values represent an
opinion from experienced practitioners, and are not derived from any specific empirical data set or analytic model.
These are offered as a way for adopters of the Controls to think about and choose Metrics in the context of their own
security improvement programs. (This is sometimes described, e.g. by NIST, for each of the Risk Thresholds as a
"lower-level metric". The "higher-level metric" is the collection of the three Risk Thresholds. When an Enterprise
chooses a specific Threshold, that becomes a "benchmark" against which that Enterprise measures progress).
Separately, for every Control, an Effectiveness Test is presented in clause 5. These provide a suggested way to
independently verify the effectiveness of the implementation for each Critical Security Control.
ETSI

---------------------- Page: 6 ----------------------
7 ETSI TR 103 305-2 V1.1.1 (2016-08)
Table 1: Critical Security Controls (Version 6): Measures, Metrics and Thresholds
Critical Security Controls (Version 6): Measures, Metrics, and Thresholds
  METRICS
Lower Risk Moderate Risk Higher Risk
ID Measure
Threshold Threshold Threshold
How many unauthorized devices are presently on the
1.1 Less than 1 % 1 % - 4 % 5 % - 10 %
organization's network (by business unit)?
How long, on average, does it take to remove
1,440 minutes 10,080 minutes
1.2 unauthorized devices from the organization's network 60 minutes
(1 day) (1 week)
(by business unit)?
What is the percentage of systems on the
organization's network that are not utilizing Network
1.3 Less than 1 % 1 % - 4 % 5 % - 10 %
Level Authentication (NLA) to authenticate to the
organization's network (by business unit)?
How many hardware devices have been recently
blocked from connecting to the network by the
1.4
organization's Network Level Authentication (NLA)
system (by business unit)?
How long does it take to detect new devices added to
1,440 minutes 10,080 minutes
1.5 the organization's network (time in minutes - by 60 minutes
(1 day) (1 week)
business unit)?
How long does it take to isolate/remove unauthorized
1,440 minutes 10,080 minutes
1.6 devices from the organization's network (time in 60 minutes
(1 day) (1 week)
minutes - by business unit)?
How many unauthorized software applications are
2.1 presently located on business systems within the Less than 1 % 1 % - 4 % 5 % - 10 %
organization (by business unit)?
How long, on average, does it take to remove
1,440 minutes 10,080 minutes
2.2 unauthorized applications from business systems 60 minutes
(1 day) (1 week)
within the organization (by business unit)?
What is the percentage of the organization's business
systems that are not running software whitelisting
2.3 Less than 1 % 1 % - 4 % 5 % - 10 %
software that blocks unauthorized software applications
(by business unit)?
How many software applications have been recently
2.4 blocked from executing by the organization's software
whitelisting software (by business unit)?
How long does it take to detect new software installed
1,440 minutes 10,080 minutes
2.5 on systems in the organization (time in minutes - by 60 minutes
(1 day) (1 week)
business unit)?
How long does it take to remove unauthorized software
1,440 Minutes 10,080 minutes
2.6 from one of the organization's systems (time in minutes 60 minutes
(1 day) (1 week)
- by business unit)?
What is the percentage of business systems that are
not currently configured with a security configuration
3.1 Less than 1 % 1 % - 4 % 5 % - 10 %
that matches the organization's approved configuration
standard (by business unit)?
What is the percentage of business systems whose
security configuration is not enforced by the
3.2 Less than 1 % 1 % - 4 % 5 % - 10 %
organization's technical configuration management
applications (by business unit)?
What is the percentage of business systems that are
3.3 not up to date with the latest available operating Less than 1 % 1 % - 4 % 5 % - 10 %
system software security patches (by business unit)?
What is the percentage of business systems that are
not up to date with the latest available business
3.4 Less than 1 % 1 % - 4 % 5 % - 10 %
software application security patches (by business
unit)?
How many unauthorized configuration changes have
3.5 been recently blocked by the organization's
configuration management system (by business unit)?
How long does it take to detect configuration changes 1,440 minutes 10,080 minutes
3.6 60 minutes
to a system (time in minutes - by business unit)? (1 day) (1 week)
How long does it take to reverse unauthorized changes 1,440 minutes 10,080 minutes
3.7 60 minutes
on systems (time in minutes - by business unit)? (1 day) (1 week)
ETSI

---------------------- Page: 7 ----------------------
8 ETSI TR 103 305-2 V1.1.1 (2016-08)
Critical Security Controls (Version 6): Measures, Metrics, and Thresholds
  METRICS
Lower Risk Moderate Risk Higher Risk
ID Measure
Threshold Threshold Threshold
What is the percentage of the organization's business
systems that have not recently been scanned by the
4.1 Less than 1 % 1 % - 4 % 5 % - 10 %
organization's approved, SCAP compliant, vulnerability
management system (by business unit)?
What is the average SCAP vulnerability score of each
4.2 of the organization's business systems (by business
unit)?
What is the total SCAP vulnerability score of each of
4.3 the organization's business systems (by business
unit)?
How long does it take, on average, to completely
1,440 minutes 10,080 minutes 43,200 minutes
4.4 deploy operating system software updates to a
(1 day) (1 week) (1 Month)
business system (by business unit)?
How long does it take, on average, to completely
1,440 minutes 10,080 minutes 43,200 minutes
4.5 deploy application software updates to a business
(1 day) (1 week) (1 Month)
system (by business unit)?
How many unauthorized elevated operating system
accounts (local administrator/root) are currently
5.1
configured on the organization's systems (by business
unit)?
How many unauthorized elevated application accounts
5.2 are currently configured on the organization's systems
(by business unit)?
What percentage of the organization's elevated
5.4 accounts do not require two-factor authentication (by Less than 1 % 1 % - 4 % 5 % - 10 %
business unit)?
How many attempts to upgrade an account to
5.5 administrative privileges have been detected on the
organization's systems recently (by business unit)?
How many attempts to gain access to password files
5.6 within the system have been detected on the
organization's systems recently (by business unit)?
How long does it take for administrators to be notified
1,440 minutes 10,080 minutes
5.7 about user accounts being added to super user groups 60 minutes
(1 day) (1 week)
(time in minutes - by business unit)?
What percentage of the organization's systems do not
currently have comprehensive logging enabled in
6.1 Less than 1 % 1 % - 4 % 5 % - 10 %
accordance with the organization's standard (by
business unit)?
What percentage of the organization's systems are not
6.2 currently configured to centralize their logs to a central Less than 1 % 1 % - 4 % 5 % - 10 %
log management system (by business unit)?
How many anomalies/events of interest have been
6.3 discovered in the organization's logs recently (by
business unit)?
If a system fails to log properly, how long does it take
1,440 minutes 10,080 minutes
6.4 for an alert about the failure to be sent (time in minutes 60 minutes
(1 day) (1 week)
- by business unit)?
If a system fails to log properly, how long does it take
1,440 minutes 10,080 minutes
6.5 for enterprise personnel to respond to the failure (time 60 minutes
(1 day) (1 week)
in minutes - by business unit)?
How many unsupported web browsers have been
7.1 detected on the organization's systems (by business
unit)?
How many unsupported email clients have been
7.2 detected on the organization's systems (by business
unit)?
How many events of interest have been detected
7.3 recently when examining logged URL requests made
from the organization's systems (by business unit)?
What percentage of devices are not required to utilize
7.4 network based URL filters to limit access to potentially Less than 1 % 1 % - 4 % 5 % - 10 %
malicious websites (by business unit)?
ETSI

---------------------- Page: 8 ----------------------
9 ETSI TR 103 305-2 V1.1.1 (2016-08)
Critical Security Controls (Version 6): Measures, Metrics, and Thresholds
  METRICS
Lower Risk Moderate Risk Higher Risk
ID Measure
Threshold Threshold Threshold
What percentage of the organization's users, on
7.5 average, will inappropriately respond to an organization Less than 1 % 1 % - 4 % 5 % - 10 %
sponsored email phishing test (by business unit)?
What percentage of systems have not been deployed
8.1 with enabled and up-to-date anti-malware systems (by Less than 1 % 1 % - 4 % 5 % - 10 %
business unit)?
How many instances of malicious code have been
8.2 detected recently by host based anti-malware systems
(by business unit)?
How many instances of malicious code have been
8.3 detected recently by network based anti-malware
systems (by business unit)?
What percentage of the organization's applications are
8.4 not utilizing application sandboxing products (by Less than 1 % 1 % - 4 % 5 % - 10 %
business unit)?
How long does it take the system to identify any
malicious software that is installed, attempted to be 1,440 minutes 10,080 minutes
8.5 60 minutes
installed, executed, or attempted to be executed on a (1 day) (1 week)
computer system (time in minutes - by business unit)?
How long does it take the organization to completely
1,440 minutes 10,080 minutes
8.6 remove the malicious code from the system after it has 60 minutes
(1 day) (1 week)
been identified (time in minutes - by business unit)?
What is the percentage of the organization's systems
9.1 that are not currently running a host based firewall (by Less than 1 % 1 % - 4 % 5 % - 10 %
business unit)?
How many unauthorized services are currently running
9.2 on the organization's business systems (by business
unit)?
How many deviations from approved service baselines
9.3 have been discovered recently on the organization's Less than 1 % 1 % - 4 % 5 % - 10 %
business systems (by business unit)?
How long does it take systems to identify any new
unauthorized listening network ports that are installed 1,440 minutes 10,080 minutes
9.4 60 minutes
on network systems (time in minutes - by business (1 day) (1 week)
unit)?
How long does it take to close or authorize newly
1,440 minutes 10,080 minutes
9.5 detected system services (time in minutes - by 60 minutes
(1 day) (1 week)
business unit)?
What percentage of the organization's systems have
10.1 not recently had their operating system or application Less than 1 % 1 % - 4 % 5 % - 10 %
binaries backed up (by business unit)?
What percentage of the organization's systems have
10.2 not recently had their data sets backed up (by business Less than 1 % 1 % - 4 % 5 % - 10 %
unit)?
What percentage of the organization's backups have
10.3 not recently been tested by the organization's Less than 1 % 1 % - 4 % 5 % - 10 %
personnel (by business unit)?
What percentage of the organization's systems do not
10.4 have a current backup that is not available to online Less than 1 % 1 % - 4 % 5 % - 10 %
operating system calls (by business unit)?
How long, on average, does it take to notify system
1,440 minutes 10,080 minutes
10.5 personnel that a backup has failed to properly take 60 minutes
(1 day) (1 week)
place on a system (by business unit)?
What is the percentage of network devices that are not
currently configured with a security configuration that
11.1 Less than 1 % 1 % - 4 % 5 % - 10 %
matches the organization's approved configuration
standard (by business unit)?
What is the percentage of network devices whose
security configuration is not enforced by the
11.2 Less than 1 % 1 % - 4 % 5 % - 10 %
organization's technical configuration management
applications (by business unit)?
ETSI

---------------------- Page: 9 ----------------------
10 ETSI TR 103 305-2 V1.1.1 (2016-08)
Critical Security Controls (Version 6): Measures, Metrics, and Thresholds
  METRICS
Lower Risk Moderate Risk Higher Risk
ID Measure
Threshold Threshold Threshold
What is the percentage of network devices that are not
11.3 up to date with the latest available operating system Less than 1 % 1 % - 4 % 5 % - 10 %
software security patches (by business unit)?
What is the percentage of network devices do not
11.4 require two-factor authentication to administer the Less than 1 % 1 % - 4 % 5 % - 10 %
device (by business unit)?
How long does it take to detect configuration changes
1,440 minutes 10,080 minutes
11.5 to a network system (time in minutes - by business 60 minutes
(1 day) (1 week)
unit)?
How long does it take to reverse unauthorized changes
1,440 minutes 10,080 minutes
11.6 on network systems (time in minutes - by business 60 minutes
(1 day) (1 week)
unit)?
What percentage of the organization's remote access
users are not required to use two-factor authentication
12.1 Less than 1 % 1 % - 4 % 5 % - 10 %
to remotely access the organization's network (by
business unit)?
What percentage of remote business systems are not
12.2 managed using the same security standards as internal Less than 1 % 1 % - 4 % 5 % - 10 %
network systems (by business unit)?
What percentage of the organization's internal systems
12.3 are not on dedicated Virtual LANs (VLANs) that are Less than 1 % 1 % - 4 % 5 % - 10 %
segmented with access control lists (by business unit)?
How many events of interest have been discovered
recently on the organization's network through analysis
12.4
of NetFlow configured on network devices (by business
unit)?
How long does it take before unauthorized network
1,440 minutes 10,080 minutes
12.5 packets are alerted on when passing through perimeter 60 minutes
(1 day) (1 week)
systems (time in minutes - by business unit)?
How long does it take to apply configuration changes to
1,440 minutes 10,080 minutes
12.6 block unauthorized traffic passing through perimeter 60 minutes
(1 day) (1 week)
systems (time in minutes - by business unit)?
How many unauthorized data exfiltration attempts have
13.1 been detected recently by the organization's Data Loss
Prevention (DLP) system (by business unit)?
How many plaintext instances of sensitive data have
13.2 been detected recently by the organization's automated
scanning software (by business unit)?
How many attempts to access known file transfer and
13.3 email exfiltration websites have been detected recently
(by business unit)?
What percentage of sensitive data sets are not
14.1 configured to require logging of access to the data set Less than 1 % 1 % - 4 % 5 % - 10 %
(by business unit)?
What percentage of the organization's business
systems are not utilizing host based Data Loss
14.2 Less than 1 % 1 % - 4 % 5 % - 10 %
Prevention (DLP) software applications (by business
unit)?
How many rogue wireless access points have been
15.1 discovered recently in the organization (by business Less than 1 % 1 % - 4 % 5 % - 10 %
unit)?
What is the average time that it takes to remove rogue
1,440 minutes 10,080 minutes
15.2 access points from the organization's network (by 60 minutes
(1 day) (1 week)
business unit)?
How many wireless access points or clients have been
discovered using an unauthorized wireless
15.3 Less than 1 % 1 % - 4 % 5 % - 10 %
configuration recently in the organization (by business
unit)?
How long does it take to generate alerts about
1,440 minutes 10,080 minutes
15.4 unauthorized wireless devices that are detected (time 60 minutes
(1 day) (1 week)
in minutes - by business unit)?
ETSI

---------------------- Page: 10 ----------------------
11 ETSI TR 103 305-2 V1.1.1 (2016-08)
Critical Security Controls (Version 6): Measures, Metrics, and Thresholds
  METRICS
Lower Risk Moderate Risk Higher Risk
ID Measure
Threshold Threshold Threshold
How long does it take for unauthorized wireless
1,440 minutes 10,080 minutes
15.5 devices to be isolated/removed from the network (time 60 minutes
(1 day) (1 week)
in minutes - by business unit)?
How many invalid attempts to access user accounts
16.1
have been detected recently (by business unit)?
How many accounts have been locked out recently (by
16.2
business unit)?
How many attempts to gain access to password files in
16.3 the system have been detected recently (by business
unit)?
What percentage of the organization's workforce
17.1 members have not completed a core information Less than 1 % 1 % - 4 % 5 % - 10 %
security awareness program (by business unit)?
What percentage of the organization's workforce
members have not completed job role specific
17.2 Less than 1 % 1 % - 4 % 5 % - 10 %
information security awareness program (by business
unit)?
What percentage of the organization's workforce
17.3 members have not passed general information security Less than 1 % 1 % - 4 % 5 % - 10 %
awareness assessments (by bus
...