ISO/IEC FDIS 20547-4

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 20547-4
ISO/IEC JTC 1/SC 27 Secretariat: DIN
Voting begins on: Voting terminates on:
2019-12-30 2020-03-23
Information technology — Big data reference
architecture —
Part 4:
Security and privacy
Technologies de l'information — Architecture de référence des mégadonnées —
Partie 4: Sécurité et Confidentialité
ICS: 35.020
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 20547-4:2019(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2019
---------------------- Page: 1 ----------------------
ISO/IEC DIS 20547-4:2019(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2019

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Symbols and abbreviated terms ........................................................................................................................................................... 1

5 Overview ....................................................................................................................................................................................................................... 2

5.1 Big data security and privacy concerns ............................................................................................................................. 2

5.2 Security and privacy objectives ................................................................................................................................................ 4

6 Security and privacy aspects of BDRA user view ................................................................................................................ 6

6.1 Overview ...................................................................................................................................................................................................... 6

6.2 Governance activities ........................................................................................................................................................................ 6

6.2.1 Purpose .................................................................................................................................................................................... 6

6.2.2 Prepare for and plan BD-S&P governance effort .................................................................................. 7

6.2.3 Monitor, assess and control BD-S&P governance activities ........................................................ 7

6.2.4 Establish BD-S&P governance objectives ................................................................................................... 7

6.2.5 Direct BD-S&P..................................................................................................................................................................... 8

guidance .................................................................................................................................................................................. 9

6.3 Management activities ..................................................................................................................................................................10

6.3.1 Purpose .................................................................................................................................................................................10

6.3.2 Prepare for and plan BD-S&P management effort ...........................................................................10

6.3.3 Monitor, assess and control the architecture management activities .............................11

6.3.4 Develop BD-S&P management approach .................................................................................................11

6.3.5 Perform management of BD-S&P ...................................................................................................................12

6.3.6 Monitor BD-S&P effectiveness ..........................................................................................................................12

6.3.7 Update the BD-S&P management plan ......................................................................................................13

6.4 Operation activities ..........................................................................................................................................................................13

6.4.1 BD-S&P solution design activities ..................................................................................................................13

6.4.2 BD-S&P solution evaluation activities ........................................................................................................18

6.4.3 BD-S&P solution enablement activities .....................................................................................................23

6.5 Security and privacy aspects of big data roles ..........................................................................................................26

7 Guidance on security and privacy operations for big data ....................................................................................29

7.1 General ........................................................................................................................................................................................................29

7.2 Guidance at organization level ...............................................................................................................................................30

7.2.1 Introduction ......................................................................................................................................................................30

7.2.2 Standard guidance on requirements ...........................................................................................................31

7.2.3 Standard guidance on risk management .................................................................................................32

7.2.4 Standard guidance on controls .........................................................................................................................32

7.2.5 Standard guidance on lifecycle operations ............................................................................................32

7.3 Guidance at ecosystem level .....................................................................................................................................................32

7.3.1 Introduction ......................................................................................................................................................................32

7.3.2 Guidance on data processing chain ..............................................................................................................33

7.3.3 Guidance on risk management .........................................................................................................................34

7.3.4 Guidance on lifecycle operations ....................................................................................................................35

8 Security and privacy functional components ......................................................................................................................37

8.1 Overview ...................................................................................................................................................................................................37

8.2 Functional components for both security and privacy......................................................................................37

8.3 Functional components for privacy ...................................................................................................................................38

8.4 Multi-layer functions for security and privacy .........................................................................................................39

Annex A (informative) Example of security and privacy threat classification .......................................................41

Annex B (informative) Example of security and privacy control classification ....................................................42

Annex C (informative) Example of ecosystem and resulting coordination of security and

privacy operations ...........................................................................................................................................................................................45

Annex D (informative) Examples of security and privacy controls per BDRA roles ........................................51

Bibliography .............................................................................................................................................................................................................................56

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work. In the field of information technology, ISO and IEC have established a joint technical committee,

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the

different types of ISO documents should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of

any patent rights identified during the development of the document will be in the Introduction and/or

Any trade name used in this document is information given for the convenience of users and does not

For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,

as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the

Technical Barriers to Trade (TBT) see the following URL: www .iso .org/ iso/ foreword .html.

The committee responsible for this document is Technical Committee ISO/IEC JTC 1, Information

technology, Subcommittee SC 27, Information security, cybersecurity and privacy protection.

Big data refers to the massive amount of digital information collected in various forms from different

sources of digital and physical environments. This data is not only generated by traditional means

of information exchange, but also from sensors embedded in physical environments, such as city

surroundings, transportation vehicles, critical infrastructures, etc. The collection and processing of big

data provides additional challenges not inherent in the traditional digital information exchange setting.

This document was developed in response to worldwide demand for a common baseline security and

privacy aspects for big data architectures to facilitate interoperability in big data systems without

The big data paradigm blurs the security boundaries between data collection, storage, and access

– areas traditionally addressed independently – that must now be confronted holistically with a

comprehensive security and privacy foundation, tightly coupled to all architecture components.

Effective standardization of security and privacy is paramount to the development of mutual trust and

This document specifies the security and privacy aspects applicable to the Big Data Reference

Architecture (BDRA) including the big data roles, activities, and functional components, and also

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 20547-3, Information technology — Big data reference architecture — Part 3: Reference

For the purposes of this document, the terms and definitions given in ISO/IEC 20546 apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

Big data has the key data characteristics of volume, velocity, variety and variability, and also the

key data processing characteristics of volatility, veracity and value. These characteristics introduce

additional risks and thus challenges on the security and privacy aspect of big data.

— Volume of data is at risk associated with massive amounts of data in various layers. For example,

multi-tiered, distributed storages and transmission on various networks with different protocols.

— Velocity of data has risks associated with faster flow at which the data is created, stored, analysed

or visualized. Security controls might be a burden on velocity and easily omitted.

— Variety of data brings more complexity from a diversity of sources under control of various actors.

Complexity inevitably leads to vulnerability. An emergent phenomenon introduced by big data

variety is its ability to infer identity from anonymized datasets by correlating with apparently

— Variability of data has risks associated with faster changes in data rate, format/structure, semantics,

and/or quality. It might become more difficult to apply security controls on data security and

— Volatility of data could affect to keep audit trails and make security management difficult.

— Veracity of data brings higher requirements to integrity, consistency and accuracy. Associated risks

Big data application boom brings more serious security and privacy issues on data, such as frequent

incidents of data loss and personal data leakage, illegal data transactions underground, which cause

data abuse and Internet fraud, and endanger social stability and national security.

From technology platform perspective, due to the continuous emergence of a variety of big data

technology, new technical architecture and support platform, and big data software, the following

Massive, multi-sourced, heterogeneous, dynamic and other big data characteristics lead to the

difference of data application security from a closed environment. Big data applications generally

use the open distributed computing and storage architecture with complex underlying support to

provide massive data distributed storage and high-performance computing services. These new

technologies and architectures make the network boundaries of big data applications become

blurred, so that the boundary-based traditional security protection measures are no longer valid.

Meanwhile, under this new situation, the Advanced Persistent threat (APT), Distributed Denial

of Service (DDoS), machine learning-based data mining and privacy discovery, and other attacks

make the traditional defence, detection and other security controls expose serious deficiencies. For

instance, providing secure data management and threat intelligence, providing secure data storage

for big data as well as secure log data generation, transmission, storage, analysis and disposal

becomes very difficult. Additionally novel technical approaches for privacy-preserving, machine

learning, cryptographic mechanisms for data-centric security and access control are necessary.

For more information on the requirements of big data security and privacy, refer the security and

privacy technical considerations in different use cases provided by ISO/IEC TR 20547-2:2018.

— Security and privacy need to be provided for the distributed computation and data store

This requires privacy-preserving and secure distributed computation and information

dissemination. Big data requires scalable and distributed solutions for secure data storage as

well as for audits and investigations for data provenance. Data integrity for streaming influx of

data from various sensors and other end-points has to be provided. Real-time analytics for threat

intelligence require processing of large amounts of security related information such as traffic

— Platform security mechanisms need to be improved in the context of big data security and privacy.

In general, existing big data applications use the big data management platforms and technology,

such as Hadoop-based HBase/Hive, Cassandra/Spark, MongoDB. At the beginning of design, these

platforms and technology are mostly considered to be used in the trusted internal network, with

little consideration of authentication, authorization, key services and security audit. Although some

software are improved, such as adding Kerberos authentication mechanism, the overall security

capability is still relatively weak. Meanwhile, the third party open source components are often

used in big data applications. Due to the lack of rigorous test management and security certification

of these components, the ability to prevent software vulnerabilities and malicious backdoors in big

Because of the variety of data types and the wide range of applications of big data, it is often

used to provide multiple services to users with different identities and purposes from different

organizations or departments. In general, access control is an effective means to achieve controlled

access to data. However, due to a large number of unknown data users and data to be accessed, it

is very difficult to pre-set roles and permissions of data access. Although user’s rights to access

data can be classified in advance, because of the numerous roles, it is difficult to define the control

of each role's actual permissions in a fine-grained way. So, it is difficulty to accurately specify the

range of data for each user to access without deploying newer access control model such as ABAC

(Attribute-Base Access Control). This also causes the issue with the data minimization principle in

When designing and applying security and privacy mechanisms such key management, identity

and access management, de-identification, etc., in big data environment, not only the security and

privacy functionalities need to be considered, but also scalability of these mechanisms need to be

taken into account in order to support processing of high volume and high velocity of the data.

From data application perspective, due to the big Vs (Volume, Variety, Velocity and Variability)

characteristics of big data, and huge value in big data, the following security and privacy capabilities

In the open network society, the huge volume of big data with immeasurable potential value

makes it more favoured and easier to become a significant target of network attacks. In recent

years, information security incidents frequently occur, for example, leakage of email accounts,

social security information and bank card numbers. The distributed system deployment, the open

network environment, the complex data application and the large amount of user accesses, all cause

the big data to face the bigger challenges in the confidentiality, integrity, availability and so on.

Due to the large amount of personal data in big data systems, when the security incidents such as

data abuse, internal theft and network attacks occur, the consequences of personal data leakage

will be more serious than ordinary information systems. On the other hand, the advantage of

generating value from the analysis and usage of large amounts of data could be compromised by

the more risk of personal data leakage during the comprehensive analysis of multi-source data

where analysts are easier to explore more personal data through correlation analysis.

Data in big data systems have a wide range of sources that could be a variety of sensors, active

uploads and public websites. In addition to reliable data sources, there are a large number of

untrusted data sources. Some attackers even deliberately falsify data in an attempt to induce

data analysis results. Therefore, it is very important to verify the authenticity of data and their

sources. However, there are many difficulties in verifying all the data authenticity because of the

performance limitation of data acquisition terminals, the lack of technology, the limited amount of

During the application of big data, the data could be accessed by a variety of users, flow from one

controller to another, and even be mined to produce new data. Therefore, in the process of data

exchange and sharing, there are the circumstance where the data ownership of a data owner

and the data use right of a data manager are separated, which implies that data could be out of

data owner's control, and brings the risks such as data abuse, vague ownership of data, unclear

responsibilities of data security supervision, so that the rights and interests of data owners could

Big data involves ecosystems, or networks of organizations which collaborates to collect, analyse and

— collaboration between stakeholders to ensure that overall ecosystem security and privacy

requirements and individual organization’s security and privacy requirements are consistent;

— collaboration between stakeholders to ensure that the overall ecosystem risk management and the

— collaboration between stakeholders to ensure that the individual organizations ensure a consistent

Big data applications have security and privacy objectives. Table 1 describes examples of security

objectives. Table 2 describes examples of privacy objectives. Note that Table 2 describes alternative

Purpose legitimacy and Ensuring that the purpose(s) complies with applicable law and relies