ETSI GR IP6 009 V1.1.1 (2017-03)

IPv6-based Industrial Internet leveraging 6TiSCH technology

DGR/IP6-0009

General Information

• Status: Published
• Publication Date: 28-Mar-2017
• Technical Committee: IP6 - IPv6 integration
• Current Stage: 12 - Completion
• Due Date: 12-Apr-2017
• Completion Date: 29-Mar-2017

Standards Content (Sample)

ETSI GR IP6 009 V1.1.1 (2017-03)






GROUP REPORT
IPv6-based Industrial Internet leveraging 6TiSCH technology
Disclaimer
The present document has been produced and approved by the IPv6 Integration (IP6) ETSI Industry Specification Group (ISG)
and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

---------------------- Page: 1 ----------------------
2 ETSI GR IP6 009 V1.1.1 (2017-03)



Reference
DGR/IP6-0009
Keywords
6TiSCH, IPv6, network

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2017.
All rights reserved.

TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI GR IP6 009 V1.1.1 (2017-03)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Executive summary . 5
Introduction . 6
1 Scope . 7
2 References . 7
2.1 Normative references . 7
2.2 Informative references . 7
3 Abbreviations . 13
4 Converging Networks for the Industrial Internet . 15
4.1 On Operational Technology . 15
4.2 Enabling the IT/OT convergence . 15
4.3 The path to the IT/OT Convergence . 16
4.4 The case of Low-power Lossy Networks . 17
5 What is Deterministic Networking? . 18
5.1 Common definitions (from Web encyclopaedia) . 18
5.2 The train analogy (to control loop traffic) . 19
5.3 The bus analogy (to deterministic circuit switching). 19
5.4 The vacation place analogy (to time-sharing) . 20
5.5 The casino analogy (to statistical effects) . 21
5.6 Transporting OT traffic . 22
6 Enabling Determinism in a Network . 22
6.1 The precursors . 22
6.1.1 On Fast Reroute . 22
6.1.2 On SDN and Traffic Engineering . 23
6.2 Expected benefits in wired networks . 23
6.3 Making Ethernet deterministic? . 24
6.4 Making wireless deterministic? . 24
7 The IETF DetNet architecture . 27
7.1 Positioning of work . 27
7.2 The architecture in a nutshell . 27
7.3 Networking in DetNet . 28
7.4 Controlling a Deterministic Network . 30
7.4.1 Reporting the topology to the controller . 30
7.4.2 Implementing the needs of the application . 31
7.4.3 Automating the network operation . 33
7.5 Limits and perspectives . 33
8 The art of low-power wireless sensor network. 33
8.1 A highly predictable wireless . 33
8.2 WSNs in Industrial Process Control . 35
8.3 6TiSCH and best effort IPv6 . 35
9 The vision of 6TiSCH centralized scheduling . 37
9.1 A converged wireless network . 37
9.2 PCE vs. 6TiSCH . 38
9.3 6TiSCH base elements (time slots, schedule, chunks and bundles) . 38
9.4 Applying DetNet to 6TiSCH . 43
9.5 Forwarding along 6TiSCH Tracks . 43
9.6 Enabling the convergence. 44
ETSI

---------------------- Page: 3 ----------------------
4 ETSI GR IP6 009 V1.1.1 (2017-03)
10 Conclusion . 45
Annex A: Authors & contributors . 47
History . 48


ETSI

---------------------- Page: 4 ----------------------
5 ETSI GR IP6 009 V1.1.1 (2017-03)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Group Report (GR) has been produced by ETSI Industry Specification Group (ISG) IPv6 Integration (IP6).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Executive summary
The Industrial Internet will enable deep process optimization in multiple industries by introducing Information
Technology (IT) capabilities, such as Big Data and virtualization, to improve Operational Technology (OT) processes
while reducing the OPEX, with the convergence of the IT and OT network. At the core of this revolution, a new breed
of Deterministic Networks will provide enhancements that are required to fully emulate the traditional serial links and
field buses that are widely deployed in that space over IPv6.
Deterministic Networking is a new (to IT networks) level of guarantee for network-based services, based on time,
resource reservation, and enforcement. Deterministic Networking provides the capability to carry specified unicast or
multicast data streams for real-time applications with extremely low data loss rates and bounded latency. Deterministic
Networking technology allows for guarantees of 'worst-case' delivery. More precisely, the worst-case data loss and
latency are guaranteed in a consistent fashion as multiple services are deployed on a common converged network
infrastructure.
Deterministic Networking adds key capabilities to the Internet (wired, wireless, Layer 2 and Layer 3) to support
time-sensitive mission-critical applications on a converged enterprise infrastructure. These capabilities are required to
drive the connection of billions of things, and make available the vast amounts of data that IoE applications generate.
Deterministic Networking is a quantum step beyond existing QoS mechanisms. It implies time synchronization on all
the nodes, often including source and destination, the centralized computation of the deterministic paths from a global
perspective for a better optimization, new traffic shapers and schedulers within and at the edge to protect the network,
and new hardware for time-triggered access to the media.
6TiSCH [i.107] enables Deterministic Networking over Low-Power Radios, controlled by a central intelligence called a
PCE. At the same time, 6TiSCH allows traditional best effort IPv6 flows routed with the RPL routing protocol to utilize
the portions of the bandwidth that are not allocated to deterministic flow. This way, the collection over IPv6 of
traditionally unmeasured data can scale to vast numbers without interfering with the more critical flows for which all
the necessary resources are reserved.


ETSI

---------------------- Page: 5 ----------------------
6 ETSI GR IP6 009 V1.1.1 (2017-03)
Introduction
It all started with point-to-point copper wires, transporting analogue signals for short messages, then telephone and
television, industrial measurements and commands, anything though initially not data. Digital data networks, and then
packet networks, came last; but with the advent of determinism, the late comers now show the potential to federate all
original forms of wired and wireless communication and lead to the final convergence of all communication networks.
A generic and cheap replacement to serial cables to provide connectivity to all sorts of devices, coupled with
resource-sharing meshed networks, are now required to simplify the cabling and drive the costs down in many
industries, from transportation to manufacturing. Simple as it may seem to emulate the legacy forms of serial
communications, reproducing the various aspects of a point-to-point electric cable over a multi-hop packet network is
actually the hardest thing to do. Yet, the need is becoming more and more pressing, as:
1) managing all the existing sorts of cables and buses has become an increasingly costly complexity in many
aspects of our lives; and
2) point-to-point wires will not scale to serve the exploding needs of the Internet of Things.
A paper on "Integrating an Industrial Wireless Sensor Network with your Switched Ethernet and IP Network [i.67]" was
presented at the Emerson Exchange 2008 conference in Washington. The paper discussed how Wireless Sensor
Networks (WSNs), which are in essence cheaper and faster to deploy than traditional wired field-buses, could leverage
the entire network to connect the sensors to a centralized controlling application located afar on the carpeted floor, for
Industrial supervisory control or logging. At the same time, the paper stressed issues that are raised when integrating a
classical, often proprietary industrial automation network, with tight response time and availability constraints, into a

wider IP network based on packet-switched and Internet technologies. With this and a collection of other papers [i.69],
[i.70], [i.71] and [i.72], the realization is now coming that with techniques such as flow isolation, high availability and a
new generation of Quality of Service (QoS), the times of the convergence of these networks are finally approaching.

ETSI

---------------------- Page: 6 ----------------------
7 ETSI GR IP6 009 V1.1.1 (2017-03)
1 Scope
The present document outlines a general architecture for an Industrial Internet, providing motivation for the
deployment, and some technical guidelines with a focus on deterministic and low power technologies, for a prospective
IPv6-based Industrial Internet leveraging deterministic wireless technology. The present document elaborates on
deterministic networking, wired and wireless, for application in the Industrial Internet.
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] IETF RFC 6371: "Operations, Administration, and Maintenance Framework for MPLS-Based
Transport Networks".
[i.2] J. Araujo et al.: "High availability automation networks: PRP and HSR ring implementations"
in: Industrial Electronics (ISIE), 2012 IEEE International Symposium on, IEEE, 2012,
pp. 1197-1202.
[i.3] IETF RFC 7471: "OSPF Traffic Engineering (TE) Metric Extensions".
[i.4] D. Beller and R. Sperber: "MPLS-TP-The New Technology for Packet Transport Networks"
in: DFN-Forum Kommunikationstechnologien, vol. 149, 2009, pp. 81-92.
[i.5] IETF RFC 6119: "IPv6 Traffic Engineering in IS-IS".
[i.6] M. S. Borella et al.: "Methods for determining sendable information content based on a determined
network latency", US Patent 6,182,125, Jan. 2001.
[i.7] IETF RFC 7490: "Remote Loop-Free Alternate (LFA) Fast Reroute (FRR)".
[i.8] A. Colvin: "CSMA with collision avoidance" in: Computer Communications 6.5 (1983),
pp. 227-235.
[i.9] IEC 62439-3:2009: "Industrial communication networks - High availability automation networks -
Part 3: Parallel Redundancy Protocol (PRP) and High-availability Seamless Redundancy (HSR)".
[i.10] IEC 62591:2016: "Industrial networks - Wireless communication network and communication
profiles - WirelessHART™".
[i.11] IEC 62734:2014: "Industrial networks - Wireless communication network and communication
profiles - ISA 100.11a".
[i.12] IEC 62601:2015: "Industrial networks - Wireless communication network and communication
profiles - WIA-PA".
ETSI

---------------------- Page: 7 ----------------------
8 ETSI GR IP6 009 V1.1.1 (2017-03)
[i.13] IEC 61850: 2016 SER: "Communication networks and systems for power utility automation -
ALL PARTS".
[i.14] S. S. Craciunas and R. S. Oliver: "SMT-based task-and network-level static schedule generation
nd
for time-triggered networked systems" in: Proceedings of the 22 International Conference on
Real-Time Networks and Systems, ACM, 2014, p. 45.
[i.15] IETF draft-ietf-6tisch-6top-sf0-02D: "6TiSCH 6top Scheduling Function Zero (SF0)".
[i.16] Y. Fang and Y. Zhang: "Call admission control schemes and performance analysis in wireless
mobile networks" in: IEEE Transactions on vehicular technology 51.2 (2002), pp. 371-382.
[i.17] IETF RFC 4655: "A Path Computation Element (PCE)-Based Architecture".
[i.18] IETF RFC 5960: "MPLS Transport Profile Data Plane Architecture".
[i.19] M. Goraj and R. Harada: "Migration paths for IEC 61850 substation communication networks
towards superb redundancy based on hybrid PRP and HSR topologies" in: Developments in Power
th
Systems Protection, 2012. DPSP 2012. 11 International Conference on, IET, 2012, pp. 1-6.
[i.20] IETF draft-ietf-detnet-use-cases-11: "Deterministic Networking Use Cases".
[i.21] T. Hasegawa et al.: "Industrial wireless standardization - Scope and implementation of ISA SP100
standard" in: SICE Annual Conference (SICE), 2011 Proceedings of, IEEE, 2011, pp. 2059-2064.
[i.22] K.-i. Hwang: "Energy efficient channel agility utilizing dynamic multi-channel CCA for ZigBee
RF4CE" in: IEEE Transactions on Consumer Electronics 57.1 (2011), pp. 113-119.
[i.23] D. M. Ingram, P. Schaub, and D. A. Campbell: "Use of precision time protocol to synchronize
sampled-value process buses" in: IEEE Transactions on Instrumentation and Measurement 61.5
(2012), pp. 1173-1180.
[i.24] H. Kirrmann et al.: "HSR: Zero recovery time and low-cost redundancy for Industrial Ethernet
th
(High availability seamless redundancy, IEC 62439-3)" in: Proceedings of the 14 IEEE
international conference on Emerging technologies & factory automation, IEEE Press, 2009,
pp. 203-206.
th
[i.25] H. Kopetz et al.: "The time-triggered ethernet (TTE) design" in: 8 IEEE International Symposium
on Object-Oriented Real-Time Distributed Computing (ISORC'05), IEEE, 2005, pp. 22-33.
[i.26] W. Liang et al.: "Survey and experiments of WIA-PA specification of industrial wireless network"
in: Wireless Communications and Mobile Computing 11.8 (2011), pp. 1197-1212,
issn: 1530-8677, doi: 10.1002/wcm.976.
NOTE: Available at http://dx.doi.org/10.1002/wcm.976.
[i.27] Z. Lin and S. Pearson: "An inside look at industrial Ethernet communication protocols" in: White
Paper Texas Instruments (2013).
[i.28] J. D. MacKay: "Applications and Opportunities for the IEEE 1588 Standard in Military
Applications", tech. rep., DTIC Document, Jan. 2007.
NOTE: Available at www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA474269.
[i.29] S. Mangold et al.: "Analysis of IEEE Std 802.11 e for QoS support in wireless LANs" in: IEEE
wireless communications 10.6 (2003), pp. 40-50.
[i.30] E. Mannie: "Generalized multi-protocol label switching (GMPLS) architecture" in: Interface 501
(2004), p. 19.
[i.31] N. McKeown et al.: "OpenFlow: enabling innovation in campus networks" in: ACM SIGCOMM
Computer Communication Review 38.2 (2008), pp. 69-74.
[i.32] S. Meier and H. Weibel: "IEEE 1588 applied in the environment of high availability LANs"
in: 2007 IEEE International Symposium on Precision Clock Synchronization for Measurement,
Control and Communication, IEEE, 2007, pp. 100-104.
ETSI

---------------------- Page: 8 ----------------------
9 ETSI GR IP6 009 V1.1.1 (2017-03)
[i.33] A. Morell et al.: "Label switching over IEEE Std 802.15.4e networks" in: Transactions on
Emerging Telecommunications Technologies 24.5 (2013), pp. 458-475.
[i.34] T. Neagoe, V. Cristea, and L. Banica: "NTP versus PTP in com puter networks clock
synchronization" in: 2006 IEEE International Symposium on Industrial Electronics, vol. 1,
IEEE, 2006, pp. 317-362.
[i.35] M. Nixon and T. Round Rock: "A Comparison of WirelessHART™ and ISA100. 11a"
in: Whitepaper, Emerson Process Management (2012), pp. 1-36.
[i.36] B. A. A. Nunes et al.: "A survey of software-defined networking: Past, present, and future of
programmable networks" in: IEEE Communications Surveys & Tutorials 16.3 (2014),
pp. 1617-1634.
[i.37] A. de la Oliva et al.: "An overview of the CPRI specification and its application to C-RAN-based
LTE scenarios" in: IEEE Communications Magazine 54.2 (2016), pp. 152-159.
[i.38] R. S. Oliver, S. S. Craciunas, and G. Stöger: "Analysis of deterministic ethernet scheduling for the
industrial internet of things" in: 2014 IEEE 19th International Workshop on Computer Aided
Modeling and Design of Communication Links and Networks (CAMAD), IEEE, 2014,
pp. 320--324.
[i.39] IETF RFC 6378: "MPLS Transport Profile (MPLS-TP) Linear Protection".
[i.40] IETF RFC 5543: "BGP Traffic Engineering Attribute".
[i.41] M. R. Palattella et al.: "Standardized protocol stack for the internet of (important) things" in: IEEE
Communications Surveys & Tutorials 15.3 (2013), pp. 1389-1406.
[i.42] S. Petersen and S. Carlsen: "WirelessHART versus ISA100. 11a: the format war hits the factory
floor" in: IEEE Industrial Electronics Magazine 5.4 (2011), pp. 23-34.
[i.43] IETF RFC 5673: "Industrial Routing Requirements in LowPower and Lossy Networks".
[i.44] K. Pister and L. Doherty: "TSMP: Time synchronized mesh protocol" in: IASTED Distributed
Sensor Networks (2008), pp. 391-398.
[i.45] G. Pujolle: "SDN (Software-Defined Networking)" in: Software Networks, pp. 15- 48.
[i.46] IETF draft-ietf-6tisch-dtsecuritysecure-join-00: "6tisch Secure Join protocol".
[i.47] IETF RFC 5714: "IP Fast Reroute Framework".
[i.48] IETF RFC 2212: "Specification of Guaranteed Quality of Service".
[i.49] F. Shu et al.: "Packet loss analysis of the IEEE Std 802.15. 4 MAC without acknowledgements"
in: IEEE communications letters 11.1 (2007), pp. 79-81.
[i.50] K. Srinivasan et al.: "The β-factor: measuring wireless link burstiness" in: Proceedings of the
th
6 ACM conference on Embedded network sensor systems, ACM, 2008, pp. 29-42.
[i.51] B. Sundararaman, U. Buy and A. D. Kshemkalyani: "Clock synchronization for wireless sensor
networks: a survey" in: Ad hoc networks 3.3 (2005), pp. 281-323.
[i.52] J.-C. Tan and W. Luan: "IEC 61850 based substation automation system architecture design"
in: 2011 IEEE Power and Energy Society General Meeting, 2011.
[i.53] R. Teixeira et al.: "Dynamics of hot-potato routing in IP networks" in: ACM SIGMETRICS
Performance Evaluation Review, vol. 32, 1, ACM, 2004, pp. 307-319.
[i.54] IETF draft-thubert-6tisch-4detnet-01: "6TiSCH requirements for DetNet".
[i.55] IETF draft-ietf-6tisch-architecture-10: "An Architecture for IPv6 over the TSCH mode of IEEE
Std 802.15.4".
[i.56] IETF draft-ietf-6lo-backbone-router-02: "IPv6 Backbone Router".
ETSI

---------------------- Page: 9 ----------------------
10 ETSI GR IP6 009 V1.1.1 (2017-03)
[i.57] IETF draft-ietf-detnet-architecture-00: "Deterministic Networking Architecture".
[i.58] IETF draft-thubert-6lo-forwarding-fragments-03: "LLN Fragment Forwarding and Recovery".
[i.59] IETF draft-ietf-6tisch-minimal-21: "Minimal 6TiSCH Configuration".
[i.60] IETF draft-ietf-6tisch-minimal-security-00: "Minimal Security Framework for 6TiSCH".
[i.61] IETF draft-ietf-6tisch-6top-protocol-03: "6top Protocol (6P)".
[i.62] T. Watteyne, A. Mehta, and K. Pister: "Reliability through frequency diversity: why channel
hopping makes sense" in: Proceedings of the 6th ACM symposium on Performance evaluation of
wireless ad hoc, sensor, and ubiquitous networks, ACM, 2009, pp. 116-123.
[i.63] T. Watteyne et al.: "Industrial IEEE Std 802.15.4e networks: Performance and trade-offs" in: 2015
IEEE International Conference on Communications (ICC), IEEE, 2015, pp. 604-609.
[i.64] T. Watteyne et al.: "Mitigating Multipath Fading Through Channel Hopping in Wireless Sensor
Networks" in: International Conference on Communications (ICC), IEEE, Cape Town, South
Africa, May 2010.
[i.65] IETF RFC 3630: "Traffic Engineering (TE) Extensions to OSPF Version 2".
[i.66] IETF RFC 5286: "Basic Specification for IP Fast Reroute: Loop-Free Alternates".
[i.67] E. Ziouva and T. Antonakopoulos: "CSMA/CA performance under high traffic conditions:
throughput and delay analysis" in: Computer communications 25.3 (2002), pp. 313-321.
[i.68] Cisco, Emerson: Integrating an Industrial Wireless Sensor Network with Your Plant's Switched
Ethernet and IP Network".
NOTE: Available at http://www.controlglobal.com/assets/14WPpdf/140303-Cisco-Emerson-
WirelessNetworks.pdf.
[i.69] Atos White Paper: "The convergence of IT and Operational Technology".
NOTE: Available at https://atos.net/content/dam/global/ascent-whitepapers/ascent-whitepaper-the-convergence-
of-it-and-operational-technology.pdf.
[i.70] Cisco White Paper.
NOTE: Available at http://www.cisco.com/c/en/us/products/collateral/se/internet-of-everything/white-paper-c11-
735380.pdf.
[i.71] Statseeker White Paper: "The IT/OT Convergence - Bridging the Gap".
NOTE: Available at http://www.isssource.com/wp-content/uploads/2015/11/111115statseeker-IT-OT-
Convergence-White-Paper.pdf.
[i.72] NexDefense White Paper: "IT/OT Convergence - Bridging the Divide".
NOTE: Available at http://ics.sans.org/media/IT-OT-Convergence-NexDefense-Whitepaper.pdf.
[i.73] IEEE/IETF Coordination meeting: "Deterministic Networking".
NOTE: Available at https://www.iab.org/wp-content/IAB-uploads/2013/01/tsn-nfinn-Deterministic-Networking-
BOF-0914-v1.pdf.
[i.74] Schneider Electric: "How the Convergence of IT and OT Enables Smart Grid Development".
NOTE: Available at
...