Deterministic networking: from niches to the mainstream? - Embedded.com

Deterministic networking: from niches to the mainstream?

There was a time before the Internet began turning into a means of distributing mass entertainment and infotainment when there was a clear delineation between two types of network operation.

One type included the real time and deterministic networks that served the specialized needs of industrial/ machine control and factory automation. The other included asynchronous networks in business and personal computing where the need was for reliable delivery and for which determinism and precisely timed delivery was a secondary consideration.

This is changing rapidly as noted in “Making networks more deterministic , ” the topic of this week’s Embedded.com Tech Focus newsletter.

Many of the early factory automation and machine control oriented networks operated according to the same rules of real time and deterministic operation of the devices they linked. Starting as ad hoc and mainly proprietary solutions, out of this came a variety of so-called controller area networks standards such as CANopen, ControlNet, DeviceNet, Modbus,  Profibus, and the fieldbus.

At the same time, network communications between non-real time systems in business and data processing began to standardize around such LAN-based topologies such as Ethernet, which used something very similar to the current TCP/IP internetworking protocol suite.

Unlike the controller area networks, the TCP/IP protocol is probabilistic and asynchronous in nature. It was – and by in large still remains – a network environment where the aim is to guarantee delivery, but not within any particular, or even predictable, time frame. Even with the development of enhancements such as the NetworkTime Protocol (NTP) for clock synchronization between computer systems,it is certainly not within the fine-grained microsecond and millisecond deterministicboundary conditions most embedded control operations need.

But Ethernet and the TCP/IP suite have one advantage that these real timenetwork protocols did not: they are ubiquitous, which provided two thingsthat embedded developers needed: 1) a network protocol that if engineered correctly would work in a variety of environments and allow communications between them, and 2) a protocol that was low in cost to implement because of its ubiquity and the resultant economies of scale.

But starting in the late 1990s and through most of the last decade there have been increasing efforts to adapt the Ethernet protocol to the real time and deterministic requirements of embedded control applications.

Initially it was just a matter of physically configuring stations or nodesinto a closed Ethernet collection and limit the number of nodes until theresponse times were within the deterministic requirements needed. This waythe time it took for each to do the time consuming handshaking to ensure deliverycould be minimized and, more importantly, predicted.

Even more fine-grained were early attempts to take the underlying TCP/IP protocol apart and use only those portions, such as UDP – that do away with the time consuming process of acknowledging receipt to guarantee delivery – in a closed environment to handle transmissions in a much more deterministic way. Several articles dealing with these techniques include:

Real time Ethernet
Speed packet throughput with compressed UDP
Internet connectivity: stacking the odds in your favor
TCP-IP for transactions

But most useful to embedded developers was the creation in 2002 of the IEEE 1588 and its precision clock synchronization protocol, the topic of asix part series on Embedded.com: “Basicsof real-time measurement, control, and communication using IEEE 1588.“ Using it as the starting point, many of the industrial control network protocolshave adapted elements of the PTP and out of that has emerged what is calledtheIndustrial Ethernet.

Joining Ethernet in adapting to a networking environment that requires more and more real-time deterministic performance have been any number of other specifications and standards. One example is the Firewire serial interconnect specification which started its life as an alternative to the Universal Serial Bus as a way of networking various peripheral and storage functions to a mainserver or desktop computer.

With the development of IEEE-1394b-2002, Firewire now has several key features that, when coupled with SAE Standard AS5643 creates a deterministic, robust, and redundant distributed system architecture that meets most of therequirements for a hard real-time control bus and networking protocol. Thedetails of its use in many demanding aerospace and defense designs are describedin “I EEE-1394and AS5643 bring deterministic networking to high reliability Mil-Aero designs , “the lead article in this week’s Embedded.com Tech Focus newsletter.

Such adaptations to real time and determinism are coming just in timefor the broader Internet, which is now increasing being used as the meansby which to deliver real time video transmissions to Internet-enabled TVs,smartphones and tablets. For example, virtually every significant Ethernet router/switchmanufacturer has incorporated the IEEE 1588 standard into their hardwarelayer.

Also, as noted in “UnderstandingIEEE’s determinisitic audio/video bridging standard, ” the 1588-basedIEEE 802.1 Audio/Video Bridging enhancement to the Ethernet standard is beingwidely used to deliver time-synchronized, low-latency audio and video whileretaining 100% compatibility with Ethernet.

Further contributing to this trend is SyncE, the topic of “Introductionto the Synchronized Ethernet . ” Taking a slightly different approachto making the TCP/IP suite more deterministic, this ITU-T standard facilitatesthe transference of clock signals over the Ethernet physical layer. It isbeing used widely in applications such as cellular networks, IPTV and VoIP,not to mention such Internet-backbone access technologies as passive opticalnetworks which require something more than traditional Ethernet protocols.

Also not immune to the need for a more deterministic TCP/IP protocol isthe new segment of embedded design activity involving in bringing wirelesslyconnected M2M and Internet-of-Things enabled devices into homes and buildingsvia new IPv6 Internet protocol extensions such as 6LoWPAN. Ironically, oneway developers are looking to satisfy this need for more deterministic operationis the same UDP subset that so occupied embedded developers attention in thelate 1990s. Several articles on Embedded.com that chronicle this trend include:

UDP and the embedded wireless IoT
Pub-sub, the IoT and 6LoWPAN connectivity
How to set up a 6LoWPAN network

I do not see this trend toward the incorporation of real time determinisminto the broader Internet slowing down. Soon, the deterministic modificationsto Ethernet servers and routers that are the backbone of the Internet willmove out of the physical layer where they now reside and move into the protocollayer. This trend will be driven not by how humans use the network, but bythe needs of the many more embedded and distributed sensors that willbe connected.

Where present protocol standards now reflect the response times of humans(one to several seconds ), the Internet of the future will be mainlypopulated and used by devices with response times in the microseconds tothe milliseconds. Even now, in the average home, the ratio of devices tohumans is on the order of 10:1. In the near future it is virtually certainthat most of the devices will be connected.

We live in exciting times. After graduating from Columbia University inthe 1970s, I came into the electronics industry only a few years after theintroduction of the microprocessor. In each of the decades since. I thoughtthe one I was in was the most exciting ever and that there was nothing that wouldtop it. This decade is no different and has not disappointed me yet. I can’twait to see what the next few years in this new exciting segment of the industrywill bring.

Embedded.com Site Editor Bernard Cole is also editor of thetwice-a-week Embedded.comnewsletters as well as a partner in the TechRite Associates editorialservices consultancy. He welcomes your feedback. Send an email to , or call928-525-9087.

See more articles and column like this one on Embedded.com. Signup for the Embedded.com newsletters .Copyright © 2013 UBM–All rights reserved.

3 thoughts on “Deterministic networking: from niches to the mainstream?

  1. ” … the 1588-based IEEE 802.1 Audio/Video Bridging enhancement to the Ethernet standard is being widely used to deliver time-synchronized, low-latency audio and video while retaining 100% compatibility with Ethernet.”
    “Widely used” are not the words I w

    Log in to Reply
  2. You are right. I should have said that it was increasingly being considered in Ethernet designs for delivery of deterministic A/V. I should have also emphasized that 802.1 is 1) a committee effort to come up with a standard that tries to meet a number of c

    Log in to Reply
  3. Excellent article. I will follow through and study this. I'm working in the Industrial Control System Security realm where we see the the Plant Data network and Process Control networks spliced together by dual-homed nodes and 3rd Party networks running th

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.