Using IEEE 1588 for synchronization of network-connected devices -

Using IEEE 1588 for synchronization of network-connected devices

When the clocks in our lives get out of sync, bad things happenwearrive late for meetings, miss the train, or record all but the lastfew minutes of our favorite television show. Wouldn't it be helpful ifclocks were always set to the correct time?

Actually, if they were networked together and using the IEEE 1588Precision Time Protocol, they could all be set to same time, typicallywithin 100 nanoseconds of each other! While that's extreme accuracy fora kitchen clock, it's perfect for clocks pumping voice, video and datathrough the heart of every network-connected device. And this simpleprotocol costs less network bandwidth and fewer dollars than otheralternatives.

Specifically, the advantages of incorporating IEEE1588-2002 Precision Clock Synchronization Protocol for NetworkedMeasurement and Control Systems (PTP) into your network connecteddesign include:

* network clocksynchronization accuracy in the sub-microsecond range
* synchronization of clocks withdiffering precision, resolution and stability
* low-cost implementation in multicast messaging networks such asEthernet
* fast re-synchronization whensystem changes occur
* simpleinstallation and maintenance.

This protocol is already widely adopted by test and measurement,power-line management and industrial automation applications. IEEE 1588principles are adopted to synchronize clocks in networks running industrialEthernet protocols such as EtherNet/IP and Ethernet Powerlink .

IEEE 1588 Version 2 (V2) is undergoing revision, and is expected tobe published by mid-2007. It features improved support for largeredundant networks and high-performance telecommunicationsapplications.

For example, electric powerutilities must synchronize across large-scaledistributed power grid switches to enablesmooth power transfer andmaintain power supply integrity across a patchwork of systems builtfrom many different technologies exposed to extreme heat and cold.Reliability, durability and longevity are particularly important forpower management, and timing accuracy is critical to maintainprotection of the power grid. It's expensive to have a GlobalPositioning System (GPS) device in every substation, especially whenIEEE 1588 could do the same job for a fraction of the cost.

The arrival of femtocell home basestations (Figure 1, below ) will let meuse mymobile phone inside my home instead of standing outdoors in -40 degrees(and I'll leave you to wonder why I don't relocate to Freescaleheadquarters in balmy Austin, Texas).

Figure1. Use of IEEE-1588 in a femtocell base station

A key ingredient for this application is a frequency-aligned, and insome cases, a phase-aligned clock that provides timing to every homebase station. Some customers consider using Network Time Protocol(NTP),GPS and other schemes to accomplish this, but many are now consideringIEEE 1588 as it becomes widely supported by embedded processorstargeting this high-volume application. This article covers theinteresting debate in more detail below.

Interest is also growing for telecommunicationsapplications where robustand stable clock distribution to all network nodes is required toenable the transition from leased T1/E1 lines toless expensive Internet Protocol (IP) networks.

However, many telecommunications applications are waiting for IEEE1588 V2 enhancements such as unicast messaging and shorter frame sizesto reduce the bandwidth consumed by the protocol messaging. Version 2also introduces new PTP devices called transparent clocks that are usedto prevent error accumulation in cascaded topologies, and Version 2provides extensions to enable redundant systems.

Timing is everything
The IEEE 1588 protocol enables rapid convergence (typically less than aminute, depending on the network topology) to sub-microsecond timesynchronization between heterogeneous distributed devices controlled byclocks of differing resolution and stability.

IEEE 1588 can be implemented solely in software to give accuracy inthe sub-100 microsecond range. This is similar to that seen with othersoftware implemented protocols such as Network Time Protocol(NTP – RFC 1305) and Simple Network Time Protocol (SNTP) methods that operate across the same network topology.

However, if timestamping is performed in the application layer,interrupts and other unpredictable software processes can introducejitter and latency which may impair the synchronization. Even the useof a very precise external oscillator won't overcome the stack jitterassociated with a software-only 1588 implementation.

Most applications require the higher accuracy achieved bytimestamping packets at the interface between the physical (PHY) anddata link (MAC) layers (often referred to as “hardware timestamping”).IEEE 1588 hardware timestamping typically improves accuracy to 100nanoseconds or better for certain network configurations, which isbetter than NTP, SNTP, Time-Triggered Protocol (TTP – SERCOS (IEC 61491) methods.

There are several ways to implement IEEE 1588 hardware timestamping.In previous years, the only option was to connect a field programmablegate array (FPGA) between the Ethernet PHY and MAC to timestamp thearrival and departure of every incoming and outgoing IEEE 1588 SYNC andDELAY_REQUEST event message.

However, now that Freescale and other silicon vendors haveimplemented the hardware timestamp function on integratedcommunications processors, customers are adopting these new devices tosignificantly improve cost, power and board footprint.

As with all time synchronization methods, better clocks give betterresults. IEEE 1588 provides the flexibility to support differentoptions to match most cost and accuracy requirements.

For example, the lowest cost option might use an on-chip systemclock for timestamping packets. An external clock oscillator can givehigher accuracy, with highest accuracy (and cost) coming from anexternal ovenized voltage controlledoscillator (OVCXO).Protocol software supporting these threeoptions should be easily configurable to select the desired clocksource type.

Remember, no matter which implementation or clock option you select,IEEE 1588 synchronization resolution time is typically less than aminute (typically faster than NTP and SNTP), and system cost, hardwarefootprint and CPU load are much lower than for all the other timesynchronization methods mentioned above.

Benefits of IEEE 1588 Version 2.
Appendix A of the IEEE 1588 Version 1 (V1) specification indicates thatthe best PTP performance can be achieved by minimizing the number ofnodes between the clock source and the slave devices.

When devices are stacked in a hierarchy, error accumulates at eachlevel and low-accuracy intermediate devices will dominate the errorexperienced by the end devices.

Since hierarchical architectures are common in public and private IPnetworks, V2 is being developed with considerable input from experts intelecomm, networking, test/ measurement, and industrial controls, whounderstand the special requirements of large distributed networks withmany hops and inherent redundancy. This experienced team is now helpingto specify support for new features such as:

* Subnanosecond accuracy , which is especially useful forsynchronizing test and measurement equipment.

* Fastersynchronization (SYNC) message rates. Unlike V1, which specifiesthat SYNC messages occur no faster than once a second, V2 defines amuch wider range of mean SYNC message rates, even allowing for ratesthat are much greater than 1000 messages per second. However, mostsystems are expected to use mean SYNC message rates much less than thisto reduce network traffic and optimize between network loading andsynchronization performance. Faster SYNC message rates generallyrequire more hardware timestamp assistance, and the CPU needs moreperformance to process the increased number of messages. Faster SYNCmessage rates are critical for telecommunications, residentialEthernet, and many control applications.

* Shorter PTPmessages, unicast messaging, new messages (path delayrequest/response/response follow-up) and message fields . Theseall reduce network bandwidth overhead and the resulting potentialnetwork queuing delays. These message optimizations are critical fortelecommunications, residential Ethernet, and many controlapplications.

* TransparentClocks (TC) , which can be used to prevent erroraccumulation in cascaded topologies. Network-induced jitter can occurwhen each packet takes a slightly different time to pass through anetwork switch.

Resulting time sync degradation can be avoided if switches implementa transparent clock at each network hop. End-to-end & peer-to-peerTransparent Clock devices calculate the delay incurred by a packetpassing through the device (called the residence time) and update atime correction field in the PTP payload to account for this delay.Thus, each switch appears to be a “wire” which does not skew the timecalculation for packets passing through them.

* FaultTolerance , which is needed to ensure no single network elementfailure can cause subtending clocks to fail. In a system with redundantgrandmasters, the redundant grandmaster must be able to detect failureof the first grandmaster, and the subtending clocks must be able toswitch from one grandmaster to another without incurring unacceptablefrequency or phase jumps, or losing the clock for subtending nodes.Fault tolerance is critical in telecom applications and many controlapplications, where safety and reliability are required.

* TLVextension to extend the protocol features/options . This allowsthe protocol to support extensions to meet the requirements of futureapplications.

* Profiles .These define what PTP features and settings are needed for differentmarket applications.

* Mapping IEEE1588 to other transport mechanisms such as 802.3/Ethernet,PROFINET, and DeviceNet.

More about Boundary and TransparentClocks
Boundary Clocks were defined in V1 to support IEEE 1588 synchronizationwithin networks containing several subnets. Boundary clocks typicallyhave more than two ports, with one port serving as a PTP slave to anupstream clock master, and the other ports serving as PTP clock mastersto downstream PTP slaves. Boundary Clocks employ a servo loop torecover the clock on the slave port, and then the recovered clock feedsBoundary Clock ports serving as PTP clock masters.

Due to the cumulative effect of multiple servo loops, cascadingmultiple Boundary Clocks within a network can significantly degradeclock accuracy of the system. Quality of the crystal oscillator used ina Boundary Clock can also impact the accuracy of the PTP clocks in thesystem. If a poor quality crystal is used in a Boundary Clock, PTPslaves attached to the Boundary Clock device will likely have inferiorsynchronization to the PTP clock master.

Transparent Clocks were added in V2 to update and append a timingcorrection field within certain message. The correction field isupdated with a value representing the “residence time” required forthat message to pass through the Transparent Clock.

Here is some additional information about Boundary and Transparentclocks:

* BoundaryClocks do not propagate Sync, Follow_Up, Delay_Req, or Delay_Respmessages.

* Boundary Clocksare suitable for topologies with a small number of switches.

* CascadingBoundary Clocks introduce the cascade effect. A Boundary Clockdistributes timing based on its local clock, and each clock depends onthe quality of all preceding clocks.

* Boundary Clocks can causeaccuracy and stability problems in highly cascaded or daisy-chainednetwork topologies because Boundary Clocks contain IEEE 1588 controlloops. Cascading Boundary Clocks is just like cascading phase lockedloops in the sense that the jitter starts to accumulate down the chain.

* TransparentClocks do not generate timestamps. Instead they update correctionfields in event messages according to the residence time of the eventmessage within the Transparent Clock.

The correction field will accumulate the value of the residencemultiplied by the number of cascaded Transparent Clocks. Ordinary andboundary clocks account for the value of the correction field in theircalculations. This results in reduced jitter and increased clockaccuracy and stability.

* There are twodifferent types of Transparent Clocks: End-to-End (E2E) andPeer-to-Peer (P2P). P2P Transparent Clocks are different than E2ETransparent Clocks in that each port on a P2P node computes the peerlink path delay with its link partner on another P2P node.

The link path delay calculation is accomplished through new V2messages: PDELAY_REQ, PDELAY_RESP, and PDELAY_RESP_FOLLOWUP (optional).P2P Transparent Clocks add the ingress link path delay time to thevalue in the correction field (in addition to the residence time of themessage).

* P2PTransparent Clocks only work with nodes supporting the peer delayfunction. This means that they work with ordinary and boundary clocksthat support the peer delay function, as well as other P2P TransparentClocks.

Figure2. Use of 1588 Syntonization in GSM wireless networks

Syntonization versus Synchronization
Certain telecommunications applications, like GSM wireless networks,requiresyntonization ,wheretwo clock frequencies are locked togetherbut the phases may be different, as shown in Figure 2, above. This is analogousto the timing information passed along T1/E1 communications circuitstoday.

The clock is passed from point A to point B across the T1/E1 link,but there is no requirement to account for the propagation delay whichresults from the clock traveling between the two points. In otherwords, the difference in phase of the two clocks measured at A and B iscaused by the propagation delay of the clock traveling down the link.

Other applications, like CDMA wireless networks, require clocks tobe aligned in both frequency and phase, e.g., when two clocks share thesame time of day and advance at the same rate. If it is important tominimize the phase difference between two points, then the delay due tothe network path delay must be known.

This is typically done by passing time-of-day information betweentwo nodes and allowing them to calculate the phase offset due to thenetwork path delay. CDMA systems typically need a precise time of dayat each node (meaning BTS) to correctly code/decode the multiplexeddata. These systems often use expensive GPS receivers to get thisinformation.

Some telecom services vendors are skeptical about relying on the IPnetwork connecting the base stations and the Base Station Controller(BSC) to consistently deliver accurate time-of-day information to allowthe data to be properly coded and decoded.

One primary concern is that customers may not control every part ofthe Ethernet network connecting their end-point equipment together.Public Ethernet networks are often shared by many customers and can besubject to complications that don't occur with leased T1/E1 lines.

These issues include sudden network reconfiguration, queuing delaysdue to traffic bursts, and delays due to traffic patterns such asheavier traffic during the day than at night.

Additionally, telecom vendors are sensitive to minimizing bothequipment cost and network downtime as the transition is made fromdistributing timing information across T1/E1 networks to distributingtiming information across and Ethernet network.

IEEE 1588 V2 seems to be a good fit for 3GPP femtocell applications,where only edge clock synchronization is required. However, it is notyet understood if it will work for all CDMA2000 femtocell applications,where clock phase and frequency synchronization is required.

Symmetricom has posteed its views in an onlinearticle , and Ericsson mentions a similar strategy in a documentthey haveposted online. An interesting thing to note in Ericsson's paper isthat the concept of timestamping at the interface of thereceiving/transmitting node is equally applicable to NTP-based networks(note the diagram in their Figure 4 where the T1 through T4 timestamppoints are the classic interface points).

We might also see SNTP-hybrid networks use hardware (MAC-layer)timestamps. Even though SNTP is a very mature protocol typically usedfor synchronizing very remote WAN networks, nothing prohibits it frombeing used in a MAN/LAN environment. Interestingly, this SNTP-hybridapproach can often be supported by the same hardware timestampmechanism added to devices to support IEEE 1588.

What IEEE 1588 solutions areavailable now?
Support is popping up everywhere for this protocol. The IEEE 1588 websitementions active device and software suppliers. The best IEEE 1588hardware solutions address market requirements by offering siliconwith:

* Multiple MACinterface options, such as support for MII/GMII/RMII/RGMII
* Flexible clockingoptions to support both low cost and high accuracy designs
* Phase andfrequency compensation circuitry to enable slave clock recovery
* RX and TXhardware timestamping close to the physical interface
* Detection of PTPframes, with a mechanism for reporting the reception of PTP frames tothe CPU
* Output triggerports to generate PTP-time based external events
* Input triggerports to timestamp input events
* PTP-time basedtimers to run timer interrupt functions based on synchronized timerather frequency of the on-board oscillator
* A one pulse persecond signal output port for testing purposes

The next gathering of the IEEE 1588 development community occursOctober 1-3, 2007, in Vienna, Austria. Protocol users are scheduled toreport on technical issues, product development and applicationexperience. The conference is scheduled to include a tutorial fornewcomers, a plug-fest to explore interoperability of various IEEE 1588implementations, and detailed discussion on V2. We hope to meet youthere!

Editor's note: Freescale's QUICCEngine and Enhanced Triple Speed Ethernet Controller (eTSEC)technologies incorporate communications interfaces to optimize IEEE1588 Precision Time Protocol (PTP) by time-stamping Ethernet packets atthe physical/datalink layer. Both are included in PowerQUICCcommunications processors such as MPC8360 and MPC8313. Freescale isalso collaborating with IXXAT Automation GmbH, to offer pre-configuredcommercial off-the-shelf total system solutions running the IEEE 1588protocol, including the IXXAT IEEE 1588 application demonstrated on theFreescale MPC8349E PowerQUICC II Pro communications processors mid-2006.

A demonstration of the IEEE 1588 specwith better than 50 nanosecond accuracy on MPC8360 PowerQUICC 2 Prowill be performed in Avnet'sBooth #624 at Embedded Systems Conference,San Jose, April 3-4, 2007. Free IXXAT evaluation software for Freescaledevelopment boards may be downloaded from References and AdditionalResources:
1) Aligning systemsclocks over networks with the IEEE 1588 Remote Timing Standard (PDF),byAlexandra Dopplinger and Jim Innis, Freescale Semiconductor and WernerAbt, IXXAT Automation.
2) IEEE 1588 website
3) John Eidson, “Tutorial on IEEE 1588,” Agilent Technologies,October, 2005.
4) “IEEE 1588 PTP Introduction,” IXXATProduct Brief..
5) “PowerQUICC Integrates IEEE 1588 Time Synchronization,” FreescaleFACT sheet (PDF) , July 21, 2006.
6) John C. Eidson, Measurement,Controland Communication UsingIEEE 1588, (Springer-Verlag London Limited 2006)

AlexandraDopplinger , P.Eng., is anIndustrial Segment Marketer atFreescale Semiconductor, Inc. inOttawa, Canada. She has marketed PowerArchitecture technology and network processors for five years. Prior tothat, she designed highly available telecom systems and holds onepatent for a redundant LAN implementation.

JimInnis is a Systems Engineerfor the Digital Systems Division atFreescale Semiconductor, Inc. in Austin, TX. He has been involved inthe specification and design of PowerQUICC devices for five years.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.