Basics of real-time measurement, control, and communication using IEEE 1588: Part 6 - Embedded.com

Basics of real-time measurement, control, and communication using IEEE 1588: Part 6

Hopefully, by this point in this series, the reader will have a goodunderstanding of IEEE 1588 technology and the ways it can be applied.It should be clear from the discussions of applications in test andmeasurement, industrial automation, and telecommunications that this isa very active field of investigation, involving many companies andresearchers around the world.

However, it is far too early to be sure that IEEE 1588 will achieve widespreadadoption, or that the additional capabilities it brings to time-basedoperations will prove sufficiently useful to displace or augmentexisting technologies.

It is clear that IEEE 1588 has the potential to make a majorcontribution to applications with hard real-time constraints. It isprobably not too much of a stretch to claim that IEEE 1588 will besuccessful in the field of industrial automation and motion control.

The area of test and measurement is less certain, and depends asmuch on how quickly the distributed architecture and peer-to-peercommunication model exemplified by the LXI specification is accepted,as it does on the IEEE 1588 standard itself.

Telecommunications applications at this point must be consideredspeculative. There are clearly areas within telecommunications whereIEEE 1588 has the potential to make significant contributions, but itis not at all clear whether it will ultimately be adopted.

Proposed Techniques to Enable IEEE1588 in Telecommunications
The telecommunication network impairments that degrade the performanceof an IEEE 1588 system are latency fluctuations and asymmetry. If onlyfrequency alignment is required, then asymmetry and the absolute valueof the latency are of no concern. However, if time transfer—epochalignment—is also required, then all impairments must be considered.

Asymmetry. Asymmetry impairments can arise from a variety of causes.Optical fiber systems supporting two-way communication will havedifferent path lengths in the two directions as a result of chromaticdispersion, even if the nominal line lengths are identical. Inpractice, companies already correct for this effect by means ofadditional fiber on the short path. This is essentially a calibrationprocess.

Far larger contributions to asymmetry can be expected from queuingeffects in switches. In addition, some network operators use ringtopologies in which traffic flows in only one direction, which leads tovastly different path lengths for the forward and reversecommunications between two arbitrary points. There are also protocolsthat are asymmetric by design, e.g., asymmetric digital subscriberloops (ADSL) .

If these factors can be rendered time-invariant by networkengineering techniques, then the needed corrections can be made bycalibrating the resulting network. This is not very appealing, but maybe the best that can be done.

It remains to be determined how well asymmetry can be controlled inactual operating environments. Algie estimates that the latency acrossa typical metropolitan area network is less than 20 ms. If this is thecase, then asymmetry-induced epoch offsets would be on the order of atmost 10 ms, which is probably not good enough for many of the proposedtelecommunications applications.

As discussed later, early data are discussed indicating that in somecircumstances the latency is substantially less. If it were possible totransfer time to the edges of the metropolitan networks using existingequipment, then it may be possible to use current IEEE 1588 techniques,such as boundary clocks, to distribute time within an enterprise.

The most difficult environment for controlling asymmetry for thetelecommunications examples discussed earlier in this series is themetropolitan area network, and the backhauls to wireless cell sites. Inbuilding distribution or distribution within equipment racks, thenetworks are much more likely to have controlled and stable asymmetryproperties.

Latency Fluctuations
Latency fluctuation impairments arise from a variety of sources. Themost common is queuing fluctuations in switches. These are particularlytroublesome because they are likely to be very traffic-dependent. Asecond source of timing fluctuation is the equipment needed totranslate between communication protocols present in the network, e.g.,the transition between a TDMA and a FDMA protocol, or from a T1 line toSONET.

On a longer time scale, changes in path routing will also introducedifferences in latency. Furthermore, these fluctuations may not be thesame on the forward and reverse paths, which will further aggravate theasymmetry problem.

The only practical solution to fluctuation is filtering the offsetand delay values computed by the slave clocks. Simple filtering is notlikely to be effective due to the time and traffic load dependence ofthe magnitude and distribution of the delay values.

More sophisticated algorithms could monitor the distributions, andmake use of the holdover properties of the local oscillator to maintaina stable time scale while the parameters for a new distribution arecomputed.

It will probably be necessary to separately monitor the forward andbackward paths, since there is no reason to assume that the effects oftraffic will be uniform in the two directions.

The observation times for these filtering algorithms will be long.If we assume that the fluctuations are uniformly distributed and have amagnitude of about 2 ms, then the NTP curve suggests that to achieve anAllan deviation of 10-8 , observation times of about 1 daywill be required.

The early data discussed later indicate that observation times onthe order of a few hours may suffice. Observations times on this ordercan be supported by quartz oscillators. If this is not the case, thenmore stable oscillators such as rubidium will be required, which willincrease the cost and make the use of IEEE 1588 in these applicationsmuch less attractive.

IEEE 1588 Timing Redundancy
Many applications in both industrial automation and telecommunicationshave redundancy requirements. Redundancy requirements are found incircumstances where loss of life, unacceptable physical damage, or lossof revenue can result due to system failure. The measures taken byGeneral Electric in the design of their turbine controllers are aclassic example of the use of redundancy to mitigate these risks.

In current telecommunications systems, carriers provide alternatepaths to ensure connectivity in the face of network outages resultingfrom broken links or equipment. Similar measures are implemented toprotect current telecommunications timing and synchronization services.

If IEEE 1588 is to be used to provide telecommunications timing,then means must be devised to satisfy these redundancy requirements, inprinciple by using the same techniques currently used intelecommunications.

For example, redundant timing in the wireless applications discussedearlier in this series would be provided by redundant IEEE 1588 timeservers, as shown in Figure 9.6 below.

Figure9.6. IEEE 1588 Redundancy in Telecommunications (courtesy of ZarlinkSemiconductor)

Here, primary and secondary time servers would provide timing toseparate IEEE 1588 grandmaster clocks. Each IEEE 1588 slave clock wouldselect the appropriate IEEE 1588 grandmaster based on a set ofalgorithms, which have yet to be defined.

The complete absence of a grandmaster, for example, due to a networkfailure between the grandmaster and the slave, would no doubt invokethe same holdover techniques currently used in telecommunications.

The difficulty is that in the current version of the IEEE 1588standard at the time of this writing, there is no provision forimplementing this form of redundancy.

The current standard allows selection of the grandmaster based onlyon the best master clock algorithm. Since the primary and secondaryreferences in a telecommunications system will be categorized by IEEE1588 as stratum 1 clocks,2 the best master clock algorithm will segmentthe IEEE 1588 topology into two disjointed regions, which is not whatis needed in this application.

The IEEE 1588 standards committee has an agenda item to providemechanisms for implementing the redundancy requirements for both theindustrial and telecommunications applications.

Measurements of IEEE 1588 onMetropolitan Networks
Since the spring of 2005, four companies have collaborated on a fieldtrial to test the viability of IEEE 1588 in distributing time over ametropolitan network. The initial results of these measurements werereported to the ITU by DaveTonks of Sematech. The topology of the trialis illustrated in Figure 9.7 below .The metropolitan network was selected by the carrier as being typicalin terms of equipment, operation, and network traffic.

Figure9.7. Topology of the IEEE 1588 telecommunications field trial

An IEEE 1588 master clock, using an Agilent 5071A cesium clock as areference, injected IEEE 1588 packets into the network. These packetswere carried on a virtual private network (VPN) to a remote office onthe network where they were returned on a different VPN.

The path consists of six switch buffers in the Cisco 6509 equipmentforming the network, plus two additional buffers in switches local tothe experiment. Other than the VPNs, no special handling of the IEEE1588 packets was used.

The returned packets were received by a slave clock that recoveredthe IEEE 1588 timing information and generated a T1 signal as an inputto the Agilent OmniBER 718 network analyzer. Both the master and slaveclocks are Semtech prototype devices. Note that the 718 is measuringthe noise of the entire IEEE 1588 system, including the master andslave clocks, as well as the noise introduced by the metropolitannetwork.

Figure9.8. MTIE field trial data for 5 June 2005 (courtesy of Semtech)

The 718 collected TIE data at 50 samples per second. The data aretransferred via a local control computer to a data store in Agilent'slaboratory in Palo Alto, California. TIE data are collected forapproximately 24 h before a new measurement sequence is initiated.Additional data collected from the slave clock allow relatedinformation, such as the actual round-trip delay as a function of time,to be archived. All collected data could be retrieved from the store bythe trial participants.

The test began on 21 April 2005, and has been collecting TIE datamore or less continuously ever since. The data were still beingcollected and analyzed as this book was being prepared. The MTIE andTDEV measurements for 5 June 2005 are shown in Figures 9.8 above and Figure 9.9 below.

Figure9.9. TDEV field trial data for 5 June 2005 (courtesy of Semtech)

In each figure, the mask shown is the plesiochronous digitalhierarchy (PDH) mask based on ITU recommendations. The observed valuesof both MTIE and TDEV data fall well below the masks. A comparison ofthe TDEV and MTIE data collected as of 5 June 2005 does not showsignificant day-to-day variation.

Figure 9.10 below , which isbased on data provided by Dave Roe of Semtech, shows the maximum MTIEvalues for each 24-h measurement period between 21 April and 5 June2005.

Figure9.10. MTIE field trial summary data: 21 April to 5 June 2005 (courtesyof Semtech)

These MTIE and TDEV data suggest that, under certain conditions,IEEE 1588 is able to transfer frequency with the necessary phasestability over metropolitan networks. Clearly, additional investigationis needed before this technique can be adopted.

Although one would expect that the 6-week trial period evaluatedhere encompassed typical network loading conditions, additional testingfor performance with loads designed to stress the use of IEEE 1588 inthis application must be done. Likewise, testing on additionalmetropolitan networks must be done to see if either carrier practicesor differences in network equipment are important.

Since some protocols require actual time transfer, rather than onlyfrequency, further experiments will be required to determine theeffects of asymmetry in telecommunication networks.Dave Tonks ofSematech presented the results of preliminary time transfer experimentsusing the trial network. These results indicate that time transfer ispossible in metropolitan networks to better than 100 ns.

Specific Concerns and LikelyOutcomes
In the author's opinion, IEEE 1588 is at a critical point in itsdevelopment. As noted, the IEEE has authorized a standards effort toextend the existing standard. The responsible IEEE working group,P1588, was in the early phase of discussions as this book was beingprepared.

The group that produced the original standard numbered no more thana dozen members. By contrast, the current P1588 committee regularlydraws between 30 and 45 members to each meeting.

Furthermore, the current committee represents a far more diverse setof interests and potential application requirements than did theoriginal group. This is an indication that IEEE 1588 is perceived asuseful technology, but carries with it the potential for fragmentationof the standard.

IEEE 1588 is an infrastructure standard. End users must be able toobtain a variety of affordable IEEE 1588-enabled devices for theirapplications. All IEEE 1588 based systems will require not only enddevices particular to the specific application, but also commoncomponents such as boundary clocks, clocks linked to a GPS system,bridges between network protocols, software tools for monitoringperformance, and, in the future, transparent clocks and other devicesyet to be conceived.

The P1588 committee must ensure that the revised standard not onlyprovides the needed enhancements, but does this in a way that keeps thestandard simple enough to enable manufacturers to make cost-effectivecommon components useful for all.

None of the application areas is large enough by itself to createthe usage numbers needed to drive down the cost of these commoncomponents to the point where IEEE 1588 is not only technicallyattractive, but also financially compelling.

This will require compromise within the P1588 group, which is alwaystricky to achieve with any group. Standards that are the union ofeveryone's design preferences are rarely successful, and certainly donot serve the end customers very well.

The outcome of the P1588 work is likely to be successful. The majorenhancements under consideration offer considerable improvements forall application areas. This should provide sufficient incentive to findcommon solutions.

While predicting the outcome of any standards committee effort, muchless the response of balloters, is an art, not a science, the earlydiscussions indicate that the P1588 committee recognizes the need forcompromise and is willing to devote the necessary time to achieve it.

There are also technology and economic concerns. Fundamentally, IEEE1588 is a standard that accurately synchronizes clocks. The accuracycurrently required by most applications can easily be met by IEEE 1588.However, accuracy requirements are likely to become more, rather thanless stringent.

It is therefore imperative that efforts to achieve low or evensub-nanosecond accuracy succeed. At this level, there are all sorts oftechnology problems, such as the PHYs, asymmetry in network links, andoscillator stability and noise, that must be addressed.

The solutions will probably require silicon, rather than FPGA orsoftware techniques. Again, cheap silicon requires volume, which isfurther reason to make sure the standard is no more complex thanneeded. Data published at the 2004 IEEE 1588 conference and more recentdata provide reason for optimism that the needed accuracies can beachieved.

Concern has been voiced that the MII or GMII interfaces between thePHY and MAC layers in Ethernet technology will be integrated intosilicon, and not be available for use by IEEE 1588. This is definitelyan issue that will be resolved based on the demand that can begenerated by the IEEE 1588 user community.

Improvements to IEEE 1588
In the short term, there are a number of improvements and extensions toIEEE 1588 that will emerge from the P1588 group. These include:

1) The transparent clock.The standard will definitely have some form of transparent clock thatwill make it feasible to construct reasonably long linear IEEE 1588topologies. Transparent clocks may also prove useful in tree topologiesinvolving large numbers of end devices as a way to reduce the number ofcascaded servos at the top layers of the hierarchy.

2) Short frames and shortsync interval. The combination of these will prove useful in achievinghigher accuracy, in providing advantageous cost optimization ofoscillator quality, and in enabling several proposed telecommunicationsapplications.

3) Layer 2 mapping. A layer2 implementation of IEEE 1588 has been requested by almost allapplication areas. It holds the potential of enabling easiersilicon-based solutions, and more efficient switch technology.

4) Gigabit implementations.Several of these have been reported to date, and more are sure toemerge. Implementations on fiber optic media will also appear,particularly if IEEE 1588 finds adoption in the power industry.

5) Wireless implementations.Although not on the current agenda for the P1588 group, it is quitelikely that serious efforts will be made over the next few years toimplement IEEE 1588 on one or more of the wireless protocols.

6) The emergence of siliconincorporating IEEE 1588. This will be driven by increasing adoption ofIEEE 1588, and broadening of the experience base to guide the design ofchips that incorporate appropriate support for applications using IEEE1588. Intel was one of the first to announce such a chip, and more arelikely to follow.

Regarding applications, the next few years should see significantnumbers of products and installations in the industrial automationarea. These will occur initially in motion control, but willsubsequently spread to monitoring and general control situations aswell. In test and measurement, the adoption in the data acquisitionmarket is very likely. Adoption in high-end test depends on the successof the LXI effort.

In telecommunications, there will be continued field trials. Ifthese prove successful, then the adoption depends on the willingness ofthe major equipment suppliers to provide the necessary devices, and theidentification of service areas where there is a clear reason to switchtechnologies. The early results reviewed earlier are encouraging, butnot yet definitive.

Final Thoughts
IEEE 1588 provides another tool to the designer of hard real-timesystems. It will be used in conjunction with the existing practices, asappropriate.

What IEEE 1588 has introduced is the ability to actually specify andexecute operations based on time to an accuracy that is appropriate forthe kinds of hard real-time problems discussed here.

It has been commented that hard real-time programming is difficultin part because computer science has abstracted away the notion oftime. Perhaps the introduction of IEEE 1588 will prove to be the neededincentive to address this issue. If so, the next few years should proveexciting.

To read Part 1, go to “The varieties of system temporalspecifications.”
To read Part 2, go to Overview ofthe 1588 clock synchronization standard.”
To read Part 3, go to “Master-slave Synchronization Hierarchy”
To read Part 4, go to  “Achievingsubmicrosecond synchronization accuracy
To read Part 5, go to “Applying1588 to wideband nets, wireless, cable and telecom.”

Used with permission of its publisher, Springer Science andBusiness Media, this series of articles is based on material from “Measurement,Control and Communication Using IEEE 1588,” by John C. Eidson andcan be purchased on line.

John C. Eidson, Ph.D., received a B.S. and an M.S. from MichiganState University and a Ph.D. from Stanford University, all inelectrical engineering. He held a postdoctoral position at Stanford fortwo years, spent six years with the Central Research Laboratory ofVarian Associates, and joined the Central Research Laboratories ofHewlett-Packard in 1972. When HP split in 1999, he transferred to theCentral Research Laboratory of AgilentTechnologies. Dr. Eidson was heavily involved in IEEE 1451.2 andIEEE 1451.1 and is the chairperson of the IEEE 1588 standards committeeand a life fellow of the IEEE.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.