Basics of real-time measurement, control, and communication using IEEE 1588: Part 3 - Embedded.com

Basics of real-time measurement, control, and communication using IEEE 1588: Part 3

All clocks in IEEE 1588 systems are organized into a master-slavehierarchy. In operation, the first task of the PTP protocol is tologically configure network communications used by IEEE 1588 into atree structure supporting this hierarchy

At the root of this hierarchy is a clock that the standard terms the”grandmaster clock”. The time scale of the entire IEEE 1588-enabled system will betraceable to the time scale of the grandmaster clock. PTP thus produces a common senseoftime throughout the system. If the time scale of the grandmaster clockis UTC, then the entire system will have a UTC time scale. Otherwise,the time scale is relative to the epoch established by the grandmaster.

Figure3.2. General network topology

Each boundary clock creates a branch point in the hierarchy tree,while the leaves of the tree are formed by ordinary clocks. Networkdevices that are not boundary clocks do not establish branch points inthe hierarchy, and except for performance degradation, are invisible tothe protocol. Figure 3.2 above illustrates several topological possibilities.

The PTP master-slave hierarchy is established on top of whateverunderlying communication topology is present. Thus, in Figure 3.2, ifthe network connections represented by the lettered lines are allpresent, then PTP will seek to logically partition the PTP messagetraffic in such a way as to produce a tree structure supporting themaster-slave hierarchy.

Many underlying communication systems run a minimum spanning tree orsimilar protocol, and will present a tree-structured communicationtopology to the PTP algorithms. The spanning tree algorithm used inEthernet systems is defined in IEEE 802.1D. A very complete discussionof spanning tree algorithms can be found in Interconnection:Bridges and Routers by Radia Perlman 

An example of such a spanning tree structure is illustrated inFigure 3.2, if the segmented lines G, H, and J are removed. However,the PTP protocol assumes that in general the underlying communicationis not tree-structured, and will present multiple paths between PTPnodes. Network devices such as an ordinary Ethernet switch notimplementing IEEE 1588, but multicasting packets to all ports areinvisible to the protocol. Thus, the network of Figure 3.2 wouldactually appear to PTP as illustrated in Figure 3.3 below.

Figure3.3. Topology of Figure 3.2 as it appears to PTP

The link G of Figure 3.2 would not be visible to PTP, exceptpossibly by observing fluctuations in latency due to the selection ofthe path between nodes, for example, between nodes 6 and 1. PTPfunctionality in nodes 6 and 3 would see nodes 1, 2, 11, and 12 asthough they were on a multi-drop link, or connected via a repeaterrather than a switch.

The role of the PTP boundary clock
PTP uses a device called a boundary clock to logically segment anetwork into a minimum spanning tree, thus reducing cyclic graphs, asshown in Figure 3.3 above , toacyclic graphs as illustrated in Figure3.4 below . This is accomplished by removing one or morecommunication links, in this case links C and E.

This network segmentation is implemented in PTP by means of analgorithm called the best master clock algorithm. This is a modifiedspanning tree algorithm, and operates based on information contained inthe Sync messages.

Figure3.4. Topology of Figure 3.3 pruned by the boundary clocks

In Figure 3.4, clock-5 is presumed to be the best clock in thesystem. The best clock in the system is termed the grandmaster clock inIEEE 1588. PTP places the grandmaster clock, in this case clock-5, atthe root of the minimum spanning tree. Branch points in the tree occuronly at boundary clocks. The actual PTP topology is determined by thestate of each clock port in the system.

PTP defines a number of states for each port of a clock. Ordinaryclocks have a single port, while boundary clocks have multiple ports.The states relevant to this discussion are master, slave, and passive,indicated in Figures 3.4 above and 3.5 below by M, S, and P in theboxes representing the various clocks.

In the example of Figure 3.4, all four ports of the grandmasterclock, clock- 5, are in the master state. The single port of eachordinary clock directly connected to clock-5 is in the slave state, forexample, clock-9 connected by path L.

The PTP spanning tree algorithm then prunes the network so that thenumber of boundary clocks between any ordinary clock and thegrandmaster is reduced to a minimum. This is done in a completelydistributed manner by the PTP algorithm executing independently on theremaining boundary clocks, i.e., clock-6 and clock-3 in this example.

These boundary clocks place one or more of their ports in thepassive state, with the single port closest to the grandmaster in theslave state. Synchronization takes place only between pairs of clockports in which one port is in the master state and the other is in theslave state.

The passive state is used by PTP to create the spanning tree. Thus,in Figure 3.4, the ports of clock-3 connected to paths C and E are inthe passive state.

Figure3.5. Topology of Figure 3.3 with two grandmaster clocks

The spanning tree algorithm works differently if there are two ormore clocks in the system that are determined to be equivalent and, inaddition, are accurately traceable to UTC. Figure 3.5 above illustrates thistopology.

In this example, clock-8 and clock-12 are both designated asgrandmaster clocks. The algorithm will then logically segment thenetwork into disjoint subnetworks by subnetworks will contain exactlyone grandmaster clock. In this example, both clock-6 and clock-3 haveports in the passive state, effectively removing paths E, D, and H fromthe network seen by PTP.

The internal architecture of a boundary clock is illustrated in Figure 3.6 below . The figure shows athree-port boundary clock. Each port is effectively serviced by amultiplexer that separates PTP traffic from all other network traffic.

Figure3.6. Internal view of a boundary clock

Non- PTP traffic is directed to the usual switch or routerfunctionality, while PTP traffic is directed to the block labeled “IEEE1588 Functions” that implement the PTP boundary clock functions. Thisblock maintains a separate section for each port, to allow each port tobe in a different state.

Generally, there is a single clock in the PTP block serving allports. If one of the ports is in the slave state, then this clock willsynchronize to a master clock connected to that port. If the clock inthe boundary clock is selected as the grandmaster, then clocksconnected to any of its ports will generally synchronize to theinternal clock of the boundary clock.

The exception to this is if one of the boundary clock ports, or theport of a clock connected to a boundary clock port is in the passivestate for the purpose of segmenting the network, as shown in Figure 3.5earlier, where

1) Clock-11 synchronizes toclock-12,
2) Clock-6 (the internal clockof boundary clock-6) synchronizes to clock-12 via the port on clock-6that is in the slave state,
3) Clock-10 synchronizes tothe internal clock of boundary clock-6 via a port on clock-6 that is inthe master state, and
4) Clock-6 does notsynchronize to clock-5 because the port on clock-6 servicing link-H isin the passive state.

One of the key features of a boundary clock is that it does notforward Sync, Delay Req, Follow Up, or Delay Resp messages. Instead,each port of a boundary clock intercepts these four message types.

To devices connected to each of these ports, the ports appear as anordinary clock. However, the timestamps issued by the ports are basedon the internal clock of the boundary clock, thus creating the clockhierarchy.

A boundary clock will forward management messages so that a complexnetwork of clocks may be managed using the multicast communicationmodel. This forwarding is limited to a finite number of boundary clockretransmissions by a field in the management message. This feature isprovided to contain the management messages within the IEEE 1588system.

Figure3.7. Simple network of clocks

Startup and Reconfiguration
One of the objectives of IEEE 1588 is to enable administration-freesystems. This means that there must be some automatic mechanism tohandle occurrences such a powe rup of the entire system or of a singleclock, the appearance or disappearance of a clock in the system, achange in the capabilities of a clock in the system, or a change in theunderlying communication topology.

Furthermore, this must be done in an autonomous and completelydistributed manner, and in such a way as to minimize any disruption toan operating IEEE 1588 system.

The mechanism for achieving this is the best master clock algorithm,coupled with timing constraints on state changes within each clock. Thebest master clock algorithm is based on the ability of pairs of clocksto decide which clock is better, based on information contained in Syncmessages. The timing constraints derive from the rate at which Syncmessages are sent, and rules on when they are to be processed byrecipients.

These rules are embodied in the PTP state machine. The combinedeffect of the best master clock algorithm, timing constraints, andstate machine rules on startup and reconfiguration is best understoodby considering several operational cases based on the system,illustrated in Figure 3.7 above ,and the greatly simplified version of the PTP state machine in Figure 3.8 below .

Figure 3.7 shows a boundary clock-1, and three ordinary clocks 2, 3,and 4, each with a port connected to the multicast network, net-A. Eachport executes an independent copy of the entire PTP protocol, includingthe state machine.

In all cases, the descriptions convey the general outline and resultof the operation of PTP. The exact details are considerably morecomplex.

Figure3.8. Simplified PTP state machine

Case 1
In this case, all clocks are powered down and a single clock, clock-2,powers up. Clock-2 enters the listening state via transition E of thestate machine in clock-2. Clock-2 listens for Sync messages from otherclocks for a protocol parameter-determined wait interval.

In this case, clock-2 is the only active clock, and at the end ofthis interval will transition to the master state via transition A andwill then begin transmitting Sync messages at a rate defined by thesynchronization interval. The synchronization interval is set by aparameter termed the “sync interval”. The default value of the syncinterval is 2 seconds.

The wait interval is named the PTP SYNC RECEIPT TIMEOUT interval,and is itself a times 10 multiple of the sync interval. Thus, in thisexample, clock-2 will wait 20 s before the transition to the masterstate. The length of the startup interval is a compromise between rapidresponse and thrashing due to missing or corrupt Sync messages.

Case 2
In this case, clock-2 is in the master state, transmitting Syncmessages when clock-3 is powered up and enters the listening state. If,after receiving at least two Sync messages from clock-2, clock-3determines (using the best master clock algorithm) that it is inferiorto clock-2, then clock-3 transitions into the slave state viatransition B and proceeds to synchronize to clock-2.

On the other hand, if clock-3 determines that it is better thanclock-2, it transitions into the master state via transition A and willthen start to transmit Sync messages.

When clock-2 has received two of these messages, it will determine(using the best master clock algorithm) that it is inferior to clock-3,and will transition into the slave state via transition D and proceedto synchronize to clock-3.

Case 3
In this case, all clocks have been on for some time, with clock-4 themaster and all others slaves. What happens if clock-4 is turned off ordisconnected?

This will result in the cessation of Sync messages being received bythe other clocks. The remaining clocks 1, 2, and 3 do nothing duringthe wait interval, PTP SYNC RECEIPT TIMEOUT, and then transition to themaster state via transition C and start transmitting Sync messages.This will result in each of the three clocks receiving Sync messagesfrom two other clocks.

Each clock examines the Sync messages from the other two clocks, anddetermines (using the best master clock algorithm) whether it is abetter or inferior clock. If it finds that it is inferior to any of theother clocks, it transitions to the slave state and proceeds tosynchronize to that clock. Since this process occurs in all clocks,eventually only a single clock will be left in the master state, andall others will be in the slave state.

Case 4
This is identical to Case 3, except that instead of clock-4 beingturned off, the network connection between clock-2 and clock-3 issevered. Since clock-3 is still receiving Sync messages from clock-4(its master), no change occurs in either clock-3 or clock-4. Clock-1and clock-2 cease receiving Sync messages from clock-4 due to the breakin the network. After the wait interval, they both transition into themaster state, and start to transmit Sync messages.

As in previous cases, one of them will determine that it is inferiorbased on the received Sync messages, and will transition into the slavestate. The net result is that after the break in the network and thepassage of enough time for all of the transitions to occur, there willbe two disjoint networks, each with a master clock.

These examples convey the general outline of the interaction of thebest master clock algorithm, timing constraints, and the PTP statemachine in a simple system either with no boundary clocks or when theboundary clocks have a connection to only a single port.

The situation is considerably more complex in networks with multipleboundary clocks, or with active connections to the ports of a singleboundary clock (for example, Figure 3.4). Recall that a boundary clocksegments a complex network. The boundary clock treats each portseparately by transmitting PTP event and general messages based on thestate of the port. Synchronization of the several ports of a boundaryclock is done internally.

For example, in Figure 3.4 links D, F, and H all operateindependently, as discussed in the four cases, with the ports on theboundary clock appearing to other ports on the same link as an ordinaryclock. Understanding the internal operation of the boundary clock indetermining its state, and therefore whether it transmits or receivesSync messages on a given port, requires the detailed analysis of thebest master clock algorithm and PTP state machine.

Next in Part 4:  How PTP'ssynchronization mechanisms work.
To read Part 1, go to “The varieties of system temporalspecifications.”
To read Part 2, go to Overview ofthe 1588 clock synchronization standard.”

Used with permission of its publisher, Springer Science andBusiness Media, this series of articles is based on material from “Measurement,Control and Communication Using IEEE 1588,” by John C. Eidson andcan be purchased on line.

John C. Eidson, Ph.D., received a B.S. and an M.S. from MichiganState University and a Ph.D. from Stanford University, all inelectrical engineering. He held a postdoctoral position at Stanford fortwo years, spent six years with the Central Research Laboratory ofVarian Associates, and joined the Central Research Laboratories ofHewlett-Packard in 1972. When HP split in 1999, he transferred to theCentral Research Laboratory of AgilentTechnologies. Dr. Eidson was heavily involved in IEEE 1451.2 andIEEE 1451.1 and is the chairperson of the IEEE 1588 standards committeeand a life fellow of the IEEE.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.