Ethernet, while extraordinarily successful as a computer network, hasnot been widely adopted for real-time streaming of audio or video.There are two primary reasons for this. First, Ethernet does notprovide the kind of isochronous and deterministic low-latency services1required by these applications. Second, the market was not large enoughto justify the work necessary.
As a result, other technologies are currently used — either IEEE1394 (FireWire, iLink, etc.) for the local cluster, or a variety ofpoint-to-point or proprietary systems for longer distances.
Times have changed, however, and there is a growing awareness ofthe need for a network that can distribute high quality digital audioand video. Most industry experts agree this must be a heterogeneousnetwork that supports both wired and wireless components. Oddly enough,the wireless networks (particularly the various religions ofultrawideband [UWB') have made strong efforts to provide these serviceseven though the wireless environment is much more challenging. Themissing link in this is the wired component.
Why Not Use Ethernet As Is?
It is actually possible to use Ethernet without any changes for A/Vnetworks, but at the cost of increased latency and requiring a numberof constraints on the network: 1. Limit the amount of traffic offeredby each device.
2. Limit the topology of the network (number of devices, number ofhops).
3. Require the network to have much higher bandwidth capabilities.
4. Limit the use of priority services to only A/V streams.
These limitations make it less likely that any particular queue onthe network will overflow and drop packets or add excessively delay.(Note that queues exist in the output ports of all devices, includingnetwork switches.) The problem with this methodology is that there isno way of enforcing the constraints. It would only take a single PCwith unusual but perfectly standards-compliant software to causeunacceptable delays and dropped packets. Similar problems could be theresult of “too many” devices or an unexpected configuration of switchesor hubs.
Perhaps Use 1394?
IEEE 1394 does provide all the services required by an A/V networksince these services were a primary design goal of the working group.The recent 1394b and 1394c efforts extended 1394 networks to 100m hops,just like Ethernet; and the 1394.1 specification allows networkscalability in a similar manner as 802.1D bridges. There is really onlyone difficulty in using 1394 as the backbone of an A/V network: 1394 isnot Ethernet.
For almost all existing computer applications, the existing andplanned wired network is (or will be) Ethernet. If a home already has awired digital network, it is almost surely Ethernet; very few 1394 homenetworks are yet deployed. If a consumer buys a computer, it almostsurely has an Ethernet port, while it only has a roughly 50% chance ofhaving a 1394 port.
So, although 1394 is a perfectly adequate (arguably, even superior)A/V interconnect, Ethernet's market leadership makes it unlikely for1394 to become the primary network in the home.
Perhaps Use Wireless?
The various 802 wireless networks all have some kind of”isochronous-like” services. Indeed, 802.15.3a and 802.11e havecompleted (or nearly completed) “protocol adaption layers” to allowthem to carry 1394 data. Perhaps these networks are the correct path?
Unfortunately, the quality of service for wireless is not adequatefor HD-quality video. The average data rate of the proposed 802.11n andUWB nets is OK (particularly at short ranges), but latency is excessive(10's of milliseconds) and normal home environments can result inmomentary packet loss. Wireless is an import part of the home A/Vnetwork, but it is not the solution for the backbone.
History Behind Residential Ethernet
In the past few years there have been a number of efforts to useEthernet or modifications of Ethernet to provide high quality A/Vnetworking. For example, the Cirrus Logic “CobraNet” has been used foraudio distribution in auditoriums, and the Gibson Guitar “MaGIC” hasbeen designed and demonstrated for use in live performances. CobraNetuses the “limited topology” concept described above, and MaGIC uses anon-standard bridge as the key bit of technology.
With the mandated HDTV switchover starting in 2004, and with therecent success of the digital audio market, it was clear that it wastime for a standardized A/V network. At the July 2004 meeting of theIEEE 802 working group there was a “Call For Interest” for aResidential Ethernet project supported by Pioneer, Nortel, GibsonGuitar, Samsung, Broadcom, NEC and others. At the 802.3 plenarymeeting, the Residential Ethernet Study Group was approved and giventhe task of setting the objectives for the project and providing thejustifications for future work.
The first meeting of the study group took place on September 30,2004, in Ottawa, Canada, and agreed on a set of objectives which werefurther refined in the November 2004 802.3 meetings in San Antonio,Texas:
1. All of the various automatic and self-configuring capabilities of802.3 will be required, not optional.
2. There will be a mechanism to request/assign resources forisochronous services (e.g. bandwidth, channel) and the default rule(s)for managing the resources.
3. Isochronous and best-effort traffic will be carried together, withsome bandwidth reserved for best-effort (“best-effort” is normalEthenet traffic).
4. Links will be 100-Mbit/s full duplex or greater to supportisochronous services (requires the use of switches to connect togethermore that two devices). All existing 802.3 physical layers that meetthis requirement are fully supported.
5. Isochronous traffic will have less than 2-milliseconds end-to-endlatency through the entire network and only 250 microseconds throughone hop.
6. Isochronous traffic will have very low jitter and wander approachingzero.
7. High quality synchronization services will provide all stations witha low jitter “house clock” to be used both for scheduling isochronouspackets and to provide a common time base for sampling and presentingstreaming data.
There are a few additional objectives for a Residential Ethernetsystem that will require work by other groups, either within 802 orother standards organizations (e.g., IEEE 1394, USB, MOCA):
1. Isochronous bridging to IEEE 1394, IEEE 802.11, and IEEE 802.15.3(perhaps even USB, MOCA, and even higher level protocols such as RTP).
2. Network-wide resource management.
3. Support for all 802.1 services, in particular 802.1Q VLANs.
4. Support arbitrary topologies within reasonable limits basicallyanything supported by 802.1D.
Following IEEE guidelines for “Study Groups”, the Residential EthernetStudy Group has focused on developing objectives and goals. However, aspart of this process, it is important to demonstrate technical andeconomic feasibility, requiring the creation of an example technicalapproach. The following architecture is derived from a series ofpresentations made by the author to the Residential Ethernet StudyGroup in September through November of 2004. As such, this design doesnot represent any official position of the Study Group, or any otherIEEE 802 organization.
There are three primary differences between the proposed ResidentialEthernet architecture and existing Ethernet (from now on the term”ResE” will be used instead of “Residential Ethernet”):
1. precise synchronization
2. scheduled priorities
3. admission controls
Although these are significant changes, they can all be implementedusing relatively small extensions to the standard Ethernet MAC (definedin IEEE 802.3) and bridges (defined in IEEE 802.1D). This”minimal-change” philosophy will allow standard Ethernet and ResEdevices to seamlessly communicate using standard Ethernet frames — butonly ResE devices will be able to send/receive/relay the newtiming-based packets or services as shown in Figure 1, below.
Figure1: Example of ResE connections
A few new terms are included in this paper:
Isochronous stream — A single stream of data to bedelivered with low latency and small jitter (e.g., MPEG transportstream, uncompressed audio or video, AAC or MP3 audio, etc.).
Talker — The original source of an isochronous stream.There is only a single talker for a stream, although a talker maysource multiple streams.
Listener — The final sink of an isochronous stream. Theremay be more than one listener to a stream, and a listener may bereceiving multiple streams.
ResE devices will periodically exchange timing information that willallow both ends of the link to synchronize their time-of-day clock veryprecisely. This precise synchronization has two purposes:
1. To allow isochronous packet scheduling (see below)
2. To provide a common time base for sampling data streams at a sourcedevice and presenting those streams at the destination device with thesame relative timing
The current expectation is that isochronous packet scheduling willbe done based on 8 kHz “cycles”. The 8 kHz rate is commonly used inmost current isochronous transports, whether the wide-area network,IEEE 1394, or even USB. The synchronization system guarantees that allstations agree on when cycles start and how they are numbered. Thecommon time base will be fine-grained than the cycle timer providing aclock similar to the 24.576 MHz version provided by IEEE 1394.
There is a single device within a ResE “cloud” that provides amaster timing signal. All other devices synchronize their clocks withthis master. Selection of the master is largely arbitrary (all ResEdevices will be master-capable), but can be overridden if the networkis used in a professional environment that already has a “house clock”.
Ethernet devices exchange capability information during theautonegotiation phase of link establishment. ResE will add “isochronouscapable” and “network sync available” flags to the mix. If both devicesare isochronous capable they will start to exchange clocksynchronization information, with the device “more distant” from thenetwork timing master becoming a the link timing slave and adjustingits clock to match that of the device “closer” to the network timingmaster. Bridges will act as a link timing master on all ports exceptthe one that points towards the network timing master (unless thebridge is the master, in which case it will act as link timing masteron all ports).
As noted above, a key part of the proposed ResE architecture is therequirement for full duplex connections (this has been an option forEthernet connections since the middle '90s, and is already arequirement for gigabit links). This means that there are no packetcollisions and all mulitport devices function as bridges (frequentlycalled switches).
The Ethernet bridge specification already defines how to routepackets directly to their destination without flooding the network, andhow multiple priorities can be handled. The proposed architecture usesthese routing and priority mechanisms in a modified way to provideisochronous services. This method might be called “scheduledpriorities” and works as follows:
1. Each cycle, all isochronous packets assembled at a networkinterface for transmission are tagged with the next cycle number andplaced in a transmit queue.
2. During cycle “n”, packets are transmitted first if they are taggedwith cycle “n-1”, then “n”, then all best effort traffic (normal legacyEthernet packets). Packets tagged with the current cycle number mustwait until the next cycle. An example of traffic transmitted from aResE device is shown below in Figure 2
Figure2: An example transmitted traffic from a ResE device
In this example, there are two isochronous streams being sent (blueand gray), as well as bursts of normal best effort Ethernet traffic(green). Note that the isochronous data that is queued during cycle 0is not transmitted until cycle 1, and isochronous data that is queuedduring cycle 1 is not transmitted until cycle 2, and so on.
During cycle 2, two best effort packets are queued and, since thetransmitter is idle, the first of those is transmitted as soon aspossible (bypassing the isochronous data queued for the next cycle).
When cycle 3 starts, the asynchronous frame is still being sent, sothe queued isochronous data is transmitted as soon as the asynchronousframe is completed. In this case, the isochronous data for one of thestreams is delayed into cycle 4 — and this is the reason that there arethree levels of scheduled priorities: delayed isochronous data,isochronous data queued for the current cycle, and then best efforttraffic.
One final note about the example above: isochronous data is notordered with respect to other isochronous data that is queued for thesame cycle. The isochronous streams are sent in a different order incycle 6, and this is perfectly acceptable.
Isochronous packets are normal 802.3 frames, with normal restrictionson format and length. The only thing that is unique will be the use ofa special value of the “Length/Type” field. This special value willallow the receiving device (or a bridge) to appropriate route the datain hardware, greatly reducing the need for latency-increasing softwareto parse the packet. Isochronous packets have a number of specialpurpose fields (Figure 3 below) . These fields include:
da , the standard 802.3 destination address. Frequently(perhaps always) isochronous streams will be sent using multicastaddresses. This will allow multiple listeners to “tune” into the”channels”.
sa , the standard 802.3 source address. It uniquely identifiesthe talker, the original source device of the isochronous stream.
lengthType, the uniquevalue to identify isochonous packets (see above).
linkCycle, which has the cycle number that was used forqueuing the packet on this particular link (see above). It isregenerated on each hop through the network.
* talkerChannel, which is used to differentiate differentisochronous streams that are sent by the same source device.
* sy, which is a stream synchronization field, useful for bridging IEEE1394 isochronous data such as uncompressed video.
* talkerCycle, which is the cycle number that was used toqueue the packet in the original source device (this number isunchanged as it passes through bridges). * contentLength is the countof content bytes.
* content, which is the isochronous data itself. Note thatpadding is sometimes necessary to meet the minimum packet lengthspecified by 802.3.
* fcs, which is the standard 802.3 32-bit CRC of all thepreceding data.
Revisions or special fields may berequired as the standard develops.
Figure 3: Diagram showing theisochronous packet format.
The basic mechanism used for single-link isochronous transportis carried over to bridges. Isochronous packets will be routed justlike best-effort traffic in existing 802.1D bridges, with thedifference that their linkCycle will be incremented by two and theywill then be placed in the isochronous transmit queue for theappropriate output port. This will have the effect of setting the delayof isochronous packets through a ResE network to two cycles per hop,plus the jitter caused by traffic on the final hop to the destination.
Under the proposed ResE architecture, delivery jitter will notaccumulate. It will be bounded to a maximum just under two cycles. Thishas the very useful benefit of bounding the size of the output queuesneeded at all output ports on the network to two cycles, no matter howbig the network gets. As an example, use the topology shown in Figure 4 below :
In Figure 4, an isochronous stream is being sent from the talker tothe listener through three bridges. There are four links in the path asshown above. Figure 5, below illustrates how the traffic might look on all four links.
Note how it takes six cycles to get through the three bridges, andthat there is very little delivery jitter on the last link since thereis no interfering traffic.
Even though the preceding mechanism can reliably deliver datawith a deterministic low latency and low jitter, it will only do so ifthe network resources (bandwidth, in particular) is available along theentire path from the talker to the listener(s). In this architecture,it is the listener's responsibility to guarantee the path is availableand to reserve the resources. The process to do this is:
1. The listener sends a “join request” control packet to talker withamount of bandwidth needed (in bytes/cycle) as well as a channel numberto use.
2. Intermediate bridges make a tentative bandwidth reservation on theport going back to the listener, update a delay count and pass oncontrol packet toward talker. Note that if the bridge is alreadyrouting the stream, it can respond on it's own, acting as a proxy forthe talker.
3. If resources are not available (i.e. the port pointing back to thelistener will exceed its capacity if it accepts the reservation), thepacket is marked as “resources not available”, but still sent totalker.
4. When the packet reaches the talker, the talker returns a “joinresponse” packet towards the listener with status information (whichincludes resource available (or not), and delay).
5. Intermediate bridges receiving the “join response, resourcesavailable” packet confirm the resource allocation. If the packet is”join response, resources not available”, the tentatively assignedresources are released. In both cases the control packet is passed backtowards the listener without alteration.
6. When the listener receives a join response control packet, it willknow whether the resources are available, and if so, that the resourceshave been reserved and the delay for the path.
The world according to GARP
Although Ethernet does not currently support this kind of reservationprotocol, the overarching architecture, IEEE 802.1, does providesomething that provides a basic mechanism: the generic attributeregistration protocol (GARP). This is currently defined as a way tocontrol and monitor the multicast address registration system for802.1D bridges, but it appears that a small enhancement might providethe features needed for the ResE admission control protocol.
Obviously, various timeouts and disconnects affect the process, butthe basic ideas have been worked out already as part of GARP and IEEEStd 1394.1-2004, the FireWire bridging protocol, so there isn't muchinvention to do.
Additional listeners also send “join requests” to the talker, butthis time an intermediate bridge can respond if it is already routingthe stream. When a device no longer needs to be a listener of a stream,the network resources are reclaimed using the following process:
1. The disconnecting device sends a “leave request” control packettowards the talker.
2. If any intermediate bridge is also routing the stream towardsanother listener, it responds with a “leave response” on its ownwithout forwarding the request, and de-allocates the bandwidthresources on the port towards the leaving device.
3. If an intermediate bridge is not forwarding the stream to anotherlistener, it just forwards the request on towards the talker.
4. When a “leave request” reaches a talker, it discontinuestransmission of the stream (usually notifying a higher layer protocol),de-allocates the resources used on its output port, and sends a “leaveresponse” control back towards the disconnecting device.
5. If an intermediate bridge receives a “leave response”, itde-allocates the resources assigned to the stream on the port towardsthe disconnecting device, and forwards the “leave response packet”.
6. When the disconnecting device receives the “leave response” packet,it can then notify a higher layer protocol that the disconnection wassuccessful.
There are various other methods used to take down a connection andrelease the allocated resources. One expected method requires that thetalker must send one packet every cycle, even if it has no content.That way any receiving device (including intermediate bridges) couldautomatically release assigned resources and notify listener(s) if theappropriate isochronous packet was missing for “X” cycles, where “X” issome small number.
Although the proposed ResE system is more complex than existingEthernet, the complexity is confined to small changes in the MAC, andsome more significant changes to 802.1D bridges. Even for bridges, thechanges are not extensive and will not add enough gates to affect thecost of a bridge over the long term.
There are easy comparisons that validate this statement: forexample, a worst case client MAC will be no more complex than aFireWire open host controller interface (OHCI), which is typically150,000 gates including multiple buffers and powerful scripted DMAengine to perform RDMA-type transactions — but will usually be muchsimpler for non-computer CE-type applications.
For a bridge the primary changes will be the extra time-scheduledpriority queue (beyond the 4 to 8 typically implemented) and therequired management functions for admission control and clockdistribution.
802 standardization will likely proceed, with activity in both the802.3 (Ethernet) and 802.1D (bridges) task groups. Once the basicapproach is stabilized a 1394 protocol adaptation layer (PAL) can bedefined so that ResE could become the backbone for 1394 localinterconnects, and even replace 1394 in some devices without requiringextensive firmware changes.
The good news is that all the basic protocol mechanisms needed areavailable, and the invention is confined to choosing which ones arebest for the ResE application. Furthermore, the customer base for ResEalready exists: all those 1394/FireWire devices being built now for DTVand high-end audio. A simple ResE/1394 bridge would be a simple thingto build, and would carry forward all shipping and planned productstowards a first-class, no excuses home network with no additional cost.Now that's nice!
About the Author
Michael Johas Teener is currently an independent consulting engineer.Prior to that he was Plumbing Architect at Apple, a title that he alsoheld from 1988 until 1996 where he was the chief architect of AppleComputer's FireWire effort. Michael has a BS from Caltech, an MS fromUCLA, and holds over 16 patents including several that are fundamentalto the implementation of 1394 and 1394b. He can be reached at