Achieving scalability with switch fabrics in CompactPCI -

Achieving scalability with switch fabrics in CompactPCI


The requirements of applications for which embedded systems are being designed vary in complexity, cost and performance widely. Presenting extreme challenges for designers of systems that must support not only a wide range of applications, but a continuously changing set of performance requirements.

Whether it's ever increasing bandwidth needs or changing communications standards, protocol based interconnects and switch fabrics offer unique benefits to addressing these issues.

CompactPCI has a rich history in embedded design; a logical step to extending its capabilities is adding a switch fabric architecture to your current product line.

Using the same interconnect across your entire design can leverage your knowledge of the protocol and its interfaces. In fact there are numerous benefits to choosing a single system-wide interconnect included: controlling development costs, design simplification, quick debug and system verification.

Scaling from Interconnect to Switch Fabrics
Many applications require complex switch fabric architectures but let's start by looking at scalability and where it starts, as a simple interconnect. When choosing an interconnect between two devices it is important to consider that the interface not only meets your current requirements, but is extendable to match you future needs.

An interconnect that can also be used throughout your system at multiple bandwidths and cost points is also important. RapidIO technology was designed specifically as a widely applicable, flexible, extensible system interconnect for embedded infrastructure equipment including networking, storage, and communication systems.

RapidIO can be used as a simple chip-to-chip interconnect, using 8 or 16 bit parallel interfaces, or 1X to 4X serial interfaces at speeds ranging from 1.25 Gbps to 3.25 Gbps per link. Future interfaces are planned for speed including 5/6.25Gbps per link.

Parallel RapidIO has an extremely low latency connection that has high bandwidth and is ideal for onboard processor interconnection. Serial RapidIO has the advantage of low pin count and a multiple width and speed options which can be matched with the bandwidth requirements of each link.

In both cases whether choosing parallel or serial RapidIO, only the physical layer of the protocol is changed. All the other aspects of the interconnection stay the same requiring no software changes to the system.

The fact that multiple speeds and widths can work together seamlessly in a system positions RapidIO well for a flexibility that enables upgrades and system longevity.

The RapidIO protocol can also be transmitted over anything from serial to parallel interfaces, from copper to fiber media. This enables further flexibility to develop innovative solutions to a broad base of applications.

The use of RapidIO as a point-to-point interconnect does not require the implementation of device addressing portion of the protocol. The transport layer specification of RapidIO defines the device addressing models for the technology.

The transport specification is independent of any RapidIO physical or logical layer specifications. When developing more complex systems, where thousands of devices are in a system; the transport layer is key to flexibility and redundancy in the system.

RapidIO can support virtually any system topology. The device ID's do not convey any intrinsic information about where they are located in the system. It is the responsibility of the interconnection fabric to discover where the devices are located and to forward packets to them based on target device ID only.

The RapidIO network learns during the system discovery phase of a system to bring up where devices are located. Switches are programmed to understand the direction, although not the exact location of all devices.

When device locations change, as might occur during hot swap or failover situations, only the switches need to be reconfigured to understand the new system topology. These advantages of RapidIO fabrics provide virtually unlimited flexibility, scalability and many ways to design redundancy into a system.

CompactPCI Serial RapidIO
The CompactPCI Serial RapidIO (PICMG 2.18) specification adds serial RapidIO as an alternative board-to-board interface for the standard CompactPCI platform. This provides a significant improvement in bandwidth compared with traditional CompactPCI interfaces: PCI, H.110 and Ethernet.

These traditional interfaces are not required, but are also not precluded. So, a heterogeneous CompactPCI system, supporting both new high bandwidth as well as legacy, can be easily constructed. The PICMG 2.18 standard chose to use the existing 2mm CompactPCI connector. This both ensures a level of mechanical compatibly and keeps the connector costs reasonable.

After performing careful signal integrity simulations it has been determined that 1.25Gbps is the maximum reliable speed for a serial RapidIO differential pair on this platform. Each board can have up to four bidirectional 4X serial RapidIO links running at 1.25Gbps for a total of 40Gbps of aggregate bandwidth between the board and backplane.

For Serial RapidIO technology, two transmitter specifications allow for solutions ranging from simple board-to-board interconnect to driving through two separate connectors and across a backplane.

The two transmitter specifications were designated: a short run transmitter and a long run transmitter. The short run transmitter is used mainly for chip-to-chip connections either on the same printed circuit board or across a single connector such as that for a mezzanine card.

The minimum swings of the short run specification reduce the overall power used by the transceivers. A user can further reduce the power by lowering the termination voltages. The long run transmitter uses larger “voltage swings” that are capable of driving across backplanes.

This allows a user to drive signals across two connectors and common printed circuit board material. To ensure interoperability between drivers and receivers of different vendors and technologies, AC coupling must be used at the receiver input.

Why RapidIO?
RapidIO technology has become the de-facto interconnect and fabric standard for embedded systems being deployed today, providing increased performance, improve efficiency, and lower cost.

RapidIO technology is supported by a broad ecosystem of leading vendors with multiple vendors shipping production switches, DSP's, processors, endpoints, FPGAs, boards, software and systems.

Serial RapidIO technology offers a higher speed physical layer that can be configured to match bandwidth requirements with different speed variants and numbers of lanes.

It builds on the communication industry's common roadmap at the Serial physical layer, using a variant of IEEE 802.3 XAUI today for 3.215Gbps and a variant of the OIF CEI work on 5 and 6 Gbps in the future.

The RapidIO fabric provides a robust packet-switched system-level interconnect. RapidIO can be used to develop designs that are both affordable and scalable for future performance upgrades.

The RapidIO technology provides a partitioned architecture that can be enhanced in the future. It enables higher levels of system performance while maintaining or reducing implementation costs. A RapidIO end point can be implemented in a small silicon footprint.

Proven industry-standard signaling schemes (LVDS, XAUI) are used for the physical interfaces. Error management includes the ability to detect multi-bit errors and survive most multi-bit and all single bit errors.

Even with all these capabilities, the RapidIO protocol overhead and latency are comparable to current bus technologies and significantly better than local area network based fabric technologies such as Ethernet.

Tom Cox is Executive Director of the RapidIO Trade Association which provides a wealth of information including public access to specifications, whitepapers, presentations, and how to become a member at

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.