Traditional PCI Express (PCIe) systems have relied only on unicast packets. This made sense as there was only one host at any given time and most of the communication was host-centric, i.e. the host communicated with all the endpoints one at a time. Few applications had peer-to-peer communication, but this communication was also routed through the host due to limitations in some vendors' PCIe switches.
Lately, however, systems have been developed that use multiple hosts and have multiple endpoints requiring simultaneous communication. Additionally, as PCIe technology is being used in telecommunications and embedded systems, along with the graphics, server and storage markets, the need grew for communication with more than one endpoint at the same time. In these applications, multicast is extremely useful.
Multicast allows each host to communicate to all endpoints simultaneously, eliminating the need for the host to write a unicast packet multiple times to each endpoint.
Multicast improves the efficiency of the processor by enabling it to focus on computational and other tasks it was designed to do. Some of the applications that can use multicast feature to improve their system performance are:
1. When the CPU needs to download the same configuration information for multiple endpoints
2. When the CPU needs to send the same command packet to multiple destinations
3. When the CPU needs to update all the endpoints with the same status
The PCI-SIG (PCI Special Interest Group), the standard-setting body for the PCI Express interface, ratified a multicast spec to enable component vendors and systems integrators to follow a defined implementation of the multicast feature, eliminating any proprietary nature of implementation.
Let's look at some of the multicast usage models in the major markets and the advantages it brings to system designers.
Multicast in telecommunications systems is a mandatory feature due to the sheer need of the processor to communicate with several end-points at the same time ” whether it is communicating configuration information, status information or updates (Figure 1 below ).
|Figure 1: Multicast in telecommunication systems|
The usage model in Figure 1 illustrates that not only is the CPU communicating (via multicast) with the NPUs but the AMC processor is also communicating with the RegEx and the FPGA at the same time. This shows the ability of the PCIe switch to have any port capable as the source port for a packet that needs to be multicasted.
For example, a PCIe switch with 24 ports (the highest number of ports on the market, from PLX) can have any of those ports capable of being the source port. So, for systems with multiple roots and multiple endpoints connected to each root, designers could create multicast packets within each root.
This brings to attention another feature needed in PCIe switches for telecommunication systems: a large number of ports. Typical telecom systems need to connect to a large number of endpoints ” such as NPUs, FPGAs, ASICs and RegEx points — each with its own bandwidth requirements, with some needing x8 ports, others x4 ports, and still some only x1 ports.
So, for the telecom market, a PCIe switch must have the ability to support multiple port and width configurations.
PCIe Multicast for Graphics
For dual-graphics applications, the processor needs to communicate with two graphics devices at the same time. Although this can be considered a typical dual-cast application, the issue arises when vendors of quad-graphics systems need to communicate with four of these graphics devices.
Multicast is absolutely essential in this case, as it saves the processor cycles from writing the same packet to these quad-graphics devices and, thus, increases its efficiency. Along with multicast, another important feature that a PCIe switch needs to support in this case is multiple upstream ports.
Due to the bandwidth-intensive nature of this application, a PCIe switch needs to be able to support at least two upstream ports, along with multicast and a x16 wide port (Figure 2 below ).
|Figure 2: Multicast in Ultra Gaming Machine|
In this usage model, the PCIe switch is supporting three things:
1. x16 wide port for bandwidth-intensive graphics applications
2. two upstream ports
In the ultra-gaming market, designers expect all three features from a PCIe switch.
Storage with PCIe Multicast
In the storage market, for redundancy implementations to preserve data at multiple locations or and/or multiple drives, it is important that a PCIe switch support multicast, as well as multiple upstream ports and failover ability (Figure 3 below ).
|Figure 3: Multicast in a storage system|
In the above figure, the PCIe switch offers:
1. multiple upstream ports
2. the ability to multicast within each partition of the switch
3. the ability to provide failover support for the endpoints connected to any CPU (in case that CPU fails)
In addition to these features, the PCIe switch should have the ability to support wider ports because, unlike in a telecommunication system, a storage system is bandwidth-intensive and needs multiple x8 wide ports. In a typical usage model, designers may need 12 x8 wide ports, equating to a 96-lane switch — the highest density switch on the market, available from PLX.
Impact of PCIe Multicoat on embedded systems
The embedded-systems market, because of its widely varying applications, poses a special opportunity for multicast. Whether it's a medical system or an instrumentation system or a multi-function printer, etc, multicast plays a key role in performance. A generic embedded usage model is shown in Figure 4 below .
|Figure 4: Multicast in an embedded system|
In this usage model, the master CPU could communicate with all its endpoints via multicast. While this usage model shows an ASIC and a few CPUs, due to the variety of usage models within the embedded market, the endpoints could be FPGAs, hard drives, Scan ASICs, or any commercially available endpoints from other vendors.
All the above usage models show how PCIe switches supporting multicast, along with other important features, are essential to multiple market segments. Without multicast, the performance of these systems would be adversely affected.
Since the PCI-SIG standardized multicast within the PCI Express Base Specification, PCIe technology usage has increased in many of these market segments, while multicast has become a standard feature in all PCIe switches.
(Krishna Mallampati is product marketing director at PLX Technology. He can be reached at KMallampati@plxtech.com . )