Using an interface wrapper module to simplify implementing PCIe on FPGAs
This "Product How-To" article focuses how to use a certain product in an embedded system and is written by a company representative.
Many end-applications today use an FPGA-based design as an inherent component of their solution. They often require PCI Express (PCIe) as an indispensible feature, to provide a standardized interface with other components in the system.
Historically, PCI Express has been difficult to implement in FPGA because it requires multi-gigabit SerDes and analog circuitry with stringent electrical requirements.
Additionally, PCI Express implementations requires complex digital logic including Physical, Data Link and Transaction layers with large data paths running at high frequency, thus making it difficult to implement in FPGA.
The most common methods used for implementing PCI Express in FPGAs include:
* ASSP/PCI Express Bridge chip
* FPGA with digital controller soft-IP and built-in SerDes/PHY
* FPGA with digital controller soft-IP and external discrete PHY chip
* FPGA with built-in PCI Express hard-IP
Each solution has its pros and cons and this paper will explore the different approaches to help pinpoint the best solution for an application.
Figure 1 - ASSP/PCIe Bridge Chip
ASSP/PCI Express Bridge chip
In this implementation, a bridge chip (ASSP) is typically used with a companion FPGA/CPLD (Figure 1 above). This solution has the advantage of typically being able to provide full PCI-E electrical compliance. However, some of the disadvantages can be problematic to implementation:
* Typically a two-chip solution, increasing BOM and manufacturing costs.
* Over time, ASSPs may become obsolete
* A generic and often proprietary user interface that may not be easy to work with or well suited for high performance applications.
* No upgrade path to higher-end PCI-E implementations as existing ASSPs only support x1 and x4 link in PCI-E 1.1 (Gen1) mode.
* Existing ASSPs are limited to endpoint-only designs.
* Designers must work with the ASSP feature set and errata, which may limit functionality.
|Figure 2 - FPGA soft IP and external PHY|
FPGA/digital controller soft IP & external discrete PHY
This solution is a derivative of the previous solution, but utilizes a lower-cost FPGA with no built-in transceiver. (Figure 2 above). This approach has the following advantages:
* Provides a lower cost at medium and high volume.
* Enables full PCI-E electrical compliance.
* Has no limitation to endpoint-only designs.
The solution, however, does have some drawbacks including:
* Upfront IP license adds significant costs for projects with very low volumes.
* A percentage of the FPGA real estate is consumed by the soft IP, resulting in less logic resources available to the user.
* A percentage of the FPGA I/Os is consumed for interfacing to the PHY chip (typically through a PIPE parallel interface). The PIPE interface may require a faster speed grade/higher density FPGA to accommodate the number of I/O and the PIPE frequency (250MHz in some cases).
* No upgrade path to x8 or PCI Express 2.0 (Gen2) as existing PHY chips only support the PCI Express 1.1 specification in x1 and x4.
|Figure 3 - FPGA soft IP and built-in SerDes/PHY|
FPGA with digital controller soft-IP and built-in SerDes/PHY
This solution requires FPGA with built-in multi-gigabit transceivers (Figure 3, above). The incorporation of a digital PCI Express controller provides the following benefits:
* Single-chip solution, reducing BOM and manufacturing cost.
* Easy upgrade to x4, x8, and potentially PCI Express 2.0 (Gen2), depending on FPGA transceiver capability.
* No limitation to endpoint-only designs.
However the solution exposes the following problems:
* Upfront IP license adds significant costs for projects with very low volumes.
* A percentage of the FPGA real estate is consumed by the soft IP resulting in less logic resources available to the user.
* Potential PHY limitations that may limit support for some low power states and some signaling mechanisms such as Beacon generation.
|Figure 4 - FPGA with Built-in PCI Express IP|
FPGA with built-in PCI Express hard IP
One solution that is emerging as a better approach is using hard IP, with SerDes/PHY and digital layers (MAC, Data Link, Transaction) hardened on FPGA silicon (Figure 4 above). This has some distinct advantages:
* Provides a single-chip solution, reducing BOM and simplifying design and test.
* Ensures a seamless upgrade to x4, x8 and PCI-E 2.0 (Gen2) depending on the capabilities of the hard IP.
* The entire FPGA real estate is available to the user.
However, using hard IP for PCI Express also exposes some issues, including:
* Until recently, it required the use of highest-end FPGAs, however hard IP for PCI Express is now supported in lower-cost FPGA families.
* Designers must work with the existing feature set, limitations, and errata, with no possible customization.
* The available fabric interface includes several hundreds of signals and is typically difficult to work with.
* Design migration and/or vendor migration is difficult due to the proprietary nature of each vendor's hard IP and fabric interface.
Solving Issues with an easy-to-use fabric interface
Many of the issues inherent in implementing an FPGA with PCI Express hard IP can be solved using an interface wrapper to provide a simple and robust user interface.
PLDA's EZDMA module, for example, is designed to wrap around the FPGA's PCI Express hard IP, hiding the complexity and limitations of hard IP fabric interfaces, and often working around hard IP functional issues (Figure 5, below) .
It is designed for those with little or no experience with PCI Express but also for experienced designers looking for an easy-to-use yet robust PCI Express interfacing solution. The EZDMA module provides designers with shorter design cycles by significantly reducing the learning curve associated with using PCI Express hard IPs fabric interface.
|Figure 5. EZDMA Interface.|
EZDMA also enables easy migration from PLDA's PCI Express soft IP, allowing designers the flexibility to choose the best in class solutions without incurring additional costly design time.
Additionally, PLDA incorporates the EZDMA interface in its PCI and PCI-X architectures, enabling seamless transitions from these legacy interfaces and a clear upgrade path.
The EZDMA approach allows the use of not only high-end FPGAs but also low cost FPGAs from leading providers such as Altera and Xilinx, making a hard IP solution affordable for many types of applications. Supported devices include:
* Altera Stratix IV GX, Arria II GX
* Xilinx Virtex-5 LXT, FXT, TXT, Virtex-6 LXT, SXT, Spartan-6 LXT
The EZDMA module utilizes approximately 2K to 5K LUT and from 4KB of memory depending on the specific configuration. Additional features of the PLDA EZDMA interface include:
* Familiar Master (DMA)/Target type interface, adding easy-to-use high-performance multi-channel DMA
* Scatter-gather (DMA chaining) with support for a number of DMA channels
* A memory-mapped "Slave" interface ideal for register, memory, and I/O access
* Fully configurable for FPGA resource optimization
* Support of low power modes
* Hardware-proven IP, deployed in 400+ designs
Additionally, the EZDMA module can be used seamlessly when migrating FPGA designs to ASIC and Structured ASIC.
Choosing the right FPGA - PCI Express solution
While there are several options available for PCI Express interfacing with FPGAs, the approach of hardening complete PCI Express controller IP inside today's FPGAs is rapidly gaining acceptance as a preferred method to minimize risk and reduce design time in a wide variety of applications.
Because of these advantages, this approach looks to become the mainstream solution for tomorrow's FPGAs, helping to deliver a balance of functionality, flexibility and performance.
Co-founder and CTO of PLDA Stphane Hauradou earned a Bachelor of Engineering from the Polytechnic School of Montreal and a Masters in Microelectronics from ENST in Paris. His master's thesis concentrated on the development of the first PCI controller IP for Programmable Logic Devices..