Power-up phase determinism: Using multichip synchronization - Embedded.com

Power-up phase determinism: Using multichip synchronization

The integration of multiple digital signal processing (DSP) blocks, wideband digital-to-analog converters (DACs), and wideband analog-to-digital converters (ADCs) within a single monolithic chip is now enabling the offload of power hungry FPGA resources to allow for smaller footprint, lower power, increased channel count platforms that can sample at higher rates than previously achievable. Along with this new capability comes novel multichip synchronization (MCS) algorithms within these integrated circuits (ICs), which allow users to achieve a known (deterministic) phase for all channels when powering the system or otherwise making software modifications to the system. This deterministic phase therefore simplifies the broader system-level calibration algorithms needed to achieve the synchronization of all channels at the output or input to the front-end networks attached to these ICs. This article presents experimental results that demonstrate this MCS capability while using a 16-channel receiver/transmitter platform consisting of multiple digitizer ICs, clock sources, and digital interfaces.

High Level System Block Diagram

A block diagram of the system used for this test is shown in Figure 1 and consists of four integrated DAC/ADC/DSP ICs that each consist of four 12 GSPS DACs, four 4 GSPS ADCs, and 12 digital upconverter (DUC) and 12 digital downconverter (DDC) blocks.

click for full size image

Figure 1. A high-level block diagram of the system used to demonstrate MCS and multichannel calibration algorithms. (Source: Analog Devices)

The DUCs/DDCs allow frequency translation and/or interpolation/decimation within the digital domain. Using a single 500 MHz reference clock injected onto the board, a reference-locked clock buffer is then utilized to generate the system reference signals needed for MCS as well as the clocks required for the digital interface to the baseband processor (BBP). The system also contains four separate phase-locked loop (PLL) synthesizers that generate the 12 GHz sources needed to clock each digitizing IC from the common reference. An RF front end is attached to each digitizer output/input, which creates a filtered and amplified signal to/from the edge-launched RF connectors. A full power distribution solution is implemented. All voltages required for the system are generated from a single 12 V source. All transmit signals propagate on the bottom of the board, whereas all receive signals propagate on the top of the board to achieve the most optimum channel-to-channel isolation.

Subarray Clock Tree Structure

As previously mentioned, the subarray clock tree consists of a single 500 MHz reference source that is split and sent to the reference inputs of four separate PLL synthesizer ICs, as shown above in Figure 1. This 500 MHz signal is also 10 dB coupled, amplified, and sent into another clock buffer IC that is responsible for generating the system references (SYSREFs) and BBP clocks required for the digital interface. The goal of this clock tree is threefold, as it:

  • Allows for individual channel SYSREF delays to correct any trace length mismatches between ICs.
  • Permits individual PLL/synthesizer phase adjustment, and therefore ensures synchronization across the respective digitizer IC clock sources, to compensate for any induced thermal gradients within the system.
  • Enables the user to achieve the necessary setup-and-hold requirements for the digitizing ICs.

The clock tree ICs are chosen to demonstrate that various board layout anomalies can be corrected in software and/or hardware with the help of digital and analog delay blocks present within these chips. Ultimately the result is a clock tree that can provide a SYSREF pulse to all required ICs within the same sample clock cycle of each of these ICs.

Digital Interface with Baseband Processor

The four digitizing ICs each establish a JESD204B or JESD204C digital link interface with the BBP.1,2 This interface is responsible for transmitting the ADC and DAC codes over the physical traces (SERDES) to and from the BBP. The number of differential SERDES traces used in this interface is termed the number of lanes (L) for this link. The converter bit resolution sent over the link is regarded as N’. The number of channelized data paths, also coined virtual converters, is labeled as M. The results shown in this article use a JESD204C link with M = 16, N’ = 16, L = 4 for the DAC-side links and M = 8, N’ = 16, L = 2 for the ADC-side links.

The rate at which data is transmitted and received between the digitizer IC and the BBP is coined the lane rate. The DSP blocks on silicon (that is, the DDCs/DUCs) allow the user to sample the digitizers at a different rate than the data rate sent across the physical lanes. As such, the lane rate is dependent on the digitally decimated/interpolated data rate for each data path. For this work, a 250 MSPS I/Q data rate has been utilized. For a JESD204C interface, the lane rate is defined as:

whereas for a JESD204B interface, the lane rate is defined as:

The results shown in this article use a lane rate of 16.5 Gbps for both the ADC- and the DAC-side JESD204C links.

Each JESD204B/JESD204C link can be established within different subclasses. These subclasses are separated based on whether multichip synchronization or deterministic latency is required. For this work, the data shown uses a JESD204C Subclass 1 mode, and therefore utilizes SYSREF signals to align portions of the digital data transmitted across the multiple links present within the system. Specifically, in this JESD204C Subclass 1 mode, the SYSREF signals are used to align the local extended multiblock counter (LEMC), which transmits at a rate:

where F is the number of octets per JESD frame per lane and K is the number of frames per single multiframe. For this work, F = 8 and K = 32, and therefore an LEMC rate of 7.8125 MSPS is used. Knowledge of this LEMC rate is important, as any successful MCS routine will need to demonstrate that RF frequencies that are not an integer multiple of the LEMC rate are able to achieve deterministic power-up phase.

Multichip Synchronization Method

Within this system, the wideband integrated ADC/DAC ICs provide MCS circuitry to allow power-up deterministic phase on all transmit and receive RF channels, even while using the DUC/DDC DSP blocks within the IC. This MCS feature enables users to populate a lookup table (LUT) during a factory calibration to minimize operational downtime. Any successful MCS demonstration must be capable of providing deterministic phase across all channels within the system for every attempted RF frequency, thermal gradient, and system power cycle.

The integrated ADC/DAC ICs contain 12 DUC blocks and 12 DDC blocks, as shown above in Figure 1. Each of these blocks contains an interpolation (DUC) or decimation (DDC) subblock to change the data rate of the DAC digital input signal or ADC digitized output signal, respectively. Also contained within each DUC/DDC is a complex numerically controlled oscillator (NCO), which allows frequency translation within the digital domain. Each of these NCOs is capable of real-time complex phase adjustment such that the digital signal between the DAC/ADC and the BBP can be modified to compensate for various SERDES trace length mismatches.

The MCS feature for these ADC/DAC ICs is responsible for achieving phase determinism throughout all aspects of the digitizer IC’s datapath. The workflow to achieve MCS is shown in Figure 2.

click for full size image

Figure 2. The MCS workflow involves separate features that align different parts of the datapath. (Source: Analog Devices)

The MCS algorithm can be divided into two separate functions:

  • One-shot sync: This feature is responsible for aligning the baseband data being sent across the physical lanes of all the digitizer ICs within the subarray system.
  • NCO master-slave sync: This feature is responsible for aligning all the NCOs throughout all the different digitizer ICs within the subarray system.

The one-shot sync feature first requires that the user defines the JESD link parameters (such as M, N’, L, etc.) and then configures the synchronization logic for any desired SYSREF averaging (if using continuous SYSREF pulses). Additionally, desired LEMC delays can be used to force the LEMC to be generated at a certain delay after the SYSREF edge. After this is completed, the user then enables the one-shot sync bit within each digitizer IC and then requests that SYSREF pulses be sent to each IC within the same clock cycle, as shown in Figure 3.

click for full size image

Figure 3. The MCS algorithm uses SYSREF signals to implement one-shot sync and GPIO signals to implement NCO master-slave sync to achieve deterministic phase. (Source: Analog Devices)

For this system, analog fine delays have been introduced within the clock buffer IC to allow synchronous SYSREFs to all digitizer ICs. A subsequent check can be executed to verify the one-shot sync process performed successfully by querying registers within each IC that provide information about the phase relationship between the SYSREF signal and the LEMC boundary of each IC’s link.

Once a stable phase is measured (that is, once the SYSREF-LEMC phase register reads 0), the user then knows that the LEMCs of all the digitizer ICs are aligned and the user can then proceed to the NCO master-slave sync process. For this activity, the subtasks described for the one-shot sync are contained within an application programming interface (API) provided by the chip manufacturer.

The NCO master-slave sync feature first assigns one of the digitizer ICs within the subarray to act as a master chip, as shown above in Figure 3. All other digitizers are then deemed slave ICs. The master IC is set up such that the GPIO0 pin of this device is configured as an output and routed to the GPIO0 nets of the three slave digitizer ICs. The slave GPIO0 nets are configured as inputs. The user can then choose to trigger on either the SYSREF pulse, the LEMC rising edge, or the LEMC falling edge. For the data shown in this article, the LEMC rising edge is used as the NCO master-slave sync trigger source and the GPIO nets are routed through the BBP instead of locally on the subarray. Next, the DDC synchronization bits are toggled low and then high to arm the ADC-side NCO synchronization algorithm. Likewise, the microprocessor align bit is toggled low and then high to arm the DAC-side NCO synchronization algorithm.

When this trigger is requested, at the next LEMC rising edge the master digitizer IC asserts high a master out signal through its GPIO0 net. This signal propagates to the GPIO0 inputs of each of the slave devices. At the next LEMC edge, all digitizer ICs experience an NCO reset algorithm. After this, any LEMC pulses are ignored with regards to the NCO master-slave sync algorithm. As with the one-shot sync, these NCO master-slave sync subtasks are contained within API functions for user ease of use.

Using both the one-shot sync and the NCO master-slave sync features aligns both inputs to every DDC/DUC such that the output phase offset of each receive and transmit channel is repeatable after multiple power cycles, as shown in Figure 4. The data in Figure 4 shows the calibrated phase offsets over 100 power cycles (indicated by multiple solid dots) for each receive and transmit channelizer while the system is operating under a static thermal gradient during each reboot.

click for full size image

Figure 4. The receive fine DDCs (left) and the transmit fine DUCs (right) are properly aligned when performing the MCS algorithm. (Source: Analog Devices)

As can be seen from the multiple dots present in this figure, the dots of each color for a given DDC/DUC are all tightly clustered to the same position after power cycles, thereby portraying deterministic phase for that particular channel. For the data in this test, all eight channelizer DUCs have been used on the transmit side, whereas only four of the eight channelizer DDCs are used. However, it has been confirmed that indeed all eight channelizer DDCs provide deterministic phase while using the MCS algorithm as well.

Issuing this algorithm at boot-up establishes a deterministic phase for each channel if the PLL synthesizer sample clocks and clock IC SYSREFs maintain the same phase relationship at boot-up. However, any system will undergo thermal gradients, which can result in PLL clock drift, and which can result in a different power-up phase if left uncompensated. To compensate for thermal gradient drift within the system, this platform utilizes PLL synthesizer phase adjustments.

In the next part of this article series, we’ll explore PLL synthesizer phase adjustments, scalability to multiple subarrays, and system-level calibration algorithms.


1 Del Jones. “JESD204C Primer: What’s New and in It for You—Part 1.” Analog Dialogue, Vol. 53, No. 2, June 2019.

2 Del Jones. “JESD204C Primer: What’s New and in It for You—Part 2.” Analog Dialogue, Vol. 53, No. 3, July 2019.

Mike Jones is a principal electrical design engineer with Analog Devices working in the Aerospace and Defense Business Unit in Greensboro, North Carolina. He joined Analog Devices in 2016. From 2007 until 2016 he worked at General Electric in Wilmington, North Carolina, as a microwave photonics design engineer, where he focused on microwave and optical solutions for the nuclear industry. He received his B.S.E.E. and B.S.C.P.E. from North Carolina State University in 2004 and his M.S.E.E. from North Carolina State University in 2006. He can be reached at Michael.Jones@analog.com.
Michael Hennerich joined Analog Devices in 2004. As a systems and applications design engineer, he has worked on a variety of DSP/FPGA- and embedded processor-based applications and reference designs. Michael now works for the System Development Group (SDG) as an open-source system engineering manager in Munich, Germany. In this role, he is leading ADI’s device driver and kernel development team, developing device drivers for all sorts of mixed-signal IC products and HDL interface cores. He holds an M.Sc. degree in computer engineering and a Dipl.-Ing. (FH) degree in electronics and information technologies from Reutlingen University. He can be reached at Michael.Hennerich@analog.com.
Peter Delos is a technical lead in the Aerospace and Defense Group at Analog Devices in Greensboro, North Carolina. He received his B.S.E.E. from Virginia Tech in 1990 and M.S.E.E. from NJIT in 2004. Peter has over 25 years of industry experience. Most of his career has been spent designing advanced RF/analog systems at the architecture level, PWB level, and IC level. He is currently focused on miniaturizing high performance receiver, waveform generator, and synthesizer designs for phased array applications. He can be reached at Peter.Delos@analog.com.

Related Contents:

For more Embedded, subscribe to Embedded’s weekly email newsletter.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.