Capturing and Debugging System Crashes
Scenario #3: Simplify logic analyzer measurements with enhanced features and tools
Different logic analyzer features and tools offer powerful insight for debug system crashes: Deep memory, Store qualification, FPGA dynamic probes, Compare, Application specific probes, Eye diagrams on logic analyzers, and Integrated logic analyzer and scope traces.
Deep Memory - Capturing a trace leading up to a system failure is critical to determine the root cause of a system failure. Handshaking, split transactions, pipelining, out-of-order execution, and deep first-in-first out data storage (FIFO) all mean the flow of data related to a problem can be distributed over thousands, if not millions of bus cycles.
How often a problem occurs can also vary from once every bus cycle to once in several weeks. Since the actual cause of the crash can happen prior to the actual crash, it is helpful to use the deepest memory a logic analyzer module has to offer when setting up a measurement. Many logic analyzer modules allow for after-market memory upgrade licenses.
Store Qualification. . This capabiliy allows the engineer to use the available acquisition memory more efficiently, rather than filling it with unwanted activity such as idle cycles or wait loops. Store qualification determines if an acquired sample should be placed in memory or discarded.
The simplest method to set up storage qualification is by setting up the default storage, which means "unless a sequence step specifies otherwise, this is what should be stored."
As an example, a user may want to only store samples if the system isn't in an idle state. By default, the storage is set to store all samples acquired. A user can also set the default storage to store nothing, which means that no samples will be stored unless a sequence step overrides the default storage.
Sequence step storage qualification. This means that within a particular trigger level only certain samples will be stored. This signifies that until a "go to" or "trigger" action is used to leave this sequence step, the storage qualification applies.
This is useful when different storage qualification for each sequence step is required. For example, in a microprocessor system, a user may want to store nothing until ADDR = 1000 and then only store samples with ADDR in the range of 1,000 to 2,000 for the rest of the measurement.
Establishing a sequence step storage requires the use of an additional branch, and it always overrides the default storage, but only for the conditions specifically mentioned in the sequence step storage. It is important to account for the interaction between default storage and sequence step storage.
FPGA dynamic probes. These are a result of collaborative development between a logic analyzer and FPGA companies (features vary by logic analyzer vendor and FPGA model).
Using an FPGA dynamic probe, a user can view internal FPGA signals without routing each signal to the periphery of the FPGA. Perhaps the user's design has an internal FPGA signal that occurs at a regular interval when the system is operating normally. That signal can be easily accessed to use in a trigger to capture a system crash.
Moving probe points internal to an FPGA used to be time consuming, but using an FPGA dynamic probe can help measure a different set of internal signals within seconds without design changes. It is important to note that FPGA timing stays constant between sets of internal signals that are probed using a dynamic probe.
Feature rich FPGA dynamic probes automatically map internal signal and bus names from the FPGA design tool to the logic analyzer, eliminating mistakes and saving hours of set-up time.
Compare captured data to reference data on a logic analyzer with a compare display. Set-up the measurement to highlight differences between a known-good device under test and the last trace captured. Or stop repetitive runs and send an email after a specified number of differences are found in a trace.
A comparison tool may provide the first indication of a problem causing a system crash, when troubleshooting failures on systems are expected to run exactly the same each time they boot (or at least for a test that can be defined or bracketed by markers).
Application specific probes provide non-intrusive probing plus logic analysis setup, triggering, and decoding for specific applications. They also enable time-correlated analysis and sequenced event triggering across multiple buses making it easy to follow transactions, data, and packets as data flows through the system. Application specific probes that are available include (but, are not limited to):
PCI Express® (PCIe)
Advanced Switching Interface (ASI)
Serial ATA (SATA) and Serial Attached SCSI (SAS)
Serial RapidIO
Parallel RapidIO
SPI 4.2 (System Packet Interface, POS PHY L4)
InfiniBand
I2C
FlexRay
SPI (Serial Peripheral Interface)
Eye diagrams on logic analyzers can provide insight into signal integrity issues across an entire bus simultaneously in minutes. The results can be viewed as individual signals or as a composite of multiple signals or buses. Eye diagrams can be used to:
* Observe skew between signals
* Find and fix inappropriate clock and signal thresholds
* Identify signal integrity issues related to rise-time,fall-time, or
data valid window widths
* Acquire signal integrity insight rapidly under a wide variety of
operating conditions
Expanding the bus can identify individual signals with problems for further parametric analysis.
Integrated logic analyzer and oscilloscope traces in the logic analyzer waveform display help, validate, correct, logical and timing relationships between the analog and digital portions of a system. Automated wizards and minimal connections using standard LAN and BNC cables simplify measurement set-up.
An incorrect value on a bus captured on a logic analyzer can trigger a scope for investigation of the analog characteristics of the signal. Alternatively, a glitch captured on a scope can trigger the logic analyzer.
As an example, in Figure 5 below, the logic analyzer triggered on a read cycle with suspect data on a DDR3 system. Integrating an external high performance scope probing the DDR3 system clock, data strobe, and a data signal allows inspection of multiple analog characteristics of the suspect data burst.
 |
| Figure 5: View scope feature is included on the 16800 and 16900 series Logic Analyzers. In this DDR example, logic signal names highlighted in purple and scope signal names highlighted in blue. |
The user can zoom out to accurately measure the time between the valid read command and the start of the data burst associated with this command. Or zoom in to measure the preamble (time the data strobe drives low) prior to the data burst.
Logic analyzers offer a wide selection of powerful features and tools that enable insightful techniques for troubleshooting elusive system crashes. When troubleshooting a system crash, the user should follow these steps:
* Gather information about normal system operation
* Recreate the failure conditions
* Consider the nature of the system crash (for example, did the system
clock stop?)
* Determine which tools are available to troubleshoot the system
* Plan the logic analyzer trigger using the concepts outlined and
adjust accordingly.
Conclusion
The concepts, techniques, and logic analyzer tools covered in this
article apply broadly for troubleshooting system crashes. When
considering the nature of the system crash, it is critical to
understand if the clock stops or continues.
If the system clock dies when the system crashes, modern logic analyzers provide 'trigger on user stop' trigger macros and 'stop' buttons to capture events that lock up a system.
If the system clock keeps running after the system crashes, logic analyzer triggers need to be more creative to capture the events leading up to the system crash. Time out triggers to capture signals that stop toggling or change from a known behavior allow the user to view events leading up to the crash.
Jennie Grosslight, Technical Marketing Lead Engineer, has 19 years of experience at Agilent Technologies with logic analysis strategy and solutions. Her areas of expertise include; system engineering, high-speed hardware design and validation, product marketing, application support, and project management. She earned her B.S.E.E. from the University of Colorado at Colorado Springs in 1989. In addition to spending time with her daughter, Jennie enjoys yoga, hiking, and water sports.
Loading comments... Write a comment