Do-it-yourself embedded Linux development tools
Debugging
Debugging embedded Linux is tricky, because the technique can be different depending on whether you are debugging applications, drivers, or kernel code. The only common element is the gdb client front end, so we'll start there. A gdb client will usually run on the host platform, although technically you can run it on the target, too. It can connect to the gdb server running on the target (more on that later) through a serial port or tcp/udp protocol. Note that you can't use the x86 gdb client that comes with your Linux distribution. You'll need the one from the cross-compiler toolchain for your embedded CPU.
Simple gdb debugging session is illustrated as:
gdb>
file vmlinux
target remote 192.168.0.10:2828
Ctrl-C
bt
This example loads vmlinux image symbols, connects to a remote target, interrupts the running code, and prints backtrace (for more information about gdb commands, see the gdb user manual at sourceware.org/gdb/current/onlinedocs/gdb_toc.html). Using a command-line interface is handy for quick tasks, but for serious debugging, most users prefer a graphical interface. I recommend two graphical gdb wrappers, DDD and Insight.
Insight has a slicker user interface, but because it incorporates gdb, you'll need a different binary for every embedded CPU architecture you work with. DDD is a bit more clumsy as it's a GUI wrapper, but it can work with an external gdb executable (using the -debugger parameter) allowing you to use a gdb binary from your cross compiler toolchain. The gdb server will vary depending on what you're debugging. For applications, you'll need to run a gdbserver executable on the target in the following way:
gdbserver 192.168.0.10:2828 your_application
gdbserver will run the binary your_application that you're going to debug and wait for gdb client to connect on the specified port. It can also attach to a running process if executed with -attach command-line argument.
Kernel debugging is trickier for various reasons, not the least of which is the fact the kgdb (kernel gdbserver equivalent) may not be integrated into your kernel, so you may need to download a kgdb patch from kgdb.linsyssoft.com, apply the patch, and recompile the kernel. Doing this will allow you debug the kernel through a serial port or over Ethernet. When debugging kernel loadable modules using kgdb version 1.8 and earlier, you'll have to load a module object file into gdb manually using the loadmodule.sh script or add-symbol-file gdb command so that gdb will be aware of your module's the symbol table.
If you're creating a board support package or doing some low-level kernel programming, you'll probably need a JTAG probe. When choosing a probe, ensure that it supports Linux, although most probes nowadays do. A Linux-friendly JTAG probe should support the Linux target and host; support a remote gdb protocol for debugging; support the debugging of kernel code, applications, and dynamically loadable modules; and support a Linux MMU.
The latter one is tricky because during debugging, you may want to access a virtual memory page that's not currently mapped. The probe will have to either know how to extract this mapping information from Linux internal data structures or make Linux remap this virtual address. Linux host and remote gdb support is not essential, but it is convenient, as it lets you debug with JTAG using the same gdb client frontend.
Tracing
Debugging lets you to catch simple bugs, but unfortunately the hard ones are usually affected by timing and probably won't show up in the debugger. If this is the case, you consider using the Linux Trace Toolkit (LLT), which can be downloaded from www.opersys.com/LTT. It allows tracing various event types for multiple processes and the kernel itself, and it can present this information graphically to help debug complex multiprocess systems. To use LTT, you'll have to patch the kernel and install LTT daemon and utilities. Note that LTT daemon requires read-write filesystem access to store the trace file. I don't recommend using the jffs2 filesystem to store such information in flash as this could severely effect system performance and timing. A RAM disk is the best option, provided you have enough memory. If this isn't the case I suggest using NFS.
The most simple LTT session looks like:
trace 60 trace_file
traceview trace_file
Both commands are a helper scripts. The first enables tracing for 60 seconds and uses trace_file as a base name for the trace results file. The second executes the graphical trace visualization tool (for more information, read the LTT reference manual at www.opersys.com/LTT/ dox/ltt-online-help/index.html).
Loading comments... Write a comment