Graphics Libraries for Embedded Systems

Graphics are no longer just for the desktop. Graphical interfaces are finding their way into more and more embedded systems. This article covers GUI development tool features that are useful to embedded developers and compares several graphics environments that are available for embedded systems.

Drawing Libraries

You should first decide what level of functionality you wish to buy from a third party vendor. At the lowest level you could purchase a drawing library, which provides the ability to render lines, boxes, arcs, and text. Such a library stores no state, so once you have drawn a box, you cannot access the same box again to change an attribute such as its color or height. You may well have to draw your own buttons and sliders and manage redrawing them if they are manipulated by the user. Such a library will not detect which of the shapes has been selected with the mouse or other pointing device. These libraries are useful if you want the absolute minimum overhead and require support for only simple graphics output with little user interaction with the display. Many attributes are required to draw something as simple as a box. Is the box filled? What is the line thickness? What color is the box? Are the corners rounded? This list of questions could be mapped into a large number of parameters to the drawBox() routine. To avoid long lists of parameters which would consume CPU cycles, as well as requiring more work on the part of the programmer, most libraries allow a pointer to a context to be passed to each drawing function. The context defines many of the parameters described above. If many similar boxes will be drawn, the context need not be changed between calls. If an attribute such as color changes, then the context can be altered for that single attribute before the next call is made. A context can be shared between different drawing primitives. Some of the attributes will not always apply, such as ýfillý when drawing a line. In those cases, the redundant attribute is simply ignored.

The next level of functionality is an object-oriented library that provides a set of graphical objects capable of maintaining their own state. When a box is created and rendered, the application can maintain a handle for the box and later adjust a single parameter. The objects may be boxes and lines and other non-control shapes. Control objects such as buttons, sliders, and tick-boxes may also be available. The non-control shapes are useful to construct diagrams, such as a schematic showing the process being controlled by the user interface. If allowing the user to control a simple dialog of tick-boxes and menus is more important, you'll need the control objects. A library may supply shapes or controls, or both. Make sure that the set of objects supplied map reasonably well to the set that you require. The two most important jobs managed by a library at this level are redraw management and event management. Redraw management means knowing where objects are relative to one another, so when a change to one object is made, any other objects that are exposed by the change are redrawn. Event management means that events that can be associated with a location on the display are directed to the objects that are in that location. For example, a mouse click event should be sent to the object under the mouse. A keyboard event should be sent to the object that currently has the keyboard focus. If there are many overlapping objects, then the redraw and event management routines will be quite complex, so having a toolkit manage this for you is highly advantageous. However, the redraw routine can easily become so heavily used that it becomes a bottleneck in your program. If you don't have source code, you may have no way of optimizing this routine. Most shapes are treated as rectangles for the purposes of establishing what area of the screen needs to be redrawn, which can cause problems for irregular shapes. For example, a line drawn from the top left of a window to the bottom right overlaps with the entire window. If you change the color of the line, the entire window may be redrawn, even though most of the objects did not overlap with the line itself; they simply overlapped with the smallest rectangle that could surround the line. If you use a lot of irregular shapes, you may want to evaluate your redraw routine to ensure that it will handle your case well. The redraw routine can also be optimized if you know that there will be no overlap between the objects.

The next level of functionality is the window manager, in which the environment provides windows that can each be controlled by independent applications. This environment may also provide a method of launching certain common applications and mechanisms for switching between them. Typically this environment isn't desirable on an embedded system, though in some cases an imitation of a particular desktop environment may simplify porting an application to the embedded system. For example, WinLight provides a Windows 3.1 work-alike, and a number of X11 window managers are available for embedded systems. In a number of cases once the application is ported, you'll want to remove the windows frame and remove the ability to resize or iconify the window to present the application to the user as the only one that can be run. The embedded system could be configured to boot straight into this application, and possibly never exit unless rebooting. As you move along this evolutionary path, a number of things change. You lose control of the details, but with that you lose responsibility for them. If you're on a tight schedule, this is a good thing. You may no longer be able to adjust the exact width of the bevel on the edge of a button or the precise way the button reacts to being selected, but you can trust that the button implemented by the third party manages these details in a reasonable manner. If you decide your application needs a greater degree of control, you have two options. One is to move to a lower level library and implement all the required functionality from there. A more attractive option may be to see if the higher level library has hooks that allow you to manage the details. The Xt intrinsics toolkit (supported by VxWindows and in part by Photon) allows the application to create new widgets that did not previously exist in the library. This allows you to implement your own custom button. You can draw it using more primitive drawing routines. You can also implement your own event management. Having implemented the object's appearance and behavior, some toolkits will allow you to add your custom object to their hierarchy of objects. If you only require customization of certain components, this approach may work well. This mechanism allows the application to construct application-specific widgets that operate differently to anything that existed before. A railroad management system may have a widget that displays a train on a track between two stations. User-supplied information about the location of the train may be reflected in its position, possibly using different colors to represent whether the train is on schedule. Such a widget is going to provide far more satisfactory feedback than anything that could be constructed from the standard set of buttons, menus, and sliders. Extra work is always required to provide customized controlsýhow much work may depend on support from your graphics library.

At each of these levels, support for code generation may come from a tool that allows the programmer to draw a screen using a drag-and-drop interface and then generate code that would draw that screen using the supplied library. The only code generation supported by any of the tools here is at the top level, for object-oriented toolkits, but there is no reason that code generation could not be provided in a simple drawing library. Once you know what level of functionality you require, there are a number of other issues that you need to examine. Obviously, if the hardware platform and real-time operating system (RTOS) decisions are already made, then your choice of tools is already restricted. In some cases, graphics will be so critical to the project that you can justify choosing the graphics library first and then choosing a processor to match. Because a lot of these types of tools follow an API for X11 or MS-Windows, you may need to consider whether this will affect the learning curve of the team. Interactive graphics APIs tend to be large and have complex calling structures that allow objects to be accessed from C, a non object-oriented language. The API also has to support callbacks, which involves manipulating pointers to functions. An API typically has to have functions to manage fonts, color pallets and multiple drawing planes. This makes learning an API a time-consuming activity. If you have a number of programmers available who are familiar with a particular API, you should try to leverage that asset. Despite the growing popularity of C++ in the graphics world, none of the products described here provide a C++ interface. Perhaps this reflects the relatively slow growth of C++ in the embedded systems community. Most of the APIs provided suffer from a lack of type checking. Functions that can take variable length lists of parameters are more flexible, but it isn't possible to recognize a missing or extra parameter at compile time. This mechanism is similar to printf(). The two low-level libraries, MAUI and OpTIC, are the only products reviewed that avoid this pitfall. A number of the toolkits that conform to such ýwell-knowný APIs as X11 or MS-Windows refer to existing documentation or books already available for the desktop programmer. This is shoddy service. If you spend thousands of dollars on a product, you have the right to a user manual and should not be sent plodding off to your local bookstore to see if they stock the book describing the product you bought. The danger here is that as the desktop world gets one revision ahead of the API supported by your toolkit, such books could become difficult to locate. So try to take a look at the documentation of the library you're considering, either in book form or on-line, and complain if it doesn't contain details of every function call available to the programmer. In embedded systems, the amount of ROM and RAM required is a big concern. The choice of library will have an impact here. Assuming that no file system is available to you, bitmaps will be built in to the executable. Excessive use of many different bitmaps can lead to a system where the data for the bitmaps dwarfs the space occupied by the executable code. Bitmaps can greatly enhance the appearance of a display, so consider compression or using bitmaps with fewer colors when space gets tight. Fonts are effectively collections of bitmaps and suffer from the same problem-you may be able to strip out some of the characters if they're never used. A number of the libraries described here are positioning themselves in the set-top box market. It should be noted that the ability to merge a video signal with graphics is fundamentally a hardware issue. If your hardware allows this, any of the following toolkits will allow you to display graphics with the video signal in the background. However, none of these products will allow you to capture, record, or otherwise manipulate the video signal, because the video information itself isn't passed to the software. The environment used to review the following products consisted of a Windows 95 Pentium PC as the host and a 486 PC as the target. The two were connected over a serial link. Because the target supported standard VGA graphics, the video drivers in all cases were already available. Other graphics hardware may require you to write your own drivers. All of the libraries reviewed provide hooks that allow you to implement your own driver if your choice of graphics hardware isn't supported.

GESPAC produces a family of tools including a window manager, application builder, a drawing library, and a set of gadgets such as buttons and bar graphs you can use in your application. These tools run on x86 or 68K processors, and require the OS-9000 or OS-9 operating systems from Microware. G-Windows, the drawing library, provides boxes, lines, and ellipses, as well as control of windows. Events can be polled but callbacks aren't handled at this level. G-View provides a higher level of control. The screen can be designed using wedit, a drag-and-drop application builder. This tool allows the programmer to draw the background of a screen with lines, curves, and bitmaps. A large number of gadgets are available, though access to them is via a two-level menu which isn't very intuitive. Luckily, usability problems in the wedit program do not necessarily end up as usability problems in the end-user application. The choice of gadgets indicates that this product is targeted at industrial control applications. The programmer is provided with buttons, needle indicators, and strip charts, as well as gadgets to manage text and bitmaps. There is even a gadget which indicates state, called an LED (light-emitting diode). There is also a polar deviation gadget for monitoring trends. If you're trying to replace a custom panel made up of individual electronic components with a GUI, and you want the look and feel to imitate those components, these gadgets will serve you well. One clever trick is that the bar graph can have an irregular shape. Displays of bottles filling up are trivial to implement. However, if you want your application to operate like a typical desktop application, you may be less pleased. No menu bar or drop down menus are supplied and the control of the pop-up menu is a bit flaky. If I click to make a sub-menu appear, then I don't expect it to disappear until I click elsewhere. I found the sub-menus disappearing on me when I least expected it, sometimes leading to three attempts before I could select the correct item. Another minor fault is that the scrollbars don't continue to scroll if the mouse button is held down on one of the scroll arrows. The user must click repeatedly to continue the scroll. These two faults will exist in any application you produce that uses two-level pop-up menus or scrollbars. Gadgets can be combined into compound gadgets. There is also an invisible gadget that you can place over any image you may construct. This effectively allows you to build your own gadgets with custom behavior. The ability to mix access to G-Windows and G-View is important here, because the G-Windows library may not give the low-level access required for customizing gadgets. Once the screen has been generated and stored as a data file, the programmer can access it via the G-View API. The source code must pass through a pre-compiler before it can be compiled. This affects the line number records stored in the debugging information, making some bug hunting more difficult. In return, the code that the user writes has the advantage of being simpler and more readable than most APIs. Simple assignments to structure members alter the object's properties with the appropriate update of the display. GESPAC recommends a minimum of 1.5MB of RAM for a run-time system. The code space used will be a minimum of 250K. Overall, GESPAC has provided a comprehensive suite of tools, but a few rough edges remain.

The tropically named Multimedia Application User Interface (MAUI) library is available from Microware for 68K and PowerPC targets. Unfortunately, because it has not been released yet for x86 targets, I wasn't able to use the toolkit. The following review is based on the user documentation alone. MAUI is supplied as a library which requires the OS-9 or OS-9000 operating system. No code generation or other tools are supplied. MAUI is more than a simple drawing library, but it isn't a fully object-oriented library either. It provides the usual selection of drawing functions for lines, arcs, boxes, bitmaps, and text, as well as some higher level structures, allowing a level of management of the display not typically found on the simpler drawing libraries. Windows and viewports provide a rectangular area that can be clipped when the application draws to them. The windows have a stacking order to allow one to obscure another. However, when a window is exposed, the onus is on the application to provide the redraw functionality. Also employed are sprites, structures that allow the application to define a sequence of images that can be displayed in turn. By changing the location as the images are selected, animation effects are created. MAUI falls short of the object-oriented toolkits in that the event management is minimal. Keyboard and pointer devices are integrated into intertask communication mechanisms; how-ever, the interpretation of the location is the responsibility of the application. If you want mouse-sensitive buttons, sliders, and tick boxes, you'll have to build them yourself, both in terms of drawing the control and detecting any pointer manipulations of the control. MAUI addresses some problems neglected by other toolkits. A set of configurable pools of memory allow heap management with reduced fragmentation. This feature is used by the MAUI toolkit but is also accessible to the application. An API to manage playing and recording sounds is also provided.

OpTIC is a drawing library that runs on Integrated Systems' pSOS operating system. As a drawing package, it provides no control over the image once it has been rendered. The library supplies the drawing primitives for lines, boxes, bitmaps, and text. The primitives for circles and the ability to fill a polygon are conspicuously absent. The drawing functions draw to a canvas which is an area of RAM representing an area on the display. An application may have a number of canvases that map to different areas of the display. If RAM is at a premium, you can use a canvas that only occupies the area you wish to manipulate, leaving the rest of the display blank or filled with a background video signal, in the case of a set-top box. The draw routines don't cause the image to be immediately rendered. The OpUpdateDisplay() function maps the canvas to the display, making changes since the last update visible. This approach helps to minimize flicker, but consumes more CPU cycles than drawing directly to the video controller. The library occupies about 40K of code space. This can be increased or decreased by about 2K per font, depending on the font size. Because all source code is available, tuning certain functions to suit your application may be possible for optimization reasons. Unfortunately, version 1.0 of the library was supplied with no makefile. A call to their technical support line rectified the situation. Once running, the library interface is quite simple to use. An optional flag allows the choice between thread-safe and non thread-safe operation. This has the advantage that you don't pay the performance price of resource locking or unlocking unless your application requires it. A small number of functions define the device driver interface. The OsdWrite() function copies the canvas to the physical display. This approach means that writing a new device driver is a simple procedure. Any new functionality that manipulated the canvas wouldn't require any changes at the device driver level. However, this simplicity in the device driver interface makes it impossible to take full advantage of advanced video cards that support line and rectangle drawing.

VxWindows, produced by VisiCom, is an implementation of X11R6 for 680x0, SPARC, MIPS, PowerPC, and x86 platforms using the VxWorks operating system from Wind River Systems. The X11 protocol divides functionality into a client/server architecture. The server is capable of rendering the graphics and runs on the processor connected to the graphics hardware. The client is an application that requests that particular images are rendered and may or may not run on the same processor as the server. The VxWindows product contains both parts. Embedded clients can communicate with a workstation to display graphics on the desktop; alternatively, a desktop machine could request that the embedded server display graphics. The X11 protocol, which operates over TCP/IP, has been proven to scale very well with applications running in one location with their user interface on another machine across the Internet. The protocol is architecture-independent, so the desktop machine doesn't even need to know that it is communicating with an embedded system. An X11 server can display a number of windows that are under the control of separate applications running on different machines. The X11 libraries provide a number of levels of functionality, from basic drawing primitives to widgets such as buttons, sliders, scrolling lists, and tickboxes. Windows come in a number of flavors: dialogs, popup windows, and scrolling panes. The higher-level libraries provided by VxWorks conform to the Motif look and feel. There is also a choice of window managers, including the Motif Window Manager. A large pool of free software, written using the X11 libraries, is available, as are many books that have been written about the librariesýa result of X11's origins in the academic community. Unfortunately, a price must be paid for all of this functionalityýthe X libraries are very large. Don't expect to do anything useful unless you have 4MB of code space available. While such a resource-hungry product may not have much to offer to the low-cost embedded system, it is important that such functionality is available on an operating system that can guarantee real-time performance. VisiCom also produces Real Time Graphics Library (RTGL), which provides simpler drawing functionality for applications that cannot support a full X11 implementation. The VxWindows product doesn't come with any code generation tools, but other products such as X-Designer, which produce X11 compliant code, should fill that gap.

Photon is a graphical extension to the QNX real-time operating system. QNX itself is POSIX-compliant and implements many Unix extensions. QNX serves as the development environment as well as the target environment. Avoiding both the step of downloading serially and the cost of emulators are big advantages. Photon implements an API similar to X11 in that it is widget/callback based, so X11 programmers will have no trouble adjusting to this environment. Photon follows the X11 architecture in that the client and server can live on different processors and communicate over a TCP/IP link. Servers are also available to allow the output to be displayed on a PC or Sun workstation. The X11 protocol itself isn't used, however, so Photon cannot communicate with other X11 clients or servers. An application builder called PhAB is available to assist in screen design. This tool allows the programmer to build a screen by drawing a set of shapes and controls into a window. The interface can be run with no compilation, allowing quick feedback to changes. Once you are finished, C code can be generated. The widgets available in Photon have an API that imitates the Xt intrinsics. You can create your own widgets via the API, and then the new widget can be integrated into the PhAB tool. Widgets already available include buttons, tickboxes, scrolling lists, scrollbars, grids, curves, drop-down menus, and a tree widget that allows file manager-style control of a hierarchy. Support is available for associating HTML-based help with each widget. A translation tool and support for multi-byte characters means that internationalization of the final product should be straightforward. The widgets are not thread-safe, but this shouldn't be an issue, as requests are generally queued to the Photon task. Within the Photon task, signals cannot be handled until any widget manipulation is complete, so signals related to hard real-time events should be handled outside of the Photon task. The window manager is a Motif work-alike. You may wish to disable the window manager, and many of the other operating system features, when your final embedded system is running. However, the window manager is essential during development. The QNX kernel footprint in ROM is small, about 8K. Once you add graphics it gets a little biggerýabout 70K for the Photon process. A simple application built with Phab with a small number of shapes and menus results in an executable of 250K when it has been linked with all of the necessary libraries. A full X11R5 implementation is also available from QNX. The only disappointment with QNX/Photon is that it is only available for Intel x86 targets.

WinLight is a small footprint Windows 3.1 work-alike that can run on embedded targets that would be overwhelmed by the demands of the full Windows 3.1 operating system. It is possible to run the same executable under WinLight and Windows 3.1, and by extension, Windows 95 and NT. The look and feel when running WinLight isn't identical to Windows 3.1. The good and the bad news is that only a subset of the Windows 3.1 API is supported. This is good news because you aren't paying for functionality that you may not use in the embedded environment, such as OLE functionality. The bad news is that in general, applications currently running under Windows 3.1 will not run under WinLight without significant modification to the source code. Winlight typically runs on ROM-DOS, which is also available from Datalight, though it can run on other flavors of DOS. This requirement is an advantage if you want to exploit other commonality with the PC architecture, but it does make WinLight unsuitable for systems with hard real-time requirements. WinLight requires about 512k including ROM-DOS, and a simple executable is about 40k in size. The possibility that you will want your embedded system to run a combination of applications is unlikely, though WinLight can support this option. It is also unlikely that you will want to embed an application that is currently running on the desktop; so what is the motivation for using WinLight? Programmers familiar with the Windows API are plentiful, an important motivator in using the Windows API on an embedded system. The other advantage is that the development environment itself will be familiar to the developers. They can use Microsoft Visual C++ or Borland C++ to develop the code, and Windows 3.1/95/NT to run and debug it. When the GUI becomes stable, it can be run under WinLight on the same hardware. Only then is it blown into ROM for the embedded device. Typically, WinLight and its applications are stored in a ROM disk and copied to RAM for execution. Discipline will be required to prevent unsupported Windows calls being inserted while developing with Windows 3.1. WinLight does provide a tool to detect such calls at run time under Windows 3.1, so the programmer doesn't have to run WinLight itself to detect such problemsýalthough a separate set of header files would have allowed such problems to be caught at compile time. The fact that the executable files run unaltered on WinLight and Windows 3.1 means that in cases where it is necessary to run the front end on Windows as well as the embedded target, there is no need to manage two source trees or to use #ifdef statements to switch between APIs. Serious applications for this may be rare, but it can be very impressive to send out marketing material with a floppy that potential customers can pop into their PCs and run a replica of the end product. The Windows 3.1 version could also be used for training purposes. Sample device drivers are supplied to assist you should you need to write your own. This is a valuable feature because many of the embedded targets will be using touch screens or custom keypads. You may be able to use existing Windows 3.1 device drivers in some cases. This library does have a number of drawbacks. The primary one is that tools and libraries such as Microsoft's MFC and AppWizard, and Borland's OWL and AppExpert, aren't fully supported, though MFC support is being improved and DataLight intends MFC to be an attractive development option for programmers. WinLight isn't yet mature enough to compete with running a full version of Windows if you wish to run typical Windows 3.1 applications. However, for those of you with lots of Windows programming experience, this will be an option to consider.

The preceding reviews don't cover all of the commercial products available in this arena, but this selection covers a good cross-section of the functionality that you will be able to purchase. Having examined the possibilities, there is no one clear answer to the question: which product should I use? If you aren't already tied into a hardware platform, then using Intel/VGA hardware definitely leaves more options open. You'll pay more in terms of ROM and RAM if you use the higher-level toolkits described; but if they reduce time to market, that may well be a price worth paying.

Contact Information

WinLight
Datalight
Arlington, WA
Phone: (800) 221-6630
(North America only)
+1 (360) 435-8086 (Rest of the world)
E-mail: sales@datalight.com
Web: http://www.datalight.com
READER SERVICE NO. 130

G-Windows and G-View
GESPAC Inc.
Mesa, AZ
Phone: (602) 962 5559
or
GESPAC SA
Geneve, Switzerland
Phone: +41 22 7943400
E-mail: info@gespac.com
Web: http://www.gespac.com/ or http://www.gespac.ch/
READER SERVICE NO. 131

OpTIC
Integrated Systems, Inc.
Sunnyvale, CA
Phone: (408) 542-1500
E-mail: info@isi.com
Web: http://www.isi.com
READER SERVICE NO. 132

MAUI
Microware Systems Corp.
Des Moines, IA
Phone: (800) 475-9000
E-mail: info@microware.com
Web: http://www.microware.com
READER SERVICE NO. 133

Photon
QNX Software Systems Ltd.
Kanata, Ontario Canada
Phone: (800) 676-0566 ext. 3000 (North America only)
+1 (613) 591-0931 (Rest of the World)
E-mail: info@qnx.com
Web: http://www.qnx.com
READER SERVICE NO. 134

VxWindows
VisiCom
San Diego, CA
Phone: (619) 457-2111
Email: vxwindows@visicom.com
Web: http://www.visicom.com
READER SERVICE NO. 135