Editor's note: Small footprint implementations of Java are now available that allow the design of graphical user interfaces for use in a wide range of ARM Cortex-M based embedded designs. Such implementations produce binary code that is more compact than C++, facilitates software reuse, and provides a high productivity level, usually 4 to 5 times better than C.
Graphical User Interfaces (GUIs) have become more important in the embedded world because they have a direct effect on end-customers. However, GUIs are not easy to design and develop because of the inherent complexity related to the variability of human factors (such as ergonomics) and the involvement of many different stakeholders. Complexity also comes from high variability of the specifications during the development and maintenance phases and embedded constraints such as memory footprint, CPU resources and cross-compiling issues.
Several strategies are possible to design GUIs for embedded devices:
use the C language even though a structured language is not the best match for developing GUIs;
- use authoring design tools which hide the implementation complexity of C but require more memory and CPU resources;
- use dedicated companion chips at the cost of reduced flexibility compared to general purpose MCUs and with extra expenses.
An alternative strategy is to use a general purpose Java technology for designing such embedded GUIs. Indeed, Java technology is a perfect match because it is Object Oriented, allows designing flexible event-driven architectures with loose coupling between entities, and features automatic data memory management for highly dynamic program executions thanks to the virtual processor and its automatic garbage collector.
In addition, Java as a general purpose technology, produces binary code that is more compact than C++, facilitates software reuse, and provides a very high productivity level, usually 4 to 5 times better than C when designing GUIs.
Typical Java example for GUI: MVC pattern
GUI software should use the Model-View-Controller (MVC) pattern (Figure 1 ) because it is the best known solution for designing flexible and easy to maintain GUIs. The model holds data that is updated by a Controller upon user interactions with a graphical object (a widget). As shown in Figure 1, once the model has changed its data, it asks the View (via the Observer pattern) to refresh the graphical appearance of the widget. All entities are loosely coupled, so a new widget using the existing Model data can be added without impacting the other parts of the system.
Java platform concepts for embedded systems
Java technology is based on two main concepts: a programming language and a virtual processor. Once the Java source code is compiled, the binary output – the bytecode – is executed by a standardized virtual processor called a Virtual Machine (VM). The VM runs on by the host processor such as a Cortex M core based microcontroller. The bytecode is therefore analogous to the object code produced by a C compiler, but targeted for a canonical processor rather than a specific microprocessor.
Since Java binary code is portable across a wide range of hardware platforms, it is easy and convenient to build a sibling embedded platform that runs on a workstation to offer simulation capability that is a keystone in the GUI prototyping process. The simulated platforms (Figure 2 ) can simulate properties such as RTOS scheduling and provide mockups that extend the simulated platforms with specific capabilities not originally available from the workstation, such as a device front panel (LCD display, buttons, LEDs) sensors, actuators, special communication protocols, etc.
Since the application bytecode is executed by both platforms, the simulated platform becomes a powerful tool for debugging and testing the GUI application from the highest level (functional test coverage) to the lowest one (binary test coverage) and allows the analysis of the application memory footprint requirements during the development phase.
Java embedded platform design for microcontrollers
The main criteria for choosing an embedded platform for designing GUIs on Cortex M series are:
- low requirements in Flash and RAM consumption;
- high speed execution engine;
- seamless integration with legacy code (RTOS and firmware).
Shown in Figure 3 below is an example of a typical platform optimized for Cortex-M3/M4 core based microcontrollers and providing the complete implementation of the MicroUI and MWT graphical libraries ( Refer to the ESR Consortium for API specifications – ESR002 and ESR011, www.e-s-r.net).
As shown in Figure 4 below, “Green threads” are Java threads that are internally managed by the Java platform instead of being natively managed by the RTOS. The Java platform thus defines a multi-threaded environment for the whole Java world that runs within one single RTOS task. The Java CPU consumption is fully controlled by configuring the RTOS task, allowing easy arbitration between the different tasks of the entire application. Indeed, the maximum amount of CPU power given to the Java world is fixed, whatever the number and/or the activities of the (Java) green threads.
The “Native thread” implementation illustrated in Figure 5 makes use of the RTOS for Java thread scheduling, hence allowing the mixing of Java threads and native (C/asm) tasks’ priorities.
There are several ways to interface Java to C or ASM in order to access drivers or legacy firmware. We will look at a simple solution called SNI (ESR012, Simple Native Interface). JNI is another way to interface native code but is much more complex and has superfluous features such as object manipulations that are not useful for designing GUIs for microcontrollers.
The SNI APIs principles (Figure 6 ) are straightforward: they allow direct calls of C functions from Java and of Java methods from C. Arguments of types such as integer or float and can be passed to transfer data. In the code example below is an illustration of how Java code can call C code:
In the same way, below is a graphical Java library integration example of how SNI allows sharing of data between both Java and C worlds, without buffer copies, using immortal object concepts that also allow DMA systems to share data with hardware peripherals.
MicroUI (ESR002) and MWT (ESR011)
MicroUI deals with one or several character-oriented displays and/or dot matrix LCDs (black & white, greyscale and color). It offers 2-D graphic APIs (basic shapes, image manipulation, etc.) and I/O for common devices such as pointers (touch, multi-touch, and mouse), buttons, LEDs and audio. MicroUI CPU usage can be highly optimized: display updates are clipped whenever possible, duplicated events are skipped and LCD refresh activity can be tuned to balance graphic activity against the overall application activity.
MWT provides a higher level framework than MicroUI to manage graphical widgets, containers, layouts, navigation, look & feel, etc. It provides the basis for the construction of specific GUI widget libraries.
MicroUI APIs to interface to the BSP of microcontrollers
The display interface supports many different organization structures according to the chosen hardware (displays with or without chip on glass and access types) and GUI characteristics (static ones or dynamic ones, requiring one or several buffers).
- Switch mode is usually used with on-chip LCD controllers and an external RAM memory device; the graphical stack uses a dual buffer as a back buffer and screen buffer. The LCD switches from one buffer to the other one to display the graphical content.
- Copy mode is usually used with LCD having a Chip-On-Glass (COG) controller; the graphical stack uses a buffer in RAM as a back buffer and LCD buffer as the screen buffer. The screen buffer is updated by a memory copy from the back buffer using the microcontroller DMA system.
- Direct mode also deals with Chip-On-Glass controller, featuring a display buffer directly accessible as a SRAM. The Graphic stack can draw directly in that buffer.
- Custom mode is all other solutions such as a display buffer accessible over a SPI link; the graphic stack needs to be designed with special routine to deal with custom pixel map addressing.
Interfacing to the display buffer is fairly simple; developers just need to implement a small C API such as:
Click on image to enlarge.
As shown in Figure 7 , GUIs often deal with a large number of events coming from a mouse or a touch pad. It is therefore important to use an optimized event system manager to quickly route events from drivers to Java listeners. In the event system, events are stored in a pump and are dispatched in the Java event system as integers (not objects) to reduce memory usage and increase processing speed.
Java technology is an appropriate solution for designing GUIs as it features both an Object Oriented language and a software processor that offers advanced mechanisms for dealing with highly variable software programs and event driven architectures.
Java platforms optimized for devices based on the ARM Cortex M Series have low memory footprints and high speed execution thanks to strong architectural choices in the Java platform design. These platforms can provide simulation capabilities for prototyping and testing GUI designs and are easy to integrate to legacy C software (RTOS and libraries) to offer seamless migration to higher level languages for GUIs.
Providing a high level of portability across a wide range of hardware/software platforms, developers can easily capitalize on their investments and deploy their GUI software to virtually any device such as low-end devices (simple RTOS, Linux, etc.) and smart 3rd party devices (IOS, Android).
Régis Latawiec is Executive VP Business Development, at IS2T Corp. He has a master’s degree in electronics and a Masters in Business Administration. With 20 years experience in hardware and software embedded design development, he was involved in GSM base-band IC design with Alcatel and he managed the Atmel MCU Application Group in France. As Marketing Director, he managed an Atmel 32-bit Flash-based MCU product line from its initial design stage to the commercial launch and first sales. At IS2T, Régis is in charge of product portfolio management and business development.He can be contacted at .
This article is based on material Régis Latawiec presented as part of a class at the 2012 ARM Technical Conference titled “Small footprint Java-based GUI design for ARM Cortex-M processors (ATC-310)’’ .