Building eye-catching GUIs for your embedded MCU designs - Embedded.com

Building eye-catching GUIs for your embedded MCU designs

This article describes how integration of 2D Direct Memory Access (DMA2D) in microcontrollers creates new possibilities for graphics use in user interfaces by placing limits on device and application coding costs, at the same time improving core processor performance. It describes the key features of DMA2D and how developers can implement these features in application software, with code examples that engineers can use in their own applications. The examples cover techniques for:

  • Handling of bitmaps (display, overlaying and moving)
  • Using transparency (pixel-level, global and blending)
  • Dynamic versus Read-Only objects
  • Management of objects and touch sensing
  • Use of layering (Z-order)
  • Transformations of graphic objects
  • Use of memory spaces including external SDRAM resources.

Chrome-ART acceleration for MCU GUI applications
Thanks to the ease of implementing touch sensing and the improved price points for displays and 32-bit MCUs, graphical user interfaces (GUI) have become increasingly present in MCU applications. Because they are attractive, flexible, and interactive, GUIs are an excellent way to enhance both application functionality and the user’s experience.

In spite of this evolution, some technical challenges have persisted. Typical GUI implementations on microcontrollers (unlike high-end processors targeting mobile appliances) have been limited by how much of their processing power, peripherals, and memory resources could be allocated to managing the display and graphics.

Relatively limited memory, processing power, and resulting code complexity all conspired to make implementation costly and responsiveness of the GUI relatively limited.

The demonstration described here illustrates how integrated 2D Direct Memory Access (DMA2D) and the right combination of supporting peripherals can open a new world of possibilities for graphics use in general purpose microcontroller applications (Figure 1 ). For our example we used ST Microelectronic’s STM32F429.

Figure 1: Multiple bitmaps used in a dynamic GUI on a general purpose MCU

DMA2D integration is the keystone to this graphics potential in the MCU we have selected. Through a set of registers, the STM32F429 manages graphics with minimal code and impact on processor performance. Graphics capabilities supported by DMA2D include:

  • Handling of bitmap files (up to 32 bits per pixel)
  • Management of image overlays, with or without transparency
  • Handling of object display to include multiple objects and their positions, sizes and layering (Z-order)

The DMA2D hardware
Chrom-ART Accelerator is the commercial name for STMicroelectronics’ implementation of the DMA2D peripheral on the STM32F429. Figure 2 shows the internal structure of the DMA2D and integrated features that make it particularly advantageous for managing graphics.
Its features include:

  • Two input pixel format converters (PFC). These blocks are able to read and to decode bitmap files
  • The blender, which computes and mixes the data from the two input PFC
  • The output PFC, which decodes the information to be sent to the destination
  • The FIFOs used for both the inputs and the outputs connected to a specific DMA controller

Integrating blending, encoding, and decoding functionality in the microcontroller itself relieves these requirements from the application code and the core processor.

Figure 2: Internal structure of Chrom-ART 2D Direct Memory Access (DMA2D)

DMA2D assistive apps on the STM32F429
In addition to DMA2D, the STM32F429 provides other integrated controllers that facilitate the optimal implementation of a GUI. These include:

  • An SDRAM controller for the addition of cost-effective RAM resources for stocking graphical elements
  • An LCD-TFT controller to enable writing of graphics directly to the display for improved efficiency and performance
  • A digital camera interface

In the past, these types of peripherals and features were reserved for high-end or application-specific processors. Notably for DMA2D, this is the first time we see it integrated in a standard, general-purpose MCU like the STM32F429. The appearance of these peripherals in a general purpose device represents a major evolution for microcontroller users and opens a range of application possibilities.

The EvoPrimer software/hardware environment
The examples described here were developed using the EvoPrimer development platform (Figure 3), its CircleOS software environment, and DMA2D API. This has some impacts on the manner in which the DMA2D is used in the demonstration.

You can watch a video of this demonstration . Source code, board schematics, bill of materials and documentation are available to download.

Figure 3: EvoPrimer features the STM32F429ZI microcontroller and 2 Mbytes of additional SDRAM

EvoPrimer features the STM32F429ZI microcontroller and 2 Mbytes of additional SDRAM. The demonstration application uses the microcontroller’s DMA2D and SDRAM controller. EvoPrimer already includes a controller for its color TFT touchscreen and no camera is available on the platform, so neither the integrated LCD-TFT controller nor camera interface are used here.

For this demonstration, a software API encapsulates the functionality of the DMA2D. This implementation is based both on the EvoPrimer hardware platform and the CircleOS software environment. In addition to an API for STM32 peripherals, CircleOS provides native capabilities for drawing objects, lines, text, and pixels. However, the source files specific to the DMA2D are designed to be easily extracted and re-used without CircleOS.

In this article we will talk about virtual spaces called “screens” and displayed “objects”. In the application, “Screen” refers to a virtual area that can be shown on the LCD display, a bit like a desktop. A screen has a visible zone the size of the EvoPrimer LCD display (240 x 320 pixels).
“Object” refers to a graphic rectangle in bitmap format that can be read from memory and written to the LCD display using the DMA2D. An object is declared as a rectangle, but any form could be considered if transparency is used in the bitmap. In this demo application, these objects such as icons or a background image are attributed to a virtual “screen” in order to be displayed. Objects are placed in the screen by X, Y coordinates. The coordinates can be outside the displayed area of the screen. The visible area can also be moved like a window to display objects in virtual space. This is demonstrated in detail in the next sections.

Using the API provided in the DMA2D files of CircleOS is a simple way to manage the graphics capability of the DMA2D to include use of two or more virtual screens, with up to 256 graphic objects per screen  and display and movement of objects anywhere on the two-dimensional space of a screen.

Keep in mind that, in its simplest form, an STM32F429 application can use the DMA2D to write a bitmap directly to the LCD. The implementation in the demonstration application is a little more complex, but allows us to explore a wide range of possibilities for DMA2D use in a graphical interface.

Figure 4 shows how DMA2D and SDRAM are used in the demonstration application. The application calls bitmaps (objects) from memory and includes the information for their positions (X, Y coordinates), layer (Z-order) and attributes them to a screen.

Figure 4: DMA2D blends the objects and stores the resulting object directly to SDRAM

The DMA2D blends the objects and stores the resulting object directly to SDRAM. In this demonstration application, the DMA is used to transfer the object to the display. SDRAM is also used to store dynamic objects such as texts or forms that are created using CircleOS native draw capabilities for rendering forms, text, lines or pixels.

In the C program, a bitmap must be declared as a byte array initialized by the contents of the binary file (translated into an .h text file). This example describes the code used to display the 640 x 320 pixel bitmap (photo_x8 with 8-bit indexed color palette) shown here, on a display that is 320 x 240 pixels. Because the image is larger than the size of the LCD panel, it will automatically be cropped by the DMA2D API, as shown in Figure 5 .

Figure 5: If an image is larger (shaded) than the LCD panel (clear display), it will automatically be cropped

Below is the code to declare for both this background image and a bird character (bird_A888) that is used later in the demonstration:

To display the picture photo_x8 as the background of a screen, all you have to do is to create an object, create a link between the object and the bitmap, then make the bitmap visible.

The palette for the bitmap is loaded automatically by the DMA2D, based on the header information that is decoded by the application software (in this case the API).

The last command (DMA2D_ScreenRedraw) scans the list of objects assigned to a screen and repaints all the visible ones. Note that the size of the background image (640×420 pixels in this example) can be bigger than the size of the screen (320×240 for our demonstration platform).

The bitmap can be positioned based on provided coordinates. 0 for X and Y axes are the lower left had corner of the LCD. To fill the entire display area of the LCD, the coordinates X and Y must be negative (or equal to 0). Positive coordinates will leave an area that has not been drawn starting in the lower left corner of the display to the coordinate where the lower left corner of the background image is displayed.

If you want to center the image, you can “move” the background before redrawing:

DMA2D_ObjectMove(BG_Image, PosX_BG = -160, PosY_BG = -120 ); //set the position

Because the overall object is too big to be displayed in our screen, it is automatically cropped at the borders of the LCD panel.

Overlaying bitmaps
The DMA2D allows other objects to be placed on the background (Figure 6 ).Overlaying objects on a background or on other objects is particularlyadvantageous with the DMA2D because only the areas affected by amodification are redrawn.

Figure 6: DMA2D allows other objects (grey butterflies) to be placed in the background

Withoutthe DMA2D, changing the position of a displayed object requires eitherpixel-by-pixel redrawing of the full display, or complex applicationcode (and a lot of processor time) to manage this functionality in thesame manner as the DMA2D. With our STM32F429, the DMA2D peripheralwrites only the affected pixels. So, to add an object over thisbackground, we simply create it, set the position (relative to thedisplay and not the background) and make it visible:

Moving Objects
Tomake an object move through the display (for example to make our birdfly), it is only necessary to periodically recall the last line with anew, different position (x_bird, y_bird):

TheDMA2D hardware peripheral is addressed only when calling theDMA2D_ScreenRedraw function. The rest of the API functions justconfigure the descriptions of the objects that will be used by theDMA2D.

Note that it is possible to animate the flying bird bymanaging several bitmaps, each showing the bird in a different state ofmovement. Then we periodically change the bitmap assignment. This is howthe butterfly is handled in the demonstration:

Thanksto the integration of the DMA2D peripheral, the display changes thatallow us to animate movement are represented by just a few lines ofsimple code.

Transparency
Transparency of an object that is overlaid on the background is handled at both the pixel level and globally.

Alpha component for the pixel First, transparency may be native to the original bitmap. A 32-bitbitmap file contains 8-bit transparency information for each pixel(24-bit color plus an 8-bit alpha). The transparency is also referred toas the “alpha” component. If you choose a file in a 32-bit format fordisplay using the DMA2D, the original transparency for each pixel ismanaged by the DMA2D. If you choose any other, smaller format, the alphaconstant is not defined in the bitmap and therefore is assumed to beequal to 255 (an opaque object) when displayed using the DMA2D.

Inaddition to the native transparency of a specific bitmap, the DMA2D canbe used to manage transparency according to a global alpha constant.Global constant alpha. Global alpha constant is used to define theglobal transparency of an object when it is displayed. If an object hasboth the original bitmap transparency and a global transparency, theresulting alpha value for each pixel will be the combination of the twoparameters. The two alpha values are multiplied and the result dividedby 256:

Thismeans that a fully transparent (invisible) pixel in the bitmap (alpha =0 in the bitmap file) remains invisible regardless of the globaltransparency of the object.

Blending
Figure 7 shows how alpha factors are combined when objects are overlaid. TheDMA2D allows complex combinations in blending, which are not exploredhere.

For the purpose of this demonstration, each time an objectis displayed, the API assumes that the “current background” is opaque(alpha = 255). “Current background” is the term that we use for theresult of the display of all the previous layers/objects.

Figure 7: When objects are overlaid the result is a blending

Thefinal pixel is the result of the combination of all the alpha values ofthe original bitmaps, weighted by the global alpha values of eachobject. Note that if the object at the top level (the highest z-order)is opaque (alpha=255 for both the object and the pixel), this object isdisplayed without any interference from the lower objects.

Thefollowing code shows how to create, in an editable object, a fullytransparent part, and an opaque part that will be globally tunable bythe global transparency ratio:

Editable objects: dynamic vs read only
Upto this point, we have only handled graphics that were edited on a PCand saved in BMP format. The bitmap was included as an “.h” file intothe FLASH that is treated by the application as read only memory.

TheAPI allows objects to be handled in RAM as well. These objects can bedynamically edited by the application. In the image in Figure 8, thetext is written directly onto a dynamic object.

Figure8: When text is written directly onto a dynamic object in an image, theEvoPrimer API allows objects to be handled in RAM or in FLASH

The red ellipse itself has been drawn using the standard library functions as shown in this code:

Editing a dynamic object The DRAW commands shown above typically pertain to the whole screen.The coordinates x = 0, y = 0 are the coordinates of the lower leftcorner of the LCD. After calling DMA2D_ObjectSelect(), these DRAW_xxxcommands are assigned to the currently selected object. The coordinatesare considered relative to the object and (0,0) matches with the lowerleft corner of the rectangle containing the object.

When adynamic object is declared, a move command is applied to the wholeobject. This means that the object will move with its contents (theellipse and the text).

Note that the object is always stored in RAM as a 32-bit image. Each pixel has a 24-bit color and an 8-bit alpha component.

Z-Order With DMA2D, objects are sorted to define the order of drawing. They arestored in an array, and the index in this array is named Z-order.Graphically, the Z-order specifies the relative position of the objectlayer (eg. Its position on the Z-axis). “Z-order == 0” corresponds tothe first drawn object (typically the background).

The APIprovides functions to increment/decrement the objects Z-order. Twoothers functions allow placement of the object at the upper layer, or asthe background as shown in Figure 9 .

Figure 9: The EvoPrimer API provides functions to increment/decrement the objects Z-order

Touchscreen and objects
Whenimplementing objects in a touchscreen interface, the functionDMA2D_ObjectFind returns a pointer to the object that is the owner ofthe pixel located at (x,y). When several objects are layered at the sameposition, the uppermost layer is considered.

So, when the touchscreen is pressed, it's quite simple to detect which object has been touched:

Object transformations
Awide range of object transformations are possible. Transformations ofobjections are enabled by two basic, generic actions – either specifyingthe contents of the destination pixel, or specifying where a sourcepixel will be copied to a destination object.

A few examples show the variety of object transformations that result from these two principles, including:

  • Copying the contents of an object onto another object
  • Rotating an object on itself (without using an intermediate buffer)
  • Shrinking/expanding the size of an object
  • Modifying some colors/alpha characteristic for any pixel

Theseexamples are not exhaustive. CircleOS allows specification, on onehand, of the source and destination objects (or a part of an object),and on the other hand the transformation operation to apply. Thetransformation is fully defined by the programmer, and only the corefunctions are embedded in CircleOS.

Managing memory capabilities with the STM32F429
Large Internal Memory New microcontroller families embed very large internal FLASH memory.The STM32F429 contains up to 2MB of internal FLASH. In FLASH, a 320 x240 pixel image requires a bit more than 75 K bytes when the colors areindexed in a 256-byte palette. Therefore, a few background images can bestored without problems.

SDRAM Controller Storingbitmaps in FLASH is easy as long as there are not too many bigpictures. But storing an editable object in RAM is more difficult tomanage, since the size of the internal SRAM is always much smaller thanthe size of the internal FLASH.

Instead of embedding a very largeinternal SRAM, the new STM32F429 features an internal controller for anexternal SDRAM. Because SDRAM is very cheap, this is a simple way toprovide a few megabytes of dynamic RAM that can be used for largedynamic graphic objects.

Screen Images in RAM Forthe purposes of this demonstration with EvoPrimer, an image of thescreen is kept in the SDRAM. This is because it is not convenient toaddress the RAM of the LCD on this hardware platform. This image is sent(using the DMA) to the LCD when a DMA2D_ScreenRedraw has been executed.This mode allows management of several screen images in SDRAM, andperforming of rapid switches between these screens.

In the caseof CircleOS, one Megabyte (half of the SDRAM) is reserved for the use ofthe DMA2D. Two screen images occupy 300 K bytes (2 x 320 x 240 x 16-bitpixels). The rest (724 K bytes) can be used for objects.

The DMA2D revolution in general purpose MCUs
Therevolutions that the DMA2D represents are both in its adaptation to aspecific use (in our case the display of graphics on the LCD) and itsinclusion in a general purpose microcontroller. As shown in thisdemonstration, the DMA2D enables a range of GUI possibilities (extensiveuse of bitmaps for graphic objects, management of movement,transparency and layering) that is a vast improvement over thepossibilities presented by microcontrollers without a DMA2D.

Inaddition, a savvy implementation of the DMA2D reduces the cost of GUIimplementation. It makes hardware more cost-effective throughaffordability of the general purpose MCU, low-cost external memory, andelimination of external LCD controller. It also reduces development andmaintenance cost with code that is simple and accessible.

The source code used for this demonstration is currently available at www.stm32circle.com .The DMA2D Chrom-ART accelerator by ST is also the topic of anapplication design competition that will end in February, 2014. Completecontest information is provided at www.stm32circle.com .

Francis Lamotte is vice president of KEOLABS. He is responsible for the company’sresearch and development. With more than 20 years of experience intechnology research with partners that include STMicroeletronics andNXP, Francis guides KEOLABS’ R&D activities and serves in anoversight role, defining the business and technology priorities for thecompanies' microcontroller tool activity. Prior to KEOLABS, Francisdirected Raisonance S.A.S., the microelectronics tool company that hefounded in 1988. 

1 thought on “Building eye-catching GUIs for your embedded MCU designs

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.