Using OpenVG for 2D graphics in auto infotainment displays

Hugo Osornio, Luis Olea, and Ioseph Martinez, Freescale Semiconductor - March 27, 2013


OpenVG is an API designed for hardware-accelerated 2D vector graphics. It was designed to help manufacturers create more attractive user interfaces by offloading computationally intensive graphics processing from the CPU onto a GPU to save energy. It is being used across many platforms, including a number of embedded applications. In this article we will describe the basic capabilities of the OpenVG spec and how to use it in automotive infotainment and instrument display applications.

OpenVG handles two main parent objects:
Bitmap: a memory organization used to store/draw an image. Every x and y coordinate will have a corresponding pixel. Each pixel is assigned an alpha, red, green and blue (ARGB) value, thus a 200 x 200 resolution image will require (200*200*(color depth)) bytes of storage. Bitmaps are particularly useful for photorealistic graphics. The coverflow application depicted in the Use Cases section contains image handling code and functions.
Vector: a mathematical formula that requires start and end points. Depending on how those points are connected, other parameters may be required, such as curve control points, arc radiuses, and arc angles. This feature makes a vector fully scalable, as it does not rely on per-pixel data to represent an image.

Figure 1 depicts how both circles appear to be the same when viewed at a normal size, but when zooming in, the bitmap image begins to lose fidelity, while the edge remains sharp in the vector image.


Figure 1: Vector vs bitmap

Vectors do not hold an advantage over bitmaps in all scenarios. Though bitmaps appear sharp at a fixed size, they become pixilated when scaled. Bitmaps are also faster to process but require more memory. Vectors maintain their image quality at any scale/size and occupy less memory. Vector images can only come from vector design tools, while bitmaps can be a photograph, or an image from any source or pixel-based tool.

Advantages of OpenVG
OpenGL can accomplish many graphical tasks, such as fonts, 2D GUIs and HMI. However, its strength lies in polygon and texture pushing power. OpenGL is bitmap oriented, and simple graphics can take precious time from that powerful OpenGL core.

OpenVG closes the gap on graphical needs. On its own, it can provide for great applications, and in combination with OpenGL, it can manage fonts, vectors, and most 2D and pseudo 3D elements, while an OpenGL core dedicates its time to doing what it does best.

When used stand-alone, OpenVG can accomplish a lot with low current and memory consumption. When combined with OpenGL, OpenVG manages the tasks the OpenGL core was not designed to perform.

Raster images (bitmaps)
Handling images in OpenVG is straightforward. Follow these steps to draw an image on your rendering surface:
  1. Declare a VGImage object.
  2. Create a VGImage and assign the handler to your VGImage object.
  3. Fill the image with the bitmap data.
  4. Set the matrix mode to VG_MATRIX_IMAGE_USER_TO_SURFACE in order to transform images (3 x 3 afine transformations).
  5. Draw your image.

Apply the same steps in pseudo code:

  VGImage ovgarticle_image
  ovgarticle_image = vgCreateImage(VGImageFormat format, VGint width, VGint height, VGbitfield allowedQuality)
  vgImageSubData(VGImage image, const void * data, VGint dataStride, VGImageFormat dataFormat, VGint x, VGint y, VGint width, VGint height)
  vgSeti(VG_MATRIX_MODE, VG_MATRIX_IMAGE_USER_TO_SURFACE);
  vgDrawImage(ovgarticle_image)


To perform this operation, you must be familiar with certain bitmap aspects, such as stride, bit depth, and set API properties such as blending.

Paths
Paths form the core of vector-accelerated graphics. To fully grasp this concept, you must understand the following:

A simple triangle path can serve as an example. First, you must define the segments that will integrate the whole path. The segments of a path are described as a series of commands stored in an array of bytes. Each command (Table 1) will consume one or more points.



Table 1: OpenVG commands for drawing a triangle



Figure 4: Drawing a triangle with OpenVG

Complex paths involve the use of arcs and bezier curves.

The steps to create and draw a path in OpenVG include:


Page 2 of 2
Coverflow animation
One of the most common yet eye-catching animations is the 2D perspective transform often used to represent album covers on audio playback devices. This type of animation gives the perception that the images are moving in a 3D space by moving and rotating on the x, y and z axes. Rather than implementing this animation via 3D GPU or a 3D processing engine, this type of transform can be defined in a 2D space, and OpenVG provides the necessary API to achieve this type of animation.

The mathematical concept is called “3D projection” and consists of mapping a set of 3D points onto a 2D plane. The concept is simple. Imagine a flat, squared object such a piece of paper or a picture. You can put this picture in front of you in such a way that it looks like a rectangle. If you bring it closer to your eyes it will look bigger, and if you pull it away it will look smaller. Now, if you rotate the picture using its vertical axis it won’t look like a rectangle anymore. It will have some acute angles and can be made to look like a rhombus or a straight line. This is accomplished by the 3D object projecting different points (vertex) into your eye. A more technical clarification of this concept is available in a Wikipedia entry on 3D projection.

With graphics, spatial image manipulation always ends up with a matrix. To scale a matrix, you must multiply your image points by a matrix. To do something as simple as translating an image, you also need a matrix multiply. To create a coverflow animation, again you need a matrix. All spatial manipulation in OpenVG is accomplished by providing a transformation matrix. A simple example of drawing an image rotated in the y axis is as follows:

  vgLoadIdentity();
  vgLoadMatrix(perspectiveMatrix);
  vgDraw(image);


The result would be something like Figure 5.


Figure 5: Example of an image rotated in the y axis

The key is obtaining the “perspective matrix” needed to achieve the 3D effect. This requires some additional math. The matrix can be created by using the equations described in a Wikipedia entry on 3D graphics (http://en.wikipedia.org/wiki/3D_projection). There are other facilities provided by the OpenVG API that include additional calculations required to come up with the proper matrix for the perspective transform:

  vguComputeWarpQuadToSquare
  vguComputeWarpSquareToQuad
  vguComputeWarpQuadToQuad


These utility functions from the OpenVG API allow you to compute a matrix of a projective transform from a quadrilateral to a quadrilateral, squared image to any given quadrilateral or vice versa. This allows you to move the vertex of an image to any desired position. Calculations are simplified by using these functions when performing coverflow animations.

The code to perform a perspective transform is implemented by the following:

//Input parameters
// H_angle: angle in radias on which the image is rotating (image rotates on the Y axis)
// width: width of the original squared image
// FOV: Field of view, ratio between the projection plane and the actual object
// pivotX: Pivot point in X; indicates which point in x to use for rotation.

// Author: Davor Bogavac, Freescale Semiconductor
void PPMatrixX(VGfloat H_angle, VGfloat width, VGfloat height, VGfloat FOV, VGfloat pivotX)
{
  VGfloat matrix[9]; // Matrix that will be used for the multiplication
  VGfloat Xw; // Holds the proportional width due to the rotation
  VGfloat Yo; // Holds the proportional height on the reduced side
  VGfloat Rx; // Delta width ( Rx + Xw = width)
  VGfloat Po,Pon; // ratio of reduction affected by X pivot point offset

  Xw = cos(H_angle) * width; // transformed width
  Rx = width - Xw; // width delta
  Yo = sin(H_angle) * (height/2) * FOV; // height reduction due to horizontal angle
  Po = pivotX/width; // ratio of reduction affected by X pivot point offset
  Pon= 1.0f - Po; // inverse ..

  //Compute the matrix for the main image
  vguComputeWarpQuadToQuad(
    (Po * Rx), height + (Po * Yo) + rD,
    (Po * Rx) + Xw, height - (Pon * Yo) + rD,
    (Po * Rx) + Xw, (Pon * Yo) + rD,
    (Po * Rx), (Po * -Yo) + rD,

    0, height,
    width, height,
    width, 0,
    0, 0,
    matrix1
  );
vgMultMatrix(matrix); //Multiply the main image matrix
}

Fonts
There are two types of fonts in OpenVG: raster and vector. The OpenVG API provides facilities for both types and has functions for defining and registering a font (vgCreateFont). Once this is complete, using the font is easy with the vgDrawGlyphs function. A vector vs. raster font comparison is outlined in Table 2.



Table 2: Vector vs. raster font comparison

The programmer must consider the information in Table 2 and decide which type of fonts to use in their application. Often, this can be a combination of both, depending upon different graphical needs. For example, zoomable text could be applied using vector fonts, while text that does not change in size, such as button labels, can be a raster font.

OpenVG use case: automotive displays
Freescale and Echilibra (see note at end of article) have developed a completely vector-based automotive cluster demo that takes advantage of the complete Freescale ecosystem, as well as Vybrid controllers  that use a GC355 OpenVG 1.1 GPU from Vivante and i.MX 6 SoloLite, 6Quad and 6Dual applications processors. Design files and assets, even fonts, were imported directly from Inkscape and Illustrator.

In both demo versions (Figure 7), the speedometer relies on large fonts to convey speed information, which involves scaling up the current font before drawing it to the render surface. Not even OpenGL  provides that kind of capability.

A simple animation engine was developed to handle scaling, rotation and translation. Each path and raster image on both clusters can be easily animated using that engine.

Complex animations such as GPS turn-by-turn navigation were easily accomplished via the vgInterpolatePath function. This function receives start and end paths (with similar segments), a destination path and the amount of interpolation you want to execute. The destination path data is interpolated between start and end paths by the amount selected. This allows you to create a complex path transformation by defining only two path states (Figure 6).



Figure 6: An OpenVG complex path transformation defined with only two path states





Figure 7: Two speedometer images built using Open VG

How the images were created
A primary advantage of OpenVG is the ability to easily create animations with the provided API. Once you have an EGL surface on which to draw, you just need a couple of lines to bring an image to the screen and a couple more to animate it.

In automotive applications, one common requirement is to implemenet crisp, realistic looking needles or gauges. In OpenVG this is a sraightforward operation that can be achieved by rendering at the highest frame rate possible (for example 60 fps).

  vgRotate(angle);
  vgDrawPath(path);


Other transition effects can be achieved by using common transformations such as scaling. Scaling doesn’t just allow you to increment or decrease the size of a form, it also allows you to do so independently on the x and y coordinates. In the case of the needle animation, needles fade in or zoom in. One animation involves making needles appear to grow larger while maintaining their width:

  vgRotate(angle);
  vgScale(0,yGrow);
  vgDrawPath(path);


To properly perform this animation, the needle must be completely vertical.

Reflection is another stunning effect that can be easily implemented using OpenVG. A reflection can be added to a horizontal image with some level of transparency for a glossy surface effect. A predefined function to flip an image does not exist. Instead, a flip matrix must be loaded before drawing the image:

  float matrix[] = {0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0};
  vgLoadMatrix(matrix);
  vgDrawImage(image);


Some of these effects are demonstrated on Freescale’s Qorivva MPC5645S platform architecture in in a YouTube video (http://youtube.com/watch?v=Vkqr6EsqcIM).

Note:
Echilibra is a startup company based in Guadalajara, México. Its main areas of focus are Design of UI interfaces and Embedded Systems. Although the link to their website is live, they are still in the process of adding content as of 3/27/13. 

Hugo Osornio has been rolling on the Automotive Industry since 2008. He started as an Infotainment and Safety Engineer at Delphi and then joined Freescale in 2010 as an Automotive Systems and Applications Engineer. He mainly supports Driver Information Systems products and applications. His current areas of interest are Computer Graphics and Embedded OS.

Luis Olea obtained a bachelors degress in Electrical Engineering in 2009. He spent a little over a year working as a Powertrain Software Engineer at one of Freescale's customers R&D centers, before deciding to join Freescale in March 2011. He currently works with Freescale's automotive devices in different applications (body, cluster) and in several activities (customer support, validation, documentation).

Ioseph Martinez has been working on graphics since 2008 specializing in Display Controllers and 2D GPU. He has worked developing display drivers, converter tools, graphic demo applications and leading an applications engineering team focused on automotive cluster applications. He is currently focused on Freescale’s Vybrid controller solutions.