Get on the Internet of Things fast with an embedded Web app server: Part 2
As noted in Part 1 in this series, embedded Web Applications use the same HTTP communications mechanism as your bank account or the latest game on your smartphone. And why not? The HTTP protocol is simplicity itself, consisting of only nine operations to transport requests and responses.
Given this simplicity, it is not surprising that developers frequently underestimate the amount of time required to produce the code used to respond to those operations. Creating only a basic web server takes months of additional coding and produces a suboptimal solution precisely because the HTTP protocol offers nothing more than those simple operations. It’s for this reason that developers need to turn to web application servers.
This second part lifts the hood on how web application servers are set up and explains how to implement this optimal solution, providing links to a downloadable tutorial so you get a chance to play.
Embedded web applications require the remote browser to be able to ‘talk’ to the system or device under control. A web server provides no such capability, so everything between the simple processing of HTTP and the application code itself becomes overhead to be developed as a part of the system. This is an unwelcome diversion from the cool application-specific code that makes the device do what it needs to do: control hydraulics on tanks, position satellite dishes, monitor incubator temperatures, or control production plant, lighting, heating, nightclub lasers, and so on.
Application servers provide that missing link, delivering configurable functionality that makes the connection between the simple HTTP operations and the complex matters the application needs to deal with. This low-level generic work is already done, and coding can focus on adapting the requests and responses to your specific technology.
Figure 1: An Application Server includes the basic functionality of a web server to process HTTP operations, and complements it with proven, efficient mechanisms to interface to the application. A development framework and libraries make the developer experience as pain free as possible. This example shows a possible application in a road tunnel lighting system, varying light intensity to match outside weather conditions.
Figure 1 shows how this is achieved. The application server is a compiled C code application, and provides the functions that make life so much easier by supplementing basic HTTP handling.
Several of these functions are there to handle scripting languages. To understand the benefit of the application server as a whole, it is therefore essential to know why there is such emphasis on scripting languages such as Lua. If this neat technique can handle the C and C++ we know and love, then why bother developers with a new language? To answer that, let us consider the development cycle for an embedded web application.
Lua delivers the user experience
The necessary work can be broken down into the following categories:
- Managing data
- Parsing requests
- Assembling responses
If C is used for embedded application development, even its level of strict typing becomes a disadvantage because there is so much conversion from text to other types implicit in the whole mechanism. Lua’s flexible approach to data types addresses that.
Additionally, in C and C++, memory must be explicitly reserved and released as needed and that requires considerable coding effort. In contrast, Lua’s automated garbage collection removes that development overhead. With application servers and scripting environments, the application server abstracts the web server details, providing access to request and response objects and their associated APIs. Application-specific code only has to deal with high-level behavior and data.
If the C language has an Achilles’ heel, it is string manipulation. It is awkward, time consuming, and easy to get wrong. In many places where C excels, such as in hardware interfaces, the amount of textual information to be handled is small, and so the advantages outweigh this irritation.
However, creating dynamic HTML user interfaces typically involves a great deal of string manipulation and so, for these applications, scripting languages like Lua are far less error-prone and tedious. Unlike C, a single line of script can implement string concatenation with ease and automatically handle lower-level machinations. Scripts also access and manipulate data without requiring a definition of the right data type or organization.
Lua doesn't totally eliminate C from the equation. Instead, C is retained for what it does best. Scripting environments intentionally restrict scripts from getting down to the hardware level, so C routines, working similarly to drivers, connect the hardware to the web application.
Scripts compile just-in-time and execute on an embedded virtual machine, and that creates some additional overhead in comparison to compiled C code. However, the impact of that overhead on response time is negligible compared with the overhead of the network connection. Lua web applications typically respond with lightning speed, even when run in small, web-based embedded devices.
While this speed of response is important in user interfaces, it’s criticality becomes even more essential for applications where functionality depends on it. Take the control of a tank turret, for example. That turret needs to turn instantly once commanded.
Running together, the Application Server and Lua scripting engine deliver applications that run as much as 20 times faster than they would in a typical LAMP setup.
Why neither CGI nor LAMP works well for embedded
Some basic embedded web servers address this problem by using C callback functions instead. Confusingly, these are also sometimes referred to as CGI because they serve a broadly similar purpose.
Those choosing not to use CGI or any of these variations will often meld a Linux OS, Apache web server, MySQL database, and PHP scripts, a combination often referred to as a LAMP bundle. This solution works well for large-scale web server applications. However, when it comes to small, embedded applications driving web-based devices, LAMP requires more resources than are available.
As a result, such applications run intolerably slowly. LAMP is also restrictive in that typically it cannot be used with an RTOS.
Lua development how-to
Figure 2 pictures a development environment that lets the developer focus on application logic via high-level data structures in Lua scripts. Unlike most languages, Lua easily extends, not only with software written in Lua, but also with software written in other languages. It is therefore easy to use C for low-level access to the hardware.
To optimize our turret application, programming begins with writing C functions and compiling them using standard C tools. Depending on the OS requirements, you either compile them into a monolithic system or build a separate executable process. To connect Lua to the C functions, Lua bindings to the C functions link the application’s communication to the low, hardware level of the tank turret’s controller. As shown in Figure 2, once built, manual coding or an automation tool builds the bindings that are included in the C project.
Figure 2: You can write your applications as Lua scripts that interact with your own hardware. A C/C++ function made available to Lua is called a Lua binding. Bindings can be crafted manually or by using tools that produce bindings from your C header files.
The Lua scripts themselves are written using a straightforward, easy-to-learn procedural syntax, similar in nature to languages such as C, C++, and Java. Code Example 1 provides a Lua example showing how that tank turrent controller would work, and how the Lua bindings interface to the C code.
Code Example 1 – Implementing Lua Bindings. A C library for Lua defines a special function which is called to register all C functions of the library and store them in appropriate places.
Example 1 code comments
Lua’s just-in-time compiler automatically compiles new changes to the scripts. Suppose that the algorithm for positioning the turret is complex, and you need to experiment with new techniques to make it as precise and responsive as possible. Whenever you navigate or refresh the browser, the virtual machine compiles any changed script before it sends the requested response to the browser, thereby automating the script compilation and execution and giving you near instant feedback on the tweaks made to the algorithm.
This contrasts starkly to a C program that requires cross compilation on a desktop computer, linking, and finally uploading of the executable to the device. Figure 3 shows how this benefits the developer of both Lua and Lua Server Pages (LSP).