Get on the Internet of Things fast with an embedded Web app server: Part 2 - Embedded.com

Get on the Internet of Things fast with an embedded Web app server: Part 2

As noted in Part 1 in this series, embedded Web Applications use the same HTTP communications mechanism as your bank account or the latest game on your smartphone. And why not? The HTTP protocol is simplicity itself, consisting of only nine operations to transport requests and responses.

Given this simplicity, it is not surprising that developers frequently underestimate the amount of time required to produce the code used to respond to those operations. Creating only a basic web server takes months of additional coding and produces a suboptimal solution precisely because the HTTP protocol offers nothing more than those simple operations. It’s for this reason that developers need to turn to web application servers.

Part 1 discussed how the combination of a pplication servers, script pages, and a synchronous JavaScript technologies combine to provide an optimal approach to address these challenges. The embedded web application developer is free to focus on their application-specific code.

This second part lifts the hood on how web application servers are set up and explains how to implement this optimal solution, providing links to a downloadable tutorial so you get a chance to play.

Application servers
Embedded web applications require the remote browser to be able to ‘talk’ to the system or device under control. A web server provides no such capability, so everything between the simple processing of HTTP and the application code itself becomes overhead to be developed as a part of the system. This is an unwelcome diversion from the cool application-specific code that makes the device do what it needs to do: control hydraulics on tanks, position satellite dishes, monitor incubator temperatures, or control production plant, lighting, heating, nightclub lasers, and so on.

Application servers provide that missing link, delivering configurable functionality that makes the connection between the simple HTTP operations and the complex matters the application needs to deal with. This low-level generic work is already done, and coding can focus on adapting the requests and responses to your specific technology.

Figure 1: An Application Server includes the basic functionality of a web server to process HTTP operations, and complements it with proven, efficient mechanisms to interface to the application. A development framework and libraries make the developer experience as pain free as possible. This example shows a possible application in a road tunnel lighting system, varying light intensity to match outside weather conditions.

Figure 1 shows how this is achieved. The application server is a compiled C code application, and provides the functions that make life so much easier by supplementing basic HTTP handling.

Several of these functions are there to handle scripting languages. To understand the benefit of the application server as a whole, it is therefore essential to know why there is such emphasis on scripting languages such as Lua. If this neat technique can handle the C and C++ we know and love, then why bother developers with a new language? To answer that, let us consider the development cycle for an embedded web application.

Lua delivers the user experience
The necessary work can be broken down into the following categories:

  • Managing data
  • Parsing requests
  • Assembling responses

If C is used for embedded application development, even its level of strict typing becomes a disadvantage because there is so much conversion from text to other types implicit in the whole mechanism. Lua’s flexible approach to data types addresses that.

Additionally, in C and C++, memory must be explicitly reserved and released as needed and that requires considerable coding effort. In contrast, Lua’s automated garbage collection removes that development overhead. With application servers and scripting environments, the application server abstracts the web server details, providing access to request and response objects and their associated APIs. Application-specific code only has to deal with high-level behavior and data.

If the C language has an Achilles’ heel, it is string manipulation. It is awkward, time consuming, and easy to get wrong. In many places where C excels, such as in hardware interfaces, the amount of textual information to be handled is small, and so the advantages outweigh this irritation.

However, creating dynamic HTML user interfaces typically involves a great deal of string manipulation and so, for these applications, scripting languages like Lua are far less error-prone and tedious. Unlike C, a single line of script can implement string concatenation with ease and automatically handle lower-level machinations. Scripts also access and manipulate data without requiring a definition of the right data type or organization.

Lua doesn't totally eliminate C from the equation. Instead, C is retained for what it does best. Scripting environments intentionally restrict scripts from getting down to the hardware level, so C routines, working similarly to drivers, connect the hardware to the web application.

Scripts compile just-in-time and execute on an embedded virtual machine, and that creates some additional overhead in comparison to compiled C code. However, the impact of that overhead on response time is negligible compared with the overhead of the network connection. Lua web applications typically respond with lightning speed, even when run in small, web-based embedded devices.

While this speed of response is important in user interfaces, it’s criticality becomes even more essential for applications where functionality depends on it. Take the control of a tank turret, for example. That turret needs to turn instantly once commanded.

Running together, the Application Server and Lua scripting engine deliver applications that run as much as 20 times faster than they would in a typical LAMP setup.

Why neither CGI nor LAMP works well for embedded
Some basic embedded web servers address this problem by using C callback functions instead. Confusingly, these are also sometimes referred to as CGI because they serve a broadly similar purpose.

Those choosing not to use CGI or any of these variations will often meld a Linux OS, Apache web server, MySQL database, and PHP scripts, a combination often referred to as a LAMP bundle. This solution works well for large-scale web server applications. However, when it comes to small, embedded applications driving web-based devices, LAMP requires more resources than are available.

As a result, such applications run intolerably slowly. LAMP is also restrictive in that typically it cannot be used with an RTOS.

Lua development how-to
Figure 2 pictures a development environment that lets the developer focus on application logic via high-level data structures in Lua scripts. Unlike most languages, Lua easily extends, not only with software written in Lua, but also with software written in other languages. It is therefore easy to use C for low-level access to the hardware.

To optimize our turret application, programming begins with writing C functions and compiling them using standard C tools. Depending on the OS requirements, you either compile them into a monolithic system or build a separate executable process. To connect Lua to the C functions, Lua bindings to the C functions link the application’s communication to the low, hardware level of the tank turret’s controller. As shown in Figure 2, once built, manual coding or an automation tool builds the bindings that are included in the C project.

Figure 2: You can write your applications as Lua scripts that interact with your own hardware. A C/C++ function made available to Lua is called a Lua binding. Bindings can be crafted manually or by using tools that produce bindings from your C header files.

The Lua scripts themselves are written using a straightforward, easy-to-learn procedural syntax, similar in nature to languages such as C, C++, and Java. Code Example 1 provides a Lua example showing how that tank turrent controller would work, and how the Lua bindings interface to the C code.


Code Example 1 – Implementing Lua Bindings. A C library for Lua defines a special function which is called to register all C functions of the library and store them in appropriate places.

Example 1 code comments

Lua’s just-in-time compiler automatically compiles new changes to the scripts. Suppose that the algorithm for positioning the turret is complex, and you need to experiment with new techniques to make it as precise and responsive as possible. Whenever you navigate or refresh the browser, the virtual machine compiles any changed script before it sends the requested response to the browser, thereby automating the script compilation and execution and giving you near instant feedback on the tweaks made to the algorithm.

This contrasts starkly to a C program that requires cross compilation on a desktop computer, linking, and finally uploading of the executable to the device. Figure 3 shows how this benefits the developer of both Lua and Lua Server Pages (LSP).Lua Server Pages provide the ‘missing link’
Lua bindingspermit a neat interface between script (such as Lua) and C, but thenature of HTTP commands means that we must also interface between Luascript and HTML. Lua Server Pages (LSP) and the bindings for themprovide that ‘missing link between application tasks and the functionsin the application server managing the HTTP operations. Just as for theLua code, the just-in-time compilation mechanism makes development abreeze, as illustrated in Figure 3 .

Figure3: By using Lua applications and server pages, the results of anymodifications to code are seen almost instantly. Compilation both fromLSP to Lua and from Lua to executable code happens seamlessly andautomatically. This is in stark contrast to high level languages such asC or C++, which require a build cycle before they can be uploaded foruse.

The LSP scripts are the last component inthe event-driven mechanism. They ensure that there is an appropriateresponse each time the web server interprets an incoming HTTP operation.For example, suppose a remote control device adjusts an incubatortemperature control. The browser sends a HTTP GET request to theapplication server on the incubator temperature controller, whichresponds with the HTML form for the “adjust temperature” page. Thebrowser then typically sends an HTTP POST request when the newtemperature is selected (Figure 4 ), and the application server makes the necessary adjustment and responds with another HTML form.

Figure4: The HTML forms contain input elements that allow the user to enterand send information to the server, such as the temperature set pointfor an incubator. The values of the input elements are transmitted tothe server side when the user clicks a submit button.

Thebeauty of the LSP scripts is that they permit the server side logic tobe incorporated into the HTML form itself. The Lua Server Page (LSP) in code example 2 shows how this looks in practice.

Code Example 2: LSP is implemented using Lua script bracketed by tags within HTML.

Example 2 Script code comments

Applicationservers can handle C/C++ code in addition to scripts. They includesupport for different plug-ins, secure transactions, and various typesof I/O. They are very flexible; for example, they offer file storage andeven database functionality if required, and yet they need not have afile system at all. An API that remains consistent between differentoperating systems gives ease of portability, perhaps allowing the use ofthe same code for an entire range of incubator monitors, each usingdifferent hardware and OS solutions but controlled by the same handset.

Crisp user interface from asynchronous data handling
Anapplication server coupled with Lua scripts for high-levelfunctionality and C source code for hardware interfaces creates adevelopment environment that easily takes advantage of the simplicity ofHTTP operations while maximizing your programming efficiency. Awell-designed web application server does away with the lengthyhigh-level build cycle of code for everything but hardware interfaces.In so doing, it optimizes even the development work.

But is this enough to give clients what they expect to see?
Today’sdemanding customers expect attractive displays and smartphone-likefunctionality from all of their devices, even those that are compact andlow-powered, such as a dedicated hand controller for the positioning ofa satellite dish. Such expectations provide a challenge for embeddeddevelopers.
Web application servers help provide a slick graphicalinterface without requiring expensive hardware. The web app can senddata to and retrieve data from a server asynchronously (in thebackground) without interfering with the display and behavior of theexisting page. The use of AJAX (Asynchronous JavaScript and XML) or JSON(JavaScript Object Notation) techniques provide this functionality andensure an impressive response.

Web applications can easily beenhanced by JavaScript code to deliver and present real-time data fromthe server. In this exchange, the server initially generates the bulk ofthe user interface when responding to the initial GET request. Afterthe browser loads a server-generated HTML page, it executes the embeddedJavaScript code inside the HTML response data. This Javascript codecan, for example, connect back to the server and fetch data in realtime. Figure 5 shows how this instantly updates the client withouthaving to refresh the page.

Figure5: JavaScript code in the dynamically created HTML page uses Ajax whenconnecting back to the server. In this example, we submit an AJAXrequest for information about a satellite dish signal strength whileprocessing an overview form, which is dynamically updated concurrentlywith the form being presented on the client side. The form submission inresponse to change from the client side is just as before, as in thisexample with the request to change the angle of the dish.

TheXMLHTTPRequest object enables JavaScript code to send HTTP commandsdirectly to a server, requesting, for instance, the strength ofsatellite signal. The browser's XMLHTTPRequest object enables theJavaScript code to use a callback function which allows it to return thesignal strength data asynchronously. Because of this, there is noblocking while the client browser waits for that information.

Fromthe user’s perspective, the client browser continues seamlessly withoutany apparent ‘freezing’. The presentation of the remainder of the formcontinues seamlessly and the real time data signal strength data appearswhen it becomes available.

The XMLHTTPRequest API is weird andcumbersome, but luckily many JavaScript libraries such as the jQuerylibrary makes it very easy to use this object. jQuery is ideally suitedsince it is small and makes it easy to write compact JavaScript codewithout having to learn the details of the browser's Document ObjectModel, as illustrated by code Example 3 .


Code Example 3: Using Ajax to respond to the satellite dish client application request for a signal strength update

Example 3 Script code comments

Embedded web apps: a different environment
Theunderlying HTTP communication technology for Web-based applications isthe same, whether embedded or otherwise. But the difference inenvironment for the embedded sector makes a massive difference to theoptimal deployment of that technology.

In this environment, youwant developers to spend their time on application-specific code whetherit controls tunnel lights or a satellite dish or drives safety-criticalapplications like a tank turret or incubator, where speed andefficiency are vital.

Using an application server specificallydesigned for embedded web applications ensures that generic HTTPcommunications are tackled with approaches which are optimized for thedemands of limited resource, deeply embedded applications.

Whenused in conjunction with Lua Server Pages and AJX/JSON asynchronous datahandling, the developer gains three technologies that combine topowerfully create the same quality of dynamic application they’ve cometo expect from advanced smart phone and desktop applications.

Withthese tools, teams can focus on application development that givesusers the interface they want, while letting the team dedicate most ofits energy to the applications only they do best.

Embedded web apps the easy way
To be used in combination with this two part series on Embedded.com, we have created four online tutorials you can download using browsers running on Windows XP, Vista, and Windows 7 & 8. Theywill give you the fundamentals for writing your own web applicationsand interfacing these applications to your hardware. It's not technicalto run; just download and run the executable. After unpacking, a blackconsole window will open with some instructions. The demo will open yourbrowser automatically after 30 seconds if you do not do anything.

Afterdownloading the self-extracting zip file, start the self-extracting zipfile and follow the self extracting zip wizard. Unzip to any directory.The self extracting zip file should automatically start the unpacked‘startdemo.bat’ batch script, which in turn starts the demo server in acommand window. Closing this window closes the server. You canrestart the demo at any time by restarting the ‘startdemo.bat’ batchscript. Uninstall by deleting all files unpacked by the self extractingzip file.

Read Part 1

Wilfred Nilsen , Founder & CTO of Real Time Logic ,has25 years' experience in designing embedded software. Powered by avision of a connected embedded systems, he designed the BarracudaApplication Server, tailoring it for the small footprint, real-timeneeds of embedded microcontrollers and microprocessors.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.