Using AWS Jobs to upgrade and configure IoT devices

June 17, 2019

Sergey Zhukov-June 17, 2019

Amazon Web Services (AWS) is one of the most popular framework environments for the Internet of Things (IoT) alongside Microsoft Azure and Google Cloud IoT. Smart devices are connected to the framework using the Internet and interact with it using the MQTT protocol. Besides interacting with devices, the framework also provides great opportunities for data storage and processing, data representation to a user, data analysis (including artificial intelligence methods), access control with a powerful system of privileges, and a lot more.

For storing data, the AWS environment provides (besides different relational and non-relational DBMS) a cloud-based hierarchical file storage system called Simple Storage Service (S3). Each file in S3 storage can have a universal resource locator (URL), accessible from outside. In this case, the file can be accessed via a web browser given appropriate access privileges. If the file content is an HTML page, then, using this file, an interactive user can access both AWS framework options and intelligent devices connected to it. The capabilities of this page are specified by the JavaScript code it has inside (this code can activate functions of the application programming interfaces (APIs) of the framework as a whole and its separate components).

Lambda functions

Besides webpages, program code in the AWS framework environment can be stored as lambda functions. These are special named pieces of code, written in one of the following languages: Python, Java, C#, or Node.Js. They are stored in the cloud and are invoked on certain events. An event can be initiated by a webpage (like calling a certain HTTP REST API on a certain URL), by another lambda function, or by an intelligent device (via sending an MQTT message of a certain type). In all of these cases, events can have parameters. Lambda functions are used as middleware for the interaction between intelligent devices, AWS resources (e.g., databases), and the webpages that the user directly interacts with (Figure 1).

click for larger image

Figure 1. Architecture of AWS component interaction (Source: Auriga)

There are hard limits for AWS lambdas, for example, the execution time of handling a single request is limited, the amount of memory that a lambda can use when handling a single request is limited. If any limit is exceeded, execution of the lambda is aborted. These limits are configured by the user when creating the lambda but cannot exceed certain values.

An IoT device connects to the cloud using the TCP protocol, which provides data integrity and buffering; in the case of a slow connection, the protocol takes care of accumulating data on the sending side and pushing it through the pipeline when it becomes possible. Also, an AWS protocol on top of TCP takes care of re-establishing the TCP connection persistently in the case of connection loss.

However, the connectivity issues between an IoT device and the cloud do not normally affect lambdas due to the specific unidirectional nature of the MQTT protocol. When communicating with an IoT device, a lambda just sends an MQTT message and does not wait for a response; if and when the response arrives, it is the responsibility of a different lambda function to handle it, and send another MQTT message to the IoT device, if needed.

AWS Jobs

One of the AWS framework components is job service (AWS Jobs). It is used for creating and executing long-lasting actions (jobs) on one or several IoT devices connected to AWS and for managing these jobs. In comparison with other AWS services, the AWS Jobs service has appeared rather recently.

Access to the AWS Jobs service is provided via the programming console as well as programmatically using the set of API functions.

A certain subset of these functions can be used by the intelligent device itself (can be invoked by sending MQTT messages). The functions accessible over the MQTT protocol execute actions necessary for accessing jobs and their parameters from the device side: GetPendingJobExecutions, StartNextPendingJobExecution, UpdateJobExecution, DescribeJobExecution, etc.

Other functions are defined over the HTTPS protocol and are intended to be called from JavaScript code on webpages, from the program code of lambda functions, and by users in the interactive mode. These functions are used mostly for the creation and deletion of jobs and job execution management: CreateJob, DeleteJob, DescribeJob, ListJobs, ListJobExecutionsForThing, etc.

In the AWS Jobs terminology, the main information about a job is stored in its job document. This is a JSON document that is passed from the framework to the target device and describes what should be done. Normally, a job document includes the name of the operation and a URL (or URLs) that refers to the location of data-job parameters.

This URL can be “pre-signed” by an AWS user. In this case, the URL allows access to a certain object for the intelligent device with the privileges of the user who has pre-signed it (so the device can have access to the data it normally can’t access). Pre-signed URLs have a limited lifetime and expire after that lifetime is over, making the object inaccessible again.

A job document can be created on the fly during the creation of a job or can be stored as a file in the S3 file storage of the AWS framework. A link to this file can be specified during job creation.

Other job attributes include the following:

  • Target device or device group. If a device group is targeted, the job runs on all devices that are members of the group.

  • Snapshot or continuous job. A snapshot job is completed after finishing on the selected device or group of devices. A continuous job always applies to a group of devices; it continues to exist after it finishes on existing devices and runs on devices that are later added to the group.

When a job is being executed on a specific device, it has a state. A limited set of states is defined by the framework: QUEUED, IN_PROGRESS, FAILED, SUCCESS, CANCELED, REJECTED, REMOVED. The current state is changed by a request from the device (e.g., calling the UpdateJobExecution function) or when a user calls one of the job management functions (e.g., cancels the job using the CancelJob function). Normally, the job execution state is IN_PROGRESS while the device is executing the job, and it becomes SUCCESS or FAILED after the device completes the job.

The state diagram for job execution is shown in Fig. 2 (here, the transitions initiated by the device are shown in blue, and those initiated by other AWS components are shown in green).

click for larger image

Figure 2. Diagram of states during AWS Jobs execution (Source: Auriga)

Continue reading on page two, Software Update Implementation >>

< Previous
Page 1 of 2
Next >

Loading comments...