Using AWS Jobs to upgrade and configure IoT devices
Amazon Web Services (AWS) is one of the most popular framework environments for the Internet of Things (IoT) alongside Microsoft Azure and Google Cloud IoT. Smart devices are connected to the framework using the Internet and interact with it using the MQTT protocol. Besides interacting with devices, the framework also provides great opportunities for data storage and processing, data representation to a user, data analysis (including artificial intelligence methods), access control with a powerful system of privileges, and a lot more.
Besides webpages, program code in the AWS framework environment can be stored as lambda functions. These are special named pieces of code, written in one of the following languages: Python, Java, C#, or Node.Js. They are stored in the cloud and are invoked on certain events. An event can be initiated by a webpage (like calling a certain HTTP REST API on a certain URL), by another lambda function, or by an intelligent device (via sending an MQTT message of a certain type). In all of these cases, events can have parameters. Lambda functions are used as middleware for the interaction between intelligent devices, AWS resources (e.g., databases), and the webpages that the user directly interacts with (Figure 1).
click for larger image
Figure 1. Architecture of AWS component interaction (Source: Auriga)
There are hard limits for AWS lambdas, for example, the execution time of handling a single request is limited, the amount of memory that a lambda can use when handling a single request is limited. If any limit is exceeded, execution of the lambda is aborted. These limits are configured by the user when creating the lambda but cannot exceed certain values.
An IoT device connects to the cloud using the TCP protocol, which provides data integrity and buffering; in the case of a slow connection, the protocol takes care of accumulating data on the sending side and pushing it through the pipeline when it becomes possible. Also, an AWS protocol on top of TCP takes care of re-establishing the TCP connection persistently in the case of connection loss.
However, the connectivity issues between an IoT device and the cloud do not normally affect lambdas due to the specific unidirectional nature of the MQTT protocol. When communicating with an IoT device, a lambda just sends an MQTT message and does not wait for a response; if and when the response arrives, it is the responsibility of a different lambda function to handle it, and send another MQTT message to the IoT device, if needed.
One of the AWS framework components is job service (AWS Jobs). It is used for creating and executing long-lasting actions (jobs) on one or several IoT devices connected to AWS and for managing these jobs. In comparison with other AWS services, the AWS Jobs service has appeared rather recently.
Access to the AWS Jobs service is provided via the programming console as well as programmatically using the set of API functions.
A certain subset of these functions can be used by the intelligent device itself (can be invoked by sending MQTT messages). The functions accessible over the MQTT protocol execute actions necessary for accessing jobs and their parameters from the device side: GetPendingJobExecutions, StartNextPendingJobExecution, UpdateJobExecution, DescribeJobExecution, etc.
In the AWS Jobs terminology, the main information about a job is stored in its job document. This is a JSON document that is passed from the framework to the target device and describes what should be done. Normally, a job document includes the name of the operation and a URL (or URLs) that refers to the location of data-job parameters.
This URL can be “pre-signed” by an AWS user. In this case, the URL allows access to a certain object for the intelligent device with the privileges of the user who has pre-signed it (so the device can have access to the data it normally can’t access). Pre-signed URLs have a limited lifetime and expire after that lifetime is over, making the object inaccessible again.
A job document can be created on the fly during the creation of a job or can be stored as a file in the S3 file storage of the AWS framework. A link to this file can be specified during job creation.
Other job attributes include the following:
Target device or device group. If a device group is targeted, the job runs on all devices that are members of the group.
Snapshot or continuous job. A snapshot job is completed after finishing on the selected device or group of devices. A continuous job always applies to a group of devices; it continues to exist after it finishes on existing devices and runs on devices that are later added to the group.
When a job is being executed on a specific device, it has a state. A limited set of states is defined by the framework: QUEUED, IN_PROGRESS, FAILED, SUCCESS, CANCELED, REJECTED, REMOVED. The current state is changed by a request from the device (e.g., calling the UpdateJobExecution function) or when a user calls one of the job management functions (e.g., cancels the job using the CancelJob function). Normally, the job execution state is IN_PROGRESS while the device is executing the job, and it becomes SUCCESS or FAILED after the device completes the job.
The state diagram for job execution is shown in Fig. 2 (here, the transitions initiated by the device are shown in blue, and those initiated by other AWS components are shown in green).
click for larger image
Figure 2. Diagram of states during AWS Jobs execution (Source: Auriga)