Why the Yocto Project for my IoT Project?
Software developers in the embedded and Internet of Things (IoT) spaces face a dizzying array of options which it comes to choosing Operating Systems (OS) to support their designs. Decisions made regarding the hardware and tools will obviously have a major impact on the success of your project. Assuming you have the flexibility to choose the components that are best for your design, and that there are no limitations due to historical decisions or backward compatibility, then it pays to invest in upfront research to understand the options and make the best choice for your project and design team.
When the focus of your system is primarily electro-mechanical with simple control logic, a Linux design may be overkill and using an RTOS or control loop may be more appropriate. When the focus of the system is more on data processing and networking, it may make sense to consider Linux for your design. If your system has elements of both, you can certainly implement the electro-mechanical aspects in Linux as the Linux kernel arguably has device support for a wider range of peripherals than most other systems. A more complex system may require a dual-OS setup using either asynchronous multiprocessing on a multi-core chipset or multiple chips. Pairing a small 8- or 16- bit microcontroller running an RTOS or custom-designed control loop software with a 32- or 64- bit microcontroller running Linux is a common design solution providing the best of both approaches.
One compelling reason to use Linux in your design is its ubiquity. Books, videos, and other training materials are easily available, and finding a blog or a conference presentation that describes a use case similar to your design is just a web-search away. Resources such as stackoverflow.com and quora.com are full of interesting, timely information that can help during your education and design cycles. Additionally, online courses from vendors such as edx.org, coursera.org, and udacity.com are available (many for free) covering a wide variety of Linux-related topics.
The amount of available software to run on Linux systems is staggering. For any portion of your design that relies on externally defined protocols, APIs, etc., you can likely find an implementation that runs on Linux. For many such software components, you will find a vibrant developer community providing bugfixes, support, and feature development. Integrating these bits into your Linux design is routinely straightforward, but will vary depending on your choice of distribution or build system. Fortunately, the common distribution and build systems also have active communities who can provide assistance with items that are not integrated by default.
There is also no shortage of talented developers with Linux expertise. Due to the wide industry adoption of Linux-based designs, finding engineers with relevant experience is relatively painless. From the hiring team's perspective, finding engineers who are already trained in the technologies you need is a distinct advantage. From the engineer's perspective, working on technologies you find interesting and being able to apply your expertise to multiple potential employers is a great benefit. In short, matching hiring needs to talent is greatly simplified with Linux and related technologies.
There are a wide variety of distribution and build systems you can use to develop your embedded Linux system. Many desktop distributions can be pared down for use in limited resource environment and systems such as Ubuntu have varieties specifically targeted at IoT devices. The Raspberry Pi platform uses a customized Debian image as its primary target OS image. Additionally, there are several options for cross-compiling a full Linux system; Yocto, OpenWRT, and Buildroot are the three most common. These systems are generally hosted on a standard desktop Linux distribution and will cross-build a Linux system for your chosen target device. Changes that are needed will first be built into a target image on the Linux desktop and deployed to the target installation media from there.
In our discussions below, we will describe the Yocto Project in more detail and explain why we believe it is a good choice for embedded Linux designs intended for the IoT.
Development for embedded Linux blurs the line between embedded and desktop development and can provide the best of both models. Many embedded Linux systems host functionality such as an SSH server, a GUI, and on-target tools, thereby allowing for a development workflow similar to traditional desktop development. Code written in scripting languages can be directly modified in the deployed system and changes tested without a costly build/deploy cycle. For compiled code, the compute capability of many such systems is limited and on-target builds can be painfully slow. Using a cross-compilation environment for such tasks is appreciably faster and feels more like traditional embedded development providing the developer with all the tools and functionality available on the development host OS. The utility of having a full-blown Linux system on the target, with all the ready-made software as well as the large number of software developers familiar with the platform, outweigh cost concerns that might otherwise force the selection of a lower-priced hardware platform.
Setting up a cross-compilation environment is notoriously tricky. Getting the right combination of toolchain components and libraries can be an exercise in frustration. Matching the kernel headers with both your desktop distribution kernel and your embedded kernel can be a challenge. There are a variety of options for obtaining a pre-built cross-toolchain; if you are fortunate, there may be one available through the package manager on your desktop development system. If the packages from your distribution provider do not work for you, using a build system such as crosstool-NG may be necessary.
Developer connectivity to embedded target boards is generally through serial ports and Ethernet. Fortunately, embedded Linux has broad support for these devices and generally poses no obstacles for your design team. Driver support in Windows and MacOS host systems can be problematic as these device drivers may not be included in the default OS and may require the installation of third-party drivers.
JTAG support is somewhat scattered in embedded Linux and is mostly useful for low-level kernel debug. Additionally, it is highly dependent on vendor support from the semiconductor manufacturer as well as the JTAG equipment maker. Debugging application and user-space code is generally better accomplished with widely available and ported tools such as GDB.
Projects in the embedded and IoT spaces use a wide variety of chip architectures and boards. The Linux kernel has been ported to the vast majority of commercially available parts and will, in all likelihood, already support your chosen chipset. Most of the semiconductor vendors develop kernel code and drivers directly for support of their products, making it easy to adopt Linux for your design. Not all vendor contributions are available in the stock kernels from kernel.org however, so you may need to do some research and integration work to gather all the software components necessary for your chosen platform.
Yocto Project Description
The Yocto Project defines itself as "an open source collaboration project that provides templates, tools, and methods to help you create custom Linux-based systems for embedded products regardless of the hardware architecture." It's roughly analogous to the concept of desktop Linux distributions; however, that is a bit of an over-simplification. Yocto is more properly referred to as a meta-distribution. It is a collection of recipes, configuration values, and dependencies that are used to create a custom Linux runtime image tailored to your specific needs.
The Yocto Project focuses on mechanisms and leaves you, the developer, free to impose policy based on your design. Where desktop distributions will select options such as packaging format and init systems, Yocto provides configuration points to allow you to select these based on your needs. By default, users can select deb, rpm, ipk, or tar package formats; or easily add a new one by adding a custom bbclass file. Similarly, selecting either the sysvinit or system init systems is a simple configuration change in your local build files.
A Yocto build starts by building many of the native utilities needed by the build system itself. This is performed to minimize the dependencies on the host OS and to ensure a known set of package versions. Most configurations then proceed to build a tested and known good cross-compilation environment before moving on to building the binaries for the target platform. Due to the large number of packages built for a Yocto configuration, this can consume significant resources and time on the host system.
The package definitions include various phases (e.g., fetch, compile, install, package) and must specify any runtime or build-time dependencies. The build is controlled by an executive called bitbake, which is responsible for processing the dependencies and scheduling the tasks. The tasks are executed in parallel as much as will be allowed by the host system resources and the interdependencies between the tasks. Additionally, the intermediate outputs of each of the build stages are maintained using a shared state mechanism to allow for reuse across similar builds, and this can result in a significant speed improvement.
Yocto advantages: Features
Yocto's focus on mechanism over policy allows wide latitude for system developers. This approach ensures that you are free to set the policy based on the needs of your design rather than having to adapt to decisions enforced by the system software provider. Yocto provides sensible defaults to allow developers to get started quickly but the mechanisms to change these policies are well documented and simple to adopt.
The Yocto layer infrastructure is an integral part of the Yocto model. Layers group related functionality into separate bundles, which can be added to your build setup only as required. This greatly reduces the complexity of each individual layer and allows them to be developed at their own pace. Layers are included in your build with a priority ordering, allowing higher priority layers to override and modify settings from the base layers.
Platform layers generally support adding boards to Yocto; they will generally contain custom kernel and bootloader components as well as any board-specific runtime packages and configurations. The following list shows platform layers provided by many of the semiconductor industry heavyweights. These layers provide support for a wide range of commercially available chips and boards and should be used as a starting point for any design containing any of these parts.
Feature layers add functionality that is not generally valuable to all Yocto users, but that is deemed too complicated to include in the base layers. The following list shows some of the feature layers available today.
In a traditional desktop Linux distribution model, you generally install from a CD or USB key and then perform additional package installations and configurations in the running target system. The Yocto workflow is different; it consists of running a full build on your development machine, the output of which is an image containing the entire target system. With a bit of care, you can eliminate most, if not all, of the configuration steps needed in the running target allowing for a more predictable software load and reducing the number of dimensions in your test matrix. Yocto can even configure your runtime software to use a read-only root filesystem, thereby resulting in systems that always start from a known-good state; of course, this requires efforts on your part to ensure that state is saved properly to a persistent data store.
Additionally, the Yocto project follows a strict release schedule. This predictability is crucial for projects that are based on Yocto and allows the development teams to plan their activities. Developers can choose which Yocto branch on which to base their activities as a function of their needs. The development branch will ensure access to the latest features while the stable branches will reduce the pace of changes while still allowing for updates to address CVE (common vulnerabilities and exposures) issues and other potential security concerns.
Yocto Advantages: Industry and community support
The Yocto Project website has the following to say about the ecosystem surrounding Yocto:
Nothing is as important to an open source project as the ecosystem of individuals and organizations around it."
This is easily one of the primary advantages of Yocto. The value of community cannot be stressed highly enough; support forums and expertise are readily available, as is an active development community pushing Yocto forward. As an open source project, you can modify it yourself to suit your needs or find someone to do it for you; and there is no concern of a software vendor changing strategies and leaving you to start over.
Yocto is also widely adopted in the industry. In addition to semiconductor vendors, there are a number of vendors providing commercially supported Yocto software and services. The Yocto project participants page shows the variety of industry players that are using Yocto. Additionally, industry initiatives such as the GENIVI Alliance and Automotive Grade Linux help to push Yocto adoption into the automotive industry.
Yocto Advantages: Tools and SDK
As discussed above, getting a properly supported cross toolchain can require a large effort. The Yocto project includes recipes to build a fully functional toolchain as part of your project build. This toolchain, which has been tested by the Yocto community across a wide variety of architectures and platforms, will suffice -- in most cases -- to support your project. If needed, this toolchain can be easily customized using standard Yocto recipe mechanisms. Additionally, if your target chip supports features that are not enabled in the default toolchain (e.g., custom instructions or optimizations), specifying platform-specific tuning parameters is a simple addition in your platform layer. Finally, if you have the need to use a prebuilt toolchain provided by a third-party, there are mechanisms built-in to Yocto to allow this. Support for several providers of commercial toolchains is available in the default Yocto source code and can be extended to additional providers with relatively modest efforts.
Yocto can also bundle the tools and libraries needed for your platform into an SDK that can be delivered to application developers. This is simpler than requiring such developers to run full Yocto system builds and allows for an optimized application development workflow while ensuring that these developers have the proper versions of all components needed for their work.
There is a fairly steep learning curve with Yocto. The terminology alone can be daunting (e.g., what is the difference between Yocto, Poky, and OpenEmbedded?). The number of options for configuring your target can make it difficult to assess the best choices. Getting a basic system up and running can be achieved quickly by following any of the tutorials available with a simple web search. However, understanding what changes need to be made for your particular design may require a nontrivial amount of research and investigation. To be sure, there are valid reasons for the complexity and we feel that the advantages discussed above make it worth the effort. Once you have completed the initial ramp-up, further design efforts will proceed much more quickly.
For developers who are used to traditional desktop and server software development, the Yocto workflow can be confusing. In a desktop development environment, the host OS provider generally provides a mechanism to install new packages. In Ubuntu for instance, new packages are easily installed using the apt-get command. The packages are available in pre-compiled binaries from servers accessible through an Internet connection. In Yocto, you will need to modify your configuration and rebuild to add additional packages; there are no Internet-hosted package libraries generally available to pull from.
Working in a cross-build environment can feel unfamiliar and requires developers to understand both the host and target systems. Being able to develop code, compile (or simply run in a scripting environment), and quickly test in your target system can be faster than running a bitbake build and deploying a new image to your target. To be clear, Yocto does support development tools on the target to support this kind of workflow; however, once the required code has been developed and debugged, there is an additional step needed to integrate those changes back into the Yocto build environment. An intermediate approach is to make the changes in the Yocto environment, but to then deploy only the updated packages. The Yocto build system outputs packages in a standard format (rpm, deb, ipk, or tar) that can be installed into a running system using utilities on the target such as rpm or ipk. These approaches work fine to optimize development cycle times early in the lifecycle; however, as you get closer to product release, you generally want to minimize or eliminate the differences between your test images and your production images -- manually installing packages or modifying code on a running system effectively creates a custom image.
Finally, the build times for a full Yocto build can be significant. This is, in general, unavoidable due to the large number of packages that need to be built for a fully functioning Linux system; however, this can be greatly mitigated by the Yocto shared state mechanism.
IoT Specific Needs
The Yocto Project is an excellent choice for IoT projects. All the advantages discussed above are valid for designs in the IoT space. Sean Hudson states:
We recommend the Yocto Project, which is the basis for our own Mentor Embedded Linux (MEL), to customers because of its flexibility, wide industry support, large community, and support for long term maintenance. For the Internet of Things (IoT), the need to support diverse architectures, multiple device configurations, and target software combinations make it a great fit for creating a Linux basis for IoT devices.
Member of Technical Staff and Embedded Linux Architect
Mentor, A Siemens Business
Yocto Project Advisory Board Founding Member
Current OpenEmbedded Board Member
Wikipedia defines the IoT as:
The inter-networking of physical devices, vehicles (also referred to as "connected devices" and "smart devices"), buildings, and other items embedded with electronics, software, sensors, actuators, and network connectivity which enable these objects to collect and exchange data.
There are, broadly speaking, two types of IoT devices: small systems hosting sensors or actuators, and large systems such as IoT gateways, Network Attached Storage (NAS) boxes, home automation routers, etc. Linux is likely overkill for the small systems as many of them run an 8- or 16- bit microcontrollers that cannot easily support Linux. For the larger devices, Linux in general, and Yocto in particular, is an excellent option. More and more, these types of devices are prototyped (and sometimes implemented) with easily available consumer hardware devices such as the Raspberry Pi or the BeagleBone. The Yocto Project has excellent support for these (and many other similar) boards, and is an excellent way to get started quickly on your IoT project.
Many connected devices are deployed and managed as a large fleet of devices. In these cases, the predictability of software contents described above is a major advantage. Manually updating devices is resource-intensive and prone to error, and having a definitive list of devices and their software contents is a first step in being able to secure the fleet. If deployed devices can make significant modifications to their software, it can be difficult to know, with any certainty, which updates are required.
IoT systems tend to use software, libraries, and protocols that are not necessarily in wide use in desktop Linux distributions. Many of the common packages such as MQTT and AMQP are readily added to your Yocto build using standard Yocto layers:
- MQTT: http://cgit.openembedded.org/meta-openembedded/tree/meta-python/recipes-devtools/python/python-paho-mqtt_1.2.bb
- AMQP: http://cgit.openembedded.org/meta-openembedded/tree/meta-oe/recipes-connectivity/rabbitmq-c/rabbitmq-c_0.7.0.bb
Additionally, industry initiatives such as IoTivity are directly supported by Yocto using a feature layer:
Developers have a variety of good options for Linux distributions and build systems. The Yocto Project is a solid option for most embedded projects and serves as an excellent starting point. The benefits of the Yocto build system as outlined in this article are among the reasons it is so widely used in the embedded software industry. Product development in the IoT space is proceeding at a blistering pace and many of those devices have feature sets that benefit from a Linux implementation. Yocto's wide variety of features, expandability and fine-grained control over the contents of the target image make it ideally suited for projects in these spaces and is certainly worth some investigation as you start your design.
Drew Moseley is currently part of the Mender.io open source project to deploy OTA software updates to embedded Linux devices. He was raised in Tampa, Florida, and attended the University of Florida. Drew has spent his career in embedded software and developer tools with stints at Intel, Motorola, Red Hat, Monta Vista, Mentor Graphics, and -- most recently -- at Mender.io. He has spent the last 10 years working on embedded Linux, with the last 6 years focused on Yocto. Drew is a retired fencer, fencing coach, and referee. In his spare time, he enjoys working on electronics projects, minimizing his life, and spending time in the Florida sun.