Pick the right multicore virtualization use case for your design

Virtualization technology provides an environment that enables the sharing of hardware resources (CPUs, memory, I/O) on a single computer system, allowing multiple operating systems to simultaneously share the system. The potential benefits to sharing a system by using virtualization are substantial cost and power savings as well as great flexibility.

As shown in Figure 1, below , virtualization is enabled by a software abstraction layer called a hypervisor (or virtual machine monitor) which creates and manages virtual machines which are isolated environments in which guest operating systems run.

Figure 1. A Hypervisor manages multiple virtual machines

The hypervisor layer operates at a privilege level higher than that of the guest operating systems, thus enabling the hypervisor to enforce partition boundaries, enforce system security, and ensure that virtual machines are isolated and cannot interfere with each other.

Hypervisor implementations can offer a wide range of capabilities and services ranging from simple partitioning of system hardware to the sharing and oversubscription of CPU or I/O resources. With the growing use of processors with multiple CPU cores in embedded systems, virtualization technology is increasingly being used to manage the complex software environments running on these devices.

This article will describe a number of different virtualization use cases for embedded systems, and discuss hardware and software virtualization technologies that can be used to address these use cases.

Use Case 1 – Hardware Consolidation
A key trend that is driving the need for virtualization technologies in multi-core embedded devices is system consolidation (Figure 2 below ). Systems that in a previous version consisted of multiple discrete processors or even multiple boards can now take those functions and potentially combine them onto a single processor that has multiple cores, providing significant savings for a system’s bill-of-materials and power consumption. A number of the other use cases discussed in this article are variations on this theme of consolidation.

Click on image to enlarge.

Figure 2. Embedded virtualization means consolidation

Hardware consolidation typically involves combining multiple software domains as well. The discrete domains being combined may be comprised of homogeneous or heterogeneous operating systems. A common architecture (Figure 3, below ) found in systems that do network processing involves two domains: 1) a data plane which is responsible for network packet processing (for example forwarding or routing packets) and 2) a control plane, which responsible for management functions such as managing routing tables.

Click on image to enlarge.

Figure 3. Network processing in data and control planes

An architecture like this is a natural candidate for using virtualization to consolidate the control and data plane functions on a single processor.

Use Case 2 – Utilization
Improving computer hardware utilization has been the classic use case for virtualization. The first mainframe operating systems in the 1960’s were inherently single-user, and batch processing was used to efficiently use the mainframe compute resources—one job after another was fed into the mainframe for processing.

At the time allowing a person to interactively program and use a computer would have been extremely inefficient with the mainframe’s CPU being substantially idle as a user slowly typed at a terminal.

To enable multiple users to interactively use a mainframe at the same time, in the mid-1960’s IBM introduced the concept of a virtual machine monitor which was used to “time share” the mainframe for multiple users— each user had a virtual machine (VM) capable of running their own private instance of the OS.

The virtual machine monitor time sliced processing cycles among the virtual machines, thus accomplishing the goal of allowing direct user interaction with the computer while still maintaining a high level of utilization.

The need to improve hardware utilization manifested itself again in the early 2000’s with the emergence of data centers based on commodity servers where it was found that many servers running Windows and Linux were greatly underutilized.

Virtualization provided a means run many virtual machines on a single server, improving system utilization, and resulting in huge power and cost savings.Utilization is frequently less of a concern in embedded systems, but there are situations as shown in Figure 4 below where a software domain may require less than the processing power of an entire CPU and the system can benefit by consolidating several such domains onto a single CPU.

Figure 4. Virtualization provides better system utilization, resulting in power & cost savings.

For example, it may be possible to determine that a peak workload in a particular virtual machine requires only a fraction of a CPU available cycles, which allows the remaining cycles on that CPU to be available for allocating to other virtual machines.

Suppose a system designer wants add a web-browser accessible administration interface, based on Linux to a network appliance. It would be reasonably straightforward to implement a VM running Linux dedicated to providing this administration service.

However, a configuration GUI would be typically invoked infrequently meaning that CPU cycles would be only needed occasionally. Thus, the physical CPU which runs the VM could be shared by other VMs.

This use case requires a capable scheduler in the hypervisor to make effective use of the CPU by scheduling the allocation of CPU cycles to the different software domains sharing the CPU.In summary, virtualization can enable sharing of a CPU and flexible fine-grained control over CPU utilization– how CPU cycles are allocated to virtual machines.

Use Case 3 – Sandboxing
Embedded systems are typically fixed function devices designed for a specific purpose, and naturally are quite different than general purpose computers which allow a variety of end user installed applications.

Software in embedded systems will commonly be “closed” with limited ability to change or add new software.However, there is an increasing interest by OEMs to provide ways for customers to extend the capabilities of systems by adding 3rd party or other custom software.

An OEM may provide an SDK that allows creating of this type of software, which then can be installed on the embedded device.To introduce “untrusted” software into a system while maintaining system integrity requires isolating or partitioning any untrusted software components away from the rest of the system.

Clickon image to enlarge.

Figure 5: Sandboxing Use Case

A hypervisor can provide the needed security to enforce the boundaries of the “sandbox” in which the untrusted software operates.

Use Case 4 – Dynamic Resource Management
Beyond simple consolidation, virtualization technologies have the potential to provide a great deal of flexibility around how CPU and I/O resources are used.

CPU Considerations. The software architecture of a given hypervisor may offer a number of different models for how CPU cores are allocated to virtual machines. If the performance requirements dictate it, cores may be “dedicated”, where a virtual machine gets dedicated/exclusive use of one or more cores.

Cores may be also shared. For the utilization use case discussed above, the hypervisor’s scheduling algorithm can offer flexible capabilities to configure the priority of virtual machines sharing a core, ensuring that virtual machine with the highest priority dynamically gets scheduling priority. Some schedulers allow control over which cores a particular virtual machine runs on, allowing fine grained customization of how a core’s cycles are allocated to virtual machines.

Schedulers may also allow a specific configuration of the maximum percentage of a core’s cycles that a given virtual machine is permitted. So, for example two virtual machines on a core could be allocated each 80% and 20% of the core. This flexibility allows creating configurations so that during a peak load that adequate cycles are available to the virtual machines that need them.

Cores can also potentially be hot plugged and unplugged dynamically from a virtual machine to allow runtime allocation of CPUs based on when where they are needed. Unused or under-utilized CPUs can also be placed into a low power state for power savings.

Virtualizing I/O . Hypervisors typically provide two approaches to I/O:

1) Dedicated I/O. Dedicated I/O is where a device (e.g. a PCI NIC card) is assigned to a virtual machine which gets direct, exclusive use of the device. The advantage of dedicated I/O is performance– native I/O performance speeds may be possible.

2) Virtual I/O. Virtual I/O provides a way for virtual machines to share an I/O device. For example, multiple virtual machines may each be given a virtual network device they can use for network I/O. The hypervisor emulates the virtual devices and bridges them to a physical network interface.

Dedicated I/O devices can potentially be hot unplugged or hot plugged if the bus they reside on (e.g. PCI Express) supports hot plug, which would allow resources to be added or removed without bringing a virtual machine down.

Clickon image to enlarge.

Figure 6: Dynamic Resource Management Use Case

In cases where virtual I/O is used to share a physical device like a network interface, some hypervisors provide the capability to limit the traffic bandwidth to/from the virtual machine.

Use Case 5 – High Availability
For many mission critical applications, any downtime or data loss has the potential for devastating effects. On approach to solving this problem is to use redundant hardware.

For example, a high availability system (Figure 7, below ) may have 2 identical hardware boards. One is considered ‘active’ and handles the normal system workload.

The second board is ‘standby’ and is prepared immediately to take over the functioning of the system if the active board fails. The standby board maintains the necessary synchronization of state with the active so that the standby can cleanly take control and keep the system alive if the active fails.

Experience has shown that a substantial majority of system failures are due to software causes. This leads to the possibility of creating high-availability systems using virtualization where the active and standby systems are virtual machines instead of redundant hardware.

If there is a system failure a “failover” event occurs and the standby VM becomes the “new- active”, takes control and keeps the system alive, while the “old-active” is reset and moves into the standby role.

This use case is another variation of consolidation through virtualization, where the system maintains high reliability while saving cost and power.

Clickon image to enlarge.

Figure 7: High Availability Use Case

A system with VMs configured in an active/standby relationship also enables the ability to upgrade a system’s software without downtime. Software in the standby VM can be upgraded without affecting the system’s normal operation.

A software controlled failover is triggered causing the standby to become the new-active. The software in the old-active VM can then be upgraded without affecting the running system.

Software Requirements for Virtualization
Consolidation use cases can theoretically be implemented without a using a virtualization layer—an approach sometimes referred to as “unsupervised asymmetric multiprocessing (AMP)”. For example, on a 2 core system OS “A” could directly run on core 0 and OS “B” could directly run on core 1, all without a hypervisor.

However, unsupervised AMP has substantial drawbacks including a complete lack of enforced isolation between operating systems and a system that is fragile and complex.

Complexities include:  boot sequencing, cooperation in sharing global hardware resources (e.g. interrupt controller), resetting partitions, error handling, and system debugging. No untrusted software could be allowed in the system. Using a virtualization layer (i.e. a hypervisor) to implement these solutions has significant benefits and can address the drawbacks listed above. Some of the key capabilities needed in a hypervisor include:

* Isolation. A hypervisor must maintain isolation between virtual machines. A truly secure system also requires hypervisor managed IOMMU hardware which controls the address space that I/O devices can read and write.

* Partitioning CPUs, memory, I/O devices. For performance reasons, in many consolidation cases embedded operating systems will need direct access to memory and I/O devices.

To enable this, a hypervisor must support the capability to divide and dedicate hardware resources such as cores, memory, and I/O devices to different virtual machines. The hypervisor must also provide a means to manage interrupts for dedicated devices.

* Sharing CPUs (Scheduling). For the “utilization” use case, an effective scheduler is needed in the virtualization layer to schedule multiple VMs on a CPU. Features such as priority scheduling, pinning VMs, and limiting CPU cycles allocated to a VM could be important.

* Sharing I/O devices. Some uses cases will require the sharing of hardware I/O resources. This requires the hypervisor to provide virtual I/O services that enable sharing of resources I/O such as network and disk.

* Inter-VM communication. Hypervisors may offer the ability for virtual machines to communicate with each other by sharing memory and signaling using doorbell interrupts.

* Minimal OS modifications. Consolidation could involve moving an older software stack, legacy OS or RTOS to the new virtual machine environment. Ideally no guest software changes are required to do this.

However, the architecture of some hypervisors can possibly require substantial changes to a guest OS to run in the virtual machine environment. The requirement to modify an OS to run on a hypervisor is referred to as “para-virtualization”.

* Debugging. Debugging a system with several software domains simultaneously running can be complex. A hypervisor must provide mechanisms that allow virtual machines to be debugged.

* Real-time. Real-time constraints involve requirements around guaranteed latencies in the system. A virtualization layer can potentially impact the real-time characteristics of a system and both the real time requirements of the system and the real time capabilities of the hypervisor must be well understood.

Processor Requirements for Virtualization
Performance is critical in many embedded systems. When using virtualization to combine multiple software domains on a single processor, the performance requirements and constraints must be well understood—factors such as CPU utilization interrupt latency, determinism, and the working set size of memory.

The virtualization layer is in place to ensure the integrity of the system. It does this by running at a higher hardware privilege level than that of guest software in virtual machines.

Certain privileged operations invoked by guest software (e.g. an MMU update) or other exceptions (e.g. a hardware interrupt) may trap to the hypervisor to be handled. These events result in additional overhead.

The key to good performance in a system using virtualization is to minimize traps to the hypervisor. The ability to achieve high performance levels will in large part depend on the virtualization assist capabilities in the physical CPU cores.

Some of the latest embedded cores (such as the Freescale Power architecture based e500mc, e5500, and e6500 ) provide hardware assist capabilities such as:

* 3 processor privilege levels : user, supervisor, and hypervisor
* Extended Virtual Address Space . Virtual address includes a partition ID, which allows the CPU’s MMU to be efficiently shared by the hypervisor and multiple virtual machines at the same time.
* Direct CPU operations. To improve performance, CPUs allow guest software to perform key operations safely with no traps to the hypervisor, including:

o enabling/disabling interrupts
o applications performing system calls
o accessing key CPU registers
o MMU updates

* Direct Interrupts and Exceptions. Interrupts from I/O devices can be sent directly to a virtual machine without hypervisor mediation.

Figure 8: IOMMU support for virtualization

As shown in Figure 8 above, to preserve system integrity and at the same time allow virtual machines to directly access DMA-capable I/O devices, the processor hardware must include an IOMMU. An IOMMU sits between I/O devices and the rest of the system and authorizes all accesses made by the device.

Software Technologies for virtualization use cases
A number of software technologies are available to enable virtualization in embedded system, and several examples will be reviewed briefly: OS-level virtualization: “Type 1” hypervisors, and “Type 2” hypervisors.

OS-Level Virtualization . OS-Level virtualization or “OS virtualization” uses the capabilities of an operating system kernel to enable multiple isolated instances of user-space, without using a hypervisor. Each user space instance has its own private, isolated set of standard operating system resources such as a filesystem and network interfaces, and applications run in this isolated “container”.

One example of OS-level virtualization is the open source Linux Containers (LXC) project which is based on Linux. LXC is a set of user space tools that work in conjunction with Linux kernel features to create and manage containers.

Linux containers could be an appropriate solution to a consolidation, sandboxing, or dynamic resource management use case when all the software domains involved are Linux applications. It is not possible to boot an operating system in a container.

Linux containers provide isolation to applications running in the containers with minimal overhead. They use the “control groups” (cgroups) capability of the Linux kernel to limit and control CPU, memory, and I/O resource usage.

“Type 2” Hypervisors. There are two general categories of hypervisor– commonly referred to as “Type 1” and “Type 2” hypervisors. Type 2 hypervisors use an operating system as the basis for the virtualization layer. Virtual machines run alongside other OS applications. An example of a type 2 hypervisor is KVM (Kernel-based virtual machine).

KVM is an open source software virtualization technology based on the Linux kernel that enables Linux to act as a virtual machine monitor. KVM is a Linux kernel driver that works together with the QEMU user space application to create and run virtual machines.

KVM is available for embedded Power architecture processors with an ARM port in-progress. As the basis for a virtualization layer, KVM inherits all the benefits of Linux— an open source, highly reliable kernel widely used in mission critical systems.

Key Features of KVM include:

* Fully capable virtual machines, allowing a variety of operating systems to be run
* Makes use of the full scheduling capabilities of the Linux kernel to schedule VMs– priority, CPU affinity, isolcpus
* Protection and isolation. Virtual machines are fully isolated from each other
* Virtual networking. Multiple virtual machines can share a physical network interface
* Virtual disk. Each virtual machine can have one more virtual disks, with a variety of formats supported
* Virtual serial port / console
* Direct mapped memory– partitioning and assignment of physical memory regions to virtual machines. Physical memory regions can be shared between partitions.
* Direct mapped I/O devices—assignment of SoC I/O devices to virtual machines enabling partitioning of a system
* GDB debug stub for debugging virtual machines
* ePAPR compliant boot architecture

“Type 1” Hypervisors . A “type 1” hypervisor runs directly on system hardware, and is not part of a general purpose operating system like type 2 hypervisor. An example is the Freescale Embedded Hypervisor.

It is a small, efficient hypervisor for the Power architecture that enables the secure partitioning of a system’s resources– an approach referred to as 'supervised AMP'. A system’s CPUs, memory, and I/O devices are statically partitioned, with each partition being capable of running a guest operating system. The hypervisor uses no scheduler and simply spatially partitions CPUs.

Key Features:
* Partitioning. Support for grouping CPUs, memory, and I/O devices into partitions.
* Protection and isolation. Partitions are fully isolated from each other.
* ePAPR-compliant APIs for: interrupt controller management, virtual character devices, inter-partition doorbells
* Hypervisor APIs available for: partition management, error management, power management
* GDB debug stub for debugging partitions.
* High-Availability. Mechanisms are available for configuring partitions in an active/stand-by arrangement.
* ePAPR compliant boot architecture

Virtualization technology provides an environment that enables the sharing of hardware resources (CPUs, memory, I/O) on a single computer system, allowing multiple operating systems to simultaneously share the system.

Virtualization use is growing in the embedded space, focusing mainly on multicore processing. There are many embedded system use cases were virtualization plays an important and flexible role. The potential benefits to sharing a system by using virtualization are substantial cost and power savings as well as greater flexibility.

Stuart Yoder is a software architect at Freescale Semiconductor . He serves as technical lead for hypervisor and virtualization software for Freescale's QorIQ(tm) multicore communications platforms. Yoder has worked in the field of system software development for 20 years. Prior to Freescale, he held positions at IBM, Intel, and several semiconductor startups. Yoder earned a Bachelor of Science degree in Computer Science and Engineering from Letourneau University.

Rob Oshana is Director of Software R&D for the Networking and Multimedia group at Freescale. He can be reached at robert.oshana@freescale.com.

1 thought on “Pick the right multicore virtualization use case for your design

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.