Manage multiple processes and processors in a deterministic multicore design

Chris Main, TenAsys Corporation

February 22, 2012

Chris Main, TenAsys Corporation

Implementing global objects
Global objects provide a means for implementing the above and have the following characteristics:

• Each instance of the RTOS is launched on each core of a multicore processor. In this particular RTOS, system functions are implemented as objects, each referred to by a handle. Each instance of the RTOS operates independently of the others, with its own separate memory, scheduler, I/O devices, and associated interrupt management. Processes that run on one instance of the RTOS have no impact on processes that run on another instance of the RTOS.
• An efficient, asynchronous message-passing system provides the common transport to support the IPC and management layers across multiple instances of the RTOS.
• The system APIs are extended transparently to allow a process on one instance of the RTOS to access objects on another instance of the RTOS. This is the same method by which processes communicate within a single instance of the RTOS, but the scope of the object handles has been extended by means of the IPC layer so that a handle may refer to an object elsewhere. Each instance of the RTOS has a global object manager process to manage this. This management process acts as an agent for remote processes interacting with local objects.
• The system-management layer (called DSM, or Distributed System Manager, in the example shown in Figure 6) allows processes to register their dependence on other processes in order that they're notified if another process in the relationship is terminated or compromised. The DSM layer also monitors nodes (instances of the RTOS), so that if one is shutdown or fails, processes on other nodes are notified.
• In this example, which implements the example system described earlier, an instance of the Windows OS has been added and the functionality is also extended to that operating system.

Click on image to enlarge.
Referring to the application in Figure 1, we can now have the video camera application (which we'll call FB3) run by Process C on Core 3 and the PLC (FB2) run by Process B on Core 2 of a multicore platform. As before, the video application needs to inform the PLC when it has completed its video capture and analysis and tell the PLC whether the results are successful or not. If both applications were run on a single processor, passing such information would be easy to do using a mailbox transport object.

With global objects and the associated global object network, the same mechanism can be used with the addition of three program steps. First, the PLC (Process B) needs to make the Mailbox "global" so that processes running on other nodes/processors can discover it. Then two additional instructions must be executed to allow a remote process, such as Process C, to find a global object, in this case the Mailbox, so that it can write to it. Table 1 shows a listing of the commands that would be executed by both processes, with the additional instruction shown in bold and underlined.


Click on image to enlarge.
Note also that the DSM registers the connection between the processes when the Mailbox is established. So if the Process B that is the PLC application faulted, the video app, Process C, would be notified and would take appropriate action.

The concept of passing information from one independent functional block to another is extendable with memory objects. The steps to setup a memory object are similar to those of setting up a Mailbox. Global memory objects are better suited for the transfer large blocks of information. An example would be passing position-correcting information directly from the video application to the motor drive upon partial failure of results from the video capture and analysis.

Because memory objects are often subscribed to by more than two processes, it's necessary to allow a memory object to remain present even though the process that created it might have terminated and has no longer any need for it to be present. Global objects provide such a mechanism with the inclusion of an Index. For example, if a memory object is expected to be subscribed to by four processes, the process that creates the memory object assigns it an initial Index =4. Every time an object unsubscribes from the memory object the index is decremented. Only when the last process unsubscribes does the memory object get removed.

Global objects extend to a Windows environment as well. They can be used to allow applications that require an advanced Windows-based human interface to communicate information with RTOS-based functional blocks. In our example system (Figure 1), a Windows application could read video information directly from the memory object that is put there by the video application and display it for visual inspection by an operator.

The above example shows how global objects can be used by independent FBs to communicate information and how the ability to pass large blocks of information can even extend the system capabilities. Moreover, because they provide a mechanism for enabling distributing FBs to communicate, the use of global-objects-based communications can also be used to "break-up" an FB like a motor-drive across two processors if needed.