Building an effective real time publish-subscribe framework for a distributed embedded design: Part 2

Rajive Joshi, Ph.D., Real-Time Innovations, Inc.

August 14, 2006

Rajive Joshi, Ph.D., Real-Time Innovations, Inc.August 14, 2006

There are some fundamental conceptual differences between DDS and JMS which can deeply impact data-centric design especially as regards to the monitoring and coordination of activities amongst diverse distributed controllers and sensors.

Particular attention in any embedded design involving real time delivery of information must be paid to differences in the areas of data modeling, dataflow routing, device discovery, and data typing.

Data modeling: Autonomous messages vs. Data-objects
P-S middleware an be distinguished in their use of data models, which ranges from (1) messaging or eventing, where the data payload is opaque to the middleware; to (2) data-object centric, where the data payload is interpreted and managed by the middleware.

Messaging or eventing P-S middleware treat a message on a topic as an event with an optional data payload that is opaque to the middleware. Data-object centric (or simply data-centric) P-S middleware allow an application to identify 'data-objects' to the middleware. The 'data-objects' are unique in the 'global data space' of the distributed system across all participants. Each participant is regarded as having a local cache of the underlying global data-object.

A message on a topic is regarded as an update to the underlying data-object that can be identified and managed by the middleware. Local changes to a data-object are propagated by the middleware; the middleware can distinguish between messages or update samples from different data-objects and manage their delivery to the interested participants on a per data-object basis.

JMS does not support an underlying data model; it is a pure "messaging" or "eventing" middleware, and treats a message as an event with an optional data payload that is opaque to the middleware.

A JMS message is a self-contained autonomous entity, representing an event with optional data payload. In its lifetime, a message may be (re)sent multiple times across multiple processes. A JMS client on the way may examine it, consume it, forward it, or generate new messages to accomplish its task. A message is uniquely identified with messageId, and carries with it its deliveryMode, priority, expiration, correlationID, redelivery flag, reply destination, and so on in the header fields.

Message payload contents are not interpreted or managed by the JMS provider; each message is a unique and distinct entity. Data modeling capabilities, if needed, will have to be provided at the application layer, in the JMS client software.

Figure 6. DDS provides a relational data model. The middleware keeps track of the data-objects instances, which can be thought of as rows in a table.

DDS is data-object centric middleware, and supports a relational data model commonly used in database applications, as shown in Figure 6, above. In database terms, a topic corresponds to a Table; the table schema corresponds to the topic type. Certain type fields (columns) can be marked as keys (primary keys) in the type description (table schema).

A data-object instance is identified by its keys, and corresponds to a row in the table. Underlying this data model is an implicit assumption of a shared global data space in which the data-objects live. The global data space is defined by the communicating applications in the DDS domain. Each participant is viewed as having access to a local cache of the topics (tables) in the global data space.

A DataWriter can write (or update) one or more data-object instances (or rows) in its local cache. The updates are propagated by the middleware to the associated DataReaders for the topic, and are delivered as samples to be applied to the local cache on the receiving end.

The DDS middleware can distinguish between different data-object instances based on the keys, and can manage the delivery of samples on a per data-object instance basis. Since the keys are embedded in the data type, relations between data-object instances are also implicitly managed by the DDS middleware.

DDS also supports unkeyed topic types, which are effectively equivalent to messaging (or eventing), as supported by JMS.

Unlike JMS, where messages are first class objects, DDS messages are user defined types and do not carry any 'per message' user settable headers or fields. However, the user is free to define the message data type, and therefore can specify needed fields.

As a consequence of this difference, DDS data delivery has the potential to be higher performance than JMS messages delivery, because the extra overhead of mandatory headers per message is not required with DDS.

Dataflow routing: Specific destinations vs. Matching endpoints
JMS destinations (Queue or Topic) are logical "message stores or channels", uniquely defined and managed by the middleware, as shown in Figure 7, below. A destination may be configured statically in the middleware using JMS vendor provided configuration tools; or it may be created dynamically using temporary destinations.

Figure 7. JMS destinations are logical message stores or channels configured using administrative tools supplied by the JMS vendor.

In either case, they represent unique well-defined "channels" in the middleware. A destination and can hold any type of message (since JMS is opaque to the payload). A consumer is attached to a specific destination from which it will receive messages. A producer can specify the destination at the time of sending a message. A destination acts as a "mini-broker" managing the delivery of the messages sent to it. A dataflow is established between a producer and a consumer via the destination as the intermediary.

A DDS topic represents an association between compatible DataWriters or DataReaders bound to the topic, in the global data space. A topic has a name, type, and associated QoS. An endpoint (DataReader or DataWriter) is tightly bound to a specific topic and may additionally specify different desired QoS. A dataflow between a DataReader and DataWriter is only established when the type and QoS offered by the DataWriter is compatible with that requested by the DataReader (Figure 8, below).

Figure 8. DDS topics represent a name, type, and QoS. DDS provides a spontaneous connection mechanism, which automatically connects matching DataReaders and DataWriters.

The DDS requested/offered mechanism establishes dataflows only between matching endpoints associated with a topic in the global data space. DDS notifies the application of incompatible endpoints, when a dataflow cannot be automatically established. Thus, DDS middleware truly acts like an "information bus", where dataflows are dynamically established and removed.

Unlike JMS, where a producer sends to a specific destination, a DDS DataWriter (producer) never specifies a destination; in DDS the dataflows are automatically managed by the DDS middleware based on matching subscriptions. A DDS middleware implementation can take advantage of this behavior by supporting direct data transfer from a DataWriter to a DataReader, without involving an intermediary; thus it has the potential for better performance and scalability than JMS.

Discovery: Administered vs. Spontaneous
JMS discovery is administered and centralized. JMS discovery requires that the producers and consumers be able to find and bind to the destinations (and not each other). There are two mechanisms for JMS destination discovery.

* Static destinations are discovered via JNDI APIs, which bind logical destination names to destination objects. The static destinations accessible this way must have been previously configured in the JMS middleware (server) using vendor supplied administrative tool (Figure 7, above).

* Destinations (including temporary destinations) may also be discovered via the replyTo attribute of received messages. In order to discover a destination using this mechanism, a static destination must have already be previously established.

Since JMS discovery is administered, the static destinations must be determined and configured before a client can use them. Determining what static destinations to use is a critical aspect of a distributed system design, and must be considered carefully prior to deploying a system based on JMS.

Evolving the system configuration for new requirements also requires careful planning and administration. Destinations take up physical resources, so destinations no longer needed in distributed system must be purged, and new ones added as needed over the lifetime of a distributed system based on JMS.

DDS discovery is spontaneous and decentralized. DDS requires that endpoints be able to find each other to determine if they are compatible and whether a dataflow should be established (Figure 8, above). Thus, discovery is implicit in the dataflow routing mechanism.

DDS provides APIs for an application to access the internal middleware discovery meta-data by means of built-in topics. The internal meta-data that can be accessed by a user application includes information such as participants joining/leaving a domain, creation/deletion of topics, data readers, and data writers in a domain.

The DDS DomainParticipant.get_builtin_subscriber() method can be used to monitor the following builtin-topics: DCPSParticipant, DCPSTopic, DCPSPublication, DCPSSubscription.

Since DDS discovery is spontaneous, the topics can dynamically change over the lifetime of a deployed distributed system based on DDS, without any administrative impact. Endpoints on new topics are discovered automatically, and dynamic dataflows established in a plug-n-play fashion. The spontaneous discovery mechanism of DDS can also potentially scale better as the span of a distributed system grows.

Predefined message types vs. Arbitrary user data types
JMS provides five predefined message types, to conveniently specify different types of message payloads. Since JMS destinations are not typed, any type of payload can be produced and consumed on a destination.

If a consumer has a different idea of the message payload than the producer, it will manifest as runtime typecasting exception when the consumer tries to access the payload using a different message type.

Also, the user data payload must be converted into one of the available message types, thereby involving conversion overhead between user data type and JMS message types at both the producer and consumer ends.

DDS does not provide any predefined message or data types. Instead it uses the data types defined in the programming language. Typically these are specified using interface definition language (IDL) in a programming language neutral way. Middleware vendor provided tools are used to generate a programming language type, and corresponding type support classes.

For example, given a user type Foo, type specific FooTypeSupport, FooDataWriter, and FooDataReader are generated with APIs as per the DDS standard. This approach has several advantages: it allows for higher performance by eliminating a potential extra conversion between a user type and a middleware type; it potentially enables the user to plugin their own data serialization ad deserialization scheme.

Also, since DDS topics are strongly typed, the middleware can detect a type mismatch between the endpoints and notify the application.

User Experience Similarities
Despite the fundamental paradigm differences, the DDS and JMS user experience is somewhat similar, making it relatively easy to understand and switch back-and-forth between the two programming models. Figure 9 below illustrates the key steps in writing a JMS client (application). Figure 10 below illustrates the key steps in writing a DDS application.

Figure 9 JMS programming model.

As shown in Figure 9 above of the JMS programming model, the key steps are typically as follows. First, decide the JMS messaging domain to use: Point-to-Point (PtP) or publish-subscribe (Pub/Sub). Next, get a reference to the ConnectionFactory for that JMS domain; typically this is done via JNDI or some other non-JMS API. Then, from a ConnectionFactory, a Connection object is created; it represents the link between the client application and the JMS provider.

From the Connection, Session objects can be created. When creating a Session object, decide if the session is transacted or not, and the acknowledgment mode to use. Next, obtain a reference to a Destination on which messages will be produced, or from which messages will be consumed.

A Destination reference is obtained using some non-JMS API, usually JNDI. In order to produce messages on the destination, create a MessageProducer from the Session. Next, create the desired type of Message object, and specify the data payload to send. Then call the send() method on the MessageProducer to send the message to the Destination. In order to consume message from a Destination, from the Session create a MessageConsumer associated with the Destination. Register a MessageListener with the MessageConsumer, and implement its onMessage() method to receive incoming messages sent to the destination.

Figure 10. DDS programming model.

Figure 10 above summarizes the DDS programming model. The key steps are typically as follows. First, get a reference to the DomainParticipantFactory singleton, and create a DomainParticipant object associated with a domain id. A DDS domain comprises of communicating peer participants associated with the same domain id. (Note that "domain" has an entirely different meaning in the context of DDS than when used in the context of JMS). Then, for each type to be used by the application, use the middleware vendor provided scheme to generate type support classes from a high level type description (for example in IDL).

For a type "Foo", the classes "FooTypeSupport", "FooDataWriter" and "FooDataReader" classes are automatically generated for use with the DDS middleware implementation. Register the data types to be used by the application with the DomainParticipant. This is done using the "FooTypeSupport.register_type()" method generated for each user type. After registering the types, create the Topics on which data will be exchanged by the application. A Topic is created from the DomainParticipant, and is bound to one of the registered types.

In order to produce data on a Topic, create a Publisher from the DomainParticipant, and from the Publisher, create a DataWriter bound to a previously created Topic. Then, to produce a data sample, call the DataWriter's write() method with the new sample value. In order to consume data, create a Subscriber from the DomainParticipant, and from the Subscriber create a DataReader bound to a previously created Topic.

Register a DataReaderListener with the DataReader to be notified when data is available. When data is available, call the read() or the take() methods on the DataReader to access the received samples. In DDS, one can also specify desired quality of service (QoS) and a status listener on each of the entities, including DomainParticipant, Topic, Publisher, DataWriter, Subscriber, DataReader to tune the behavior and operation of the middleware.

While the user experience in using these two APIs is somewhat similar, there are some key differences.

For example, JMS offers five predefined message types, whereas DDS allows the use of arbitrary user data types. In DDS the type specific DataWriter and DataReader facades are strongly typed and generated using a vendor specific scheme. In JMS, Destinations are not typed; whereas DDS Topics are strongly typed.

In JMS, the destination to which a message is to be sent can be specified at the time of sending; in DDS the topic is implicitly associated with a DataWriter. In DDS one can find out the availability of a new update first, and then access the data; in JMS the notification and the message arrive together.

Some of these differences arise from the fundamental paradigm differences between the two APIs. A programmer must be aware of these differences when switching between the two APIs or when mapping a distributed application architecture to the underlying middleware technology.

To come in Part 3: Practical considerations when using DDS and/or JMS
To read Part 1 go to: The basics of DDS and JMS

Rajive Joshi, Ph.D., is principal engineer at Real-Time Innovations, Inc.

1) Data Distribution Service for Real-time Systems, v1.1,

2) J2EE Java Message Service (JMS)

3) RTI Data Distribution Service

Loading comments...