Building a effective real-time distributed publish-subscribe framework: part 3 -

Building a effective real-time distributed publish-subscribe framework: part 3


InPart 1 and Part 2 in this series ofarticles weexamined the functional similarities and differences between DDS andJMS, and their implications for real time distributed applicationdesign. Beyond this, a number of practical factors come into play whenchoosing a middleware technology for building a real time distributedsystem: DDS, JMS or some combination. What you must deal with when itcomes time to implement an actual design involve severalimplementation-related issues:

(1) the most appropriatedistributed architecture;
(2) whether or not you needpoint-to-point delivery and what kinds;
(3) the most appropriateunderlying software platform;
(4) interoperability;
(5) the transport mechanismsyou will be using;
(6) the middleware performanceyou will need;
(7) scalability; and
(8) the right balance betweensupport for real-time operation and for the very real needs of theenterprise applications your system will be interfacing to.

However, such decisions are not just either/or considerations.Often, it will require that you use some combination of the newprotocol frameworks.

JMS APIsare described in terms of a client/provider interaction; the clientbeing the user application, distinct from the JMS provider or themiddleware server. Most popular JMS provider implementations havecentralized server-based architecture; some use a cluster of serversfor fault-tolerance and load balancing. In a centralized server-basedarchitecture, a message must pass via the server, which introducesextra latency and a potential resource bottleneck.

DDS APIs are described in terms of a peer-to-peer interaction; datais transferred directly from a DataWriter to a DataReader. DDSimplementations generally use a decentralized peer-to-peerarchitecture. For example, the “RTI Data Distribution Service” from Real-Time Innovations, Inc.  has acompletely symmetric architecture. There is no single point of failureand data transfer latency is minimized. Participants can freely join orleave a domain, without needing special configuration. This is inkeeping with the goals of DDS to enable robust, high-performance, lowlatency distributed applications.

Point-to-point delivery
In the JMS PtP messaging domain, a destination (Queue) may havemultiple consumers (QueueReceivers )and producers (QueueSenders ).A message is processed by exactly one of the attached consumers. Thus,a message is delivered point-to-point, from the producer to one of themany available consumers. The policy for selecting a consumer is leftup-to the middleware provider. Upon message redelivery (if any), amessage may get dispatched to a different consumer.

The point-to-point delivery mechanism in the JMS PtP messagingdomain makes it very easy to distribute processing load across multipleidentical consumers, thus providing a simple means for load balancing.However, since PtP behaves as if the messages are put in a singlelogical queue and handed over to one of the available consumers, thismessaging domain will generally be less scalable that the Pub/Subdomain.

DDS does not support a point-to-point delivery mechanism. All thematching consumers associated with a topic will receive updates. The PARTITION QosPolicy may be used topartially achieve point-to-point delivery. A PARTITION QosPolicy,specifies a set strings that introduce a logical partition among thetopics visible by a Publisher and a Subscriber.

A DataWriter within a Publisher only communicates with a DataReader in aSubscriber if (in addition to matching the Topic and having compatibleQoS) the Publisher and Subscriber have a common partition name string.A change of this policy can potentially modify the “association” ofexisting DataReader and DataWriter entities. It may establish new “associations” that did not existbefore, or break existing associations.

However, Point-to-point delivery may be accomplished by: (1)assigning a unique partition name to every consumer (Subscriber,DataReader pair); and (2) switching a producer (Publisher, DataWriterpair) among those partition names. The consumer selection policy can beconfigured in a variety of ways, at the application level.

JMS, as the name implies, was developed to provide a portable vendorneutral Java API for a wide range of messageoriented middleware (MOM) implementations. JMS requires the Javaplatform. Some vendors provide JMS like APIs for other programminglanguages, but there is no established standard. JMS vendors may alsoprovide proprietary APIs in other languages, native to the underlyingMOM implementation. Also, note that JMS applications require non JMSAPIs to bootstrap the client application. The standard practice is touse JNDI APIs, which are well established for Java EE programming.

DDS is an ObjectManagement Group (OMG) standard defined in a language and platformneutral manner. OMG defines standard platform specific mappings tocreate language specific bindings. Therefore, standardized DDS APIs areavailable for all OMG supported programming languages; C, C++, and Javabeing the most popular. Since DDS is a standard in C, C++, and Java, itis available on a wide variety of platforms, including popularreal-time operating systems (RTOS), desktop and server operatingsystems, and Java platforms.

JMS is an API only standard, and does not define an on-the-wireinteroperability protocol. JMS only requires limited messageportability on the client side: a Message created by provider A shouldbe usable with provider B. Beyond this, a producer written usingprovider A cannot be expected to deliver messages to a consumer writtenusing provider B.

Currently DDS is also an API only standard. However, there is activeprogress being made at the OMG towards standardizing on a DDSon-the-wire interoperability protocol.

JMS being an API only specification does not specify a transport model.However, since JMS message delivery is reliable (messages are notdropped unless the provider fails) and ordered, a JMS implementationcan benefit from a reliable transport such as TCP.Being connection-oriented, TCP also fits naturally into theclient/provider scheme.

Centralized server based JMS implementations generally use TCP. Theyrely on TCP to guarantee reliable and ordered delivery required by theJMS APIs. Some UDP based implementations do exist; they implement thereliable and ordered message delivery semantics on top of UDP.

Like JMS, DDS being an API only specification does not specify atransport model. However, DDS does not depend on reliable and ordereddelivery of messages. In fact, the “best-effort” delivery QosPolicy isnaturally suited to an unreliable low-latency transport such as UDP,whereas “reliable” delivery may benefit from the use of a reliabletransport like TCP.

However, since DDS middleware must support both delivery schemes, itcannot make any assumptions about the reliability properties of theunderlying transport. Therefore, DDS middleware is less reliant on thecapabilities provided by a particular class of transport, and may beable to work well with a variety of transport classes. As an example,the RTI Data Distribution Service implementation provides a pluggabletransport architecture, wherein any kind of transport can be pluggedin, including UDP, TCP, shared memory, and various specializedtransports.

JMS has a provision for optionally specifying a (usernme, password)when creating a Connection. Beyond this, security issues are left up tothe JMS middleware vendor and the client application.

DDS provides an extension mechanism that can be useful in creatingsecure applications. For example, a consumer application can use aDataReader's USER_DATA QosPolicy to present security credentials to aDataWriter in the producer application. The producer application canauthenticate the security credentials; these may potentially containauthorization rights.

If the consumer's security credentials are not acceptable, theDataReader entity can be permanently ignored using the DomainParticipant.ignore_subscription()method. A secure transport provided by the middleware vendor can ensurethat the data is transferred securely.

JMS implementations minimally require the use of an external means forconfiguring and administering Destinations and ConnectionFactories. Itis common practice to use JNDI to access theseobjects. Vendor provided proprietary tools must be used to configurethe JNDI registries, before they can be used by an application.Evolving an application over its lifetime requires coordination withJNDI administration.

DDS implementations are expected to spontaneously discover eachother; the DDS APIs do not rely on external means for establishing thediscovery information. For example, with RTI Data Distribution Service,the user application only needs to link in a middleware library; noadditional configuration is required to discover and establishdataflows with peer applications. As a result, DDS applications areplug-n-play, and require “zero” system administration.

Middleware performance can be characterized along several aspectsincluding: (1) the end-to-end latency, i.e. the time required to send amessage from a producer to a consumer; (2) the throughput, i.e. themaximum amount of data per unit time that can be transferred from aproducer to a consumer.

While it is impossible to make any specific comments aboutperformance—this can vary significantly from one middlewareimplementation to another (regardless of the supported APIs)—it ispossible to make some general observations regarding potentialmiddleware performance, based on the different choices made by the JMSvs. DDS APIs.

As compared with JMS, DDS has several features that can potentiallyminimize the end-to-end latency. These include:

(1) a “best-efforts”delivery mode that does not require acknowledgements;
(2) reduced message overhead,since meta-data such as message headers and properties are notspecified per message, but rather on a per endpoint basis;
(3) ability to use arbitrarydata types which eliminates the need for converting back-and-forthbetween user and middleware provided types; (4) notification of dataavailability does not include the actual data, thus avoiding theoverhead in setting this up;
(5) ability to support”zero-copy” data access so that an application can access the receiveddata directly in the middleware internal buffers without requiring acopy;
(6) direct peer-to-peer datatransfer from a DataWriter to DataReader without needed anintermediary. Thus, DDS middleware can potentially have better (lower)latency performance,

As compared with JMS, DDS has several features that can potentiallymaximize the throughput. These include: (a) reduced message overhead as in(2) above; (b) reducedprocessing in the data path as a result of (3) and (4) above; (c) direct data transfer as in (6)above, which eliminates a potential resource bottleneck and loadingpoint. Thus, DDS middleware can potentially have better (higher)throughput performance as well.

Independent studies have observed DDS implementations that provide afactor of ten performance improvement of over JMS implementations.

Scalability refers to the ability to maintain performance levels asmore nodes are added to a distributed system. For example,publish-subscribe scales better compared to “remote-procedure-calls(RPC)”, due to the loose coupling between participants. As withperformance, it is impossible to make any specific comments aboutscalability—this can vary significantly from one middlewareimplementation to another (regardless of the supported APIs). We somegeneral observations regarding potential middleware scalability, basedon the different choices made by the JMS vs. DDS APIs.

As compared with DDS, JMS has certain features that can potentiallylimit its scalability compared to DDS. These include:

(1) PtP messaging domain,which specifies that a message be delivered to exactly one consumer,and behaves as if the messages are put in a single logical queue andhanded over to one of the available consumers;
(2) centralized server-basedarchitectures, generally used by JMS implementations will be lessscalable than decentralized peer-to-peer architectures that supportdirect data transfer between endpoints.

<>Real-time versus enterpriseapplication needs
DDS and JMS vary in their support for the needs of both real-time andenterprise applications. DDS has a variety of features that directlymeet the needs of real-time applications, and have no counterparts inJMS. This is not surprising since DDS was developed while keepingreal-time requirements in mind. The real-time specific features of DDSinclude:

(1) a low-latency best-effortsdelivery mechanism;
(2) QoS policies forpredictable delivery;
(3) QoS<> policies forresource management;
(4) status notifications; and
(5) potential for lower latencyand higher throughput as discussed earlier.

In addition, the availability of DDS on high-performance RTOSes andthe ability to utilize low latency transports (for example UDP insteadof TCP) can further minimize end-to-end latency and support predictableoperation. Combined together, they make possible DDS implementationsthat enable high-performance real-time distributed applications.

JMS has a variety of features that directly meet the needs ofenterprise applications, and have no counterparts in DDS. This is notsurprising since JMS was originally developed to provide a Java adaptorfor a variety of enterprise messaging middleware implementations.

The enterprise specific features of JMS include: (1) full transaction support; and (2) explicit user applicationmessage acknowledgements. Combining message acknowledgements withpersistent and durable delivery allows enterprise applications toguarantee message delivery.

In addition, Java EE, widely used in enterprise environments,supports a Message-driven Bean. A message-driven bean is a dataconsumer integrated into the Enterprise Java Beans (EJB)framework. Producers are written directly using the messaging API.While the message-driven bean specification does not assume the use ofJMS, it is the most commonly messaging technology supported by Java EEvendors.

JMS implementations generally also support for JTA, so that amessage application can participate in a distributed transaction. Also,the JNDI APIs generally used by JMS applications are included in theJava EE specifications.

Thus, JMS is well integrated into enterprise application frameworks;given its legacy this is hardly surprising. However, DDS can also beused in enterprise environments; a message-driven bean using DDS canpotentially simplify Java EE integration.

Using DDS and JMS together
It should be obvious that the choice of DDS or JMS as the middlewaretechnology has a significant impact on a data-centric design. While DDSand JMS offer some capabilities that are similar, there also offer someunique capabilities. Thus, a data-centric design may employ both incomplementary ways. There is nothing precluding the use of JMS and DDStogether in the same application.

In developing a distributed system using both JMS and DDS, certaincapabilities can facilitate development and integration. These include (1) DDS-JMS bridging; (2) JMS/DDS bindings; and (3) DDS for JMS discovery.

JMS-DDSbridging. DDS-JMS bridging involves creating a “bridge” that isboth a DDS and JMS application. A bridge allows JMS and DDSapplications to interoperate.

A bridge forwards JMS messages as DDS data updates, and DDS dataupdates as JMS messages. A “bridge configuration” file can specify themapping between DDS and JMS topics, types, and QoS. Such a bridge willincur data conversion and mapping overhead, and introduce a singlepoint of failure between the DDS and JMS domains. It will be limited tosupporting the “least common denominator” i.e. only the overlappingcapabilities of DDS and JMS. It may not be always possible to provideend-to-end data delivery semantics. However, it can be useful forintegrating disparate sub-systems using JMS or DDS.

JMS/DDS bindings .JMS/DDS bindings wrap a DDS middleware with JMS APIs. JMS/DDS bindingscan be useful for porting applications written to a JMS API to a DDSdomain, or for developing JMS applications that are interoperable withDDS applications, or simply to enable a potentially higher performanceJMS implementation.

Since DDS provides finer grained data distribution and management ofdata flows with many more QoS, it is possible to efficiently map andimplement JMS APIs on top of DDS. JMS features not directly provided byDDS, such as PtP messaging semantics, full transactional semantics, andclient acknowledgement, can be implemented on top of the DDS APIs. Theremaining JMS features can be mapped into DDS APIs.

Note that implementing a DDS API on top of JMS is not a practicalidea, since JMS does not provide the fine granularity and low-levelprimitives needed to provide an efficient DDS implementation.

DDS for JMSdiscovery. JMS clients rely on non-JMS APIs for creating theConnectionFactory and Destination objects. Typically these objects areobtained by performing a JNDI lookup; these must have been alreadyregistered with the JNDI directory.

An application can alternatively utilize DDS to discover theseobjects. A JMS provider, they could use DDS to announce the configuredConnectionFactory and Destination objects to the client applications. AJMS client application would receive the available objects, select theones it is interested in, and bootstrap the JMS APIs. This approach isuseful in a mixed DDS and JMS environment, where DDS and JMS are usedsimultaneously to distribute different types of information. JMSapplications can benefit from the use of DDS's spontaneous discoverymechanism.

DDS and JMS differ in their ability to cater to the key data-centricdesign requirements. DDS is newer standard based on fundamentallydifferent paradigms than JMS, with regards to data modeling, dataflowrouting, discovery, and data typing. However, these differences provideapplications designers with powerful new architectural possibilities.

If you are designing or integrating distributed data-centricapplications, DDS and JMS merit careful consideration. Using one orboth can considerably simplify a data-centric design and integration,and help maintain the focus on application issues, rather than becomingbogged down by communication and data delivery concerns.

To read Part 1 go to: Thebasics ofDDS and JMS
To read Part 2 go to: Thedifferences between DDS and JMS

Rajive Joshi, Ph.D., is principalengineer at Real-TimeInnovations, Inc.

1) DataDistribution Service for Real-time Systems, v1.1,

2) J2EE Java Message Service(JMS)

3) RTI DataDistribution Service

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.