John D'Emic's blog about programming, integration, system administration, etc...

Monday, January 25, 2010

Mule and JMS as Decoupling Middleware

I just finished reading Michael Nygard's excellent "Release It!". His war stories echo a lot of my own experience, particularly when I was doing ops work full-time. Its a definite must read for developers and system administrators alike.

In Chapter 5 Michael describes the "Decoupling Middleware" pattern. He suggests using a messaging broker to decouple remote service invocation. This allows you to leverage the features of the messaging broker, like durability and delayed re-delivery, to improve the resiliency of communication with a remote service.

The following demonstrates how to use Mule and JMS to decouple interaction with Twitter's API. "Tweet Service"
transactionally accepts messages from a JMS queue and submits them to Twitter's REST API:


<model name="Twitter Services">

<service name="Tweet Service">
<inbound>
<jms:inbound-endpoint queue="tweets">
<jms:transaction action="BEGIN_OR_JOIN"/>
</jms:inbound-endpoint>
</inbound>
<http:rest-service-component
httpMethod="POST"
serviceUrl="http://user:password@twitter.com/statuses/update.json">
<http:requiredParameter key="status" value="#[payload:]"/>
</http:rest-service-component>
</service>
</model>


Let's consider some of the benefits of this indirection:


  • The service can be taken down / brought up without fear of losing messages - they will queue on the broker and be delivered when the service is brought back up.

  • The service can be scaled horizontally simply by adding additional instances (competing consumption off the queue)

  • Messages can be re-ordered based on priority and sent to the remote service

  • Requests can be re-submitted in the event of a failure.



The last point is particularly important. Its common to encounter transient errors when integrating with remote services, particularly web-based ones. These errors are usually recoverable after a certain amount of time. If your JMS broker supports it, you can use delayed redelivery to periodically re-attempt the request. HornetQ supports this by configuring address-settings on the queue. The following address-settings for the "tweets" queue specifies 12 redelivery attempts with a 5 minute interval between each request:


<address-setting match="jms.queue.tweets">
<max-delivery-attempts>12</max-delivery-attempts>
<redelivery-delay>300000</redelivery-delay>
</address-setting>


Its incidentally easy to employ a "circuit-breaker", another one of Michael's patterns, into the above using an exception-strategy. I'll demonstrate that in another post.