OpenMQ, Second Thoughts

 OpenMQ was fairly painless to get going with Mule.  I opted to set it up in our staging environment as a conventional cluster with 2 nodes. In this scenario, clients can maintain a list of brokers to connect to in the event one of them fails.  Loadbalancing between brokers might also be supported, but I haven't gone too far down that rabbit hole yet.  

Nor have I gone down the rabbit hole of HA clusters, which support failover of message data between brokers but require a shared database.  Amongst other things, we're using JMS to distribute monitoring alerts.  To do HA in production in a sensible manner, we'd need to back OpenMQ against our production MySQL cluster.  Since we're sending monitoring data about the same production MySQL cluster over JMS,  if the MySQL cluster failed we'd never hear about it.  We're not (currently) using JMS for any sort of financial or mission-critical data so losing a few messages in the event of a failover isn't too big of a deal for us as long as its a rare occurrence.  

Getting clients to connect to OpenMQ was a little more painful. It manages all of its "objects" (connection factories, queues, topics, etc) in JNDI.  We don't really use any sort of distributed JNDI infrastructure, so I started off using the filesystem JNDI context supplied with OpenMQ. In this scenario, all your OpenMQ objects are stored in a hidden file in a directory.  This works fine in development or testing situations when your clients and broker all have access to the directory.  Its obviously not an option for production, unless you do something painful like make tarballs of the JNDI filesystem directory and scp them or around or export it over NFS.    

According to the documentation, the "right" way seems to be by using an LDAP directory context to store the JNDI data (someone please correct me if I'm wrong about this.)  In this case, you store your OpenMQ objects to LDAP.  Each client then loads the appropriate connection factory, queus, etc from LDAP.  This is nice in the sense that configuration data for the connections (broker lists, etc) are maintained outside of the clients.  Presumably this allows you to add brokers to a cluster,  etc w/o having to restart your JMS clients.  

Despite the bit of complexity, this again was pretty straightforward. I just needed an LDAP directory to store the JNDI data in.  It (briefly) occurred to me to use our Active Directory deployment.  My assumption was, however, that this would involve modifying Active Directory's schema which I've never done before and heard nightmare stories about(it would also involve making changes to the production AD deployment - which is treated akin to a live hand grenade in the company.)

I ultimately opted to use OpenLDAP.  This was painless.  The only thing I had to do was include the supplied java.schema in the the slapd.conf and restart the service.  A short while later I was able to get Mule and JMeter sending messages through it.  The OpenMQ command line stuff worked great while doing some preliminary load testing.  The queue metrics in particular were really nice - it lets you watch queue statistics the same way you'd watch memory statistics with vmstat or disk statistics with iostat.  I am pretty impressed so far...