Getting Started: Messaging

Happy 2012! For the next installment of our getting started series we'll explore the messaging abstractions available to your Clojure apps when deployed on Immutant. Because Immutant is built atop JBoss AS7, it includes the excellent HornetQ messaging service built right in. Hence, there is nothing extra to install or configure in order for your applications to benefit from asynchronous messaging.

Destinations are either Queues or Topics

Two types of message destinations, or endpoints, are supported: queues and topics. A queue exhibits point-to-point semantics: a message sent to a queue will be delivered to a single recipient. A topic provides publish-subscribe semantics: messages sent to a topic will be delivered to all subscribed recipients. In both cases, the message producers have no direct knowledge of the message consumers.

Use the start function to define a messaging destination. A simple naming convention designates an endpoint as either a queue or a topic: if its name begins with /queue, it's a queue; if it begins with /topic, it's a topic.

(require '[immutant.messaging :as msg])
(msg/start "/queue/work")   ; to start a queue
(msg/start "/topic/news")   ; to start a topic

You can invoke start from anywhere in your application, but typically it's done in the immutant.clj initialization file, as described in an earlier tutorial.

While start has a complement, stop, you needn't call it directly. It will be invoked when your application is undeployed. And it's important to note that start is idempotent: if an endpoint has already been started, likely by a cooperating application, the call is effectively a no-op. Similarly, a call to stop will silently fail if the endpoint is in use by any other application. The last to leave will turn the lights out.

One Way to Produce Messages

publish

Messages are sent to a destination, whether queue or topic, via a single function, publish, to which is passed the destination name and the message content, which can be just about anything. A number of optional key-value parameters may be passed as well.

  • :encoding may be either :clojure (the default), :json (useful with non-clojure consumers) or :text (no encoding)
  • :priority may be an integer between 0-9, inclusive. Convenient keyword values :low, :normal, :high and :critical correspond to 0, 4, 7 and 9, respectively. 4 is the default.
  • :ttl time-to-live may be specified in milliseconds, after which time the message is discarded if not consumed. Default is 0, i.e. forever.
  • :properties is a hash of arbitrary message metadata upon which JMS selector expressions may be constructed to filter received messages.

Some examples:

;; A simple string
(msg/publish "/queue/work" "simple string")
;; Notify everyone something interesting just happened
(msg/publish "/topic/news" {:event "VISIT" :url "/sales-inquiry"})
;; Move this message to the front of the line
(msg/publish "/queue/work" some-message :priority :high :ttl 1000)
;; Make messages as complex as necessary
(msg/publish "/queue/work" {:a "b" :c [1 2 3 {:foo 42}]})
;; Make messages consumable by a Ruby app
(msg/publish "/queue/work" {:a "b" :c [1 2 3 {:foo 42}]} :encoding :json)

Three Ways to Consume Messages

receive

Block on a call to receive, passing a destination name and optionally, the following:

  • :timeout an expiration in milliseconds, after which nil is returned. Default is 0, i.e. wait forever
  • :selector a JMS expression used to filter messages according to the values of arbitrary :properties. For documentation on JMS selector syntax please see the javadoc for javax.jms.Message.

listen

Pass a destination name and function to listen and the decoded content of a message sent to that destination will be passed to the function. Options include:

  • :concurrency the maximum number of listening threads that can simultaneouly call the function. Default is 1.
  • :selector same as :receive

message-seq

Create a lazy sequence of messages via message-seq, which accepts the same options as receive.

Some examples:

;; Wait on a task
(let [task (msg/receive "/queue/work")]
  (perform task))

;; Case-sensitive work queues?
(msg/listen "/queue/lower" #(msg/publish "/queue/upper" (.toUpperCase %)))

;; Contrived laziness
(let [messages (message-seq queue)]
  (doseq [i (range 4)] (publish queue i))
  (= (range 4) (take 4 messages)))

Language Interop

One of our initial goals for Immutant messaging was simple interop between Ruby and Clojure applications deployed on a single platform. TorqueBox Ruby processors already grok the :json encoding and will automatically decode the message into the analogous Ruby data structures, so as long as you limit the content of your messages to standard collections and types, they are transparently interoperable between Clojure and Ruby in either direction. See the overlay post for more details on TorqueBox/Immutant integration.

Of course, the :json encoding enables other JVM-based languages -- anything you could conceivably cram into a war file -- to join in the fun, too. For non-JVM languages or external endpoints, something like the Pipes and Filters API's provided by Clamq could be used since we expose our JMS connection factory as immutant.messaging.core/connection-factory.

Anything Else?

Another advantage we get from AS7 is its clustering support. Once we work out some small integration bits, message distribution across a cluster of dynamic nodes will be automatically load-balanced and fault-tolerant, with minimal to no configuration required.

Of course, we still have other messaging features on our roadmap, e.g. XA transactions, durable subscribers and synchronous request/response, and we're looking for ways to make container-based deployment more developer-friendly, so there's still much to do. Feel free to follow along on Twitter, IRC, or the mailing list.

Getting Started: Installing Immutant v2

Greetings! This article covers the new, improved method for installing Immutant, and replaces the first in the getting started series of tutorials with Immutant. This entry covers setting up a development environment and installing Immutant. This tutorial assumes you are on a *nix system. It also assumes you have Leiningen installed. If not, follow these instructions, then come back here.

Installing the lein plugin

We provide a lein plugin for creating your Immutant applications and managing their life-cycles. As of this post, the latest version of the plugin is 0.3.1. Check clojars for the current version.

Let's install it as a global plugin:

 $ lein plugin install lein-immutant 0.3.1
Copying 3 files to /a/nice/long/tmp/path/lib
Including lein-immutant-0.3.1.jar
Including clojure-1.2.0.jar
Including clojure-contrib-1.2.0.jar
Including data.json-0.1.1.jar
Including fleet-0.9.5.jar
Including overlay-1.0.1.jar
Including progress-1.0.1.jar
Created lein-immutant-0.3.0.jar

Now, run lein immutant to see what tasks the plugin provides:

 $ lein immutant
Manage the deployment lifecycle of an Immutant application.

Subtasks available:
install    Downloads and installs Immutant
overlay    Overlays features onto ~/.lein/immutant/current or $IMMUTANT_HOME
env        Displays paths to the Immutant that the plugin can find
new        Creates a new project skeleton initialized for Immutant
init       Adds a sample immutant.clj configuration file to an existing project
deploy     Deploys the current project to the Immutant specified by ~/.lein/immutant/current or $IMMUTANT_HOME
undeploy   Undeploys the current project from the Immutant specified by ~/.lein/immutant/current or $IMMUTANT_HOME
run        Starts up the Immutant specified by ~/.lein/immutant/current or $IMMUTANT_HOME, displaying its console output

We'll only talk about the install and run tasks in this article - we covered the application specific management tasks in our second tutorial, and cover overlay in its own article.

Installing Immutant

Now we need to install an Immutant distribution. We've yet to make any official releases, but our CI server is setup to publish an incremental build every time we push to the git repo. The latest incremental build is always available at http://immutant.org/builds/immutant-dist-bin.zip. The previous version of this tutorial walked you through downloading and installing the latest incremental manually, but with the new install task, that's no longer necessary - the install task will download and install the latest incremental build for you. Let's see that in action:

 $ lein immutant install
Downloading http://repository-projectodd.forge.cloudbees.com/incremental/immutant/LATEST/immutant-dist-bin.zip
done!                                                                           
Extracting /a/nice/long/tmp/path/lib/immutant-dist-bin.zip
Extracted /Users/tobias/.lein/immutant/releases/immutant-1.x.incremental.51
Linking /Users/tobias/.lein/immutant/current to /Users/tobias/.lein/immutant/releases/immutant-1.x.incremental.51

Part of the install process links the most recently installed version to ~/.lein/immutant/current so the plugin can find the Immutant install without requiring you to set $IMMUTANT_HOME. If $IMMUTANT_HOME is set, it will override the current link.

If you want to install a specific incremental build, specify the build number (available from our builds page) as an argument to lein:

 $ lein immutant install 50

You can also have it install to a directory of your choosing (if you want the latest build, specify 'latest' as the version):

 $ lein immutant install latest /some/other/path

~/.lein/immutant/current will be linked to that location.

Running Immutant

To verify that Immutant is properly installed, let's fire it up. To do so, use the lein immutant run command. This is a convenient way to start the Immutant's JBoss server, and will run in the foreground displaying the console log. You'll see lots of log messages that you can ignore - the one to look for should be the last message, and will tell you the Immutant was properly started:

 $ lein immutant run
Starting Immutant via /Users/tobias/.lein/immutant/current/jboss/bin/standalone.sh
...
(a plethora of log messages deleted)
...
09:18:03,709 INFO  [org.jboss.as] (Controller Boot Thread) JBoss AS 7.1.0.Beta1 "Tesla" started in 2143ms - Started 149 of 211 services (61 services are passive or on-demand)

You can kill the Immutant with Ctrl-C.

Wrapping up

If you've done all of the above, you're now ready to deploy an application. We covered that in our second tutorial.

Since Immutant is still in a pre-alpha state, none of what I said above is set in stone. If anything does change, We'll update this post to keep it accurate.

If you have any feedback or questions, get in touch!

Getting Started: Deploying a Web Application

Welcome back! This article covers creating a basic Ring web application and deploying to an Immutant. It is the second installment in a series of tutorials on getting started with Immutant. If you haven't read the first installment, go do so now, since it covers installation and setup. This tutorial assumes you are on a *nix system.

Creating an Immutant Clojure application

In our previous article, we installed the lein plugin. Let's take another look at the tasks it provides:

~/immutant $ lein immutant
Manage the deployment lifecycle of an Immutant application.

Subtasks available:
new        Create a new project skeleton initialized for immutant.
init       Adds a sample immutant.clj configuration file to an existing project
deploy     Deploys the current project to the Immutant specified by $IMMUTANT_HOME
undeploy   Undeploys the current project from the Immutant specified by $IMMUTANT_HOME
run        Starts up the Immutant specified by $IMMUTANT_HOME, displaying its console output

We talked about run last time. This time, we'll cover new and deploy. To do so, we'll build a basic application that demonstrates the current web features. To get started, let's create an Immutant project:

~/immutant $ lein immutant new immutant-demo
Created new project in: /Users/tobias/immutant/immutant-demo
Look over project.clj and start coding in immutant_demo/core.clj
Wrote sample immutant.clj

The new task creates a Leiningen project and gives it a sample Immutant configuration file (immutant.clj). It is equivalent to calling:

~/immutant $ lein new immutant-demo && cd immutant-demo && lein immutant init

We'll come back to immutant.clj in a sec. Now, let's add a ring handler to our core namespace:

(ns immutant-demo.core)

(defn ring-handler [request]
  {:status 200
    :headers {"Content-Type" "text/html"}
    :body "Hello from Immutant!" })

Configuring the application for Immutant

When the Immutant deploys an application, it looks for a file named immutant.clj at the root and evaluates it if it exists. This file is used to configure the Immutant services you want your application to consume. It's the single place you defines all the components required by your application, and saves you from having to keep external configuration files in sync (crontabs, message queue definitions, init scripts, etc).

The file has example code for configuring web endpoints and messaging services, but we're just going to deal with web endpoints in this article. Edit your immutant.clj so it looks like:

(ns immutant-demo.init
  (:use immutant-demo.core)
  (:require [immutant.messaging :as messaging]
            [immutant.web :as web]))

(web/start "/" #'ring-handler)

We'll come back to what web/start is doing after we get the application running.

Deploying your application

Before we can start up an Immutant, we need to tell it about our application. We do that by deploying (for this to work, you need to have IMMUTANT_HOME set - see the previous article for details):

~/immutant/immutant-demo $ lein immutant deploy
Deployed immutant-demo to /Users/tobias/immutant/current/jboss/standalone/deployments/immutant-demo.clj

This writes a deployment descriptor to Immutant's deploy directory which points back to the application's root directory. Now the Immutant can find your application - so let's fire it up.

Starting Immutant

To launch an Immutant, use the lein immutant run command. This will start the Immutant's JBoss server, and will run in the foreground displaying the console log. You'll see lots of log messages that you can ignore - the one to look for should be the last message, and should tell you the app was deployed:

~/immutant/immutant-demo $ lein immutant run
Starting Immutant via /Users/tobias/immutant/current/jboss/bin/standalone.sh
...
(a plethora of log messages deleted)
...
13:04:39,888 INFO  [org.jboss.as.server.controller] (DeploymentScanner-threads - 2) Deployed "immutant-demo.clj"

Now, let's verify that our app is really there. Immutant runs on port 8080 by default, so let's hit it and see what happens:

~ $ curl http://localhost:8080/immutant-demo/
Hello from Immutant!

Yay!

You can kill the Immutant with Ctrl-C.

Context Paths

Remember our call to web/start earlier? Let's talk about what that is doing. To do that, however, we need to first talk about context paths. The context path is the portion of the URL between the hostname and the routes within the application. It basically tells Immutant which requests to route to a particular application.

An Immutant can host multiple applications at the same time, but each application must have a unique context path. If no context path is provided when an application is deployed, it defaults to one based on the name of the deployment. The deployment name is taken from the name of the deployment descriptor, which in turn is taken from the name of the project given to defproject in project.clj. So for our sample app above, the context path defaults to /immutant-demo. You can override this default by specifying a :context-path within an :immutant map in your project.clj. Let's go ahead and do that:

(defproject immutant-demo "1.0.0-SNAPSHOT"
  :description "A basic demo"
  :dependencies [[org.clojure/clojure "1.2.1"]]
  :immutant {:context-path "/"})

Now, when you call lein immutant deploy, the context path will be picked up from your project.clj and included in the deployment descriptor and any web endpoints your application stands up will be accessible under that context path.

Which brings us back to web/start. web/start stands up a web endpoint for you, and takes two arguments: a sub-context path and a Ring handler function. The sub-context path is relative to the application's context path, so a context path of "/ham" and a sub-context path of "/" makes the handler function available at /ham, whereas a sub-context path of "/biscuits" makes the handler function available at /ham/biscuits. Make sense?

You can register as many web endpoints as you like within an application - they just each need an application unique sub-context path. If we add this to our core.clj:

(defn another-ring-handler [request]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body "Pssst! Over here!"})

And this to our immutant.clj:

(web/start "/biscuits" #'another-ring-handler)

Redeploy the application to pick up the :context-path from immutant.clj:

~/immutant/immutant-demo $ lein immutant deploy
Deployed immutant-demo to /Users/tobias/immutant/current/jboss/standalone/deployments/immutant-demo.clj

Then fire an Immutant up again with lein immutant run, we can see they both work:

~ $ curl http://localhost:8080
Hello from Immutant!
~ $ curl http://localhost:8080/biscuits
Pssst! Over here!

web/start has a companion function for shutting down a web endpoint: web/stop. It takes the sub-context path for the endpoint, and can be called from anywhere. You aren't required to shut down your endpoints - Immutant will do that on your behalf when it is shut down or the application is undeployed.

Wrapping up

I hope you've enjoyed this quick run-through of deploying a web application to Immutant. Since Immutant is still in a pre-alpha state, none of what I said above is set in stone. If anything does change, I'll edit this post to keep it accurate. I've posted the demo application we've built if you want to download it.

If you have any feedback or questions, get in touch! And stay tuned - our next tutorial will cover using Immutant's messaging features.

Getting Started: Installing Immutant

This article is out of date - see our new tutorial for the latest instructions for installing Immutant.

Greetings! This article is the first in a series of tutorials on getting started with Immutant. This entry covers setting up a development environment and installing Immutant. This tutorial assumes you are on a *nix system. It also assumes you have Leiningen installed. If not, follow these instructions, then come back here.

Installing the lein plugin

We provide a lein plugin for creating your Immutant applications and managing their life-cycles. As of this post, the latest version of the plugin is 0.2.0. Check clojars for the current version.

Let's install it as a global plugin:

~ $ lein plugin install lein-immutant 0.2.0
Copying 3 files to /a/nice/long/tmp/path/lib
Including lein-immutant-0.2.0.jar
Including clojure-1.2.0.jar
Including commons-exec-1.1.jar
Including fleet-0.9.5.jar
Created lein-immutant-0.2.0.jar

Now, run lein immutant to see what tasks the plugin provides:

~ $ lein immutant
Manage the deployment lifecycle of an Immutant application.

Subtasks available:
new        Create a new project skeleton initialized for immutant.
init       Adds a sample immutant.clj configuration file to an existing project
deploy     Deploys the current project to the Immutant specified by $IMMUTANT_HOME
undeploy   Undeploys the current project from the Immutant specified by $IMMUTANT_HOME
run        Starts up the Immutant specified by $IMMUTANT_HOME, displaying its console output

We'll only talk about the run task today, and save the rest for the next tutorial.

Installing Immutant

Now we need to install an Immutant distribution. We've yet to make any official releases, but our CI server is setup to publish an incremental build every time we push to the git repo. The latest incremental build is always available at http://immutant.org/builds/immutant-dist-bin.zip.

At this point, I'd love to tell you to install Immutant via:

lein immutant install # coming soon!

But that currently won't work. Until we actually implement that, you'll have to download and setup Immutant manually. Go ahead and download our latest incremental build.

Once you have the zip file downloaded, unzip it somewhere handy. For this walk-through, I'm going to put it in ~/immutant/. When unzipped, the distribution will have the build number in the directory name. It's handy to have a consistent path to the distribution, allowing us to download other incremental builds in the future with minimal fuss. To do that, link the distribution to current. Here is my shell session doing just that:

~ $ mkdir immutant
~ $ cd immutant
~/immutant $ unzip -q ../downloads/immutant-dist-bin.zip
~/immutant $ ln -s immutant-1.x.incremental.15 current
~/immutant $ ls
current                     immutant-1.x.incremental.15
~/immutant $ ls -l
total 8
lrwxr-xr-x  1 tobias  staff   27 Nov  7 22:41 current -> immutant-1.x.incremental.15
drwxrwxr-x  3 tobias  staff  102 Nov  7 13:20 immutant-1.x.incremental.15

Now we'll set a environment variable pointing to the distribution:

export IMMUTANT_HOME=~/immutant/current

This will tell the lein plugin where to find Immutant. Once we implement lein immutant install, this requirement will go away.

Running Immutant

To verify that Immutant is properly installed, let's fire it up. To do so, use the lein immutant run command. This is a convenient way to start the Immutant's JBoss server, and will run in the foreground displaying the console log. You'll see lots of log messages that you can ignore - the one to look for should be the last message, and will tell you the Immutant was properly started:

~/immutant $ lein immutant run
Starting Immutant via /Users/tobias/immutant/current/jboss/bin/standalone.sh
...
(a plethora of log messages deleted)
...
22:46:31,663 INFO  [org.jboss.as] (Controller Boot Thread) JBoss AS 7.x.incremental.182 "Ahoy!" started in 2990ms - Started 136 of 200 services (61 services are passive or on-demand)

You can kill the Immutant with Ctrl-C.

Wrapping up

If you've done all of the above, you're now ready to deploy an application. We'll cover that in our next tutorial.

Since Immutant is still in a pre-alpha state, none of what I said above is set in stone. If anything does change, I'll edit this post to keep it accurate.

If you have any feedback or questions, get in touch!