Today we'll get a Clojure application running in Immutant on OpenShift, persisting its data to a PostgreSQL database. We'll use Poorsmatic, the app I built in my recent talk at Clojure/Conj 2012.

Poorsmatic, a "poor man's Prismatic", is a truly awful content discovery service that merely returns URL's from Twitter that contain at least one occurrence of the search term used to find the tweets containing the URL's in the first place.

Got that? Don't worry. It doesn't matter.

Because Poorsmatic was contrived to be a pretty good example of many of Immutant's features, including topics, queues, XA transactions, HA services, and a few other things. In my talk I used Datomic as my database, but here we'll try a different approach, using Lobos for database migrations, the Korma DSL, and OpenShift's PostgreSQL cartridge for persistence.

Create an app on OpenShift

To get started on OpenShift you'll need an account, the command line tools installed, and a domain setup. Below you'll see references to $namespace -- this corresponds to your domain name.

Once you've setup your domain, create an app. Call it poorsmatic.

$ rhc app create -a poorsmatic -t jbossas-7

We're specifying the jbossas-7 OpenShift cartridge. That will create a sample Java application in the poorsmatic/ directory. But we don't want that. Instead, we'll use the Immutant Quickstart to add the Immutant modules to AS7 and replace the Java app with a Clojure app:

cd poorsmatic
rm -rf pom.xml src
git remote add quickstart -m master git://github.com/openshift-quickstart/immutant-quickstart.git
git pull --no-commit -s recursive -X theirs quickstart master
git add -A .
git commit -m "Add Immutant modules and setup Clojure project"

At this point, we could git push, and after a couple of minutes hit http://poorsmatic-$namespace.rhcloud.com to see a static welcome page. Instead, we'll configure our database and add the Poorsmatic source files before pushing.

Add the PostgreSQL cartridge

To add a PostgreSQL database to our app, we add a cartridge:

$ rhc cartridge add postgresql-8.4 -a poorsmatic

And boom, we have a database. We have to tweak it just a bit, though. So we're going to log into our app using the ssh URI from the output of the app create command (available via rhc app show -a poorsmatic or from the My Applications tab of the web UI). Here's the URI it gave me:

$ ssh a4117d5ebac04c5f8114f7a96eba2737@poorsmatic-jimi.rhcloud.com

Once logged in, we need to modify PostgreSQL's default configuration to enable distributed transactions, which Poorsmatic uses. We're going to set max_prepared_transactions to 10 and then restart the database:

$ perl -p -i -e 's/#(max_prepared_transactions).*/\1 = 10/' postgresql-8.4/data/postgresql.conf
$ pg_ctl restart -D $PWD/postgresql-8.4/data -m fast
$ exit

If you forget to do this, you'll see errors referencing max_prepared_transactions in the logs.

Add the Poorsmatic source to your app

We'll use git to pull in the Poorsmatic source code. You can use the same technique to get your own apps deployed to OpenShift:

$ git pull -s recursive -X theirs git://github.com/jcrossley3/poorsmatic.git korma-lobos

Note that we specified the korma-lobos branch.

Configure the app to use PostgreSQL

You'll see Leiningen profiles in project.clj that determine which database both the lobos and korma libraries will use. One of these profiles, :openshift, refers to the name of the PostgreSQL datasource configured in your .openshift/config/standalone.xml provided by the quickstart.

We'll activate the :openshift profile in deployments/your-clojure-application.clj:

{
 :root (System/getenv "OPENSHIFT_REPO_DIR")
 :context-path "/"
 :swank-port 24005
 :nrepl-port 27888

 :lein-profiles [:openshift]
}

Add your Twitter credentials

Finally, because Poorsmatic accesses Twitter's streaming API, you must create an account at http://dev.twitter.com and add a file called resources/twitter-creds that contains your OAuth credentials in a simple Clojure vector:

["app-key" "app-secret" "user-token" "user-token-secret"]

You may be concerned about storing sensitive information with your app, but remember that OpenShift secures your git repo with ssh public/private key pairs and only those people whose public keys you've associated with your app have access to it.

Push!

Now we can commit our changes and push:

$ git add -A .
$ git commit -m "Database config and twitter creds"
$ git push

And now we wait. The first push will take a few minutes. Immutant will be installed and started, your app deployed, the app's dependencies fetched, the database schema installed, etc. You should login to your app and view the logs while your app boots:

$ ssh a4117d5ebac04c5f8114f7a96eba2737@poorsmatic-jimi.rhcloud.com
$ tail_all

Eventually, you should see a log message saying Deployed "your-clojure-application.clj", at which point you can go to http://poorsmatic-$namespace.rhcloud.com, enter bieber and then watch your server.log fill up with more meaningless drivel than you ever dreamed possible.

And you may even see some bieber tweets. ;-)

Reload the web page to see the scraped URL's and their counts.

The REPL

You may have noticed the nREPL and Swank ports configured in the deployment descriptor above. They are not externally accessible. They can only be accessed via an ssh tunnel secured with your private key.

Run the following:

$ rhc port-forward -a poorsmatic

Depending on your OS, this may not work. If it doesn't, try the -L option:

$ ssh -L 27888:127.11.205.129:27888 a4117d5ebac04c5f8114f7a96eba2737@poorsmatic-jimi.rhcloud.com

But replace 127.11.205.129 with whatever rhc port-forward told you (or ssh to your instance and echo $OPENSHIFT_INTERNAL_IP). And obviously, you should use the ssh URI associated with your own app.

Once the tunnel is established, you can then connect to the remote REPL at 127.0.0.1:27888 using whatever REPL client you prefer.

Tune in next time...

Immutant's clustering capabilities yield some of its coolest features, e.g. load-balanced message distribution, highly-available services and scheduled jobs, etc. But clustering is a pain to configure when multicast is disabled. OpenShift aims to simplify that, but it's not quite there yet. In a future post, I hope to demonstrate those clustering features by creating a scaled OpenShift application, letting it deal with all the murky cluster configuration for you.

Stay tuned!