This is not a comprehensive guide to Clojure, but it will get you started.
Riemann is configured using a Clojure-based configuration file. This means your configuration file is actually processed as a Clojure program. So to process events and send alerts and metrics you'll be writing Clojure. Don't panic! You won't need to become a fully fledged Clojure developer to use Riemann. We'll teach you what you need to know in order to use Riemann. Additionally, Riemann comes with a lot of helpers and shortcuts that make it easier to write Clojure to do what we need to process our events.
Let's learn a bit more about Clojure and help you get started with Riemann. Clojure is a dynamic programming language that targets the Java Virtual Machine. It's a dialect of Lisp and is largely a functional programming language.
Functional programming is a programming style that focuses on the evaluation of mathematical functions and steers away from changing state and mutable data. It's highly declarative, meaning you build programs from expressions that describe "what" a program should accomplish rather than "how" it accomplishes it.
Examples of declarative programming languages include SQL, CSS, regular expressions and configuration management languages like Puppet and Chef. Let's take a simple example.
In this SQL query we're asking for the user_id
for user_name
of Alice
from the users
table. The statement is asking a declarative "what" question. We don't really care about the "how", the database engine takes care of those details.
In addition to their declarative nature, functional programming languages try to eliminate all side effects from changing state. In a functional language when you call a function its output value depends only on the inputs to the function. So if you repeatedly call function f
with the same value for argument x
, f(x)
, it will produce the same result every time. This makes functional programs very easy to understand, test and predict. Functional programming languages call functions that operate like this "pure" functions.
The best way to get started with Clojure is to understand the basics of its syntax and types. Let's get a crash course now.
Let's step through the Clojure basic syntax and types. We'll also show you a tool called REPL that can help you test and build your Clojure snippets. REPL (short for read–eval–print loop) is an interactive programming shell that takes single expressions, evaluates them and returns the results. It's a great way to get to know Clojure.
irb
. Or in Python when you launch the python
binary interactively.
We can install REPL via a tool called Leiningen. Leiningen is an automation tool for Clojure that helps you automate the build and management of Clojure projects.
In order to install Leiningen we'll need to have Java installed on the host. The prerequisite Java packages on Ubuntu and Red Hat for Reimann will also be sufficient for Leiningen too.
We're going to download a Leiningen binary called lein
to install it. Let's download that into a bin
directory under our home directory.
Here we've created a new directory called ~/bin
and changed into it. We've then used the curl
command to download the lein
binary and the chmod
command to make it executable. Lastly, we've added our ~/bin
directory to our path so that we can find the lein
binary.
~/bin
directory assumes you're in a Bash shell. It's also temporary to your current shell. You'd need to add the path to your .bashrc
or the similar setup for your shell.
Next we need to run lein
to auto-install its supporting libraries.
This will download Leiningen's supporting Jar file.
Finally, we can run REPL using the lein repl
sub-command.
This will download Clojure itself (in the form of its Jar file) and launch our interactive Clojure shell.
Let's use this interactive shell to look at some of the syntax and functions we've just learnt about. Let's start by opening our shell.
Now let's try a simple expression.
The nil
expression is the simplest value in Clojure. It represents literally nothing.
We can also specify an integer value.
Or a string.
Or Boolean values.
Whilst interesting these values aren't very exciting on their own. To do some more interesting things we can use Clojure functions. A function is structured like this:
Let's look at a function in action by doing something with some values: adding two integers together.
In this case we've used the +
function and added 1
and 1
together to get 2
.
But there's something about this structure that might look familiar to you if you've used other programming languages. Our function looks just like a list. This is because it is! Our expression might add two numbers together but it’s also a list of three items in a valid list data structure.
This is a feature of Clojure called homoiconicity, sometimes described as: "code is data, data is code". This concept is inherited from Clojure's parent language: Lisp.
Homoiconicity means that the program's structure is similar to its syntax. In this case Clojure programs are written in the form of lists. Hence you can gain insight into the program's internal workings by reading its code. This also makes metaprogramming really easy because Clojure's source code is a data structure and the language can treat it like one.
Now let's look more closely at the +
function. Each function is a symbol. A symbol is a bare string of characters, like +
or inc
. Symbols have short names and full names. The short name is used to refer to it locally, for example +
. The full name, or perhaps more accurately the fully qualified name, gives you a way to refer to the symbol unambiguously from anywhere. The fully qualified name of the +
symbol is clojure.core/+
. The clojure.core
being the fundamental library of the Clojure language. We can refer to +
in it's fully qualified form here:
Symbols refer to other things; generally they point to values. Think about them as a name or identifier that points to a concept: +
is the name, "adding" is the concept. When Clojure encounters a symbol it evaluates it by looking up its meaning. If it can't find a meaning it'll generate an error message, for example:
Clojure also has a syntax for stopping that evaluation. This is called quoting and it is achieved by prefixing the expression with a quotation mark: '
.
This returns the symbol itself without evaluating it. This is important because often we want to do things, review things, or test things without evaluating.
For example, if we need to determine what type of thing something is in Clojure we can use the type
function and quote the function like so:
Here we can see that +
is a Clojure language symbol.
Clojure also has a variety of data structures. Especially useful to us will be collections. Collections are groups of values, for example a list or a map.
Let's start by looking at lists. Lists are core to all Lisp-based languages (Lisp means "LISt Processing"). As we discovered above Clojure programs are essentially lists. So we're going to see a lot of them!
Lists have zero or more elements and are wrapped in parentheses.
Here we've created a list containing the elements a
, b
and c
. We've quoted it because we don't want it evaluated. If we didn't quote it then evaluation would fail because none of the elements, a
, b
, etc are defined. Let's see that now.
We can do a few neat things with lists, for example add an element using the conj
function.
You can see we've added a new element, d
, to the front of the list. Why the front? Because a list is really a linked list and focusses on providing immediate access to the first value in the list. Lists are most useful for small collections of elements and when you need to read elements in a linear fashion.
We can also return values from a list using a variety of functions.
Here we've pulled out the first element, second element, and using the nth
function, the third element.
This last, nth
, function shows us a multi-argument function. The first argument is the list, '(a b c)
, and the second argument is the index value of the element we want to return, here 2
.
0
.
We can also create a list with the list
function.
Another collection available to us is the vector. Vectors are like lists but they are optimized for random access to the elements by index. Vectors are created by adding zero or more elements inside square brackets.
Like lists, we can again use conj
to add to a vector.
You'll note the d
element is added at the end because a vector isn't focussed on sequential access like a list.
There are some other useful functions we can use on lists and vectors, for example to get the last element in a list or vector.
Or count the elements.
Because vectors are designed to look up elements by index, we can also use them directly as functions, for example:
Here we've retrieved the value, 2
, at index 1
.
We can create a vector with the vector
function or convert an existing structure, like a list, into a vector with the vec
function.
There's a final collection related to lists and vectors called a set. Sets are unordered collections of values, prefixed with #
and wrapped in curly braces, { }
. They are most useful for collections of values where you want to check a value or values is present.
You'll notice the set was returned in a different order. This is because sets are focussed on presence lookups so order doesn't matter quite so much.
Like lists and vectors we can use the conj
function to add an element to a set.
Sets can never contain an element more than once, so adding an element which is already present does nothing. You can remove elements with the disj
function.
The most common operation with a set is to check for the presence of a specific value, for this we use the contains?
function.
Like a vector, you can also use the set itself as a function. This returns the value if it is present or nil
if it is not.
You can make a set out of any other collection with the set
function.
Here we've made a set out of a vector.
The last data structure we're going to look at is the map. Maps are key/value pairs enclosed in braces. You can think about them as being equivalent to a hash.
Here we've defined a map with two key/value pairs: :a 1
and :b 2
.
You'll note each key is prefixed with a :
. This denotes another type of Clojure syntax: the keyword. A keyword is much like a symbol but instead of referencing another value it is merely a name or label. It's highly useful in data structures like maps to do lookups, you look up the keyword and return the value.
We can use the get
function to retrieve a value.
Here we've specified the keyword :a
and asked Clojure if it is inside our map. It's returned the value in the key/value pair, 1
.
If the key doesn't exist in the map then Clojure returns nil
.
The get
function can also take a default value to return instead of nil
, if the key doesn’t exist in that map.
We can also use the map itself as a function.
We can also use keywords as functions to look themselves up in a map.
To add a key/value pair to a map we use the assoc
function.
If a key isn't present then assoc
adds it. If the key is present then assoc
replaces the value.
To remove a key we use the dissoc
function.
We can also work with strings. Clojure lets you turn pretty much any value into a string using the str
function.
The str
function turns anything specified into a string. We can also use it concatenate strings.
Up until now we've run functions as stand-alone expressions, for example here's the inc
function which increments arguments passed to it:
This isn't overly practical, except to demonstrate how a function works. If we want do more with Clojure we need to be able to define our own functions. To do this Clojure provides a function called fn
. Let us construct our first function.
So what's going on here? We've used the fn
function to create a new function. The fn
function takes a vector as an argument. This vector contains any arguments being passed to our function. Then we specify the actual action our function is going to perform. In our case we're mimicking the behavior of the inc
function. The function will take the value of a
and add 1
to it.
If we run this code now nothing will happen because a
is currently unbound as we haven't defined a value for it. Let's run our function now.
Here we've evaluated our function and passed in an argument of 2
. This is assigned to our a
symbol and passed to the function. The function adds a
, now set to 2
, and 1
and returns the resulting value: 3
.
There's also a shorthand for writing functions that we'll see occasionally in Riemann configurations.
This shorthand function is the equivalent of (fn [x] (+ x 1))
and we can call it to see the result.
But we're still a step from a named function and we're missing an important piece, how do we define our own variables to hold values? Clojure has a function called def
that allows us to do this.
The def
function does two things:
#'user/smoker
returned as output of the def
function.smoker
is bound to a var with a value of the string "joker"
.When we evaluate a symbol pointing to a var it is replaced by the var's value. But because def
also creates a symbol we can refer to our var like that too.
Where did this user/
come from? It's a Clojure namespace. Namespaces are a way Clojure organizes code and program structure. In this case the REPL creates a namespace called user/
by default. Remember we learnt earlier that a symbol has a short name, for example smoker
that can be used locally to refer to it, and a full name. That full name, here user/smoker
, would be used to refer to this symbol from another namespace.
We'll talk more about namespaces and use them to organize our Riemann configuration in the HOWTO. If you'd like to read more about them then there is an excellent explanation at http://www.braveclojure.com/organization/.
We can also use the type
function to see the type of value the symbol references.
Here we can see that the value smoker
resolves to is a string.
Now with the combination of def
and fn
we can create our own named functions.
Firstly, we've defined a var (and symbol) called grow
. Inside that we've defined a function. Our function takes a single argument, number
, and passes that number to the *
function, the mathematical multiplication operator in Clojure, and multiplies it by 2
.
Let's call our function now.
Here we've called the grow
function and passed it a value of 10
. The grow
function multiplies that value and returns the result: 20
. Pretty awesome eh?
But the syntax is a little cumbersome. Thankfully Clojure offers a shortcut to creating a var and binding it to a function called defn
. Let's rewrite our function using this form.
That's a little neater and easier to read. Now how about we add a second argument? Let's make both the number to be multiplied and the multiplier arguments.
Let's call our grow
function again.
Ooops not enough arguments. Let's add the second argument.
We can also add a doc string to our function to help us articulate what it does.
We can access a function's doc string using the doc
function.
The doc
function tells us the full name of the function, the arguments it accepts, and returns the docstring.
That's the end of our crash course.
I recommend trying to get an understanding of the basics of Clojure to get the most out of Riemann. If you'd like to start to learn a bit about Clojure then Kyle Kingsbury's excellent Clojure from the ground up series is an great place to start. This section is very much an abbreviated crash-course of sections of that tutorial and I can't thank Kyle enough for writing it. A reading of this tutorial will add signicantly to the knowledge we've shared here. I recommend at least a solid reading of the first three posts in the series: