Useful Jenkins plugins – Parameterized Trigger

This will be the first in a series of posts about Jenkins plugins that I like and use heavily. A big part of the value in Jenkins is the great collection of plugins that exist for it, though you do have to watch out for ones that are buggy or have been abandoned.

The Parameterized Trigger plugin lets you trigger other builds and pass parameters to them which can be used in their configurations. This assumes that the downstream jobs are configured with parameters. You can do this as a build step or a post-build step. In our case, we use it in the post-build process of all our build jobs, to optionally deploy the built code to one of several test environments.

Our build jobs all have parameters of git branch name (defaults to master) and deploy environment (defaults to none). If you choose an environment when you build, it causes the job to create a package if the build (compilation and tests) is successful. During the package step, we write (echo from a shell) a Java properties file, which is just a text file of key/value pairs, one per line, in the form ‘key=value.’ We store the keys package_name, version_number, and deploy_environment in that. In the post-build phase, we add a ‘Trigger parameterized build on other projects’ step. It calls the ‘util_deploy’ job, only when the current build is stable, and sends the build properties file that I created earlier.

 

Screen shot 2013-05-07 at 2.57.24 PM

The util_deploy job has parameters of package_name, version_number and deploy_environment. It runs a shell script which takes those variables and kicks off a deploy as requested. The console output of the parent job will include a link to the deploy job, so users can quickly jump there and look at the log if something went wrong.

Parameterized Trigger is, by my informal estimate, one of the most widely-used Jenkins plugins. It was downloaded over 10,000 times in April of this year alone. It gives you a much more powerful way to chain Jenkins jobs together than the built-in ‘Build other projects’ step, because once you can actually pass data around between them you can do almost anything you want. You don’t have to create a ‘properties’ file, either. You can just choose to pass all the parameters/values your parent job has, the Subversion revision or Git commit that was used in the build, or arbitrary parameters that you define.

[follow_me]

Why I love Jenkins

This will be the first of many posts about Jenkins, I’m sure. I’ve been using it for about four years, and it is an invaluable part of my team’s workflow. More than just a build system, I think of Jenkins as a really flexible tool for performing all kinds of commanded jobs. We use it as a frontend for doing deploys, launching virtualized test environments, and managing integration test infrastructure. Herewith, a few of the reasons why I love Jenkins.

It’s easy to deploy
Getting started with Jenkins involves downloading the .war file from http://jenkins-ci.org and running ‘java -jar jenkins.war’ on the command line. In a minute or so you have a fully-functioning build system that you can access via HTTP on port 8080.

It’s easy to scale
Jenkins has a concept of slave nodes that can be used just for distributing the build workload, or for performing specialized tasks on dedicated systems. Setting one up is trivial; in the easiest scenario, make sure that the user Jenkins is running as has passwordless ssh access to the slave node, then add it through the Jenkins UI. Jenkins will ssh in, push over its own JRE if needed, then push over the Jenkins slave .war file, start it up and connect to it. Presto, you have a slave build node in about 15 seconds.

By default, any job will run on any available slave. The alternate configuration, which we use, is to reserve slaves for “tied jobs.” In this way you can target specific builds to specific slaves or groups of slaves. We need to build on a matrix of Debian Lenny, Ubuntu Oneiric, and Ubuntu Precise, with both 32 and 64 bit for Lenny. We have four instances in total, one master and three slaves. Each slave is tagged for its distro and architecture combination, and then all of our builds are “matrix builds” in Jenkins parlance. This means that some basic job setup tasks happen on the master, and then the meat of the build happens in parallel on each slave that is needed. The slaves are specified by tag in the build, so that in the future if we outgrow any particular one, we can spin up additional identical slaves, give them the same tag, and Jenkins will automatically use them where appropriate.

It’s highly extensible
Jenkins’s biggest strength, in my opinion, is its fantastic collection of plugins. The base installation can access a few popular SCMs out of the box, build using Maven or just an arbitrary shell script, send emails about build failures, and do most other common build tasks. But eventually you start to find yourself thinking, “I wish I could do THIS with Jenkins…” That’s when you go looking for a plugin, and you’re almost always rewarded.

Some of the plugins we use most heavily are:

  • Parameterized Trigger, to send arguments to downstream builds when a primary build succeeds or fails.
  • OpenID, to allow us to authenticate users using our Google Apps for Business credentials.
  • Environment Script, to set environment variables and propagate them to slaves during each build.
  • Conditional BuildStep, to trigger actions within the build job based on variables; a rudimentary if/then construct, basically.

There are lots of others that we use for little nice-to-have things as well.

The developer community is awesome
Jenkins is open source, and the developer community that surrounds it is one of the best I’ve ever interacted with. The users mailing list is really active, and by virtue of the fact that it’s on Google Groups, mining it for tips or solutions to a problem is easy. But even more awesome than that is the #jenkins IRC channel on Freenode. Not only do many of the core committers to the project, such as kohsuke (its creator), abayer, and rpetti hang out there, but they and many others are always generous with their time. Whether you’re a new user with a simple question, or you’re looking for a plugin that might help you accomplish something, or you have a possible bug to report, you’ll probably get a quick and helpful response.

In one instance I found a bug in a plugin I wanted to use which completely blocked me. I went in #jenkins and brought it up, and it turned out that abayer was the maintainer of it. Within a few minutes he had fixed the bug and sent me a custom build of the plugin to test. The fix even worked the first time! In another case, I went to the channel wondering if it would be possible to get the OpenID plugin to support authenticating with the Google Apps for Business API. This would let me configure Jenkins to allow access to anyone at work who was already logged in to their corporate Gmail account. Kohsuke turned out to be the maintainer of that plugin, and thought it would be a useful feature, so he coded it up, sent me a build to test, and it worked!

Those are just two of the many, many examples I have of people in the community helping me out, and in a very immediate way. More recently, I’ve been encouraged to get involved myself. I’ve had a pull request accepted to Jenkins core, I’ve tried to be an active support resource on IRC, and I convinced my employer, BrightRoll, to sponsor the 2013 Palo Alto Jenkins User Conference!

Take a great tool as your base, make it easy to use and easy to enhance, and foster a vibrant community around it; that’s the way to run a successful open source project!

[follow_me]