Concurrency in Hudson

How to organize and avoid concurrency problems when implementing complex projects in Hudson?

When your Hudson projects are split in multiple jobs that are chained to each other, how do you avoid concurrency issues in Hudson?

If you have a lengthy process, it will be a good practice to split it in multiple tasks, having each calling the next one, like on the (rather simplified) left image.

Problem number one: How to avoid Clean and Build (for instance) to run at the same time?

As (almost) always, “you have a plugin for that” in Hudson. This time is Locks and Latches, which will let you create a lock, shared by all the jobs you want. Only the job that has the lock will be executed.

So far, so good, but, where is the problem?

As you may guess, Clean could be called multiple times, while Build, Publish, or the same Clean are executed. How could this happen? If you trigger Clean with a SCM (like SVN) this could be an expected situation, as you have no way to prioritize the jobs in Hudson. This is explained in bug 833.

The following Groovy script checks if there is any other job waiting for an executor or being executed. This effectively prevents Clean to destroy the others jobs data. (Note: you will need the Groovy plugin and then create a “system Groovy script”.)

 1//Jobs currently building or in the queue
 2jobs = hudson.model.Hudson.instance.items.findAll{job -> job.isBuilding()} + hudson.model.Hudson.instance.items.findAll{job -> job.isInQueue()}
 3 
 4//Keep only the names from the collected job objects
 5jobs = jobs.collect{x -> x.name}
 6 
 7//get current job name
 8currentJob = Thread.currentThread().executable.toString().split()[0]
 9 
10//cut current job from the jobs list
11jobs -= currentJob
12 
13//decide if there is any running (or waiting) job
14res = true
15if (jobs.size()  > 0)
16{
17    res = false
18    println "The following jobs are being executed or waiting for an executor"
19    jobs.each{println it}
20}
21else
22{
23    println "Concurrency OK"
24}
25 
26return res

In case you have other jobs in your Hudson instance that have nothing to do with our “Clean -> Build -> Publish” project, you can restrict the concurrency checking to our project. In order to do that we will need to implement this small modification in our Groovy script:

 1//Jobs currently building or in the queue
 2jobs = hudson.model.Hudson.instance.items.findAll{job -> job.isBuilding()} + hudson.model.Hudson.instance.items.findAll{job -> job.isInQueue()}
 3 
 4//Keep only the names from the collected job objects
 5jobs = jobs.collect{x -> x.name}
 6 
 7//list of jobs that belong to the same project
 8project_jobs = ["Compile", "Publish"]
 9 
10//cut current job from the jobs list
11jobs = project_jobs.intersect(jobs)
12 
13//decide if there is any running (or waiting) job
14res = true
15if (jobs.size()  > 0)
16{
17    res = false
18    println "The following jobs are being executed or waiting for an executor"
19    jobs.each{println it}
20}
21else
22{
23    println "Concurrency OK"
24}
25 
26return res