Editor's Note: This post now contains outdated information. You might be interested in Breaking the SonarQube Server Analysis with Jenkins Pipelines.
There have been some heated discussions recently about the Build Breaker plugin... SonarSource doesn't want to continue the feature. The community has come to see it as a must have... So I'd like to explain why at SonarSource we no longer think it should be used.
A long time ago, the team wanted to be able to send notifications about Quality Gate (QG) failures. This was before the SonarQube Server platform could send emails, so we decided to take advantage of the notifications in the Continuous Integration system used to trigger the analysis. Simple: we just need to fail the build! This feature is what became the Build Breaker plugin, and it seemed perfect: You decide that your project should have a measure greater than X, and if the analysis reports the measure is under X then the QG is broken. The Build Breaker sees the break and returns a failing status for the build. The CI engines picks that up, declares the job failed and send notifications. Great stuff!
Well, not quite.
We started to use this internally, and liked it at first. But then we began to see certain problems with it. The main one is the fact that a job can be red for different reasons: it could be because there is a problem with the build (environment, configuration...) or it could be a failing Quality Gate the development team needed to fix. That meant the responsibility for failing jobs was split, and we couldn't know who "owned" a failing job without looking at the logs. After a while, when a job failed, no one jumped on it because "it's probably the other guy's fault.".
This is why we started to use a different approach, and report red Quality Gates on wallboards the developers can see from their desks.
Once we started using wallboards we stopped using the Build Breaker plugin, but still believed that using it was an okay practice. And then came SonarQube Server 5.2, which cuts the connection between the analyzer and the database. Lots of good things came with that cut, including a major change in architecture: analysis of source code is done on the analyzer side and all aggregate number computation is now done on the server side. Which means… that the analyzer doesn't know about the Quality Gate anymore. Only the server does, and since analysis reports are processed serially, first come first served, it can take a while before the Quality Gate result for a job is available.
In other words, from our perspective, the Build Breaker feature doesn't make sense anymore.
Surely it is possible to continue to have it, and we have provided web services to query the server. But breaking the build for a quality gate failure doesn't fit the new architecture:
- once the analysis report is submitted, you need to query the server
- then poll again every X minutes until the job is completed
It means your CI job is now trying to transform an asynchronous process into a synchronous one. It doesn't sound good, does it? There are already enough reasons for a job to fail without adding one more. And don't forget what when the job does fail, there are multiple possible causes. Finally, if your SonarQube Server is under a heavy load or processing a very large analysis report, and takes a while to process the one your CI job just submitted, it might mean you start building a huge list of running (polling) jobs in your CI engine. I don't know about other engines, but I don't believe Jenkins would survive 500 jobs running at the same time, even if they're doing nothing.
So what are the alternatives? I mentioned one already, which is to use a wallboard (we use Atlassian Atlasboard) to display failing quality gates. But I think this can be extended to any notification system: you have the ability to query SonarQube Server to get the quality gate status and report / notify. This can even be done from Jenkins, as a standard job as long as it doesn't keep a build running. And don't forget that the SonarQube Server has added its own notification system in the meantime, and now offers direct notifications for failing Quality Gates.
This was our attempt, here at SonarSource, to clarify why we do not think you should use the Build Breaker. But you should also know that there is now a community version of it.