SonarQube Review

Code coverage is useful, but the solution lacks mutation testing

How has it helped my organization?

We have literally thousands of rules and they are of medium effectiveness. The problem is that most people bypass the rules or turn them off. But even that is information to us. The fact that they have to turn the rules off is as much value to us as the rules themselves.

What is most valuable?

Code coverage of tests is their most valuable feature. Code coverage is of no value if it's high, but if it's a low number then that's of great value to me.

What needs improvement?

I would like to see something around mutation testing included in SonarQube. I'd like to see some mechanism of quality which has real meaning. The problem in metrics is that they're correlated. I'd like to see how they can add a feature to detect genuine quality, instead of numbers that people can game. The number can be manipulated. There are a few ways to do this, and mutation testing is one of them.

I would also be interested in more security scanning.

For how long have I used the solution?

Our company has been using this solution for over five years.

What do I think about the stability of the solution?

Stability has never been a problem. It would have to be unstable for me to experience a problem, and we haven't. So it's good.

What do I think about the scalability of the solution?

I don't really know how scalable this solution is, but I know we use it on thousands of projects, so it's probably good.

We have a pipeline. The pipeline currently runs 4000 teams through it, and all of them have SonarQube but usually with default rules. So that's pretty expensive. Now, we can't increase it because everything goes through it. We are evaluating what our best option is as we migrate our pipeline. We're migrating the pipeline and we're wondering what to do. If SonarQube did more security scanning, there's a good chance that we would use it more, in a different role. We're already using SonarQube everywhere, in some aspect.

Which solution did I use previously and why did I switch?

It was years ago. They probably evaluated other solutions. 

We're evaluating the use of different solutions at the moment, but I've just withdrawn from that task.

How was the initial setup?

In all the companies that I've worked with, nobody has ever had a problem with the initial setup. It takes time to set up. It's a big thing and you do it, but it's just a project.

What about the implementation team?

We used people in-house to deploy. We have about 100 people in our pipeline maintenance team. SonarQube has not led to any significant increase in that number. It's just absorbed as a part of the cost. There are no dedicated staff working on it.

What other advice do I have?

My advice is to focus on quality, not on tools. Work on the quality of your code and get a quality culture, but don't require the use of a tool. SonarQube is an okay tool. I'd suggest it as a default tool, but I wouldn't rave about it.

In all of my previous jobs, there has been somebody using SonarQube. They're usually very positive. I don't share that positiveness, but the reasons for that are that I don't believe you can have metrics of code quality based upon code analysis. I don't think it's possible for a computer to do it.

I don't rate any tool higher than a five or six, ever. JUnit is the only tool that gets a rating of ten. On a scale of one to ten, where ten is JUnit, I would rate SonarQube as about a five or a six.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment
Sign Up with Email