Improvements to My Organization
Most obvious: Having builds and test tasks triggered on commit helps not to break the product.
From my own experience:
We significantly reduced build times of large projects (more than 80k lines of Scala code) using build time on Jenkins as a time sample. It reduced the developer write-test-commit cycle time, and increased productivity.
Integration with GitLab reduced time used for code reviews. Jenkins posted build state and code quality reports into the merge request.
Simplified build scripts: Organisation started to integrate Pipelines as a part of a build, and built a library of common functions. It simplified and made our build scripts more readable.
Automation of chores like deployment, frequent manual tasks (like running scripts on test and production systems) reduced the time used and the number of errors made by engineers, freeing them to do meaningful work instead.
- Configuration management: It is so easy to configure a Jenkins instance. Migrate configuration to a new environment just by copying XML files and setting up new nodes.
- Programmable pipelines: In recent versions, Jenkins has a Groovy Sandbox where build scripts execute. I have never seen that powerful a tool in CI solutions yet. On other platforms you can use shell scripts, but Jenkins' solution is much better in terms of readability and portability. And given that you can create your own libraries for the Jenkins Pipelines, it becomes much more powerful and DRYer, simplifying work of DevOps and build engineers.
- Brand new Blue Ocean UI: Jenkins used to have a pretty outdated UI. Now, you can use the Blue Ocean plugin to make it nice, clean, and modern-looking. Also, it has very good integration with Pipelines (basically it is built to use Pipelines).
Room for Improvement
UI: Jenkins relies on the old version of interface for configuration management.
Developer documentation for plugins, plugin development, integrations: Sometimes it’s tricky to do pretty obvious things.
Use of Solution
Three to five years.
Rarely. I can remember only one time we lost our build info after upgrading Jenkins, somewhere around the 1.6xx versions.
Sometimes you have Jenkins restarting because of OOM errors.
No scalability issues. I used to have up to five worker nodes with one master, and it did not produce any slowdowns. I have never had bigger deployments.
Customer Service and Technical Support
I have never used technical support directly. The community, documentation, issue tracker, are pretty good, though not ideal.
TeamCity - It’s pretty limited in build runners, mostly targeting enterprise tech (Java, MS Stack, mobile apps) and the price is quite high.
Buildkite - An okay solution, but builds are shell scripts in general. It’s hard to maintain them. Also, I had weird issues with SCM integrations and Github.
GitLab CI - It’s coupled with GitLab too tightly. It’s pretty difficult to configure. It’s slow and requires a lot of resources to run.
As for me, I just start to use it. It runs builds, unless you need something more complicated.
Setup of commonly used plugins is very straightforward, but it can be more difficult to get it running with exotic technologies. Still, it’s much easier than with other solutions.
Pricing, Setup Cost and Licensing
I used the free OSS version all the time. It was enough for all my needs.
Other Solutions Considered
I was always choosing between Jenkins, TeamCity, Buildkite, and Bamboo. Most recently, hosted solutions like Codeship and Travis CI added to the list.
For business needs, Jenkins is the most relevant choice because it can be self-hosted, the price is good, it’s robust, and requires almost no effort for maintenance.
For open source projects, Travis CI is standard.
I like it very much, and I'm actively promoting it on my network.
Take your time to get used to the management flows of the application and builds. Jenkins is very powerful when you know how to cook it.
Disclosure: I am a real user, and this review is based on my own experience and opinions.