TestRail by Gurock Review

Easy to write new test cases and test plans and it is flexible


What is our primary use case?

We use the on-premises deployment model of this solution. Our primary use case is for test management. 

How has it helped my organization?

It was very easy to create a quick test plan, despite the challenges we face, like a lack of documentation and the lesser knowledge we have for testing. We were testing a new upgrade for an app and by just exploring and running things through our mobile app we found that many features that were existing already in the production environment in the production app were not documented properly. All we had to do was create an exploratory session with the feature that we have to test and then we created a test plan with a different composition like Android and the languages that were working. 

It took me less than an hour to figure out the test plan and the test execution was more managed in the sense that we did have a test case at this instance to be executed. The stated test results are not easily monitored when it's reported. Before we used to have a more complex system and we used to manage the test cases in Excel format, so we saved a lot of time with this. 

In the next release, I would like to see dynamic reporting. 

What is most valuable?

The most valuable features are the flexibility, ease of use for writing new test cases, the test plans, and the composition.

What needs improvement?

It's not easy to create a custom report. It's not straightforward. A good improvement would be if there was a way to report and create a custom report without using a plugin or scripting language.

For how long have I used the solution?

I have been using TestRail for around three months.

What do I think about the stability of the solution?

It's very stable. 

What do I think about the scalability of the solution?

We use it on a medium scale. We have 100 users licensed but have around less than 50 active users. I haven't seen any performance issues. I haven't seen any issues with scalability. It's very stable and nice.

I handle the maintenance. I have handled it now for the past three months and nothing happened. I haven't had any need for maintenance yet. One team member can handle the whole thing.

In terms of our plans to increase our usage, in another six months, we were planning to ramp up the team, but I don't think it will have an effect on this solution. 

If you previously used a different solution, which one did you use and why did you switch?

We previously used HP Quality Center for the last eight years. We switched to TestRail because it is more flexible and cost-effective. The price is very competitive, and even though HP is the ultimate solution, for a starter company or like in our case, the cost was a major factor. 

How was the initial setup?

The initial setup was kind of complex, but I have previous experience that helped. It was not really straightforward. I heard that with the new version, it's in a package and it's not so complex anymore. 

The deployment took around two days. I did the deployment alone and wasn't doing it full time. I did the PoC first and then purchased the license. 

What's my experience with pricing, setup cost, and licensing?

We purchased a license for two years. The license is on an annual basis. We got a nice discount for the second year, around 10%, which is nice. It's priced annually and per user. It's a fee-based license. One thing that Gurock does not offer is a perpetual license. There is no option to buy the tool without having to make a recurring cost. It is fine for us. The annual license is not an issue.

Which other solutions did I evaluate?

We also looked at TestLink in open source, and we looked at ALM. The functionality and the usage, and usability of TestRail is much better than any other product. It's not dependent on any ActiveX or any library. The client can use it out of the box, they just need to type in the URL and start using it. Also, the training videos are very nice. It covers everything, that would be all you have to do is just watch the videos for a few minutes and you can really just learn very quickly and easily. Other tools, like ALM, had issues with the training. 

What other advice do I have?

I would advise someone considering this solution to consider is that TestRail does not have a defect management policy. They would have to use another tool for defect management. In our case, we use JIRA, which is at an additional cost, but it's not that much. The defect management is supposed to be in a different domain because it does not fall under test management. Test case management is totally different than defect management. I believe Gurock did a great job just by separating the test management from the defect management. 

Good advice would be to start with a cloud for some time. I would rate TestRail a nine out of ten. 

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment
Guest
Sign Up with Email