Qualibrate Review

Enables us to test much more frequently and provide functional maintenance feedback quickly

What is our primary use case?

With Qualibrate, the accent for us is on SAP, because that's our biggest application and a critical application for our company. Web was also an opportunity.

How has it helped my organization?

If you do a SAP regression test and you have to do it via manual testing, you need to organize a lot of people. The difference with automation testing is I can do it every day. What we have with SAP are the HR Support Packages, the service packages. We do them every half year, but if I want to check it, I can do it every day. That is a great advantage when using Qualibrate for automated testing.

Qualibrate has enabled our business users to create automated test scripts without requiring coding knowledge. That was a main part of the decision for the business, that it is easy to use. You don't have to be educated and have experience in Java or HTML. They have to know what kinds of flows they want. I have explained that they need to make their flows in building blocks. We are now planning to bring the Qualibrate solution to the business users.

Qualibrate enables us to do more frequent testing, without putting too much pressure on timelines and resources. I'm doing it every day and I'm showing management the differences between that and once a month or once every half a year. We now have continuous testing and that's what we also want to do with all the applications in the cloud. We need to be able to test development or when a change is planned for an application. We need automated testing in the continuous delivery pipeline.

The more frequent testing has affected the number of production incidents and software quality a lot. If you do testing every day, after a weekend of development changes in some formulas we use for the desktop application, running the scripts automatically means they see the changes in what they have deployed. That is a big advantage. I can immediately see, in the morning, what they have done over the weekend in deploying applications. I report things to the project leaders and the people responsible for functional maintenance of that application. They are surprised: "How do you know that so fast?" It's because by running Qualibrate we get the answer.

With Qualibrate, when we looked at the time and resources we are saving it was €1.5 million per year, mostly because of the move away from manual testing. Maybe that's a small amount if you compare it with my last employer, when I worked for ING Bank, but we are a small company. We only have 1,800 employees. Using Qualibrate we are saving a lot of time for the regular users because people on the business side are not testers. They have to take care of daily business as a priority. I am able to take work from them and that makes it easier for them to do their regular work.

Also, where it might have taken a day to create a test document, now, if you do work with a flow, it takes you four or five minutes and you have your automated test flow and your documentation for your process. That makes Qualibrate a fine application to work with.

What is most valuable?

What Qualibrate makes very easy to do is to record a process flow. Within five minutes you have a clear document produced by Qualibrate. Instead of using Word, and copying and pasting pictures into it from printscreens, within five minutes what you have was easy to make and it's easy for users to use.

What needs improvement?

We had an issue with SAP when using PDF forms. That was something that was not supported by Qualibrate, but we solved that issue by choosing another solution. That was the only wish we had with Qualibrate. All the other functionality provided is all the functionality we need.

For how long have I used the solution?

We have been using Qualibrate for three years.

What do I think about the stability of the solution?

It's very stable. I'm working in Qualibrate almost every day and sometimes on weekends. I haven't experienced any issues. It's running smoothly.

What do I think about the scalability of the solution?

Our use of the solution is still growing. We started using it on sub-solutions but nowadays we are getting a lot of requests from the business to do dev projects or help them with their applications. Instead of their manual testing they would like automated testing. It's easy for me, as an admin user, to create a new project and guide them with my knowledge in creating flows and scenarios. The number of projects and business partners requesting use of it is growing in our company.

We now have 301 flows using Qualibrate and in a year we did about a half million test steps.

We have five licenses. I use it as a test engineer. On the business side there are functional maintainers, and some of the business users, or the people around them, are guiding the process flow by first creating the test steps we have to record. So we now have 10 people directly involved with Qualibrate.

How are customer service and technical support?

We have weekly contact with Qualibrate. If there are some wishes on our side, we can mention them there and they will be put into review for deployment. Our last request was that when you record a flow, that it would automatically create screenshots. Screenshots are much better than they were before for the documentation and for use as a user manual. That has improved now.

I'm very happy with the service level they give us, especially from Alan, Marina, and Martina. They are always available for me if I'm calling, or by email or by Teams. Their technical support is a nine out of 10. 

Which solution did I use previously and why did I switch?

We previously used a manual testing system that I had adopted from ING Bank. Phase one was to collect the information and phase two was to create your manual testing.

We started using Qualibrate in 2017 with a pilot version because our organizations had a big need for automating our process flows. We would usually do it manually and part of it is still manual. But our organizations had a question: "Can we go for automated testing?"

How was the initial setup?

I was the person who introduced Qualibrate as a choice in our company. The parent company is IOVIO, and I worked with IOVIO when I was at ING Bank.

The initial setup was very easy. It is a SaaS application. It was easy to install and easy to use. There was no problem installing it in our environment. Within an hour it was operational.

The only thing we needed to do to use Qualibrate was more due to our technical maintenance requirements. We weren't allowed to have local admin rights but because Qualibrate is using an updater. The updater has to be clear of admin rights because otherwise you can't install new features. That made for a little discussion, but happily that's over now. But there were no problems installing or setting up the whole thing. However, I have to mention that I'm experienced in test engineering, so that also enabled us to do it very quickly and easily.

What about the implementation team?

We deployed it with help from Qualibrate consultants because it was our first time using Qualibrate. They were onsite and helped us to install it on each of the laptops, easily. We had three licenses to start.

The  consultants were great. Qualibrate is a young and dynamic team, and they were willing and able to help us every time I called.

What's my experience with pricing, setup cost, and licensing?

Automated testing is not cheap. But other companies, for example, Panaya, required a minimum of 10 licenses. Qualibrate allowed us to start small, with three licenses, with a price that was competitive within the market.

Which other solutions did I evaluate?

We invited a lot of companies to do a presentation. In the end, our stakeholders were convinced that Qualibrate was the best solution for us. We looked at Tosca, Solution Manager from SAP, Panaya, and Testbirds. Because we are government organizations, and we need to be very transparent in choosing a company to provide us with automated testing, we had to invite several companies.

In the past I was a test engineer with Micro Focus LoadRunner and Solution Manager. Qualibrate is a very simple solution. You don't have to be a person who has skills in coding or creating or something like that. It makes for a very easy tool to use in Agile sprints. The difference with LoadRunner was that you had to be a coding specialist, otherwise you could not run the application. That makes a great difference. People without those skills can use Qualibrate as a tool for flow documentations, and it's easy to maintain while providing test automation as well.

While you don't need to have coding skills, if you have the skills it makes some things easier to use. There are some options within the data, in the advanced options, that you can use. There are building blocks created by Qualibrate and some code that you can use. For example, for a date field you use code they have already made for you. You can specify today plus seven days so that your script is always running smoothly.

What other advice do I have?

I would advise using Qualibrate as a testing tool in a DevOps organization. It's a fine product. Other solutions are also okay. Tosca is also a great solution, but in the end, Qualibrate makes us better.

We don't use Qualibrate's Test Planning and test Execution Scheduling features yet, but we plan to do so this year. We are now growing from three to five licenses and management wants to know what the options are for test management and test planning so that we can schedule things for when the business wants to run the scripts. It can be every day or once a week. That's the way we plan to use Test Planning.

We don't use defects at the moment because we are now organizing API management to get JIRA onboard. Our process is still growing in production.

We are totally satisfied with what we now have in our possession with Qualibrate. We are very happy with the solution as it stands now.

**Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
More Qualibrate reviews from users
...who compared it with Tricentis Tosca
Learn what your peers think about Qualibrate. Get advice and tips from experienced pros sharing their opinions. Updated: July 2021.
523,230 professionals have used our research since 2012.
Add a Comment
ITCS user