What is our primary use case?
Our most important use case was a carve out. We acquired an organization, so we needed to to carve out their SAP environment.
We use it for regression testing. We set up an automated regression test set because we have to test everything to make sure that everything still works.
It is SaaS.
How has it helped my organization?
It allows us to change more quickly because we built that automated test set to do regression testing, which takes away the manual effort of regression testing. So, if we change things, then we don't have to worry about the time that it takes for that regression testing and can do that overnight. I think it has changed the way that our organization looks at the risk of changes in an SAP environment.
The solution has enabled business users to create automated test scripts without requiring coding knowledge. Initially, it doesn't speed it up significantly because the business users still need to do the first test. For example, you do your test, then you would typically hand that over to the business units through the testing. So, we still do that, but then they record it and it becomes part of the progression tests. From my end-to-end scenario testing, I don't have to do that much anymore. For regression testing, that has pretty much been all taken out of the equation.
Qualibrate has enabled us to do more frequent testing, without putting too much pressure on timelines and resources. Because of the way that we have the intervention regression test setup, we hardly have any production incidents anymore.
What is most valuable?
The widget's ease of use is the most valuable, which means it allows you or business people to record the automated test scripts. In most cases, it is really good because it is the business people who actually know how the system is being used. The simplicity of the design is valuable, where you can record your transactions, then create your automated scripts. You can automate it at the same time, and the automation features are cool.
It definitely has a low-code approach that has enabled users to get onboard quickly and use it in their daily test creation, running, and maintenance. If we change things in SAP, we still do the Unix test. Once that is signed off, we say, "Okay, it does what it should do, and we record it in Qualibrate." Then, we use it for end-to-end testing, so it becomes part of the regression test set.
What needs improvement?
What could be improved would be the intuitiveness of the reporting engine. It does have reporting, i.e., a dashboard, but it is preconfigured, predefined KPIs and datasets. That could be improved because the datasets don't have descriptions, so you really need to know what you're doing. Whereas, it would be great if it could have more descriptions and be easy to build your own KPIs.
For how long have I used the solution?
I have been using it for two and a half years. The last time that I used it was about two months ago.
What do I think about the stability of the solution?
The stability is reasonable. We do have the odd incident, but it is pretty stable overall.
What do I think about the scalability of the solution?
It is easy to scale. If we want new users, then we can do this on the fly.
We have about eight people at the moment using it. Most are business users or functional consultants in the IT organization.
Qualibrate is being used very frequently. At this moment, we don't have any plans for new systems.
How are customer service and technical support?
The technical support is very good. If there are any issues, then they are always there to help out. They have good response times within a day, but typically within half a day. They are knowledgeable and able to help.
There is a lot of information online in the help files, but also the team themselves are always willing to answer questions and help out with explaining how things work. I think the overall support, not just technical support, is really good.
Which solution did I use previously and why did I switch?
We were using Solution Manager for test management, but we weren't using any test automation.
I heard about Qualibrate from a guy that I know who works at SAP.
At that moment, we are starting to get an idea of test automation. We have been in contact with Qualibrate, and it is really good.
How was the initial setup?
The initial setup was pretty straightforward and easy. We just had to make sure that a lot of scripting was enabled and the firewall was set up, then the Qualibrate team took care of it. It all worked.
The deployment took a couple of days. This was more because we had to organize it with the infrastructure teams. The communication with the team is what takes the time because the actual setup is really easy. We have to log a ticket and wait.
What about the implementation team?
Someone from the infrastructure team and an SAP consultant needed to do about 10 minutes of work.
As many people involved in a change, those are typically who maintain it. It is not a huge maintenance effort. Because if you change something in your system, then you should change the test as well.
What was our ROI?
If you look at it from an overall testing perspective, the solution saves a lot of time and documentation. If you look at how individual change flows to the software development life cycle, it might be a 20 percent savings, but still significant. If you look at it from an overall perspective, where you look at the time that we typically would spend on testing, then it is probably a bit more like 60 to 70 percent time savings.
It has a more reliable time to market and delivery of software. You get significantly less defects.
Our first use case with Qualibrate couldn't have been done without Qualibrate because there was limited time for the project. We could only do it because we automated it. We were only able to create the end-to-end automated regression test set because we could leverage our business users. From an ROI perspective, we saved so much time in testing from an IT perspective that it definitely pays itself back.
What's my experience with pricing, setup cost, and licensing?
Qualibrate is realistically priced. I can't compare it because I haven't looked at other tools, but I think it is good. What I like is you can simply add new users, if you want. It has a license model that comes with different types of users, which I think makes sense.
Since it is SaaS, the licensing is all based on subscriptions.
What other advice do I have?
Just try it. The only advice would be to think differently about testing and how you organize testing. It comes with an easy setup to use that business users can do.
When you look at test automation and the progression testing that is ready, testing is not really an issue anymore. You can just test everything all the time, because time is not an issue. It puts a different perspective on software development. However, that is not specifically for Qualibrate, but for test automation in general. With Qualibrate, it is just so easy.
Biggest lesson learnt: It can be easy to automate testing. I never expected that. Typically, automating was almost always very technical. Qualibrate made it easy to automate testing.
One of the things that we were really looking for is test planning, but that is a new feature that is now there. So, that is good.
Its Test Planning & Execution Scheduling features are definitely something that we want to look into using in the future.
I would rate it an eight (out of 10).
Which version of this solution are you currently using?