Judy ZemanQA Manager at Carrier Global Corp.
Renee KuijpersAdvisor Test management and coordination Concern Affairs Province of Noord-Holland at Provincie Noord-Holland
gagneetContinuous Delivery Lead at SAI Global
We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
"With autotesting, we have been able to eliminate duplication of test cases across those four areas. This has helped us knock down our number of test cases. Our test cases are also running more optimally. Therefore, it has very much helped in that sense, so we were able to eliminate a lot of test cases and get out of manual silos by running on autotesting, which is more efficient."
"It is a pretty easy tool to use as far as automated testing tools go."
"The biggest feature is the fact that it's codeless. It takes away the problem of finding people with the correct programming language, since there are multiple such languages. It saves time in introducing people to the solution because they don't need programming knowledge, they just need to be able to think logically. This makes it vastly usable by more people who are not even acquainted with IT at all."
"It does allow for good reusability. When it's designed properly and utilized properly, we can put things in a way that allows for reusability, meaning a lot of reuse of VA01, if they're very similar flows, to keep it simple."
"Certify integrates with other tools and it works very well with other machine testing applications."
"It's pretty seamless with SAP and Salesforce because they've built in the field definitions and all the things that you need. You literally turn it on and execute your script and it records it. It's very simple. Then you can go back and put in some of the other functions. For example, instead of hard-coding field selections, you put in a data table so you can run it multiple times or with multiple data. It was actually written to work very well with SAP."
"Certify's web UI testing abilities for testing of modern applications like SAP Fiori was good when we started and they developed it to be even better. We all know that web applications can change objects in DOM quite fast and it breaks tests. To counter it Certify has made object recognition more flexible and generic, so we don't have any troubles."
"The scripting methodology is easy to learn. It is easy to maintain because it is presented in a simple, narrative way. You don't need to know programming." "It has reduced our test maintenance time by more than 50 percent because we don't have to do manual test processes. We have saved over 150 man-hours monthly. It has increased our delivery times. We went from 200 man-hours down (three weeks work time frame) to approximately 40 man-hours (three days work time frame)."
"The widget's ease of use is the most valuable, which means it allows you or business people to record the automated test scripts. In most cases, it is really good because it is the business people who actually know how the system is being used. The simplicity of the design is valuable, where you can record your transactions, then create your automated scripts. You can automate it at the same time, and the automation features are cool."
"What Qualibrate makes very easy to do is to record a process flow. Within five minutes you have a clear document produced by Qualibrate. Instead of using Word, and copying and pasting pictures into it from printscreens, within five minutes what you have was easy to make and it's easy for users to use."
"The most valuable feature is that it's user-friendly."
"It has helped teams within our organization become more aware of the testing requirements in terms of risk and priority."
"The solution is script-less, so you don't need IT knowledge to use the solution in an operational way. This is the most valuable feature. It's also only one of two or three tools that can do good automation on SAP, and in my opinion, it's the best of those."
"The Model-Based Test Automation is the most valuable feature, where you can create reusable components. Even though we are using a scriptless automation tool, there still needs to be an understanding of how to create reusable components and how to keep refactoring and how to keep regression, the test scripts, at an okay level. We are coupling Tosca with some other risk-based testing tools, as well, but automation is primarily what we're using Tosca for, the scriptless, model-based technology which is driving automation for us."
"This solution is easy to use for everybody, including those who are not IT-educated."
"Tosca BI is important to make sure that our data integrity is in check and validated; to make sure our data is good. Our data is the number-one important driver for our company, so if that's not good, we have some big problems."
"We like the fact that it works across mobile, desktop, web, and APIs. Due to this, the solution has a broad range of applications."
"This tool is very easy to use and I think that anyone can come in, having no experience with it, and within four to six months be comfortable with it."
"The feature that I have found the most valuable is the recovery and cleanup process. Suppose there is a list of test cases and one test case has failed, it should not impact the other test cases. We can reuse the same test case. We can change the configuration of parameters and then use the test cases on the same thing. So, that's a useful thing. Otherwise, we have to use the cleanup process. Another useful feature is to have our own library files. We can create our objects in the libraries and reuse them. There is no need to create duplicate data for that. They have been giving some enhancements recently which means integration is also there. They've integrated with different software like Jenkins, Bamboo. So, we can also create pipeline points. They have recently given SAP and everything, which is very useful."
"It would be great if our business testers could develop their own automated test cases. With every release you do, you have to go back and touch your old test cases and bring them up to speed, or develop new test cases. In the beginning, that is a challenge because you have to have someone who is certified in the tool to help you develop these test cases."
"Performance on the web UI part, especially with some of the more comprehensive Fiori features, like the complex tables that are being used, could be improved. In those cases we have noticed a lot of execution-time increase with regards to the Certify solution."
"An area that I would like to see improved is how the permissions are applied. If you're applying permissions groups to a user, one of the options is to delete the group entirely and lose the entire permission group, rather than just deleting the permission from the user, which seems a little silly. In my opinion, that whole module of permissions is very confusing and lends itself to common errors."
"Pricing is a bit high and we would like to have the availability of a trail environment for beginners and training would be great to have and easier to expand and use by more and more consultants."
"Better automation capability would be helpful."
"With the codeless process automation across packaged applications, once in a while, if we get a weird application that's not widely used, it gets a little stickier. First, the software has to learn the fields, so you have to identify all the fields. Once you do that, as long as there isn’t any non-standard code in the application, then it works fine. But there's that one step that you have to do, a step you don't have to do with SAP and Salesforce, for example."
"When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us."
"Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify."
"What could be improved would be the intuitiveness of the reporting engine. It does have reporting, i.e., a dashboard, but it is preconfigured, predefined KPIs and datasets. That could be improved because the datasets don't have descriptions, so you really need to know what you're doing. Whereas, it would be great if it could have more descriptions and be easy to build your own KPIs."
"We had an issue with SAP when using PDF forms. That was something that was not supported by Qualibrate, but we solved that issue by choosing another solution. That was the only wish we had with Qualibrate."
"What I would really like to see is if you are running scripts in Qualibrate, and there is a defect, then you can have it automatically raise a defect in your own ticketing system."
"Not being able to mask test data in relation to testing data management, in my opinion, is also a limitation."
"The solution should work with the Linux platform. Right now, it only runs on Windows."
"There have been some setbacks because of upgrades. While Tosca has been around for a while, Tricentis has catered to smaller clients and I don't think they have done such a large, at-scale transition or transformation before or worked with a company like ours, which is doing an enterprise-wide transformation. When we go to their customer advisory-board meetings, upgrades have been an issue. They have been working a lot to make upgrades seamless."
"I would like to be able to manage different projects in one repository or have better data exchange between repositories."
"The main area where there is room for improvement is how they do upgrades. Going through this current upgrade, we were delayed a month because we are using a third-party tool. It's called Tosca Connect by Tasktop. When this latest upgrade broke that relationship between the two, it took Tricentis a month to come back with a workable solution... Their whole upgrade process needs to be better and cleaner, from an end-user standpoint."
"The solution needs to improve its simulation of mobile environments. Right now, that aspect is really lacking."
"I would like to see better integration with other testing tools."
"It would be of great help if they can fix the loading or performance issues. Sometimes, when I create the test case folder and test cases, it feels like it has loading or performance issues. When passing the objects, we can't sometimes find the exact element. We need to find out that exact location and just give the path for that, and then it works. In the pipeline, when creating Jenkins, we create the list execution for passing the execution list to the commander. I feel it is a bit late, by a fraction of seconds. I first thought it could be my mistake or a setting issue, but I worked on that, and it's not a mistake or a setting issue."
"There is a cost involved to doing it, but once you get over the initial cost, then you'll start reaping the benefits and seeing that testing is getting done more quickly and efficiently. We are still early on with it, but the expectation and what we're seeing is that we will start seeing some savings coming out on the back-end once we have this done."
"The price is in line with everyone else's in the market. They are not cheaper nor more expensive than anyone else who was in our RFP."
"The initial investment is probably a little high. It was a little hard for me to sell, but it was a one-shot deal and that's why it's so high. All we are doing now is paying annual maintenance, which we don't have to do if we don't want upgrades, but we do."
"Purchasing and licensing are okay. Go for the perpetual licenses. In that way, you own a license, then you can purchase maintenance and support on top of that, so you don't have to pay every year for it. Even if you don't want it a contract with Worksoft Certify in the future, you will have your own license of it. Then, if your usage is not that much, you can have one or two perpetual licenses. However, if you want to run your processes, you will need more licenses, e.g., using the run-only licenses. They are really cheap compared to the full licensing."
"Qualibrate is realistically priced. I can't compare it because I haven't looked at other tools, but I think it is good. What I like is you can simply add new users, if you want. It has a license model that comes with different types of users, which I think makes sense."
"Automated testing is not cheap. But other companies, for example, Panaya, required a minimum of 10 licenses. Qualibrate allowed us to start small, with three licenses, with a price that was competitive within the market."
"We signed a three-year contract and the pricing is in line with our expectations."
"Tricentis Tosca may be relatively on the higher side in terms of pricing, but their sales rep can give pretty decent deals when asked."
"We have around 200 [concurrent] licenses and the cost around $1.4 million a year."
"I am satisfied with the cost."
"My understanding is that it's an expensive product, although I don't know the specifics with regards to pricing."
"I would like to see better costing packs. There are several features but USD $11,000 for one license is expensive."
Tricentis Tosca specializes in enabling large enterprises to improve the quality of their applications by equipping them to optimize, manage, and automate their software testing.
Tosca's model-based approach to software test automation enables enterprises to achieve high automation rates while maximizing business risk coverage.
Tricentis Tosca also provides market-leading test case design and planning capabilities, test data provisioning, service virtualization, mobile testing, and more.
Tosca is fully compatible with other testing solutions and Application Lifecycle Management products. Tricentis also offers uniquely designed testing solutions for SAP, ServiceNow, Trax, Avaloq, and Oracle.
Qualibrate is ranked 13th in Test Automation Tools with 3 reviews while Tricentis Tosca is ranked 1st in Test Automation Tools with 13 reviews. Qualibrate is rated 9.0, while Tricentis Tosca is rated 8.6. The top reviewer of Qualibrate writes "Enables us to test much more frequently and provide functional maintenance feedback quickly". On the other hand, the top reviewer of Tricentis Tosca writes "Having a regression suite that we can run gives us a high level of confidence in the changes we are making". Qualibrate is most compared with , whereas Tricentis Tosca is most compared with Micro Focus UFT One, Selenium HQ, Tricentis qTest, SmartBear TestComplete and Katalon Studio. See our Qualibrate vs. Tricentis Tosca report.
We monitor all Test Automation Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.