Jens NeuhausEnterprise Architect SAP Solutions at a computer software company
Anonymous UserSoftware QA Test Engineer at a tech services company
Malli Bharathi MIT Service Delivery Manager Testing and RPA at a transportation company
We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
"The biggest feature is the fact that it's codeless. It takes away the problem of finding people with the correct programming language, since there are multiple such languages. It saves time in introducing people to the solution because they don't need programming knowledge, they just need to be able to think logically. This makes it vastly usable by more people who are not even acquainted with IT at all."
"It does allow for good reusability. When it's designed properly and utilized properly, we can put things in a way that allows for reusability, meaning a lot of reuse of VA01, if they're very similar flows, to keep it simple."
"Certify integrates with other tools and it works very well with other machine testing applications."
"It's pretty seamless with SAP and Salesforce because they've built in the field definitions and all the things that you need. You literally turn it on and execute your script and it records it. It's very simple. Then you can go back and put in some of the other functions. For example, instead of hard-coding field selections, you put in a data table so you can run it multiple times or with multiple data. It was actually written to work very well with SAP."
"Certify's web UI testing abilities for testing of modern applications like SAP Fiori was good when we started and they developed it to be even better. We all know that web applications can change objects in DOM quite fast and it breaks tests. To counter it Certify has made object recognition more flexible and generic, so we don't have any troubles."
"The scripting methodology is easy to learn. It is easy to maintain because it is presented in a simple, narrative way. You don't need to know programming." "It has reduced our test maintenance time by more than 50 percent because we don't have to do manual test processes. We have saved over 150 man-hours monthly. It has increased our delivery times. We went from 200 man-hours down (three weeks work time frame) to approximately 40 man-hours (three days work time frame)."
"The most valuable feature is the ability to automate quickly and to maintain and update scripts."
"The decoupling of the test scripts from the data and the application is a nice feature. When you are creating test scripts, for example, for a web application, you have to learn about Worksoft and how the controls of a screen can be interpreted by Worksoft. For that purpose, you create so-called maps. These maps are loosely coupled to your scripts, which means if the application is changed, the control will be changed from an identifier. You don't need to rework the entire script. You only need to do these adjustments in the map, and then you can automatically reuse the scripts. So, it is really a smart move to have the decoupling of scripts, maps, and data."
"The functionality of the data is very good. You can upload and disperse data using the solution."
"This solution is very user-friendly to learn and implement, and less technical knowledge is required to handle automation."
"The basic framework and project organization is very good."
"This is a product that is well ahead of its immediate competition in features and functionality."
"The most important feature is the Jenkins integration; it is pretty straight forward and allows us to run nightly builds."
"It is easy for a non-technical person to use and adopt."
"The most valuable feature is that the browser and the API testing is available in a single tool, and I don't have to look at other options."
"One of the features that I like is Object Recognition. It worked very well, and it allowed me to create a dynamic expert based on my requirements."
"The ability to develop scripts in Visual Studio, Visual Studio integration, is the most valuable feature."
"The feature I like most is the ease of reporting."
"The statistics that are available are very good."
"The scalability of the solution is quite good. You can easily expand the product if you need to."
"The major thing it has helped with is to reduce the workload on testing activities."
"Scripting is the most valuable. We are able to record and then go in and modify the script that it creates. It has a lot of generative scripts."
"Performance on the web UI part, especially with some of the more comprehensive Fiori features, like the complex tables that are being used, could be improved. In those cases we have noticed a lot of execution-time increase with regards to the Certify solution."
"An area that I would like to see improved is how the permissions are applied. If you're applying permissions groups to a user, one of the options is to delete the group entirely and lose the entire permission group, rather than just deleting the permission from the user, which seems a little silly. In my opinion, that whole module of permissions is very confusing and lends itself to common errors."
"Pricing is a bit high and we would like to have the availability of a trail environment for beginners and training would be great to have and easier to expand and use by more and more consultants."
"Better automation capability would be helpful."
"With the codeless process automation across packaged applications, once in a while, if we get a weird application that's not widely used, it gets a little stickier. First, the software has to learn the fields, so you have to identify all the fields. Once you do that, as long as there isn’t any non-standard code in the application, then it works fine. But there's that one step that you have to do, a step you don't have to do with SAP and Salesforce, for example."
"When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us."
"Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify."
"We would like this to be able to be used outside of SAP applications, as it would be good for other types of products."
"Katalon doesn't support testing of hybrid applications. It's a limitation."
"My recently-updated Katalon studio version hangs a lot and is not a stable version."
"We have had some issues when it comes to stability, which is something that needs to be improved."
"They need to work on documentation to make it more centralized and easier to find what you are looking for."
"It would be helpful to have an automatic save feature."
"It is difficult to identify elements on websites that use Angular or ReactJS because they don't have an option to view source code."
"There is not much support offered for the free version, which is something that could be improved."
"Now we are having difficulties in using it and had to limit our automation experts to the features that Katalon allows us to use for free. It was free access and they claimed that it would be free for all time."
"They should extend some of the functions that are a bit clunky and improve the integration."
"The pricing is an issue, the program is very expensive. That is something that can improve."
"Everything is very manual. It's up to us to find out exactly what the issues are."
"The solution has a lack of compatibility with newer technologies."
"The support for automation with iOS applications can be better."
"We moved to Ranorex because the solution did not easily scale, and we could not find good and short term third-party help. We needed to have a bigger pool of third-party contractors that we could draw on for specific implementations. Silk didn't have that, and we found what we needed for Ranorex here in the Houston area. It would be good if there is more community support. I don't know if Silk runs a user conference once a year and how they set up partners. We need to be able to talk to somebody more than just on the phone. It really comes right down to that. The generated automated script was highly dependent upon screen position and other keys that were not as robust as we wanted. We found the automated script generated by Ranorex and the other key information about a specific data point to be more robust. It handled the transition better when we moved from computer to computer and from one size of the application to the other size. When we restarted Silk, we typically had to recalibrate screen elements within the script. Ranorex also has some of these same issues, but when we restart, it typically is faster, which is important."
"The initial investment is probably a little high. It was a little hard for me to sell, but it was a one-shot deal and that's why it's so high. All we are doing now is paying annual maintenance, which we don't have to do if we don't want upgrades, but we do."
"Purchasing and licensing are okay. Go for the perpetual licenses. In that way, you own a license, then you can purchase maintenance and support on top of that, so you don't have to pay every year for it. Even if you don't want it a contract with Worksoft Certify in the future, you will have your own license of it. Then, if your usage is not that much, you can have one or two perpetual licenses. However, if you want to run your processes, you will need more licenses, e.g., using the run-only licenses. They are really cheap compared to the full licensing."
"I can only judge based on the situation that we had around six years ago when we did the tool evaluation. Worksoft was not the cheapest, but it provided the value. For 25 concurrent licenses, we paid more than €400,000, so it was not cheap. In the end, if you see how much time you are saving and compare it with others, its price is okay. We had also compared its cost with the licensing costs for HP and Tricentis, and they were at another level. Now, as we have already booked the licenses, we only have to pay an annual maintenance fee, which is 70%, and that is okay."
"The tool is free to use."
"It is freeware software, so start using it."
"There is a free version of the tool to use and support is available through the community forums, but there is a licensed version as well."
"We were using the free version of this tool."
"Katalon Studio is a free tool."
"We are using the free version of Katalon Studio at this point."
"The price is not high it's good."
"Our licensing fees are on a yearly basis, and while I think that the price is quite reasonable I am not allowed to share those details."
"We paid annually. There is a purchase cost, and then there is an ongoing maintenance fee."
Katalon Studio is a simple and powerful automation solution built by KMS Technology for testers everywhere. The tool provides features comparable to popular commercial solutions while eliminating the hassle of building an automation framework from open sources. Best of all, we are committed to keeping Katalon Studio free.
Katalon Studio is ranked 4th in Test Automation Tools with 12 reviews while Silk Test is ranked 9th in Test Automation Tools with 6 reviews. Katalon Studio is rated 7.8, while Silk Test is rated 7.0. The top reviewer of Katalon Studio writes "Powerful yet easy-to-use and focuses on the end-user as well as interface testing". On the other hand, the top reviewer of Silk Test writes "An easy to use interface with a recording feature that our business users are happy with". Katalon Studio is most compared with Tricentis Tosca, SmartBear TestComplete, TestProject, Ranorex Studio and Postman, whereas Silk Test is most compared with Micro Focus UFT One, Micro Focus LoadRunner Professional, Selenium HQ, Apache JMeter and Micro Focus UFT Developer. See our Katalon Studio vs. Silk Test report.
See our list of best Test Automation Tools vendors.
We monitor all Test Automation Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.