Anonymous UserSAP QA Manager at a manufacturing company
Anonymous UserQuality Assurance Specialist at SelectQuote
Anonymous UserQA Analyst II at Regal Entertainment Group
We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
"The biggest feature is the fact that it's codeless. It takes away the problem of finding people with the correct programming language, since there are multiple such languages. It saves time in introducing people to the solution because they don't need programming knowledge, they just need to be able to think logically. This makes it vastly usable by more people who are not even acquainted with IT at all."
"It does allow for good reusability. When it's designed properly and utilized properly, we can put things in a way that allows for reusability, meaning a lot of reuse of VA01, if they're very similar flows, to keep it simple."
"Certify integrates with other tools and it works very well with other machine testing applications."
"It's pretty seamless with SAP and Salesforce because they've built in the field definitions and all the things that you need. You literally turn it on and execute your script and it records it. It's very simple. Then you can go back and put in some of the other functions. For example, instead of hard-coding field selections, you put in a data table so you can run it multiple times or with multiple data. It was actually written to work very well with SAP."
"Certify's web UI testing abilities for testing of modern applications like SAP Fiori was good when we started and they developed it to be even better. We all know that web applications can change objects in DOM quite fast and it breaks tests. To counter it Certify has made object recognition more flexible and generic, so we don't have any troubles."
"The scripting methodology is easy to learn. It is easy to maintain because it is presented in a simple, narrative way. You don't need to know programming." "It has reduced our test maintenance time by more than 50 percent because we don't have to do manual test processes. We have saved over 150 man-hours monthly. It has increased our delivery times. We went from 200 man-hours down (three weeks work time frame) to approximately 40 man-hours (three days work time frame)."
"The most valuable feature is the ability to automate quickly and to maintain and update scripts."
"The decoupling of the test scripts from the data and the application is a nice feature. When you are creating test scripts, for example, for a web application, you have to learn about Worksoft and how the controls of a screen can be interpreted by Worksoft. For that purpose, you create so-called maps. These maps are loosely coupled to your scripts, which means if the application is changed, the control will be changed from an identifier. You don't need to rework the entire script. You only need to do these adjustments in the map, and then you can automatically reuse the scripts. So, it is really a smart move to have the decoupling of scripts, maps, and data."
"CrossBrowserTesting allows us to test our site with real-world devices in real-world scenarios and find what we're missing."
"When developing new pages that have questionable functionality or coding, we will often use CBT to test it in a browser. CBT works with our testing environment and development site."
"I can run a page through the screenshot tool, then send a URL with the results to my team."
"Each new session started with the live testing feature allows for a cleared browser and new experience to be able to not only see these attributes on the page clearly but also pass clean data."
"Supplying devices to a testing team of possibly close to one thousand testers and developers is a great undertaking but Sauce Labs has made this very easy and a welcomed solution."
"As stated earlier we use Sauce Labs for a combination of automated testing and manual testing. Therefore the most useful features are the ability to run the functional automated tests via a Sauce Labs tunnels which allows access to applications in our internal network. The second most useful feature is the manual side."
"The most valuable feature is the cross-browser feature, it has many android and iOS devices both simulators and real devices. It's easy to integrate. I also like video recording too."
"Sauce Lab analytics helped us to get detailed knowledge on test cases execution and logs."
"I have found the live test section with Sauce Labs to be extremely valuable. When you can't quite figure out why a test is failing, you can go to the live test results section within their tool and launch your test (specifying a given OS/browser, or device) manually and step through the test to see the issue more clearly, usually opening up the developer's tool console and watching the network calls and console (within Chrome) to usually find the underlying issue."
"The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later."
"Live device testing. As we all know, It's really hard and challenging to find/purchase many real devices to test because it will be costly and not all the team can be able to purchase all of the devices out there. We used to have a lot of real devices under our labs. However, it is really time-consuming to maintain those devices and make sure they are up to date with the testing requirements."
"So far, the stability has proven to be quite good."
"Performance on the web UI part, especially with some of the more comprehensive Fiori features, like the complex tables that are being used, could be improved. In those cases we have noticed a lot of execution-time increase with regards to the Certify solution."
"An area that I would like to see improved is how the permissions are applied. If you're applying permissions groups to a user, one of the options is to delete the group entirely and lose the entire permission group, rather than just deleting the permission from the user, which seems a little silly. In my opinion, that whole module of permissions is very confusing and lends itself to common errors."
"Pricing is a bit high and we would like to have the availability of a trail environment for beginners and training would be great to have and easier to expand and use by more and more consultants."
"Better automation capability would be helpful."
"With the codeless process automation across packaged applications, once in a while, if we get a weird application that's not widely used, it gets a little stickier. First, the software has to learn the fields, so you have to identify all the fields. Once you do that, as long as there isn’t any non-standard code in the application, then it works fine. But there's that one step that you have to do, a step you don't have to do with SAP and Salesforce, for example."
"When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us."
"Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify."
"We would like this to be able to be used outside of SAP applications, as it would be good for other types of products."
"This solution would benefit from faster testing and support for more devices."
"The screenshot tool defaults to a screen layout instead of a full page test. I find it a bit cumbersome that I can't have it run a full screenshot as my default."
"Being able to test on real devices via the virtual connection is wonderful, but it can cause some lag and load time issues while testing."
"If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farm"
"We have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down. If this speed could be improved then the service would be much better."
"The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution."
"They should provide a JIRA integration plugin so that we can easily log issues."
"Overall, I think Sauce Labs provides us with a valuable tool and resource. As far as what could be improved, I would say the overall test execution time. Some of the calls take a bit longer than I expect, for example in web browser tests; while the execution time isn't obnoxious, it could be improved so that overall tests/test suites finish faster."
"The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended."
"As a web product QA team, we sometimes need to spot check some new child site on multiple browsers and OS(es). It was a little time consuming for us since we need to click on each of the browser/OS combinations and start a new session to test. Every sprint, with new features and child pages being added, we mostly need to do the same steps over and over again."
"Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code."
"The initial investment is probably a little high. It was a little hard for me to sell, but it was a one-shot deal and that's why it's so high. All we are doing now is paying annual maintenance, which we don't have to do if we don't want upgrades, but we do."
"Purchasing and licensing are okay. Go for the perpetual licenses. In that way, you own a license, then you can purchase maintenance and support on top of that, so you don't have to pay every year for it. Even if you don't want it a contract with Worksoft Certify in the future, you will have your own license of it. Then, if your usage is not that much, you can have one or two perpetual licenses. However, if you want to run your processes, you will need more licenses, e.g., using the run-only licenses. They are really cheap compared to the full licensing."
"I can only judge based on the situation that we had around six years ago when we did the tool evaluation. Worksoft was not the cheapest, but it provided the value. For 25 concurrent licenses, we paid more than €400,000, so it was not cheap. In the end, if you see how much time you are saving and compare it with others, its price is okay. We had also compared its cost with the licensing costs for HP and Tricentis, and they were at another level. Now, as we have already booked the licenses, we only have to pay an annual maintenance fee, which is 70%, and that is okay."
"SmartBear offers bundles of products that work together."
CrossBrowserTesting is a cloud testing platform that gives instant access to 1500+ different real desktop and mobile browsers for testers, developers, and designers.
Sauce Labs provides the world's largest secure and highly scalable automation cloud for testing desktop, mobile web, native, and hybrid applications. Sauce Labs helps companies accelerate software development cycles, improve application quality, and deploy with confidence across 450+ browser/OS/device/platform combinations.
CrossBrowserTesting is ranked 17th in Functional Testing Tools with 4 reviews while Sauce Labs is ranked 7th in Functional Testing Tools with 10 reviews. CrossBrowserTesting is rated 9.2, while Sauce Labs is rated 8.4. The top reviewer of CrossBrowserTesting writes "Live testing gives us the ability to identify potential issues on different browsers and devices proving to be a very useful tool". On the other hand, the top reviewer of Sauce Labs writes "Helps us in reducing the number of manual testing". CrossBrowserTesting is most compared with BrowserStack, Tricentis Tosca, SmartBear TestComplete and Micro Focus UFT One, whereas Sauce Labs is most compared with BrowserStack, Perfecto, HeadSpin, Tricentis Tosca and SmartBear TestComplete. See our CrossBrowserTesting vs. Sauce Labs report.
We monitor all Regression Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.