We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
"Supplying devices to a testing team of possibly close to one thousand testers and developers is a great undertaking but Sauce Labs has made this very easy and a welcomed solution."
"As stated earlier we use Sauce Labs for a combination of automated testing and manual testing. Therefore the most useful features are the ability to run the functional automated tests via a Sauce Labs tunnels which allows access to applications in our internal network. The second most useful feature is the manual side."
"The most valuable feature is the cross-browser feature, it has many android and iOS devices both simulators and real devices. It's easy to integrate. I also like video recording too."
"Sauce Lab analytics helped us to get detailed knowledge on test cases execution and logs."
"I have found the live test section with Sauce Labs to be extremely valuable. When you can't quite figure out why a test is failing, you can go to the live test results section within their tool and launch your test (specifying a given OS/browser, or device) manually and step through the test to see the issue more clearly, usually opening up the developer's tool console and watching the network calls and console (within Chrome) to usually find the underlying issue."
"The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later."
"Live device testing. As we all know, It's really hard and challenging to find/purchase many real devices to test because it will be costly and not all the team can be able to purchase all of the devices out there. We used to have a lot of real devices under our labs. However, it is really time-consuming to maintain those devices and make sure they are up to date with the testing requirements."
"So far, the stability has proven to be quite good."
"The most valuable feature of this solution is regression testing tools."
"The solution helps improve the stability of our product. It also decreases the work of our manual quality assurance engineers."
"The solution is great as a record and playback tool. It also has valuable regression testing."
"Complete works perfectly with CUTE. That includes all dialogues, right-click menus, or system dialogues, etc., which are handled well."
"The reporting is ready to use and doesn't require any setup."
"The most valuable features are the desktop and mobile modules."
"TestComplete is simple, it's a very easy-to-use tool."
"The database checkpoints detect problems which are difficult for a human resource to find."
"If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farm"
"We have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down. If this speed could be improved then the service would be much better."
"The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution."
"They should provide a JIRA integration plugin so that we can easily log issues."
"Overall, I think Sauce Labs provides us with a valuable tool and resource. As far as what could be improved, I would say the overall test execution time. Some of the calls take a bit longer than I expect, for example in web browser tests; while the execution time isn't obnoxious, it could be improved so that overall tests/test suites finish faster."
"The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended."
"As a web product QA team, we sometimes need to spot check some new child site on multiple browsers and OS(es). It was a little time consuming for us since we need to click on each of the browser/OS combinations and start a new session to test. Every sprint, with new features and child pages being added, we mostly need to do the same steps over and over again."
"Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code."
"The artificial intelligence needs to be improved."
"The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS."
"The solution needs more training manuals or some form of online forum for learning. It needs more documentation."
"The solution needs to extend the possibilities so that we can test on other operating systems, platforms and publications for Android as well as iOS."
"The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools."
"The licensing costs are a little bit high and should be reduced."
"Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work."
"Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover."
"This is a pay-per-use service that is not expensive, and cost-efficient if you have a small team."
"The option we chose was around $2,000 USD."
"The licensing costs are in the range of $1,000 to $3,000."
"The license price for a physical machine is cheap, and for virtual machine, it is very expensive."
"Our ROI is about $10,000 a year."
"The pricing is a little above average — it could be lower."
"It costs a few hundred per year, but I am not sure. It is not at all expensive as compared to other tools."
Sauce Labs provides the world's largest secure and highly scalable automation cloud for testing desktop, mobile web, native, and hybrid applications. Sauce Labs helps companies accelerate software development cycles, improve application quality, and deploy with confidence across 450+ browser/OS/device/platform combinations.
TestComplete is a powerful and robust automated testing tool for mobile, web and desktop applications. Quickly and easily create accurate and repeatable tests across multiple devices, platforms and environments – regardless of experience level. It supports multiple languages, modern control sets and integrates with open source frameworks and tools like Selenium, SoapUI and Jenkins.
Sauce Labs is ranked 7th in Functional Testing Tools with 10 reviews while SmartBear TestComplete is ranked 4th in Functional Testing Tools with 14 reviews. Sauce Labs is rated 8.4, while SmartBear TestComplete is rated 8.2. The top reviewer of Sauce Labs writes "Helps us in reducing the number of manual testing". On the other hand, the top reviewer of SmartBear TestComplete writes "Speed, configuration consistency, and accuracy of tests with fantastic results". Sauce Labs is most compared with BrowserStack, Perfecto, CrossBrowserTesting, Tricentis Tosca and Bitbar, whereas SmartBear TestComplete is most compared with Tricentis Tosca, Katalon Studio, Micro Focus UFT One, Ranorex Studio and Tricentis qTest. See our Sauce Labs vs. SmartBear TestComplete report.
We monitor all Functional Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.