We performed a comparison between Sauce Labs, SmartBear TestComplete, and Zeenyx AscentialTest based on real PeerSpot user reviews.
Find out what your peers are saying about Tricentis, OpenText, Perforce and others in Functional Testing Tools."The most critical thing is that this software aligns with our Agile and DevOps way of doing things. It integrates with kickoff scripts through DevOps."
"The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing."
"It runs on the cloud, so you don't have physical setups to run all of this."
"Live device testing. As we all know, It's really hard and challenging to find/purchase many real devices to test because it will be costly and not all the team can be able to purchase all of the devices out there. We used to have a lot of real devices under our labs. However, it is really time-consuming to maintain those devices and make sure they are up to date with the testing requirements."
"Easy to integrate with the other platform for tracking purposes."
"The error logging is also very robust. If we run a test through Sauce Labs and there's some sort of issue, a log will appear on the screen. Log messages are usually heinous and horrible... Sauce Labs is incredibly good at saying things like, 'Hey, here is the exact issue. Fix this and you can run the test.' That helps in getting things up and running and executing the way they should."
"The most valuable feature is the ability to easily test multiple browsers and versions, as well as off-loading our local environments using the SauceCloud."
"I like the dashboard and seeing the test results. As a manager, I like to see the insights of the people using it, understanding the total path and run. I can see all of that as a manager. I also know team members love seeing the dashboard and seeing the test results in real-time."
"The most valuable feature is the integration with Azure DevOps."
"This company offers end-to-end capabilities for test suite creation and execution. One feature that I particularly appreciate is the tagging system. Tags are highly valuable, as they allow you to assign tags to your test cases. When there's an impact in a specific area, you can search for and run all test cases associated with that tag. I find this functionality very useful."
"The solution is mainly stable."
"The solution helps improve the stability of our product. It also decreases the work of our manual quality assurance engineers."
"The most valuable features of the SmartBear TestComplete are self-healing, they reduce the maintenance required. The different languages SmartBear TestComplete supports are good because some of our libraries are written in Python, JavaScript, and C#. It's very easy to put them all under one project and use them. The are other features that SmartBear TestComplete has but the competition widely has them as well."
"The ease-of-use and quality of the overall product are above average."
"The most valuable feature of this solution is regression testing tools."
"The product has many features."
"It’s been really easy to automate the same application TestComplete struggled with. I have been able to automate two of our key applications in just a few months. I haven’t even taken their training."
"If you use the PowerBuilder application, do choose AscentialTest without thinking twice."
"The most valuable feature of AscentialTest for us is that it fully supports PowerBuilder."
"AscentialTest's object recognition in snapshots is a robust feature that goes beyond standard elements, even accurately identifying objects within Datawindows."
"We encountered minor issues with stability from time to time but Sauce Labs continues to make improvements."
"The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution."
"If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farm"
"On a rare occasion, I will come into a ticket where a customer will have reached out to me after reaching out to Sauce Labs, saying, "Sauce Labs doesn't understand what I am going through. They are not being very helpful." So, I try to do clean up there. Outside of those extremely rare occasions, I have only had one or two of those support issues."
"The testing process is difficult. I need to prove the complete competency of the tool, and I am finding that challenging."
"The pricing model of Sauce Labs could be more flexible. Sauce Labs has just one price for the type of solution and a set number of devices. Other solutions have a fee for the base solution and an additional cost per device. If you're a smaller organization, you have to consider your needs. Some smaller companies still need to test various devices, so my advice is to start small and scale up as needed. We had initially planned to start big, but that would have been a big waste."
"Multi-domain SSO is a big concern for us right now, especially as we've been merged into a larger company. I suddenly have teams coming from 20 different domains, and because the main master Sauce Labs account is locked down to one SSO domain, there are teams that can't run a test right now. I've heard they're working on a solution and they've been very communicative with us about it. A solution to that would help us a lot."
"One of the challenging areas for us is the reporting and the matrix. It should be based on roles, but right now it is only available for the admin role. The admin role can really do a lot of stuff, but our infrastructure team, which holds the admin role, is not ready to hand it over to us on the testing team. If Sauce Labs had permissions associated with roles, where this role could do this activity and that role could do that activity, it would be easier for us."
"During the distribution of our regression test cases, the control IDs are not always recognized correctly."
"It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing)."
"In the cross-browser domain, it has a few snags with Microsoft Edge and Chrome; although, these problems are not critical."
"The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT."
"The licensing costs are a little bit high and should be reduced."
"In SmartBear TestComplete the integration with Jenkins could be easier. Additionally, some of the controls could have better customization options. For example, if a grid is used and it contains multiple controls within it, it can be a checkbox, radio button, or any kind of control, the way the Object Spy is operating currently there is a lot of room for improvement."
"I didn't use it very heavily. One issue that I found was that there wasn't a quick way or a button to move Visual Basic scripts to TestComplete. We have a lot of such scripts in our organization, and it would be very useful to have some option to easily move these scripts. It is currently possible to convert these scripts to TestComplete, but it is not easy. I have to write some code, but everything is not available immediately."
"The initial setup of SmartBear TestComplete was complex."
"I would like to see an improvement in the User Interface."
"Streamlining the retrieval of results from individual test set runs would be beneficial."
"Classes are not as object-oriented as I would like, but I am a programmer and not QA so I expect a lot."
"The only thing I can't wait for is for Zeenyx to add automating Mobile apps."
Earn 20 points
