We performed a comparison between Apache JMeter and Sauce Labs based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools."Very user-friendly and easy to use."
"The performance of the solution is excellent."
"We are using this for performance testing and some automation."
"It is easy to set up."
"I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance."
"It is an open-source tool that is easy to use. It can be easily integrated with multiple tools, including Selenium."
"JMeter is a free tool with a large user population, which comes in handy because we have a vast knowledge base to tap into when needed. It's also easier to hire consultants who know JMeter."
"The product helps me get the expected performance from applications or servers and reduces costs. It also enhances the performance of the services and helped them reach their ultimate capacity."
"The most valuable feature is the cross-browser feature, it has many android and iOS devices both simulators and real devices. It's easy to integrate. I also like video recording too."
"The most valuable feature is cross-browser, cross-OS, cross-mobile device testing."
"As stated earlier we use Sauce Labs for a combination of automated testing and manual testing. Therefore the most useful features are the ability to run the functional automated tests via a Sauce Labs tunnels which allows access to applications in our internal network. The second most useful feature is the manual side."
"Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. We have an integration with App Center that we're working on. They have a storage API that lets us retrieve APK and IPA, iOS and Android builds off the phone, so that we can continue testing with CI/CD. They integrate with Jenkins, and Jenkins is the main CI/CD."
"The most valuable feature is the ability to run concurrent automated tests up to a specified value, depending on what we are currently paying for."
"The Failure Analysis feature is really important for us, one of the most important aspects. What is the root cause? Is it because we have a defect or is it that we have a test case that we need to fix or modify? The Failure Analysis is one of the main functionalities that I am exploring all the time in Sauce Labs... The Failure Analysis helps us to discover which test cases we need to work on."
"The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing."
"They offer a large number of devices and browser/operating system combinations for real device tests"
"Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports."
"Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like T6."
"The reporting is not very good."
"Self-healing and page rendering for the end-users are not available in Apache JMeter."
"The UI has room for improvement."
"The UI could be better."
"The stability could be a bit better."
"The tool needs to have a better Graphical User Interface. Many of the solution's features are difficult to understand due to the complex user interface and user experience. The product needs to add plugins. It should also work on the integration with external partners like IDE and API gateways."
"Start execution time as each time a set of tests start, it will launch a new VM so it takes a bit of time."
"Multi-domain SSO is a big concern for us right now, especially as we've been merged into a larger company. I suddenly have teams coming from 20 different domains, and because the main master Sauce Labs account is locked down to one SSO domain, there are teams that can't run a test right now. I've heard they're working on a solution and they've been very communicative with us about it. A solution to that would help us a lot."
"Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code."
"Progress towards reducing application testing time can be made."
"Unable to segregate reports for tests that are currently being developed, and might not be returning useful results."
"If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farm"
"Every time that we run scenarios where we need to discover the geolocation of our customers, by default it shows as Palo Alto, California. That's a problem for us and we need a workaround for those cases... It would be helpful if we could enter a latitude and longitude into Sauce Labs configuration and say, "When you run a virtual Chrome device or an iPhone, make this your default location. Then, provide me that device so I can run my scenarios," because we have stores in different regions across the United States."
"Sometimes pasting text while using "text object" does not seem to work, and it slows down testing times quite a bit."
Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Sauce Labs is ranked 11th in Functional Testing Tools with 112 reviews. Apache JMeter is rated 7.8, while Sauce Labs is rated 8.8. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Sauce Labs writes "Robust documentation, helpful support representative, good licensing model". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Sauce Labs is most compared with BrowserStack, Perfecto, LambdaTest, OpenText UFT One and Katalon Studio.
We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.