We performed a comparison between Apache JMeter and Sauce Labs based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools."We use Apache JMeter for load testing, where we provide the throughput time."
"It's a powerful tool that is open source."
"We are using this for performance testing and some automation."
"It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy."
"We find the ease of use and the reports and graphs available valuable."
"It is easy to set up."
"I like the fact that JMeter integrates well with other tools."
"JMeter is user-friendly, and that's a notable advantage of JVTech. It's straightforward and easy to use, unlike some other load testing tools, making it very easy to understand."
"The abundance of device, platform, and browser combinations/versions that can be used in parallel."
"The most valuable feature is the ability to easily test multiple browsers and versions, as well as off-loading our local environments using the SauceCloud."
"I have found the live test section with Sauce Labs to be extremely valuable. When you can't quite figure out why a test is failing, you can go to the live test results section within their tool and launch your test (specifying a given OS/browser, or device) manually and step through the test to see the issue more clearly, usually opening up the developer's tool console and watching the network calls and console (within Chrome) to usually find the underlying issue."
"Maintaining many environments for test is a bear. These guys make it so easy with their CI integration that you can have tests going in after a few hours."
"The Failure Analysis feature is really important for us, one of the most important aspects. What is the root cause? Is it because we have a defect or is it that we have a test case that we need to fix or modify? The Failure Analysis is one of the main functionalities that I am exploring all the time in Sauce Labs... The Failure Analysis helps us to discover which test cases we need to work on."
"Testing subscription videos for on-demand software, making them better quality and bug free for our customers."
"The most valuable feature is the ability to run concurrent automated tests up to a specified value, depending on what we are currently paying for."
"One of the most valuable features is that we do not have to have the cross-platform testing vehicles in-house. Sauce Labs gives us the ability to test across platforms and that really helps give us confidence in our products."
"Self-healing and page rendering for the end-users are not available in Apache JMeter."
"What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses. As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected. What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter."
"There are certain things like we can't merge custom metrics into the JMeter reports. We're limited to JMeter metrics, and other server metrics can't be integrated with JMeter dashboard. This forces us to rely on another tool."
"The stability could be a bit better."
"In future releases, it would be helpful if there was an integration with ALM Octane."
"It has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems."
"They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."
"We would like some reporting and analysis tools to be added to this solution."
"An image comparison would be a nice feature to include in the Sauce Labs product."
"Start execution time as each time a set of tests start, it will launch a new VM so it takes a bit of time."
"User account management needs an overhauls, allowing for user groups rather than just a hierarchy structure."
"There have been various times throughout the last month or so where the service has gone down during business hours."
"Integration with Github, as well as several other similar tools, could be improved."
"This product is not super scalable, because you have a very specific number of VMs that you can use."
"We have had some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt."
"Multi-domain SSO is a big concern for us right now, especially as we've been merged into a larger company. I suddenly have teams coming from 20 different domains, and because the main master Sauce Labs account is locked down to one SSO domain, there are teams that can't run a test right now. I've heard they're working on a solution and they've been very communicative with us about it. A solution to that would help us a lot."
Apache JMeter is ranked 1st in Performance Testing Tools with 46 reviews while Sauce Labs is ranked 11th in Functional Testing Tools with 1 review. Apache JMeter is rated 7.8, while Sauce Labs is rated 8.8. The top reviewer of Apache JMeter writes "It's a free, scalable tool that's good for checking backend services". On the other hand, the top reviewer of Sauce Labs writes "Provides multiple test environments and saves time, but should have something for inspecting mobile apps screens". Apache JMeter is most compared with Postman, BlazeMeter, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Sauce Labs is most compared with BrowserStack, Perfecto, LambdaTest, Katalon Studio and Bitbar.
We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.