We performed a comparison between Apache JMeter and Sauce Labs based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools."It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users."
"It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy."
"Scripting with the solution is good."
"The product helps me get the expected performance from applications or servers and reduces costs. It also enhances the performance of the services and helped them reach their ultimate capacity."
"The most valuable feature for us is the available information on the forums and to be able to discuss and get answers from the people that are involved in using this tool."
"The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features."
"We find the ease of use and the reports and graphs available valuable."
"We use Apache JMeter for load testing, where we provide the throughput time."
"It has a wide assortment of platforms."
"Before implementing Sauce Labs, we tested physical devices that team members had to share. It was more feasible when we were all located in one office, but we couldn't leverage our offshore capacity. With this solution, we can do everything remotely, which is essential now that most of us work from home."
"There is a huge amount of open source, community provided resources."
"APPIUM for mobile testing has improved our organization by allowing us to test our website on more devices and browsers than we currently possess."
"It helped to integrate our performance testing and UAT, which helped to deliver a bug free software for our customers."
"They update for the latest browsers and mobile phones and support a lot of combinations. They have 1,000-plus desktop combinations and browser versions, which is really great. We need that at Applause. The all-in-one testing suite aspect of it is really important because most of our clients prefer to go to one place."
"Testing subscription videos for on-demand software, making them better quality and bug free for our customers."
"Sauce Lab analytics helped us to get detailed knowledge on test cases execution and logs."
"Self-healing and page rendering for the end-users are not available in Apache JMeter."
"Because so much is being done these days with authentication processes, a better system for either getting bearer tokens or some kind of token-based authentication prior to executing APIs would benefit the product. It is there, and you can do things. It is just not real clean at this point. There should be a better authentication process for JMeter or some automation or better guidelines for gaining and utilizing tokens on the fly."
"If JMeter could provide a web version of editing, that would be good."
"The tool needs to have a better Graphical User Interface. Many of the solution's features are difficult to understand due to the complex user interface and user experience. The product needs to add plugins. It should also work on the integration with external partners like IDE and API gateways."
"Apache JMeter could be a more user-friendly product from the end user's perspective."
"Apache should have a graphic interface."
"At times when we overload the application, it gets stuck...After the solution gets stuck due to overloading, we have to restart our computers. In short, the solution keeps crashing."
"What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses. As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected. What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter."
"Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code."
"Better and programmatic controls on request/response recordings and sharing with developers."
"Another aspect that could be improved is having the virtual devices boot up a little quicker. Sometimes we wait up to a minute before a device is available. It would be nice if that was 30 seconds or so."
"This product is not super scalable, because you have a very specific number of VMs that you can use."
"We had some specific features that we opened tickets on, although they were not earth-shattering. For example, the way the menus scroll could be improved because it does not have a bar, the way that people are used to, where you can move up and down."
"Sauce Labs' dashboard could be improved by adding more filters and allowing more customization options. There was one instance where the dashboard on the Sauce Labs UI didn't meet our requirements, so we had to use the Sauce Labs API to create some apps and dashboards on our own. The API endpoints could be a little more robust and customizable."
"We have had some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt."
"I would like to see improved network connectivity and it should allow playback for native apps."
Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Sauce Labs is ranked 11th in Functional Testing Tools with 112 reviews. Apache JMeter is rated 7.8, while Sauce Labs is rated 8.8. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Sauce Labs writes "Robust documentation, helpful support representative, good licensing model". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Sauce Labs is most compared with BrowserStack, Perfecto, LambdaTest, OpenText UFT One and Katalon Studio.
We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
