We performed a comparison between Apache JMeter and Sauce Labs based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools."The most valuable feature of Apache JMeter is its popularity. It is the best open-source tool with all the features needed."
"Apache JMeter is quite flexible."
"We find the ease of use and the reports and graphs available valuable."
"The reports and analysis tools are very good. They are the solution's most valuable features."
"JMeter is easy to use for a user who doesn't have too much knowledge of programming or certain languages."
"We use Apache JMeter for load testing, where we provide the throughput time."
"We are using it just for load testing. We are using its free version, and it is scalable."
"Scripting with the solution is good."
"Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. We have an integration with App Center that we're working on. They have a storage API that lets us retrieve APK and IPA, iOS and Android builds off the phone, so that we can continue testing with CI/CD. They integrate with Jenkins, and Jenkins is the main CI/CD."
"Since this is an all-in-one testing site, we are able to take advantage of the browser OS combinations, mobile emulators and simulators, and real mobile devices. This is important to us since we have a variety of users, browsers, OS, etc."
"The abundance of device, platform, and browser combinations/versions that can be used in parallel."
"The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later."
"Sauce Labs' dashboards contain multiple useful metrics in one place. Everything is represented to us visually on the dashboard, which helps us understand where to focus our attention, what the issues are, and what we need to resolve."
"The most valuable feature is the cross-browser feature, it has many android and iOS devices both simulators and real devices. It's easy to integrate. I also like video recording too."
"It runs on the cloud, so you don't have physical setups to run all of this."
"It helped to integrate our performance testing and UAT, which helped to deliver a bug free software for our customers."
"Automation is difficult in JMeter."
"The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter."
"In Micro Focus LoadRunner we can go from the UI and we can configure it. There is no such feature in Apache JMeter. There should be UI-based recording history or logs."
"The only thing is the learning curve. It's high."
"The stability could be a bit better."
"Because so much is being done these days with authentication processes, a better system for either getting bearer tokens or some kind of token-based authentication prior to executing APIs would benefit the product. It is there, and you can do things. It is just not real clean at this point. There should be a better authentication process for JMeter or some automation or better guidelines for gaining and utilizing tokens on the fly."
"Currently, the integration pipeline is implemented by using Jenkins or a similar tool platform. These are continuous integration tools. As far as I know, integration is done by using custom scripts. It would be good if the integration with a continuous integration pipeline, like Jenkins or Hudson, can be done out of the box without using a script."
"The solution could use some sort of educational features to offer tips and hints to help users navigate it better. They should improve the manuals and help files."
"There have been various times throughout the last month or so where the service has gone down during business hours."
"Start execution time as each time a set of tests start, it will launch a new VM so it takes a bit of time."
"The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended."
"We have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down. If this speed could be improved then the service would be much better."
"When we were in development, it was a bit of a pain because we have onshore and offshore development. One of our development shops is in India, and we were running tests over there. When some of the users tried to log in, it was slow for them or we didn't have enough licenses. That was during the core development before we even launched."
"This product is not super scalable, because you have a very specific number of VMs that you can use."
"Speeds up the time it takes to run end-to-end user interface (UI) tests inside their virtual machines (VMs)."
"We have faced challenges with the availability of mobile devices. There was once or twice where there were no mobile devices available."
Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Sauce Labs is ranked 11th in Functional Testing Tools with 112 reviews. Apache JMeter is rated 7.8, while Sauce Labs is rated 8.8. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Sauce Labs writes "Robust documentation, helpful support representative, good licensing model". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Sauce Labs is most compared with BrowserStack, Perfecto, LambdaTest, OpenText UFT One and Katalon Studio.
We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.