We performed a comparison between Apache JMeter and Sauce Labs based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools."The most valuable feature of this solution is that it is free."
"The solution offers a lot of plug-ins and a huge continuously developing community that is regularly offering new features and plug-ins."
"We are using this for performance testing and some automation."
"It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy."
"What I like best about Apache JMeter is its user-friendly GUI because even if you don't have very good coding knowledge or understanding, or even if you don't come from a development background, you can still use the solution with just a few clicks. This is what's unique about Apache JMeter, in comparison with other tools in the market. As Apache JMeter is open source, when there's a missing feature, you can search in several community blogs for plugins that you can use to modify Apache JMeter to meet your requirements, and this is another advantage."
"It's very easy to install, and it's very easy to code and develop the script."
"The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable."
"The thread groups, samplers, and listeners, which are all determined by the script's requirements, are the most valuable features of this solution."
"Easy to integrate with the other platform for tracking purposes."
"It provides zero maintenance browser instances."
"They offer a large number of devices and browser/operating system combinations for real device tests"
"The abundance of device, platform, and browser combinations/versions that can be used in parallel."
"I find that the multitude of browser and OS versions are very helpful for broadening testing scope."
"The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later."
"The Failure Analysis feature is really important for us, one of the most important aspects. What is the root cause? Is it because we have a defect or is it that we have a test case that we need to fix or modify? The Failure Analysis is one of the main functionalities that I am exploring all the time in Sauce Labs... The Failure Analysis helps us to discover which test cases we need to work on."
"I like the dashboard and seeing the test results. As a manager, I like to see the insights of the people using it, understanding the total path and run. I can see all of that as a manager. I also know team members love seeing the dashboard and seeing the test results in real-time."
"In this tool, automation in general is almost non-existent. Everything is done manually."
"The tool needs to have a better Graphical User Interface. Many of the solution's features are difficult to understand due to the complex user interface and user experience. The product needs to add plugins. It should also work on the integration with external partners like IDE and API gateways."
"The stability could be a bit better."
"The solution needs to improve reporting. Currently, there is not enough automation involved with the feature. For example, there should be an automatic way of saving reports."
"Given that Apache JMeter is a free and open-source tool, documentation improvement may not be a major concern, as it is mostly contributed on a voluntary basis. The essential information is already available. However, in terms of the interface, there are occasional bugs, and the tool may not address them as quickly as some users would like. Fixing defects and bugs might take a considerable amount of time, with users sometimes having to wait for several months or even a year for the next release to address specific issues."
"They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."
"The user interface is a little bit tricky."
"The memory utilization in JMeter is very poor."
"I may not know what should be improved on the platform, but I think it could offer a greater variety of testing solutions. I know there might be competing solutions that cover more things I'm unaware of, and it could expand a bit more. We've done a lot of automated testing scenarios, and that's all the rage these days, so I think Sauce Labs could look at potential ways to improve and cover other scenarios."
"The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution."
"When we were in development, it was a bit of a pain because we have onshore and offshore development. One of our development shops is in India, and we were running tests over there. When some of the users tried to log in, it was slow for them or we didn't have enough licenses. That was during the core development before we even launched."
"Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code."
"Latency, due to Sauce Labs being a cloud-based solution, has been a concern. We work in different continents and countries, but last time I checked, Sauce Labs was only offering two data centers, one in the EU and another in the US. If you're not in either of those two places, you would have latency and issues running your test cases."
"On a rare occasion, I will come into a ticket where a customer will have reached out to me after reaching out to Sauce Labs, saying, "Sauce Labs doesn't understand what I am going through. They are not being very helpful." So, I try to do clean up there. Outside of those extremely rare occasions, I have only had one or two of those support issues."
"Speeds up the time it takes to run end-to-end user interface (UI) tests inside their virtual machines (VMs)."
"Multi-domain SSO is a big concern for us right now, especially as we've been merged into a larger company. I suddenly have teams coming from 20 different domains, and because the main master Sauce Labs account is locked down to one SSO domain, there are teams that can't run a test right now. I've heard they're working on a solution and they've been very communicative with us about it. A solution to that would help us a lot."
Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Sauce Labs is ranked 11th in Functional Testing Tools with 111 reviews. Apache JMeter is rated 7.8, while Sauce Labs is rated 8.8. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Sauce Labs writes "Robust documentation, helpful support representative, good licensing model". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Sauce Labs is most compared with BrowserStack, Perfecto, LambdaTest, OpenText UFT One and Katalon Studio.
We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.