We performed a comparison between Apache JMeter and Sauce Labs based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools."It is very quick and user-friendly."
"The most valuable feature of this solution is that it is free."
"We appreciate that the solution is free to use, as an open-source tool."
"It is an open-source tool that is easy to use. It can be easily integrated with multiple tools, including Selenium."
"This solution is very user-friendly, and allows for a lot of data capture when testing."
"User-friendly and open source."
"It gives accurate results and recommendations that we can implement to enhance the performance of websites."
"The solution offers a lot of plug-ins and a huge continuously developing community that is regularly offering new features and plug-ins."
"I find that the multitude of browser and OS versions are very helpful for broadening testing scope."
"It provides a comprehensive selection of browsers and platform versions for our test automations and CI/CD pipeline process. It also provides a comprehensive set of virtual mobile devices, which we can configure for our automation and availability. These features are valuable for us when it comes to testing our applications. We have a website and mobile applications that we want to test and diversify to various browsers and mobile devices as well as restore various versions. This helps us to find bugs that users might be facing and correct them."
"Our machines are mostly Windows. Being able to test with Safari, on a Mac, and other types of browser pieces without having to manage all the infrastructure is the biggest feature that our team enjoys."
"The most critical thing is that this software aligns with our Agile and DevOps way of doing things. It integrates with kickoff scripts through DevOps."
"I have found the live test section with Sauce Labs to be extremely valuable. When you can't quite figure out why a test is failing, you can go to the live test results section within their tool and launch your test (specifying a given OS/browser, or device) manually and step through the test to see the issue more clearly, usually opening up the developer's tool console and watching the network calls and console (within Chrome) to usually find the underlying issue."
"The most valuable feature is the ability to easily test multiple browsers and versions, as well as off-loading our local environments using the SauceCloud."
"It provides zero maintenance browser instances."
"It has a wide assortment of platforms."
"I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this."
"Apache JMeter may have difficulty recognizing dynamic objects in some critical cases, which can lead to challenges in terms of object identification."
"There are certain things like we can't merge custom metrics into the JMeter reports. We're limited to JMeter metrics, and other server metrics can't be integrated with JMeter dashboard. This forces us to rely on another tool."
"JMeter output reports can be difficult to understand without training."
"They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."
"In future releases, it would be helpful if there was an integration with ALM Octane."
"At present, if the number of virtual users increases beyond 10,000 when testing, then it results in a Java heap which causes the solution to crash."
"It's not easy to get the data from one place or to do customizations."
"Sauce Labs' dashboard could be improved by adding more filters and allowing more customization options. There was one instance where the dashboard on the Sauce Labs UI didn't meet our requirements, so we had to use the Sauce Labs API to create some apps and dashboards on our own. The API endpoints could be a little more robust and customizable."
"The testing process is difficult. I need to prove the complete competency of the tool, and I am finding that challenging."
"The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution."
"The ability to install profiles on iOS real mobile devices should be included."
"I may not know what should be improved on the platform, but I think it could offer a greater variety of testing solutions. I know there might be competing solutions that cover more things I'm unaware of, and it could expand a bit more. We've done a lot of automated testing scenarios, and that's all the rage these days, so I think Sauce Labs could look at potential ways to improve and cover other scenarios."
"The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended."
"One of the challenging areas for us is the reporting and the matrix. It should be based on roles, but right now it is only available for the admin role. The admin role can really do a lot of stuff, but our infrastructure team, which holds the admin role, is not ready to hand it over to us on the testing team. If Sauce Labs had permissions associated with roles, where this role could do this activity and that role could do that activity, it would be easier for us."
"Lacks the ability to start multiple tests simultaneously."
Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Sauce Labs is ranked 11th in Functional Testing Tools with 111 reviews. Apache JMeter is rated 7.8, while Sauce Labs is rated 8.8. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Sauce Labs writes "Robust documentation, helpful support representative, good licensing model". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Sauce Labs is most compared with BrowserStack, Perfecto, LambdaTest, OpenText UFT One and Katalon Studio.
We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.