We performed a comparison between Apache JMeter and Katalon Studio based on real PeerSpot user reviews.
Find out in this report how the two API Testing Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."This solution is very user-friendly, and allows for a lot of data capture when testing."
"It is very quick and user-friendly."
"User-friendly and open source."
"The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."
"The features that I appreciate are quite basic. It is easy to ramp up the threads and start calling the application. A lot of connectors can already be found within Apache JMeter, but we are not using the entire set because the integration between the customers and platform is based on HTTP. We are just going to produce lots of HTTP sequences."
"JMeter is a free tool with a large user population, which comes in handy because we have a vast knowledge base to tap into when needed. It's also easier to hire consultants who know JMeter."
"The most valuable feature of Apache JMeter is its popularity. It is the best open-source tool with all the features needed."
"Very user-friendly and easy to use."
"One of the features that I like is Object Recognition. It worked very well, and it allowed me to create a dynamic expert based on my requirements."
"Katalon Studio's biggest advantage is its price. It's a good tool for the price, albeit with some limitations when you compare it with tools like Tricentis Tosca."
"The best feature is the split up between test cases and the test object. This allows us to easily change an error."
"The solution is easy to learn."
"I like the feature where you can define a model that represents your website and then reuse the components from the model. It makes creating more test cases easy."
"It is easy for a non-technical person to use and adopt."
"The best thing about the solution is that there is a record and playback functionality."
"The most valuable feature of Katalon Studio is that everything can be managed from one platform."
"Report generation needs to be improved. It is quite difficult to get to."
"It should start supporting the presentation layer. It currently provides performance testing specifically at the application and API level. It can be extended to the presentation layer, which includes mainly Angular and React frameworks. It should also be easy to use and easy to train people."
"They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."
"It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow."
"Apache JMeter may have difficulty recognizing dynamic objects in some critical cases, which can lead to challenges in terms of object identification."
"Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like T6."
"Currently, the integration pipeline is implemented by using Jenkins or a similar tool platform. These are continuous integration tools. As far as I know, integration is done by using custom scripts. It would be good if the integration with a continuous integration pipeline, like Jenkins or Hudson, can be done out of the box without using a script."
"The reports in Apache JMeter could improve."
"It is difficult to identify elements on websites that use Angular or ReactJS because they don't have an option to view source code."
"The tool's maintenance is very difficult since they do not follow call by value or call by reference. Due to this, any change happening is not reflected throughout all the test cases. There are some issues with data parameterization as well."
"Its stability needs improvement."
"The product’s regression testing time could be reduced."
"The product is comparatively slow."
"Currently, I'm the only one who can use it because I have a Mac. It is giving a hard time to my team members who are on Linux. My team members are using Linux, and Katalon doesn't support Linux very well. It keeps on crashing. It crashes and shuts down almost every time they save their work, so they are not able to use it. It should have good support for Linux. We don't know what the problem is. If the Katalon team can pick up this issue, it will be very helpful."
"During parallel execution, some reports are randomly switched."
"The integration process with Jira should be enhanced and facilitated."
Apache JMeter is ranked 3rd in API Testing Tools with 82 reviews while Katalon Studio is ranked 5th in API Testing Tools with 41 reviews. Apache JMeter is rated 7.8, while Katalon Studio is rated 7.8. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Katalon Studio writes "Useful multiple technology platform, scalable, but usability could improve". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and ReadyAPI, whereas Katalon Studio is most compared with Tricentis Tosca, Postman, OpenText UFT One, Testim and Eggplant Test. See our Apache JMeter vs. Katalon Studio report.
See our list of best API Testing Tools vendors and best Regression Testing Tools vendors.
We monitor all API Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
