Apache JMeter Review

Excellent for load testing in use cases with high TPS


What is our primary use case?

Our use cases are for applications such as E-commerce sites where there are multiple concurrent users. In some cases, there is also API load testing where a high number API calls are required for load testing. 

How has it helped my organization?

Apache J meter has proved to be the tool of choice for Performance testing with various features and also it comes without any cost.

What is most valuable?

A valuable feature is that when there's a high number of TPS I can achieve more transactions per second given the hyper-limitations. If the TPS is more than 5,000 TPS, the CPUs get very high (for Load runner)  and JMeter handles that well. I'd also say that the scripting flexibility like co-relation using regular expression is good, it's in the central control. Those would be the two major valuable features. 

What needs improvement?

We have some scenarios for diameter load testing where TPS requirements are very high, 30K or 40K TPS. In the telco area, this is for simulating mobile usage. However, diameter load testing can be difficult in J Meter. The only way to imitate Diameter requests and process the responses of these requests is to implement them in the code of JSR223 samplers. JMeter generally provides synchronous calls. It's something that could maybe be improved in the future, because for achieving that very high kind of TPS, more than 30K, 40K requires a asynchronous solution. It's not a common thing, it's really very specific to the telco domain and a very few projects. 

For how long have I used the solution?

I've been using this solution for about 18 months. 

What do I think about the stability of the solution?

We haven't encountered any major issues with the stability.

What do I think about the scalability of the solution?

Scalability is good. 

How are customer service and technical support?

Support is more open-sourced. There is a lot of material available online so we haven't had to contact Apache technical support and we have our own team in the company who maintain the product.

Which solution did I use previously and why did I switch?

We sometimes use LoadRunner for various purposes but currently, we use JMeter 80% of the time because LoadRunner's CPU consumption becomes very high for TPS more than 5000. Also, LoadRunner is not free so we have to pay for the license. We're using it for very few projects now. JMeter can easily achieve a high TPS but LoadRunner has a lot of high-resource requirements. 

LoadRunner's advantage is that the output report is very comprehensive. JMeter reports are only good for the people who understand the detailed logs. Out-of-box, LoadRunner has nice reports and ease of execution. In JMeter, it's also simple but requires some online training or knowledge on the loader and the setup. 

How was the initial setup?

Initial setup is quite straightforward, it takes about two hours, no more, although each project is different. It depends on the scripting that goes into the environment. In terms of the product alone, the deployment doesn't take long. If you include the hardware, it can take about a week because the hardware calls for a lot of couplers and storage units and that takes time.

What other advice do I have?

For the load test which requires high transactions per second to be achieved, in the range of 15K transactions per second, this is a very good tool to use and the biggest advantage is that it is free. It has a big user group, so we can find solutions easily if there are any challenges. It's really a very good open-source tool for any organization to look at in terms of performance tests.

I would rate this solution an eight out of 10. 

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

**Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment
Guest