Apache JMeter Reviews

We use this solution for functional API testing, and performance testing.

This solution allows us to test all the functional APIs through it, then record and run the APIs.

This solution is easier to use than any other tool in the market; there is not even a requirement to learn a lot of scripting in order to use it.

We also appreciate that the solution is open source, so it has a very big community to utilize if we need help with an issue.

At present, if the number of virtual users increases beyond 10,000 when testing, then it results in a Java heap which causes the solution to crash.

I have been working with this solution for around seven years.

The stability of this product is good, outside of the issues with adding virtual users during testing.

This solution is scalable, but only up to a certain point when adding virtual users, before performance issues begin to occur.

The initial setup of this solution was quite straightforward, and deployment only took one to two minutes.

We implemented this solution using our in-house team.

This is an open-source solution, so there are no licensing costs associated with it.

We would recommend this solution to any QA or developer as a 'must-try product'.

I would rate this solution an eight out of ten.

I use JMeter for concurrency and some backend services. We are also using JMeter to apply our tests within our framework by sending some requests to JMeter to assess our application's performance. My company is developing software based on data analytics that produces insights and graphs like Tableau or Power BI. We use JMeter to test the graphs, how the tables are rendered, and how long it takes to render some applications. 

For instance, we have a problem with one of our clients that use some complex scenarios or queries that take too much time to retrieve from the application or render to the end user. We ran a couple of tests on the application using JMeter and spotted the service or request that was taking too much time.We alerted the developers about this, and they took the appropriate action to fix this problem. Afterward, we run another cycle of JMeter to ensure everything is working as expected.

There are two modules. The analytics module is the interface for the stakeholders and company decision-makers. The number of users isn't that big, unlike the retail applications websites. It's 500 users at most. The other module is deals with data volume. We are currently doing some POCs to check whether we will benefit from JMeter in this area because it's not a concurrency issue. It's a data volume issue.

We have a hybrid deployment because we are using JMeter to assess the performance of our products. If the product is deployed on the cloud, we use JMeter on the cloud. If it's on-prem, we are using it on-prem. At my previous company, we mainly used JMeter on-prem.

I expect that we will continue to rely more on JMeter, and we have multiple DevOps pipelines using JMeter to test another module in our application.

I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance.

The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter. It's working on this, actually.

It's an excellent plugin that you can use to record the scenario from Google Chrome, and it integrates easily into JMeter. They could also make it easier to generate the built-in report. Now, you run the tests and generate the charts in a separate column. The graphs and charts that display the test metrics could be better.

I worked with another tool called Web Performance Tester, and its interface is better than JMeter's. They have intuitive graphs while you are running the tests, so you can see how things are going. It shows you the number of concurrent users logged into the system, the number of failures, response times, etc.

I worked in performance testing from 2011 until 2019. I was working with another tool, but in the last few years, I started using JMeter for a couple of projects.

It's hard to say definitively. When there's something wrong with JMeter, we're unsure whether it's something in the device because there are lots of requests coming from the machine. Maybe we didn't use it that many times from our local server. There are some bottlenecks, but in many cases, we only need to restart JMeter, and it works fine.

JMeter is scalable. I did a test using it in a distributed environment with more than one client. We used JMeter to load from the server and also the plugin tool. It works fairly well in a complex environment. I still need to research the maximum concurrent JMeter can handle. Is it 1 billion or 10 million? 

In my work experience, I need to load maybe 1 million users. For example, most of my work is on the backend of e-commerce websites with a maximum of 1,000 users. We have many clients, but the load is not that big. 

I don't think JMeter has technical support because it's an open-source application, but there is support for third-party apps that use JMeter. For instance, BlazeMeter has good technical support.

I have more experience in a tool called Web Performance Tester, but it has some limitations. For example, it doesn't support some forms. A long time ago, their support was better, but lately, their support hasn't been so good. They always ask us to switch to the consultancy service and they don't give us the solution. If we face problems, they always tell us, "Okay. You can consult us." This is not a good practice, actually.

Also, Web Performance Tester isn't well-known, so some of our clients aren't confident in it because it doesn't have a reputation.  They were more familiar with JMeter, and it's something I've used.  However, I have no problem with Web Performance Tester. It's hard to compare the two tools because it's not apples to apples. 

I joined this company in October, and they had just started using JMeter. Previously, they were using a tool that they had developed in-house. They found that JMeter has more capabilities and specificity than the tools they were using.

Setting up JMeter is straightforward. It doesn't need an installation like other tools. There is a batch file for Windows. Around four people in my organization are responsible for maintaining and managing JMeter.

JMeter is a free open-source tool. There are some third-party tools built on top of JMeter that have a license or something like BlazeMeter. I think you can also purchase some additional services.

I rate Apache JMeter eight out of 10.  Before using JMeter, you should find some tutorials because you need to be trained to use it. It's not a plug-and-play tool. This is what I did. Spend some time researching JMeter's capabilities. We met a lot of people who knew about JMeter, but when you tell them that it can do something, they're not aware of it. 

For instance, through taking some courses, we learned that JMeter has distributed performance capabilities. Some people still know that you can't make these recordable scenarios. This information hasn't reached everyone in the IT market.

We use this solution mostly for creating scripts, load testing, and performance testing. If my client doesn't have a tool or I don't have a tool, I try to leverage JMeter because it's an open-source tool.

I personally use JMeter for things which cannot be covered by our load testing tool, which is OATS, Oracle Application Testing Suite. This is a tool that was developed by Oracle but is no longer in use because Oracle has stopped developing it.

The solution is deployed on-prem and on private and public clouds.

JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases. A lot of the newer features don't work, and sometimes you have to spend a lot of time maintaining the scripts. That is something JMeter could probably look at.

For example, in JMeter 5 they brought in a lot of new controllers. But there isn't a lot of documentation available on the Apache site on how you're supposed to use those controllers. They've explained the controller functionality, but there aren't any proper use cases to show that. 

I have been using JMeter for a long time and do a lot of work with this solution.

It's not stable. With every new release, they come up with newer features, which aren't always very stable. So, stability is a concern.

It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users. That's not a problem.

Because it's open-source, a lot of organizations try to leverage JMeter into everything. I don't think JMeter is currently very adaptable to all the technologies. That is something Apache should think of – incorporating other technologies and other tech stacks, which can be scripted from JMeter and tested using JMeter.

On a scale of 1 to 5, I would rate technical support 2.5

They're not very good because JMeter is an open-source tool. You can raise a bug to JMeter, and they might fix it or they might say that, "We will actually fix this in the next release." So, because it's an open-source tool, you do not get dedicated support. You have to raise tickets and wait. If you use BlazeMeter, the support is probably a little better because you can take your issues there.

I don't think support is great if you compare it with tools like NeoLoad or LoadRunner where they have a dedicated support team. You can raise tickets. You have a direct conversation with their engineers. They can help you understand where the issue is exactly. That kind of support is not in JMeter, at least not now. BlazeMeter has it, but I have not used BlazeMeter support, so I'm not sure how it is. But from what I have seen with JMeter, support is not great.

It is straightforward, but for JMeter 5, since they have introduced SSL handshake at the slave level, that becomes a little bit of a challenge for somebody who is doing it for the first time if you're doing a distributed execution, that becomes a challenge.

But since there was not enough documentation, I had to do a lot of troubleshooting on my own, since it was new. Now there's a lot of documentation available, but initially, there was not much documentation.

JMeter is pretty easy to use if you are using it for the purpose of only developing scripts. You can just uninstall the file and you can start using it. The only challenge happens if you're trying to access the internet over any kind of proxy. Then, you'll have to start it via the proxy mode.

Otherwise, deploying JMeter is pretty easy compared to other tools. You just have to download, unzip, and you are ready to go.

We installed JMeter ourselves.

I would rate this solution 7 out of 10.

We use Apache JMeter for performance testing, including the response time of our websites for multiple users.

The product is specialized in performance testing. It gives accurate results and recommendations that we can implement to enhance the performance of websites.

Apache JMeter could be a more user-friendly product from the end user's perspective. It requires someone with technical knowledge to administer it. This particular area needs improvement.

We have been using Apache JMeter for a year.

I rate Apache JMeter's stability a ten out of ten.

We have around 12 Apache JMeter users in our organization. I rate the product's scalability an eight out of ten.

The technical support team responds quickly to queries.

The initial setup process is very complicated for the on-premises version. It would be better to use the cloud version.

The product has reasonable pricing.

It is a complete tool that integrates lots of functionality. I recommend it to others and rate it a nine out of ten.

We primarily use the solution for performance and delivery testing.

It's a free tool. It's a very common tool. There's a lot of support in the community for this. It mainly supports web API testing.

The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it.

Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there.

It's an open-source tool; we cannot ask for additional features really.

The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

I’ve used the solution since 2009.

The solution is relatively stable. There are no bugs or glitches, and it doesn’t crash o freeze. It’s reliable.

It can scale. However, what others say about scalability is that you need to have some proper calculations to be done first.

We have dealt with support. Technical support is found in more technical forums. It’s open-source, and communes are where you go for information. We have received good responses when we have inquiries. There are quite a lot of forums in general available.

Now, as we have grown as an organization or as a team, there are still questions such as, "What are the limitations of this tool?" And we put that to JMeter so we can learn what is best for the maintenance.

The installation can be a bit complex. There are quite a lot of things and issues if you go deep into it and if you're setting up JMeter. For example, direct script captioning is slightly different. It's a bit more complex, the correlation parametrization. Setting up the workload model can be complex as it is based on a Java service mechanism.

It becomes a bit challenging to manage. If I want to put a 50,000 user load, I have to be very careful how the memory is utilized. I must be very aware of the underlying system's capability to execute this. I have to make it into multiple nodes and run them in parallel. There are some calculations, and there are some good power processes that will be required when you are using JMeter. You need to understand its limitations and load work them on.

The solution doesn’t really need maintenance. They're open-source tools, so we don't expect any maintenance. What we typically do is we pick up a particular version and understand the limitations of it. We then play within those limitations.

We handled the initial setup in-house. I handled the setup myself.

The solution is free to use.

We may be using the latest version. I don’t know the exact version number.

It is on-prem. However, we can put it on the cloud as well. We install it on any machine, so it can be a local or cloud-based machine.

I’d recommend JMeter. One critical piece of advice is to plan properly. For JMeter, planning and having an understanding of the limitations are important. If you play within those limitations, it is a really great tool. There are trade rules that apply to a lot of things that we use. However, there's plenty of material available so far. Whoever is doing it, can plan it accordingly. They can create those tests, and execute those tests. That said, at the same time, be aware of the scalability of the JMeter. For example, a single JMeter line, which is running, can take 1,000 to 2,000 threats, not more than that. Again, it depends on the available hardware. If you want to scale to, say, one million or something, different parts of planning are needed.

I’d rate the solution eight out of ten.

We use JMeter for performance testing. Our testing team has 50 to 60 people.

JMeter lets us generate virtual users and T-load, per our requirements. It's easy to configure and adjusting the virtual users according to the DPS we want to achieve.

It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow.

We've been using JMeter for six months.

With an on-premises deployment, we run into some performance problems when we have a high load. It's not that stable if we increase the scripts or the load. But that could be something in our environment.

JMeter is an on-premise solution, so it has limited scalability. We are looking for a cloud-based solution that is scalable and elastic. 

I rate JMeter six out of 10.

I've used JMeter in conjunction with Selenium, Java, and Log4j for logging. I used it before
I ended that contract in August. Its version was up to date at that time.

It was used for an e-commerce site that is specialized in C-PAP or weaving machines in effect. Their max was a thousand people logged in at once. I, of course, pushed the boundaries on that, but it was to test the performance of the website, and of course, I'm had to try subsystems, database interactions, etc.

I'm a total geek, so I liked the fact that I got to program. The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable.

I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this.

If I were going to be Mr. Selfish and say anything I want, I'd say a full feature GUI that lets me drag and drop different modules in line. It could have a simple-to-use GUI.

I have been using this solution for probably a year and a half.

I didn't have any issues with the stability of JMeter itself. There were definitely issues with the program I was testing, but that's why I was testing it.

It was very easy to scale, but I was barely scratching the surface. I have spent 17 years at Microsoft, and for the performance testing that we did there, we had 8.3 million users at once, as opposed to a maximum of a thousand. If I'm scaling, I have to do it quite straightforward and simple, but it was very minimal.

Only I was using it. It was the QA department. I showed it to some of the devs, and they were very interested. A couple of them tried it, but none were actually using it day-to-day for testing out the environment.

I would rate them an eight out of 10 because sometimes, they would take two or three days to get back to me. Of course, at that point, you're like, "I need the answer; I need to answer." So, it was a little bit unrealistic in terms of expectations.

At that particular company, I was the one who was tasked with coming up with the solution, so that was the only one that I looked at simply because JMeter is industry standard, but at Microsoft, they wrote their own custom tools, so I used custom Microsoft tools.

It was rather complex. It is a complex product, but that part of it was very well-documented. I didn't have any problems with it.

Don't be shy in asking questions. Google/Bing is your friend. It is complicated. There's no reason to spend eight hours trying to figure out something, except unless you are trying to learn in-depth. There are a lot of people who've done exactly what you're trying to do, and it doesn't matter what it is.

I would rate it a 10 out of 10 because it is industry standard. It did everything I could've asked. I barely scratched the surface, but what I needed it for, it did well and in a very straightforward-to-implement way.

We primarily use the solution to manage our capabilities. Wherever the business goes, we need to work within the expected benchmark. In terms of the user bases we deal with, most of these are products from different locations with different types of functional flow-out. There are environmental integrations also running in the background, which a user is basically accessing while going through a particular workflow. 

We'll also handle the reporting section of our data management responsibilities utilizing this system.

The reports and analysis tools are very good. They are the solution's most valuable features.

The GUI could be improved. When we go into GUI mode, there are occasions where it will not sync with our expectations. There are crashes that happen that will stop the solution from performing. It seems we get minor glitches when we go into GUI mode.  

The data client architecture that we have isn't so great. If we are to consume the data, it won't clear because there is tech running on different agents. 

When I need to pull the reports from different agents, it's not user-friendly. The reporting can be difficult to handle. It's hard to increase it if you are working on a client's architecture.

It's not easy to get the data from one place or to do customizations. 

There are other solutions that allow users to model their load and structure with them. You can't do that on JMeter. 

On other solutions, like Silk Performer, you can do network packeting, which you can't do on JMeter. They should add this to the solution as a capability in the future.

The support management needs improvement. Support is coming from consultants; you will not be able to get on-premise support from all of their agents in one place. On Silk Performer, for example,  they have the capability where you can basically have a summarized report from different agents.

Personally I've been using the solution for the last 10 years. However, in the program we are in, I've only been using this tool for six months.

If you want stability, you will have trouble, because the system is not uniform. You have to do a lot of modeling. 

If you're talking about mobile platforms, JMeter will not allow you to capture the performance of applications on your mobile platform. It's an area where the solution is lacking. It doesn't have the capability to allow developers to check the performance on an AP app.

You can scale the solution, but you will run into issues if you want to run, say, 1,000 users.  You will have to configure multiple agents and then distribute your load. It's quite difficult to manage everything on a single agent.

The initial setup is straightforward. We found it quite easy. For us, it wasn't complex at all. The solution is up and running quite quickly and doesn't take long at all to set up and deploy.

We've looked at Silk Impact and Load Balancer in the past. Each of these solutions offers a few different functionalities that aren't necessarily available on JMeter. For example, on both, there are more network features available that aren't included in JMeter and you can model your load and structure with them as well.

If companies have mobile development, they should use JMeter. If they want to integrate it with the PA pipeline, JMeter will work well. However, there are other platforms and solutions as well.

Other options include Silk Performer and Load Impact. These two solutions have other capabilities that JMeter doesn't have. You can basically model your load and structure with them. You can't do that on this solution. On Silk Performer, you can also do network packeting, which you can't do on JMeter.

Then there are some network features available in Load Impact as well as Silk Performer that aren't in JMeter. 

I'd rate the solution eight out of ten. The solution has certain limitations, but it's still a good product.