Apache JMeter Review
JMeter is great as a stand-alone load generator, however when you need a distributed load you'll require high server use


Over the past year I have spent a lot of time in close collaboration with Apache JMeter. I used JMeter as a standalone platform, in collaboration with several tools out there and with my own servers. Over the next few posts I will elaborate a bit more on what I have learned from working with tools like BlazeMeter, Loadosophia as well as working with your own remote servers and Jmeter. On top of that I will share some things I consider good practices to use when working with JMeter and building elaborate test scenarios in it.

JMeter and remote servers

Meter is great as a stand-alone load generator, however the moment you need to generate a substantial load or a distributed load, you will require the use of more servers. There are several possibilities to add external servers to JMeter. You can use a load generation platform such as BlazeMeter and Flood.io or you can choose to use your own machines (Disclaimer: I have only worked with Flood and BlazeMeter as services, so I only mention these two. There may very well be other services out there I am not (yet) aware of).

Why use your own remote servers?

Although there are several commercial options to run JMeter tests from a cloud-platform, it might be useful to have your own servers running. Your own machines, for example Virtual Private Servers, will give you at least the following advantages, please bear in mind these are the main reasons for me to run private servers. It may very well be that you have different reasons for doing the same (I’d be happy to hear them btw!):

  • Availability within firewalls: Not all applications are available outside of firewalls. For quite a lot of companies it is not an option to allow external addresses, especially cloud services, within their firewalled environment. When this is the case in your organisation it can be very useful to have your own load generator hosted inside the firewall.
  • Geographical location: not each and every country has platforms available for load generation. Although platforms like Blazemeter have a lot of different locations available, it may very well be that your particular country is not available through a service provider. However getting your own, hosted, machines within a country is generally not too difficult to do. Keep in mind though whenever you want to use something like a Virtual Private Server to verify with the hosting company whether they allow load generation from within their network!
  • Control: a fairly straightforward reason is in order for you to have full, unlimited control over the load generating servers. If you have your own (virtual) servers running you can easily adjust settings, properties and other things in order to make it fully match your (customers) needs.

Why use a service?

Even though there are quite some reasons for using your own machines, the advantages of using a service like BlazeMeter or Flood.io are plenty, I will just highlight the few that for me have tipped the scales several times in favor of a service.

  • Maintenance: Setting up your own servers is time consuming and therefore not cheap. These servers need maintenance even though you quite likely do not constantly need them. A Service is just there. You buy a subscription, be it with a time constraint (monthly) or a load constraint (max amount of concurrent users) or something like that, but the service is simply at your fingertips. The moment you need it, you have it.
  • Multiple locations: the JMeter based services generally make use of the Amazon AWS cloud, thus giving you, the user, an immense amount of servers at your disposal. With this comes a huge worldwide distribution, which can be very advantageous for big world-wide used applications.
  • Support: both mentioned platforms have a very solid support base for their customers. If you have questions, if your tests seem to not run properly or you simply have issues getting something done, there generally is a good support platform available. You can use forums or the actual helpdesk, but either way, there is a good, solid, commercial support-base to help resolve your issues.

In the end, I choose per assignment what best fits my needs. Sometimes I use the VPS, sometimes I use a service. It is important you at least think about what best suits your needs for the test you are about to build and execute.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
2 visitors found this review helpful

5 Comments

it_user191529User

Well explained and very useful to know how jMeter can be used in cloud.

12 April 15
Martijn de VriezeConsultantTOP 20POPULAR

Thanks! I tried to make it as clear as possible what the benefits of either may be. Sometimes it is hard to make a clear choice.

12 April 15
it_user278154User

Hi Martijn,

Great review - very helpful.

I am trying to get JMeter / Blazemeter as the go-to performance testing tool at my current client, but I am concerned about what protocols JMeter cannot test? Obviously JMeter is great for anything HTTP based, but I would be interested to know your thoughts on its limitations? Have you come across anything that it simply cannot performance test?

Many thanks

23 July 15
Martijn de VriezeConsultantTOP 20POPULAR

Hi James,

The only thing I have once experienced was a really unfriendly proprietary protocol used to encapsulate COBOL code within a JAVA wrapper and as such made COBOL available via a browser. This was such a strange and low-level of protocol that I did not manage to throw load against it.
For the rest I have not had many problems using Jmeter to build up loads, even for more traditional client/server setups.

What protocols in particular are you concerned about?

cheers

mrtn

23 July 15
it_user278154User

Hi Martin,

Thanks for your input, very useful to know. I am working as a consultant on a relatively new client site, and am preparing to head up the team to write an RFP proposal for a new performance testing tool to be used across the company.

Getting all the technical / protocol details for the various systems is proving to be more challening than it should - they seem to be all Web based (although only support old versions of IE like 8 and 9... I know). I am very tempted to strongly recommend JMeter as the tool (probably with BlazeMeter or Flood as the cloud based service), but I am concerned that a couple of months down the line some legacy system will come out of the woodwork that JMeter would not be able to support.

Hope that all makes sense. Would be interested in any other feedback that you had.

Cheers,

James

24 July 15
Guest
Why do you like it?

Sign Up with Email