What is our primary use case?
It is all about load testing. We have to do a lot of load testing in our IT environment. In the investment banking space, you do a lot of end-to-end endurance testing, to see how to benchmark performance based on resources and the CPU memory. So, we will have to benchmark the performance based on trade volumes. Trade volumes are the basic KPI.
Endurance testing needs to be done to see how many trades can be pumped in my environment. I do endurance testing.
What is most valuable?
The record and playback feature is the most valuable feature. It's all driven by the script, so it's a script-based tool where the background tracing starts. Java's background process does a lot of tracing. The process starts in the background. It sees what peaks of volumes the process can handle. It's easy to use because it's script based, record and playback. I play certain tests. I record those tests, see how they pass through the entire environment, and see how many breaks I get.
Then we repeat that for multiple numbers. If the first time, I pumped in 1 million trades, and get 5% of the failures, then I try with 2 million trades and I see that there is a 7% failure. These scenarios are easy to do in the LoadRunner. LoadRunner has been the best tool for more than 20 years in the market.
You can do endurance testing, you can do load testing. These are the different types of load testing. It lets you see how much volume you can benchmark in your entire environment, given the resources like CPU and RAM.
The record and playback is the best feature. You need to use a console. You log in to your console and see the dashboard with how many trades failed and how many trades passed, what the peak loads endurance load were.
What needs improvement?
The performance has really improved in terms of running test cycles. The product used to crash on-premises and when it had a lot of trades being pumped in. Because it is memory intensive, it used to crash if it was running out of memory. That was the limitation of the on-premises thing. Running those states and cycles in the cloud is much faster.
Everything is frozen in the cloud. The RAM, CPU, compute, and storage are provisioned in the cloud, which is becoming easier for running these test cycles. Test cycles are highly effective. Of course, you need to have a test strategy, like volume-based load testing. Configure some test cases and run those test cases in cycles. Cloud performance is much faster. Volume-based endurance testing is easier in the cloud.
For how long have I used the solution?
It is an old solution, around 20 years old. It was called once Mercury LoadRunner and then it was acquired by HP, it became HP QTP. That was around 2007, it was known as Hewlett-Packard HP QTP. Since then it has been acquired by Micro Focus.
What do I think about the stability of the solution?
What do I think about the scalability of the solution?
We have around 100 users, they are functional experts who use this for load testing.
The people who have been responsible for setting the KPIs will decide if we will expand the solution. They are the end-users. It will depend on the dashboard and the results of the load testing.
How are customer service and technical support?
LoadRunner is a very old product in this industry. It was a little difficult to get to the right support team. Now there's a professional service group that is better than they were before.
If scaling becomes a problem, then we have to involve the support, the right kind of support to actually fine-tune and troubleshoot the entire configuration. But with the cloud version, that risk is minimized. It has adequate storage available.
What's my experience with pricing, setup cost, and licensing?
It's a subscription-based, pay-per-user license. I wouldn't say it's expensive.
Because it was an on-premise license, it was a costly affair. Only certain stakeholders could use it. Now, it is enterprise-wide, we can have any number of cloud users performing the load testing because it is pay-per-user.
What other advice do I have?
I would recommend LoadRunner. It is an industry-leading tool. You can benchmark the volumes on the dashboard and it is easy to use.
I would definitely recommend it for investment banks and certain businesses like OTC derivatives.
I would rate LoadRunner an eight out of ten. It has good documentation. The recording and playback scripts can be easily done. I would like for there to be better integration with other tools so that when you do load testing you can also do a security check.
Which deployment model are you using for this solution?