What is most valuable?
LoadRunner is one of the most versatile performance testing tools. One thing I really like about LoadRunner is its breadth of coverage. The number of protocols that it supports, and especially, for example, when it talks about SAP GUI-based performance testing. I could not think about any other tool other than LoadRunner. I've also seen LoadRunner evolve over a period of time, in the sense that it addresses new technologies. I would say it's still a very strong product and there's a lot of new products out there.
How has it helped my organization?
Think about the performance testing as a part of the whole of your testing lifecycle. It's our go-to product and that's the only product which we use within the organization right now. Not every project goes through performance testing, but any project that has performance testing is a component of the test lifecycle. It has our de facto standard.
When you think about deploying these applications to production, and our organization being a global company, we have a user footprint across the globe. LoadRunner, along with HPE Envy Network Virtualization: It's a combination where we, not only run business processes in volumes, but also similar to these network conditions, helps us identify some performance issues well in advance. It's one of those risk verifications that we do.
What needs improvement?
One thing I was desiring a few years ago, before StormRunner came in, I was hoping HPE would come out with some of the new tools that are cloud-based, SAS-based performance testing solutions. With StormRunner coming out, it has probably filled that gap.
You should be able to use LoadRunner as a single platform. You should be able to have browser based access. You should be able to run enterprise tests. HPE addressed a little bit of this by taking over the license for the controller and tying it to the load generators. But in an ideal world, just make it a single platform, everything should be like Performance Center.
For how long have I used the solution?
I've been using LoadRunner as a performance engineer since 2002. That makes it about 15 years now.
What do I think about the stability of the solution?
It's such a mature product. It's been out there for a very long time. It has come a long way. It's extremely stable in the sense that I can approach the technical support, as an example. It is because we ran into some challenges with program scripting or something, but generally not because of the application crashing. I'd list my experience with LoadRunner as it being a very stable product.
What do I think about the scalability of the solution?
In my previous experience with previous employers working with some retailers, we had to run some extremely large tests, and even at my current company, we go up to about 3000 content users. I think it's fairly scalable. We run very small tests, obviously, with StormRunner. It's can probably scale to much higher levels, but within our boundaries of demands, it's scaled very well.
How are customer service and technical support?
We use tech support, but not quite often, just when we really run into some problems. We have a really strong internal team, which so a majority of the issues are internally solved. We are either experimenting or knowledge base in everything. We do reach out to tech support, but it's very rarely done (rare cases).
Service depends upon the issue we face. Sometimes we go off on a tangent. It's an exception, but we go down the wrong path. Then, it's a long winding road to the actual resolution. In many cases, if it's a common issue and it's fairly quick, we get pretty good support. I've actually seen both.
Which solution did I use previously and why did I switch?
In 2002, was the first time I was actually really involved in bringing the tool in. Ever since, I go to new employers, the tool was already there. I help take it to the next level, mature the practice of what not, but again there's only one instance where I really had to bring the tool in, but everywhere else, we already had a tool in place.
Back then, there was no competition. It was the only choice there. After that, whenever I moved companies, the tool was already there. I was involved in the sense of bringing new protocols, new additional licenses, and so on.
How was the initial setup?
I was involved in the initial setup. It's extremely straightforward. I've probably worked on about six or seven different versions, and it's extremely straightforward and fairly simple to setup.
What's my experience with pricing, setup cost, and licensing?
I would still consider LoadRunner as an expensive tool and you get a LoadRunner and the Performance Center.
Which other solutions did I evaluate?
Once there was situation before StormRunner was out, where we did venture into a tool call. Here I was working for a retailer. We ventured in tool call with SOASTA. We were looking for a cloud-based performance testing solution, and at that time, HPE was not there. Every time I wanted to make a decision, LoadRunner was our default choice and we had nothing else running in contention.
What other advice do I have?
LoadRunner is still my first choice. There's still no competition to LoadRunner. In the sense, there are tools out there. I know without a doubt, very good tools. But again, LoadRunner to me is more versatile and has more breadth of coverage.
Most important criteria when selecting a vendor:
- We look at the product itself. In the sense, the stability of the product, etc.
- In terms of the vendor, we look at the thought leadership and the roadmap. That's something we always look into. Just to see, is the vendor a good investment. For example, are they going to stick around for a long time?
- We want them to innovate. In the sense, if you think about how quickly the technologies are changing. We want the vendor that we are looking into to focus on innovation.