OpenText LoadRunner Cloud Reviews

We used Load Runner Cloud particularly to do spike testing. 

The license is built such that it can be used across the entire year. The flexibility to use it across multiple applications for the year for various concurrencies and time periods works well for us.

The TCO has been optimized along with the total ROI. The product, being a part of the Gartner magic quadrant, helps score against some of the open-source products out there. This helps as there are certain regulators who look at technology adoption that is a part of the Gartner magic quadrant.

We used Load Runner Cloud particularly to do spike testing and it was great. 

The license can be used across the entire year. They make it flexible so that we can use it across multiple applications throughout the year for various concurrencies and time periods.

The TCO has been optimized along with the total ROI. 

The product being is part of the Gartner magic quadrant and scores well against open-source products. This helps us with certain regulators who look at technology adoption.

There are two features that I would want MicroFocus to work on.

1. It should have a feature to report with a 99.9 percentile success rate.

2. We should be creating a performance dashboard with InfluxDB OR ElasticSearch integration with Micro Focus Cloud LR. We need Micro Focus Cloud LR to send feeds to InfluxDB OR ElasticSearch for each run for:

1) Realtime PT result publishing
2) Trending between runs
3) Data mashing with Grafana/Chronograf
4) Live alerting

The feed from the Micro Focus Cloud LR instance to InfluxDB should be configured, or an integration touch point should be available for sending real-time feeds into the DBs.

I have been using and supporting Micro Focus LoadRunner Cloud for more than two years now. 

Features like spike testing and load per use are excellent features. Users who are looking at very high concurrency requirements yet need it only for a short interval can use the LR Cloud efficiently. The learning curve is very smooth, and there is enough documentation. 

We use a warehouse management system, my company is like a Dutch Amazon. We use LoadRunner to performance test the warehouse management system for peak load. So if Valentine's Day, Christmas, or whatever comes up, we try to run all functional loads through LoadRunner Cloud and make sure the peak count can be handled by the WMS. We use it almost daily. We have peak periods where the testing will increase, but nowadays it's so busy that throughout the year, it's almost the same. It's the same high load every day and we're testing continuously.

The amount of load on the system wasn't that large a couple of years ago for us to test. So the business case for performance testing wasn't pointing to investing in it, but now we have such huge, increasing numbers and increasing load that it's worthwhile setting up a structured and automated performance test and to invest in something like LoadRunner Cloud.

We use the cloud. The controllers and the dashboard servers are in the Micro Focus cloud. But we use our own load generators, which are on-premise. So we use a hybrid setup.

Before LoadRunner, we weren't even testing performance and now we're performance testing. So we now do performance testing as part of the delivery of a new release, which is an improvement compared to a couple of years ago. 

The TruClient feature is the most valuable for us. An application with testing can only be scripted using TruClient, so it's part web-based, but it also has its own protocol combined with HTTP and HTML. So many other tools do not recognize this specific proprietary protocol. Using TruClient, we can still create scripts that cover everything that we need to cover.

It supports multiple protocols, which is important to us. We also use JMeter open source combined in LoadRunner Cloud. In our company, we also use open-source performance testing tools like JMeter, Gatling, and Locust. It's important to be able to embed those scripts as well into a LoadRunner-based performance test. So it would really help us to have a single place for all kinds of test scripts and the open-source ones.

It's also very important to us that LoadRunner allows full integration into the ACI pipeline. We were focusing on a shift left on not only functional testing but also performance testing. And we're trying to integrate the performance testing in our CI/CD pipelines currently. So the better it supports this, the better and easier it is for us.

Reporting and analysis need improvement. Compared to the old school LoadRunner Windows application, the reporting and analysis are mediocre in LoadRunner Cloud.

I've been using LoadRunner since I started at this company one and a half years ago but the company has been using it for longer. 

We don't have any issues with stability. 

When it comes to efficient and cost-effective scalability, it can be expensive. It is efficient, but also expensive. So the balance would then be just okay.

It's used intensively by the three performance engineers, daily. We have plans to have other teams also use LoadRunner Cloud. So we hope it will increase.

We do our own maintenance on the on-premise part and Micro Focus does the maintenance on the cloud part.

We don't call customer support but we do online requests. We have put in service requests. They're helpful and the replies are quick and valuable. It helps us out.

We make use of virtual user hours. We buy time in the LoadRunner Cloud. It costs around $80,000.

I don't think there are any additional costs to standard licensing.

We tried Neoload and looked at JMeter. We chose LoadRunner because that was the only performance testing tool that had something like TruClient.

We're thinking about using LoadRunner Developer integration. We want to hand over the performance testing responsibility from our performance testing guilds to the actual feature teams, to the scrum teams, so they can do it themselves. The developers do their own performance testing in an early stage, and maybe that is a solution for that.

My advice would be to look at whether you need on-premise load generators or fully on the cloud. Invest in training. It can be complex. You need some training to get started.

I would rate LoadRunner an eight out of ten. 

We use LoadRunner Cloud for functional testing, where we can log in online using the portal.

Our primary use case is load testing, where we can simulate a scenario with more than 100 users accessing the system at the same time. This shows us things such as the performance of the system and how it reacts when multiple people are using it.

The most valuable feature is that we do not have to accommodate the load-testing infrastructure in our own data center. Normally for load testing, you have to have your own hardware and tools on-premises. With LoadRunner Cloud, the testing will be done remotely using their resources.

There are three modules in the system that are different products packaged into one, and they can sometimes be difficult to figure out, so they should be better integrated with each other.

The current support model is something that can be improved.

We have been using this product for five years.

This is a stable product and as far as we're concerned, there is no other choice on the market.

We are just in the process of scaling now. Our intention is to expand but it will take time to do so. We are using it for five projects right now and we need to acquire more licenses.

At this time, Micro Focus is handling the support, which is kind of messy compared to going directly to HP. There is no single point of contact between sales and support.

Sometimes, if you open a ticket with technical support then we might be waiting for a week. It is not always this way, but in general, it is not what we would expect.

We also use HP ALM for functional testing.

The initial setup of the cloud version is easy compared to using it on-premises. In the on-premises deployment, you have to set up all of the servers starting with bare metal and it will take no less than three months to deploy. However, with LoadRunner Cloud, all you have to do is buy the resources and execute it from there.

There is no monthly or yearly cost but rather, the fees are based on the amount of traffic that you use. It is difficult to predict the costs until after you have begun using it.

My advice for anybody who is implementing LoadRunner Cloud is to make sure that when you buy the package, it is comprehensive. All of the modules should be there, including the support package. Otherwise, it will be hard to use.

In summary, all of the features that we need are there and it does everything that it is supposed to it. That said, the integration of modules and support are still in need of improvement.

I would rate this solution a seven out of ten.

We use the solution primarily for executing the load tests.

We are able to see the analysis component, and it automatically generates a report from the tool itself after completing the load test. I like that feature. It has great automated reporting features where the report will be generated from the tool itself.

It's user-friendly and it supports a lot of protocols.

The solution can scale.

The stability is good. 

While executing the test it automatically refreshes and I need to see the downwards component, however, it's not allowing me to see that. since it's automatically refreshing, I'm unable to see all the lines in that tool that we are seeing in the editor window.

We did have some challenges with the initial implementation.

I've used the solution for about one year.

I like the product. It's very stable. There are no bugs or glitches. It doesn't crash or freeze. It is reliable.

We can scale the product as needed. It's very easy to do. 

It's also good for any size of the company, from small to enterprise. 

Right now, I'm the only person that uses the solution in our organization. However, we use it to execute the tests with 800 users.

I use it daily. It's used for a quarterly test. For every quarter we'll be executing the testing required.

Technical support has been good. We find them to be quite helpful and responsive. 

Positive

We initially had some problems with the setup as we had an issue while configuring load generators with the tool. In the end, everything went well. We raised it with the Micro Focus team, and they resolved our issue.

I'd rate the initial setup a four out of five in terms of the ease of implementation. 

We handled the initial setup in-house. We didn't need any outside assistance. 

I'm not aware of the cost. I don't handle that aspect of the product.

My understanding, however, is that it is a little bit pricey. 

We have looked into other tools and find that this tool, although expensive, has many useful features. 

We are just a customer and end-user of the product. 

Even though the cost is expensive, there are good features when compared to all other tools. I'd recommend it to others as something they should use. Of course, it does depend on the applications a company uses. 

I'd rate the solution nine out of ten. 

I have been using it to analyze the performance of various enterprise systems like SAP, web applications, Oracle Labs, and mobile app applications as well. The objective of the performance testing is to assess the system and whether it can withstand an actual load.

One of the main benefits is that we don't need to worry about load generators. Before we had Loadrunner Cloud, we had 10 to 20 load generators, and we needed to maintain them. There were always upgrades. We would also have to configure the load generators when we were ready to test, and at times there was stuff to clean up. None of that is necessary anymore. We can concentrate most of our time on creating scripts and configuring Loadrunner based on our objectives. It's very easy for us. We don't need to worry about what happens in the backend or how the load is distributed.

A second advantage is the reporting system. Once a test is done, even if it's in continuous integration and continuous delivery, it automatically triggers and produces reports. We can send the reports to multiple dev teams whose developers are expecting the reports.

Another advantage of the solution is seen compared to others we have used, like JMeter, and IBM solutions. LoadRunner has reduced a lot of the time involved in the scripting cycle because of TruClient. We only need to record one time and we can configure it accordingly to create various scenarios. It has reduced scripting time by about 50 to 60 percent.

When we do mobile applications, every build can be tested with LoadRunner Cloud. Once the developer checks into a particular branch, we call it a mod branch, the performance test is already integrated into it. They can trigger it through their continuous pipeline. For mobile development platforms, it's very crucial because notifications and other things need to be tested before we send to prod. Notifications are very load-sensitive because they go to every user—it could be 10,000 or 20,000 users—and we previously had issues with this. Using LoadRunner Developer and continuous integration has helped us.

Recently I have found the TruClient tool to be very useful. My team is involved in a lot of performance testing of applications including mobile platforms and different types of web browsers. In those cases, LoadRunner TruClient is very crucial to testing the performance. TruClient is one of the best features, one I use every day to create load scenarios.

The fact that the solution supports multiple protocols such as open source, VuGen, TruWeb, TruClient, and SAP is very important because these protocols help us to concentrate on what is really needed to produce performance tests. If something is not supported, you have to use other tools or find other ways of assimilating loads. For example, when you are trying to create loads for web applications, if it's not TruClient, you need to find and analyze every call: What the HTTP request is, and what the other kinds of requests are that we need to call, and then correlate all the correlation identifiers. All of that is taken care of by TruClient and other protocols. That is actually very beneficial across all the platforms. For example, a SAP application might be tested for mobile. We can use a combination of the SAP protocol and TruClient protocol. The combinations are very helpful for performance testing.

LoadRunner Cloud gives you a lot of options, even for multi-browser or multi-device testing. It has been the main tool that can do everything; complete end-to-end performance testing.

The support for large-scale testing is also an important feature in our operations. We have thousands of users and it provides the best solution. You can have an unlimited number of users, although you would need to pay for them, but that's a different story. In our organization, the maximum number of users is currently about 20,000 to 30,000. It's a one-stop solution. I can configure my load on the cloud environment and have 30,000 virtual users on the cloud. I don't need to create the infrastructure locally and I don't have to maintain it. Everything is taken care of by the solution.

In addition, its ability to run unlimited concurrent tests without worrying about hardware availability helps eliminate hardware dependency. You don't need to have the load generators on your network. You don't need to maintain those systems and you don't need to have that kind of network capability. If you're testing on-prem, but you don't have the network capacity to scale up to millions of users, LoadRunner Cloud enables you to create virtual networks and use the cloud to generate those kinds of loads. You can then analyze what the impact will be to your system when you have millions of users. LoadRunner Cloud is the best way to do that.

When it comes to the CI pipeline, there were some limitations initially, but the latest version of LoadRunner is very helpful. They can integrate into the CI/CD pipeline. We are trying to put it into a complete CI/CD pipeline, but there are still some challenges when you try to run it through different protocols. The challenges are around how you can containerize applications. There are some limitations to some protocols, such as desktop. And when it comes to database testing, there are some things that we can't do through CI/CD.

For CI/CD, the previous versions may not be the right ones, but the latest version is definitely a step ahead. We are aiming for 100 percent, but we have achieved around 60 to 70 percent in CI/CD. Still, it's very good to have that capability.

Also, it would be helpful if Loadrunner Cloud had the same kind of enterprise environment where we had multiple models and options while creating the load profile. Not all the options are available in the LoadRunner Cloud. If they could be added, it would be good.

I've been using a LoadRunner solution for more than 10 years as part of my performance testing.

The efficiency of the scalability is a 10 out of 10. I have used multiple tools and LoadRunner is the best one in terms of efficiency. When it comes to cost-effectiveness of the scalability, I would give it an eight out of 10. Even though it's cloud, and you can have thousands of users, we are paying in the tens of thousands. It's not so cost-effective for a university like ours. We still have to justify why we need to spend so much money every financial year.

I have used OpenText technical support a couple of times and they have been very supportive. They're good.

The only problem with setup is that there might be some problems with the firewalls, configuring SSH and other things. We were initially using OpenText SiteScope but we had some issues getting all the metrics. With New Relic and Dynatrace, we stopped using SiteScope. Other than that, we didn't have any issues. It's pretty straightforward. You install LoadRunner, configure your virtual users, and create the environment.

Our initial deployment of the on-prem solution took a week to get everything done, including setting up the firewall, configuration files, and the protocols. The migration to Loadrunner Cloud is nothing. You can start triggering whatever code you had on-prem in the cloud. There really isn't any migration involved. It's pretty straightforward.

We have definitely seen return on our investment with Loadrunner Cloud. As I noted, we used to have many load generators and they are no longer used. That is saving us about $30,000. And Loadrunner costs 50 percent of what it used to cost us to run the same test in the cloud. We don't need a Windows Server license anymore or networking capabilities specifically for testing. Those are the kinds of savings we have seen from moving to the cloud.

Also, in the past, we used to write code. But with TruClient, while you need a performance tester, you don't need a programmer to write scripting. If you know the system, and if you know the objectives of performance testing, you can do the performance tests. No programming skills are needed. That also gives us leverage. We can use someone with performance testing capabilities, even though he might not be the best programmer. That has also reduced our costs by $10,000 a year.

I have extensively used JMeter as a performance testing tool. JMeter is free and also enables codeless scripting. Even without coding you can try running scripts. But where there is no comparison between JMeter and LoadRunner is when it comes to distributing load. LoadRunner stands out. With JMeter, it's very difficult to distribute the load. 

When it comes to creating reports, Loadrunner is best. You will spend most of your time analyzing what's happened with the test, analyzing bottlenecks and pain points with the performance parameters. But in JMeter, you have to manually collect everything: collate the results and produce the reports. Then you need to do a detailed analysis to find the bottlenecks and resource patterns. It is very difficult, but it's free. If you have the skill set and the time, you can use JMeter. But if you are time-constrained, and you want to actually concentrate on performance testing, use LoadRunner.

LoadRunner Cloud provides application performance and management tools to an extent, but not to the extent of New Relic or Splunk. We predominantly use New Relic to monitor application performance and in some cases we use Dynatrace as well. But LoadRunner Cloud doesn't have complete application performance monitoring metrics.

LoadRunner's Developer integration enables developers to script and run tests without leaving the developer ecosystem. It gives a complete IDE where you can develop the code and add your script. For example, if you are using a Java platform to code, and it has all the libraries and the IDE, you can integrate your load testing into your development. For us, because we don't have a single development cycle or ecosystem—we constantly move to various methods—that's where the IDE has limitations. It supports certain languages but it doesn't support everything. If we are using Go, for example, we might need to add certain libraries, so that's where it isn't helpful. But if you are purely on Java or core platforms, it will definitely help.

The Developer integration enables developers to add performance testing to their day-to-day tasks, but shift-left depends on your company's development strategy. If your whole culture supports shift-left, so that your quality assurance is embedded from the start of the development cycle, then shift-left works out. But in our case, we use a lot of packaged applications like Workday or SAP where we don't have much opportunity to work with their code. We do a lot of configurations and integrations. In that case, shift-left doesn't work as well. But whether you follow shift-left or shift-right, Loadrunner Cloud works.

For non-functional testing, LoadRunner is the best tool. I can recommend it to other people, to create specific tests from the smallest load to the highest level of load. I don't think any other commercial tool has that capability to create load performance testing. There is no other tool that gives this kind of experience for a load-testing professional. From end-to-end, starting from creating the load scenarios, to running them, and then reporting, LoadRunner is the best tool. You save a lot of time and, with LoadRunner Cloud, you are saving a lot of money. Go for it.

It is all about load testing. We have to do a lot of load testing in our IT environment. In the investment banking space, you do a lot of end-to-end endurance testing, to see how to benchmark performance based on resources and the CPU memory. So, we will have to benchmark the performance based on trade volumes. Trade volumes are the basic KPI.

Endurance testing needs to be done to see how many trades can be pumped in my environment. I do endurance testing. 

better decision making was done Business KPI & technical KPI can be set by automating loy of endurance testing where given resources thresholds capacity can be measured against  business kpis

The record and playback feature is the most valuable feature. It's all driven by the script, so it's a script-based tool where the background tracing starts. Java's background process does a lot of tracing. The process starts in the background. It sees what peaks of volumes the process can handle. It's easy to use because it's script based, record and playback. I play certain tests. I record those tests, see how they pass through the entire environment, and see how many breaks I get. 

Then we repeat that for multiple numbers. If the first time, I pumped in 1 million trades, and get 5% of the failures, then I try with 2 million trades and I see that there is a 7% failure. These scenarios are easy to do in the LoadRunner. LoadRunner has been the best tool for more than 20 years in the market.

You can do endurance testing, you can do load testing. These are the different types of load testing. It lets you see how much volume you can benchmark in your entire environment, given the resources like CPU and RAM. 

The record and playback is the best feature. You need to use a console. You log in to your console and see the dashboard with how many trades failed and how many trades passed, what the peak loads endurance load were. 

The performance has really improved in terms of running test cycles. The product used to crash on-premises and when it had a lot of trades being pumped in. Because it is memory intensive, it used to crash if it was running out of memory. That was the limitation of the on-premises thing. Running those states and cycles in the cloud is much faster. 

Everything is frozen in the cloud. The RAM, CPU, compute, and storage are provisioned in the cloud, which is becoming easier for running these test cycles. Test cycles are highly effective. Of course, you need to have a test strategy, like volume-based load testing. Configure some test cases and run those test cases in cycles. Cloud performance is much faster. Volume-based endurance testing is easier in the cloud.

It is an old solution, around 20 years old. It was called once Mercury LoadRunner and then it was acquired by HP, it became HP QTP. That was around 2007, it was known as Hewlett-Packard HP QTP. Since then it has been acquired by OpenText.

It is very stable.

We have around 100 users, they are functional experts who use this for load testing.

The people who have been responsible for setting the KPIs will decide if we will expand the solution. They are the end-users. It will depend on the dashboard and the results of the load testing.

LoadRunner is a very old product in this industry. It was a little difficult to get to the right support team. Now there's a professional service group that is better than they were before. 

If scaling becomes a problem, then we have to involve the support, the right kind of support to actually fine-tune and troubleshoot the entire configuration. But with the cloud version, that risk is minimized. It has adequate storage available.

It's a subscription-based, pay-per-user license. I wouldn't say it's expensive. 

Because it was an on-premise license, it was a costly affair. Only certain stakeholders could use it. Now, it is enterprise-wide, we can have any number of cloud users performing the load testing because it is pay-per-user.

I would recommend LoadRunner. It is an industry-leading tool. You can benchmark the volumes on the dashboard and it is easy to use.

I would definitely recommend it for investment banks and certain businesses like OTC derivatives.

I would rate LoadRunner an eight out of ten. It has good documentation. The recording and playback scripts can be easily done. I would like for there to be better integration with other tools so that when you do load testing you can also do a security check.

The best benefit would be budget. It's a really inexpensive if you are testing your application every spring or every month. Even for less frequent applications that you have to test regularly. You can hook up your infrastructure to StormRunner Cloud and you can get the best of both worlds.

In larger enterprises, we have different departments using Performance Center and StormRunner or Selenium. So, StormRunner can act like an umbrella and plug-in everything, get it executed or done across the world. Since they use both HPE Cloud, AWS, and Microsoft, it uses more access points geographically to test it making it best of the class.

StormRunner itself is a pretty good hybrid product of Performance Center. Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore. Whenever we have to test lab scale applications, and in a scenario where we don't have to test every day, we don't have to build the machines and pay for it. Instead, we can get the infrastructure from on-demand from StormRunner, and its ability to run it anywhere just by opening it in browser is the best part. The evolution of StormRunner starting with supporting LoadRunner along with the open source technologies like JMeter and Selenium. Their unit testing tools are actually very advanced in the region of the product. That's why I would recommend anyone to use StormRunner, even though something is not supported now, eventually it's going to be supported by StormRunner. That's the kind of credibility that's needed for any customer when it comes to relying on a product or going for a new product.

More insight into test results and allowances. It might be a tailor-made requirement for me, but I would like to download them offline and do my own customizations on the reports. Right now, we have some standard templates that generates reports. But if I had to do some customizations, include something else and create a report, it's not easy. So, if I can an order to download the raw data I can make custom report. That would be the case with every customer because even though they tested one product it might be part of a big project and they need to have other information included in it along with the report, so feasibly it is good to have.

Also, it's evolving, where there's too many features for me to handle and it's too much on my plate to any make ROI out of it.

There is a steep learning curve for the product, too.

Over the last couple of years, it's been evolving. We started using more in the last one year and we see it's pretty stable. Adding new features to StormRunner is going to slow down once it has every feature. That has to do with stability and I don't see any drawback in that.

We haven't really hit any scalability issues by using StormRunner, but I just now heard that they can support two million users, which is kind of astonishing. The max I have used is 40,000 users, but two million is good, too. It's big step up from 40,000.

At the same time, you don't have to get the infrastructure built and set it up. For example, let's say you had to test it for one million users. You don't have to procure all of the machines you need for the one test you're going to do. You can do it on demand, that's the best thing. Two million is still very much overboard, but it's good to know. It's good to know that there's no limitation nearby.

I'm not a big fan of the tech support to be frank. SaaS tech support (HPE tech support or Micro Focus), there's a gap between the people who access our tickets and the people who know the product. So, there is always a blind exchange of information within Micro Focus and most of the time get frustrated with the kind of ticket updates we get. What happens is the person handling the ticket might not be an expert in the product so we end up redoing everything. Communicating everything about our setup and infrastructure and the customer engagement for each ticket. Therefore, we started involving our technical correspondent with Micro Focus, but we're told that it's a pretty messy situation over there.

I always thought it would be good to have something like this from HPE because we rely a lot on HPE. Then, StormRunner was released and I knew where exactly it was going and what's it would be for us.

I was involved in the initial setup. It was straightforward and there was a lot of good information available, but I did not need it. I didn't go through any support to set it up. The documentation itself was good.

The other product we considered was BlazeMeter. We eventually chose StormRunner. I think only those two are pretty much in the market - nothing else.

I like that StormRunner incorporates the idea of accepting and adapting all open sources. It is my understanding that they are planning to continue accepting, supporting, and adapting all open sources.

For someone evaluating StormRunner and similar products, there are two parameters I would tell him to evaluate:

1. Is the application under test? Is it customer facing?
2. Is how often do you test it?

Most important criteria when selecting a vendor: We are a retail business. The first thing I look for is any customer use case, or case study for the vendor. I look to see if they have proven that they are good resource for any existing retail customers. I look for any kind of case study in those lines before I jump in.

The solution is used for load performance testing activities.

The product supports a wide variety of technology compared to any other tool.

The product must improve support. The support team provides delayed responses. Many times, there are a lot of conversations between the support team and the technical team without any conclusions.

I have been using the solution for 14 to 15 years.

The tool is not perfectly stable.

Around 30 to 40 people use the tool in our organization.

The installation is not difficult, but it is a pretty lengthy process.

The product must improve its price. The solution’s price is considerably high. It is the highest in the market.

My company decided to use the product due to the technology support it provided. I would recommend the solution to someone with technology that other tools couldn’t support. Overall, I rate the solution a six out of ten.