We just raised a $30M Series A: Read our story

Micro Focus LoadRunner Cloud OverviewUNIXBusinessApplication

Micro Focus LoadRunner Cloud is the #4 ranked solution in our list of top Performance Testing Tools. It is most often compared to Micro Focus LoadRunner Enterprise: Micro Focus LoadRunner Cloud vs Micro Focus LoadRunner Enterprise

What is Micro Focus LoadRunner Cloud?

Micro Focus LoadRunner Cloud is a simple, smart and scalable cloud-based load and performance testing solution for agile and DevOps teams. It helps you to detect performance issues and ensure that your mobile and web apps are ready for the load. It provides an Intelligent and predictive analytics for fast problem detection and scale up to millions of user in minutes from different regions.

Micro Focus LoadRunner Cloud is also known as StormRunner Load, LoadRunner Cloud, and Micro Focus StormRunner Load.

Micro Focus LoadRunner Cloud Buyer's Guide

Download the Micro Focus LoadRunner Cloud Buyer's Guide including reviews and more. Updated: October 2021

Micro Focus LoadRunner Cloud Customers

Alfa Bank, N Brown Group, University of Copenhagen, McGraw-Hill, Cognizant

Micro Focus LoadRunner Cloud Video

Archived Micro Focus LoadRunner Cloud Reviews (more than two years old)

Filter by:
Filter Reviews
Industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
Rating
Loading...
Filter Unavailable
Considered
Loading...
Filter Unavailable
Order by:
Loading...
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Search:
Showingreviews based on the current filters. Reset all filters
CS
IT Project Analyst at a tech services company with 10,001+ employees
User
Enables us to check response times for applications from locations around the world

Pros and Cons

  • "The most valuable feature is having load generators in countries where we don’t have access to them."
  • "An area for improvement is analytics on why response times are slow from certain countries."

What is our primary use case?

We use it to get response times around the world for external applications. We use it on top of our own virtual users for remote site testing. We include it in our scenarios in our performance center and use the results in the LoadRunner analysis tool.

How has it helped my organization?

We can determine response times for our external applications, from most places around the world, for a small fee.

What is most valuable?

The most valuable feature is having load generators in countries where we don’t have access to them.

What needs improvement?

An area for improvement is analytics on why response times are slow from certain countries. Slowness can be caused by the laptop, the connection, the VPN, the internal network, the country's internet, etc.

What do I think about the stability of the solution?

It has been stable every time I have used it. We don’t put much stress on it since we have our own performance center and load generators.

How was the initial setup?

In terms of integration and configuration of the product in our AWS environment, it is the easiest Micro Focus tool I have ever used.

What's my experience with pricing, setup cost, and licensing?

The pricing is very reasonable and the licensing is straightforward.

You decide on the number of virtual users you will need to support multiple runs or for a year and purchase that quality of virtual-user hours. We chose to procure this solution via the AWS Marketplace because of remote site testing and low cost.

What other advice do I have?

I rate this solution a ten out of ten. It addresses a need in performance testing tools and is easy to purchase and use.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
AK
Senior Performance Engineer at a retailer with 1,001-5,000 employees
Real User
The best feature is that we don't have to build and maintain infrastructure anymore

Pros and Cons

  • "Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore."
  • "There is a steep learning curve for the product, too."

How has it helped my organization?

The best benefit would be budget. It's a really inexpensive if you are testing your application every spring or every month. Even for less frequent applications that you have to test regularly. You can hook up your infrastructure to StormRunner Cloud and you can get the best of both worlds.

In larger enterprises, we have different departments using Performance Center and StormRunner or Selenium. So, StormRunner can act like an umbrella and plug-in everything, get it executed or done across the world. Since they use both HPE Cloud, AWS, and Microsoft, it uses more access points geographically to test it making it best of the class.

What is most valuable?

StormRunner itself is a pretty good hybrid product of Performance Center. Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore. Whenever we have to test lab scale applications, and in a scenario where we don't have to test every day, we don't have to build the machines and pay for it. Instead, we can get the infrastructure from on-demand from StormRunner, and its ability to run it anywhere just by opening it in browser is the best part. The evolution of StormRunner starting with supporting LoadRunner along with the open source technologies like JMeter and Selenium. Their unit testing tools are actually very advanced in the region of the product. That's why I would recommend anyone to use StormRunner, even though something is not supported now, eventually it's going to be supported by StormRunner. That's the kind of credibility that's needed for any customer when it comes to relying on a product or going for a new product.

What needs improvement?

More insight into test results and allowances. It might be a tailor-made requirement for me, but I would like to download them offline and do my own customizations on the reports. Right now, we have some standard templates that generates reports. But if I had to do some customizations, include something else and create a report, it's not easy. So, if I can an order to download the raw data I can make custom report. That would be the case with every customer because even though they tested one product it might be part of a big project and they need to have other information included in it along with the report, so feasibly it is good to have.

Also, it's evolving, where there's too many features for me to handle and it's too much on my plate to any make ROI out of it.

There is a steep learning curve for the product, too.

What do I think about the stability of the solution?

Over the last couple of years, it's been evolving. We started using more in the last one year and we see it's pretty stable. Adding new features to StormRunner is going to slow down once it has every feature. That has to do with stability and I don't see any drawback in that.

What do I think about the scalability of the solution?

We haven't really hit any scalability issues by using StormRunner, but I just now heard that they can support two million users, which is kind of astonishing. The max I have used is 40,000 users, but two million is good, too. It's big step up from 40,000.

At the same time, you don't have to get the infrastructure built and set it up. For example, let's say you had to test it for one million users. You don't have to procure all of the machines you need for the one test you're going to do. You can do it on demand, that's the best thing. Two million is still very much overboard, but it's good to know. It's good to know that there's no limitation nearby.

How are customer service and technical support?

I'm not a big fan of the tech support to be frank. SaaS tech support (HPE tech support or Micro Focus), there's a gap between the people who access our tickets and the people who know the product. So, there is always a blind exchange of information within Micro Focus and most of the time get frustrated with the kind of ticket updates we get. What happens is the person handling the ticket might not be an expert in the product so we end up redoing everything. Communicating everything about our setup and infrastructure and the customer engagement for each ticket. Therefore, we started involving our technical correspondent with Micro Focus, but we're told that it's a pretty messy situation over there.

Which solution did I use previously and why did I switch?

I always thought it would be good to have something like this from HPE because we rely a lot on HPE. Then, StormRunner was released and I knew where exactly it was going and what's it would be for us.

How was the initial setup?

I was involved in the initial setup. It was straightforward and there was a lot of good information available, but I did not need it. I didn't go through any support to set it up. The documentation itself was good.

Which other solutions did I evaluate?

The other product we considered was BlazeMeter. We eventually chose StormRunner. I think only those two are pretty much in the market - nothing else.

What other advice do I have?

I like that StormRunner incorporates the idea of accepting and adapting all open sources. It is my understanding that they are planning to continue accepting, supporting, and adapting all open sources.

For someone evaluating StormRunner and similar products, there are two parameters I would tell him to evaluate:

  1. Is the application under test? Is it customer facing?
  2. Is how often do you test it?

Most important criteria when selecting a vendor: We are a retail business. The first thing I look for is any customer use case, or case study for the vendor. I look to see if they have proven that they are good resource for any existing retail customers. I look for any kind of case study in those lines before I jump in.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Learn what your peers think about Micro Focus LoadRunner Cloud. Get advice and tips from experienced pros sharing their opinions. Updated: October 2021.
541,708 professionals have used our research since 2012.
it_user368772
Technical Sales at a tech services company with 501-1,000 employees
Consultant
It's fast, easy to use, has a user-friendly UI, and you can split users. It also works well with SiteScope and other HP products.

Pros and Cons

  • "It's fast, easy to use, has a user-friendly UI, and you can split users."
  • "It doesn't provide custom reports. You can only use the default reports which contain irrelevant data or is missing data that we need."

How has it helped my organization?

Its ability to work well with other HP products, particularly SiteScope, allows us to maintain consistency in our infrastructure. The different solutions work seamlessly together and free up resources that would otherwise be spent on conflicting vendors and products.

What is most valuable?

It's fast, easy to use, has a user-friendly UI, and you can split users. It also works well with SiteScope and other HP products.

What needs improvement?

It doesn't provide custom reports. You can only use the default reports which contain irrelevant data or is missing data that we need.

Also, a lot of our customers have issues logging into the cloud.

For how long have I used the solution?

One to three years.

What do I think about the stability of the solution?

It's been stable.

What do I think about the scalability of the solution?

It scales fine because our customers are usually larger.

How is customer service and technical support?

Technical support is something we're not so happy with.

How was the initial setup?

The initial setup is straightforward.

Disclosure: My company has a business relationship with this vendor other than being a customer: We're resellers.
it_user739599
Support director of academic systems
Vendor
Helps validate the capacity of our systems as number of users increases

What is most valuable?

It's very valuable because with StormRunner we can validate the capacity of our systems. Specifically, we use it for one system that is from two years ago. We have an increment of the users, about 300% more, so we need to assure that the quality of the service keeps up with the users. StormRunner helps us to validate that.

What needs improvement?

Right now, we are just starting to use it. I don't have any comments on that.

For how long have I used the solution?

Less than a year.

What do I think about the stability of the solution?

Yes, it's stable. We don't have any issue right now with the performance or anything like that.

What do I think about the scalability of the solution?

It's very flexible. We can have load testing for 100 users, maybe…

What is most valuable?

It's very valuable because with StormRunner we can validate the capacity of our systems. Specifically, we use it for one system that is from two years ago. We have an increment of the users, about 300% more, so we need to assure that the quality of the service keeps up with the users. StormRunner helps us to validate that.

What needs improvement?

Right now, we are just starting to use it. I don't have any comments on that.

For how long have I used the solution?

Less than a year.

What do I think about the stability of the solution?

Yes, it's stable. We don't have any issue right now with the performance or anything like that.

What do I think about the scalability of the solution?

It's very flexible. We can have load testing for 100 users, maybe 1,000 or even more, so yes, no problem with that.

How is customer service and technical support?

It's very good. We also have support from our partner, Micro Focus, so it's basically good on that.

How was the initial setup?

No, it was straightforward. It was very easy to implement and to have a load test.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user739593
Snr Performance engineer
Vendor
Allows us to utilize cloud technology, saving time and money

Pros and Cons

  • "This solution is SaaS based so we can utilize cloud technology, which is less time consuming and saves a lot of of money."
  • "Sometimes, you are utilizing one of the low generators, then all of a sudden if you discontinue from one project, it actually deletes the entire low generator."

How has it helped my organization?

This solution is SaaS based so we can utilize cloud technology, which is less time consuming and saves a lot of of money.

What needs improvement?

Sometimes, you are utilizing one of the low generators, then all of a sudden if you discontinue from one project, it actually deletes the entire low generator. You have to go through all the setup again. This kind of problem is on and off. It comes and goes.

For how long have I used the solution?

We have just been using it for about two months now.

What do I think about the stability of the solution?

This is a new product, so there are a couple of flaws, sometimes things act up weirdly. We reach out to the support group, they solve the issues. It's an ongoing process. We understand that.

How is customer service and technical support?

Since this is a new solution, they need to improve the support. They also have some lack of expertise. We understand that.

How was the initial setup?

It is complex. You need to convince the leadership and the IT security team. You need to solve all these issues. There are a lot of things that you have to go through.

Which other solutions did I evaluate?

We chose this one because of the name. It's established, the last decade. We used to be with this company. Micro Focus is more than seven or eight years, we're good with them, they are good with us.

What other advice do I have?

It's a new product and we are facing some issues. We believe, they are going to work out at their end. They are going make it work and fix everything, whatever the flaws are now.

Overall, we are using this to save money and utilize the cloud technology.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user671397
Test manager at a financial services firm with 10,001+ employees
Vendor
We can configure browser settings for different operating systems and on different versions without the need to install every version on each machine.

Pros and Cons

  • "The most valuable feature is the ability to configure browser settings for different operating systems and on different versions without the need to install every single version on each machine and to manage them."
  • "I'd like to see more ability to dive more deeply into the configuration."

What is most valuable?

The most valuable feature is the ability to configure browser settings for different operating systems and on different versions without the need to install every single version on each machine and to manage them. This is the most useful feature that I'm interested in.

I'm also interested in mobile center services related to StormRunner functions because we are going to determine which will be the best solution for developing mobile testing. And we are interested in outsourcing mobile testing, so this kind of environment will be useful as we are not engaged in the configuration of infrastructure. Also, I think it will be useful because Storm Runner functionality does not need specific configuration because the configuration basis is in the UFT and the UFT can be in our environment. We do a lot of configuration because we develop a custom framework related to UFT. So, I appreciate that configuration can be managed by us, while the browser can be released on the remit version.

What needs improvement?

I'd like to see more ability to dive more deeply into the configuration.

What do I think about the stability of the solution?

I think the product is still immature because we have some problem reaching the mobile center connection related to a proper configuration on our machine. In fact, we are in touch with the R&D team to fix the configuration for our environment. I also think that it will be useful to have an object recognition tool because the specific configuration on the browsers are to be managed so as to fix our specific test. I need to recognize what was the configuration on the remit browser and I need a tool to analyze the impact of the browser configuration on the test.

What do I think about the scalability of the solution?

I think the scalability is fine with this kind of solution because I have no problem with how many machines will be available to run the test; it's a matter of the cloud provisioning. It will be useful to run the tests in multiple environments in parallel. That will be a good tool for us when we have to make tests with a shortage of time.

How is customer service and technical support?

I have not used technical support recently. HPE support is always prompt to solve the issues. That is the main thing I appreciate regarding support.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user567969
Test Architect at a engineering company with 1,001-5,000 employees
Vendor
It's easy to set up a load test, to start it and end it. It needs a lot more LoadRunner Analysis features.

What is most valuable?

It's in the cloud. Everything goes in the cloud. We needed something that's easy to set up in the cloud so we can use it directly there.

It's very easy to set up a load test, to start it and end it. It is valuable that it is very easy to use.

How has it helped my organization?

We are not benefitting a lot at this time because it has its bad parts.

What needs improvement?

It needs more features; a lot more features. Everything that's in LoadRunner Analysis should be built up in StormRunner; so we can use everything, every functionality and every feature all together.

What do I think about the stability of the solution?

It's stable. It's not perfect, but it's stable.

What do I think about the scalability of the solution?

It is not really scalable.

How is customer service and technical support?

We didn't use it that much. It was a demo session set up especially for us.

How was the initial setup?

It's very east to set up. I did it directly myself just once.

Which other solutions did I evaluate?

We compared it to LoadRunner and Performance Center because we use that currently for load and performance tests. We cannot use this solution in the cloud. StormRunner is very rudimentary at this time, but it's a great concept. We hope that it will be thrilling.

We chose StormRunner because of the cloud. That's a great part of it.

What other advice do I have?

Wait with StormRunner.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user426591
Lead Performance Architect at a tech company with 1,001-5,000 employees
Vendor
Scripts developed in JMeter can be uploaded instead of spending time creating new ones. It needs to be configured with SiteScope or AppDynamics.

What is most valuable?

JMeter script integration. This integration feature is a good approach provided by HPE. Sometimes, we need to run scripts developed in an open source tool like JMeter. Instead of spending time on creating new scripts, the scripts developed in JMeter can be uploaded and based on the protocol license because it is a DEV user protocol, it provides a valuable feature when integrated with StormRunner.

What needs improvement?

Monitoring and sample access to servers needs to be provided, so that we can explore this and keep up-to-date.

The licensed version / free edition needs to be configured with SiteScope or AppDynamics, which many customers might not have. The LoadRunner feature of Windows Resource Monitor should come in handy in such scenarios. It is not only for Windows-based monitoring but also for UNIX-based monitoring. Dependency on the other tools or HP tools (SiteScope) would be reduced.

It needs to be more mature.

The client breakdown is a separate report and is not part of the existing load test report. The client breakdown is basically the response time observed from different browsers. We need to take a snapshot and place it in the load test report and that is not available when the load test report is generated from StormRunner.

Also, for the VuGen Web protocol, it supports only 12.50 and above, which is tough and compatibility is lost.

For how long have I used the solution?

More than a year and half

What was my experience with deployment of the solution?

I did not encounter any issues during deployment, but there are not enough tutorials available for the TruClient protocol and also on the reporting side.

What do I think about the stability of the solution?

I have worked extensively on the various module supported by STRL2.1v and provided my suggestions/improvements/recommendation to the HP R&D Team

How is customer service and technical support?

Customer Service:

Good

Technical Support:

The level of customer service / technical support is satisfactory, with a few times good.

For example, when trying to learn the monitoring feature of StormRunner, I requested access to the Web Tours application that is installed on the server so I could explore that feature in depth. However, access was denied (monitoring-only access was provided). My question was: When both the application and the tool belong to HPE, why was access denied?

What other advice do I have?

The end user needs to have strong skills in any of the load testing tools, specifically if they already have experience with LoadRunner.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user469167
Solution Architect at a tech company with 501-1,000 employees
Vendor
We use it for spike testing before an application goes into production or before a big event.

Valuable Features

We've seen in the past where we have customers that use other performance testing products such as LoadRunner and Performance Center that they often have the need to do spike testing. Their testing for 1,000 virtual users running at the same time but before the application goes into production or before a big event, they need to scale up to 10,000, 20,000, 50,000, and so forth. We want to make sure that they can test to those specific limits and StormRunner allows us to do that.

Room for Improvement

I think the biggest issues that I've seen, and this is a personal view of mine, is that most of the HPE products have a common look and feel to them. I'd really like to see it be a little more customizable to a use and user standpoint. For example. I happen to be colors blind, so I'd like to see more vivid colors on the UI, and things like that. It would make it a little more flexible from the customer standpoint.

Use of Solution

I was formally an HP employee, so I've used the HP products in the past. I also was a former Mercury Interactive employee which was acquired by HP.

Scalability Issues

It's been extremely scalable as far as the testing that we've done with our customers. They've all been really satisfied with the scalability of the HPE products.

Customer Service and Technical Support

We've gone through the standard customer support and it’s been good. It's not quite to the level that an R&D would be, because they really get really under the covers to fix things.

Other Advice

It really doesn't need to be as flexible as some other solutions because it's right out of the box, it's relatively easy to use. You open up, you create a test in ten minutes and then you're able to run it.

Disclosure: My company has a business relationship with this vendor other than being a customer: We're partners.
it_user468132
Consultant/Partner at a tech consulting company with 501-1,000 employees
Consultant
Virtual user hours is much more dynamic than virtual user days or static licenses.

What is most valuable?

It's the license and structure because it's so dynamic. Virtual user hours is much more dynamic than virtual user days, and even more dynamic than static licenses. I did a test a short time ago where we could keep the test to half an hour so we could really optimize this virtual user hour. That was wonderful.

How has it helped my organization?

It's used to improve our customers, but we don't use it in-house.

What needs improvement?

I'm a consultant and this was used for a small project. My customer bought around five consulting days from me. We ran into some undocumented limits around the number of concurrent users. It was really just set up to keep the guys out that wanted to do denial of service using the demo account. It was not clear that this was the limit it would hit. That's expensive for a company. Out of five days, maybe half a day. That's 20% of the budget went into trying to run a test. The test fails, contact support, ask what's going on. Make them move the limit and then re-run. That was bad. It should just be upfront - these limits exists, be sure before you start that you don't want to move them.

What do I think about the stability of the solution?

Quite good.

What do I think about the scalability of the solution?

I trust it. I haven't tried it with thousands of virtual users. As true client, yes of course. We had arl 6000 true client users. That's 60 gigabyte memory. It's not a PC, so it's not large.

How are customer service and technical support?

Quite responsive.

Which solution did I use previously and why did I switch?

We were previously using LoadRunner for our customer, but we needed something that could be used in the cloud. I advised the customer to buy StormRunner Load as given the time frame it was clear from the beginning that we needed to use the true client scripts. We didn't want to set up the memory foot print for them so we had to go to cloud.

How was the initial setup?

It was difficult because the Amazon stuff didn't work. I don't like to use Microsoft stuff. I was forced to put it onto Azure Cloud because that was the one that just worked out of the box. The Amazon launching died somewhere in the process. There was another error there as well.

Which other solutions did I evaluate?

I'm an HPE partner. I don't look anywhere else.

What other advice do I have?

For all the in house stuff where we run into firewalls, it just gets bad. For public websites, it's perfect.

The flexibility makes it a viable solution.

Disclosure: My company has a business relationship with this vendor other than being a customer: We're partners,
it_user468114
Senior Performance Analyst at a educational organization with 1,001-5,000 employees
Vendor
The reason we chose it and the most valuable feature is the amount of virtual users we can run in a test.

What is most valuable?

The reason we chose it and the most valuable feature is the amount of virtual users we can run in a test. We have run up to 900,000 users.

How has it helped my organization?

We do online testing for the States as we have contracts with them. We have to be able to support a certain number of students, so this helps us confirm and validate that we can support the number of students that are taking online testing at one time.

What needs improvement?

I think the reporting could be better. I think you could have a little more variety of reports. Right now, you get an email saying, "Here's your report". You can't put in the analysis tool. You can't export it to Excel. Those are the things that would make it a little better. That's probably my biggest pain point, it is the reporting.

What do I think about the stability of the solution?

I think it's fairly stable. There's some features that I think could be made better but, for what they have, I think what they have works well.

What do I think about the scalability of the solution?

I think it scales really well. I like that you can add users on the fly or remove users on the fly. That's a great feature.

How are customer service and technical support?

I have used the support. We were one of the first users for Beta, so we have a relationship with the developers, so we tend to just go right to them when we have questions or problems.

They've been great. A lot of the times, it's not really a problem with StormRunner, it's just our understanding of what it's supposed to do, so they are very great about explaining the reasoning and what will be coming in the future. If we don't think that's really what we want, they'll explain how it's in the roadmap and how it's going to be made better.

Which solution did I use previously and why did I switch?

We were using another HPE product, Performance Center. We haven't totally switched, but StormRunner Load is our option for the test where we need a lot of virtual users where we can run it in the cloud. Whereas, on Performance Center, you have to have load generators in-house, or what HPE provides. That's why we use it. It's for when we want to use a lot and perform a big test.

How was the initial setup?

I wasn't involved in the deployment.

Which other solutions did I evaluate?

Not really because we're mostly an HPE shop and this was working with HPE they to meet our needs, so they provided this and it's worked out great.

What other advice do I have?

I don't like their reporting. Other than that, I think it's great. It does what it's supposed to do and I know it's just the beginning, so there's going to be a lot more features added.

You should look into it. I know there are other solutions that have more bells whistles, but,when it actually comes down to function, I think StormRunner does the job.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Performance Test Architect and HP ALM Expert at a tech services company with 10,001+ employees
Consultant
I have been able to run several tests from outside the corporate LAN which in the past used to be an infrastructure nightmare.

What is most valuable?

  • Ability to get started on the fly
  • Reusability of existing HTTP/HTML scripts

How has it helped my organization?

I have been able to run several tests from outside the corporate LAN, which in the past used to be an infrastructure nightmare as it required opening up ports, disabling firewalls, etc.

What needs improvement?

It needs the ability to execute other protocols, especially the TruClient for Firefox without having to convert them.

For how long have I used the solution?

We've used it for over a year.

What was my experience with deployment of the solution?

There have been no issues with the deployment.

What do I think about the stability of the solution?

We have had no stability issues.

What do I think about the scalability of the solution?

It's scaled to our needs.

How are customer service and technical support?

Customer Service:

Customer Service is immaculate.

Technical Support:

Tech Support is immaculate.

Which solution did I use previously and why did I switch?

There was no solution in place previously.

How was the initial setup?

There was no set-up required.

What about the implementation team?

It's done in-house, but there's no actual implementation needed.

What's my experience with pricing, setup cost, and licensing?

There pricing and licensing options are very aggressive, but the holiday discount brought down the price further.

What other advice do I have?

I think that StormRunner Load is the way forward and if HPE manages to support all protocols currently supported by LoadRunner/Performance Center eventually in StormRunner, it would be a massive game changer.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user230619
Performance Architect at Gallop Solutions
Vendor
Enables developers and testers to quickly automate the user actions at the browser level, which also helps measuring the end user experience by measuring the browser rendering times.

What is most valuable?

StormRunner Load reduces the overall time for performance testing engagement and is best suited for the products in agile process. Easy to automate and execute the tests, StormRunner Load provides a feature to compare the current test results with the previous results to see if the current build is better than the previous or not.

With the ability to integrate with monitoring tools like SiteScope and APM tools like New Relic, performance bottlenecks in the applications can be identified very quickly and easily.

Protocol Support

StormRunner Load supports Web, Web Services, SAP Web and JMeter scripts

Easy Automation

StormRunner Load uses Truclient and TruAPI (which uses node.js) which enables developers and testers to quickly automate the user actions at the browser level, which also helps measuring the end user experience by measuring the browser rendering times as well. Performance testers can upload their LoadRunner scripts that are developed using LR 12.XX to StormRunner Load directly.

High Scalability

StormRunner Load is highly scalable and supports upto 1 million concurrent users.

On Premise Load Generators

StormRunner Load introduces the concept of On Premise Load Generators to compare the application performance within the network to various other geographical locations.

Configure SLAs

SLAs can be configured in StormRunner Load during the test execution. The tool highlights all the transactions that did not meet the SLAs and further drill down can help the users understand the geographical locations and times where the SLAs are not met.

Real time anomaly detection

During the test execution, StormRunner Load detects any anomalies where the test results are deviated for a specific transaction / geographical location / script / counter to identify the issues real time and further drill down provides more details about the anomalies.

Integration with Monitoring Tools

StormRunner Load integrates with monitoring tools like SiteScope to provide the hardware and application level metrics during the performance tests. StormRunner Load can also integrate with New Relic and other similar APM tools that helps identifying the server side bottlenecks so that organization can reuse investments

How has it helped my organization?

HP StormRunner Load has provided us a) flexibility for on demand load generation b) reduced over all TCO on our load testing efforts c) Reduced the skill level to develop load test scripts.

What needs improvement?

Custom load patterns for the performance tests: Currently StormRunner Load supports only linear ramp up of the user load and doesn’t provide the user load to follow a step incremental load pattern or any other pattern that a performance tester wants to.

Load distribution among on premise LGs: StormRunner Load currently supports load generation from only one of the On premise load generators and the user load cannot be distributed to multiple on premise LGs.

Simulation of latencies: There is no provision to simulate network latencies to simulate realistic geographical locations and mobile networks.

Ability to create custom reports: StormRunner Load provides only the default reports and there is no provision available currently to generate the custom reports

Backward compatibility: StormRunner Load expects the LoadRunner scripts that are generated from LR 12.00 and above. Any scripts developed using the older versions are not supported.The work around to overcome this problem is to open the scripts in LR 12.00 or higher and save it again before uploading to StormRunner Load..

For how long have I used the solution?

We have used the product for more than 14 weeks now and tested on some of our internal applications in the organization that would require performance testing.

What was my experience with deployment of the solution?

HP StormRunner Load doesn’t need any installation or deployment unless you wanted to configure few on premise load generators. Configuring on premise load generators is pretty straight forward. All that you need is to have an account with HP SaaS to start using StormRunner Load.

What do I think about the stability of the solution?

HP SaaS web application is pretty stable and no issues were seen during the evaluation process.

What do I think about the scalability of the solution?

There were no issues with the scalability, however, the time taken to initialize the test is directly dependent on the number of users to be simulated.

How are customer service and technical support?

Customer Service:

5/5.

Technical Support:

The support provided to us very good which helped us to evaluate the solution for our context.

Which solution did I use previously and why did I switch?

We are in the process of adopting a SaaS based option, this led us to evaluate a bunch of tools.

How was the initial setup?

It is simple - just registering with HP SaaS platform called pronq.com.

What about the implementation team?

We evaluated through our in-house resources.

What was our ROI?

Our estimated ROI is positive we need a couple of months to gain insight into actual ROI.

What's my experience with pricing, setup cost, and licensing?

HP StormRunner load is SAAS offerings so the initial setup costs are very minimal when compared to the traditional approach (no or minimal capex ), and the day to day costs would be purely driven by the duration and number of virtual users required to simulate the traffic.

Which other solutions did I evaluate?

We have evaluated HP StormRunner Load against other market competitors that include Blazemeter, SOASTA’s Cloud Test, NeoLOAD, VSTS, Load Complete and StressStimulus.

What other advice do I have?

HPStormRunner load is an apt product a) if your application stack has Web 2.0 traffic b) your requirement is for a service that allows for pay as you go model c) proven and reliable load testing solution with superior support.

This review was co-authored by Sravanthi Naga & Ramanjaneyulu Narra, Performance Architects at Gallop Solutions. Gallop is a US based Co-located Independent Testing Services operating since 2003 with offices in Philadelphia & California. World's largest and leading organizations have relied on Gallop's specialist independent software testing services for more than a decade and have achieved significant market acceleration, returns on investments in their software quality initiatives.

Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
it_user273663
QA Load Tester with 1,001-5,000 employees
Vendor
I was able to simulate a real scenario without using our regular tools, instead uploading scripts into the cloud which gave me unlimited resources.

What is most valuable?

The platform is easy to use for uploading and backing up scripts.

How has it helped my organization?

I had to do overload testing on our homepage. I couldn’t use our regular tools because I wanted to simulate a real scenario and StormRunner Load does this. It enabled us to upload the scripts into the cloud and gave me unlimited resources.

What needs improvement?

The monitoring abilities were lacking. The reporting didn’t present the information in a clear manner for me. It took me four days to understand why I couldn’t upload the scripts from my computer to the cloud, and I discovered I was missing an important feature that StormRunner Load is missing, called Corver, and this converts the script for the tests.

For how long have I used the solution?

I used it once in a testing session and have been using it for one month in a live environment.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

There was a bug and we opened a ticket with HP, and it was fixed after a few days.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

Customer Service:

It was great, there was an issue and I worked with Kim from HP who was great.

Technical Support:

It was great.

Which solution did I use previously and why did I switch?

No previous solution used.

How was the initial setup?

It was straightforward, we just needed to register on the site. In comparison to other HP products, it was very user-friendly.

What about the implementation team?

We did it in-house.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user272247
Software Team Lead with 501-1,000 employees
Vendor
The performance-evaluation feature helped us find a "missing" metric that affected our response time.

What is most valuable?

The charts dashboard, especially the results screen, is the most valuable feature. I’ve also configured the tests, so I’ve also used the test definition page, which is very easy to use, self-explanatory, and simple. The real value comes from the results screen.

How has it helped my organization?

StormRunner Load helped us evaluate the performance of the product version we were about to launch. It showed us bottlenecks, where our product reaches its limits, and different characteristics that affect this. For example, we had metrics we thought were affecting our response time. Initially we thought there were two metrics, but we found another significant metric we should have taken into account. Without StormRunner Load we wouldn’t have known about this "missing" metric.

What needs improvement?

I think the report page could be improved, although we have used the reporting feature. It wasn’t accurate enough to use as a report that I could send my managers. I had to take additional screenshots to help fill in the reporting. I actually, ultimately, generated a report on my own.

For how long have I used the solution?

I've been using it for two months.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

Which solution did I use previously and why did I switch?

There was no previous solution in place.

What was our ROI?

This solution immediately helped us to determine the priorities for what to develop next and where we would reach our limits. Because we had a clear prediction, we knew when it would be urgent to develop features, in various cases. Before StormRunner Load, we didn’t have a clear developer roadmap, and now its very clear where we should invest, our weakest links, and what we need to do to strengthen weak links.

If you’re using third-parties for networking, cloud based servers, load balancer, VPN providers etc., all of them should be informed about the expected load. We had a VPN provider that was expecting thirty users per hour and calibrated accordingly, and the minute we started our load test it shut down. This third-party was not ready. Once we informed them that we were increasing our load, they increased their ability to support the load test.

Which other solutions did I evaluate?

No, this was my first choice.

What other advice do I have?

It is very cost effective to start with a low number of virtual users for a short amount of time in order to verify that your company system is running properly. Start with a small load test, run it until completion, verify that you see the logs you intended to see, and all people involved in the environment are ready to see what they expected.

Disclosure: My company has a business relationship with this vendor other than being a customer: I used to work for HP.
it_user251874
Performance Analyst at a educational organization with 1,001-5,000 employees
Vendor
StormRunner is one of the fastest-growing cloud based performance testing tools for running high concurrency tests very quickly.

What is most valuable?

  • Load generator setup is not required so we can be running high concurrency tests in few minutes.
  • Scenario setup for running tests was very easy.
  • Distribution of the load from different locations was very easy.
  • SiteScope and New Relic monitoring tools can be easily integrated with StormRunner Load.
  • SLA configuration was available to monitor response times.
  • Navigation breakdown report - to create a client navigation breakdown report.
  • Real time graphs for monitoring results.
  • Email reports.
  • Benchmark- comparing previous test results with current tests.
  • Anomalies detection.

How has it helped my organization?

Being able to setup and run 300,000 load tests using LoadRunner Performance Center takes around 1 to 2 hours when creating four scenarios of 75,000 each, including allocating/blocking load generators required for testing, initializing the fusers and running the test.

Running a 300,000 test was very easy using StormRunner. We just had to upload the script and create scenarios for 300,000 and run the test in 20 minutes.

Running back to back high concurrency test was very easy using StormRunner.

What needs improvement?

  • Improved PDF reports and creating custom reports
  • Real world- custom scenario setup
  • Update existing scripts online without uploading them again
  • Run time settings update in scenario

For how long have I used the solution?

I've used it for five months.

What was my experience with deployment of the solution?

None - there's no deployment required for running a load test.

What do I think about the stability of the solution?

During the initial days of high concurrency testing (300,000 and 600,000) we had issues with the StormRunner infrastructure e.g. graphs “disappeared” in the middle of a test, and HP had to give an online fix. Also, there were cases where tests appeared as “initializing” even though it actually started and HP again had to give an online fix. The StormRunner R&D team was very proactive in fixing the issues in very small turnaround time and supporting us running the high concurrency tests.

What do I think about the scalability of the solution?

Scalability issues were seen during initial days of testing but it was resolved by StormRunner R&D team.

How are customer service and technical support?

Customer Service:

8/10.

Technical Support:

8/10 - their team was very proactive in resolving the issues and provided hotfixes to support us running performance tests whenever required.

Which solution did I use previously and why did I switch?

We were using Performance Center to run performance tests, so we had to switch to StormRunner to run high concurrency tests.

How was the initial setup?

As StormRunner Load is a web-based application, it's simple setup, just registering users through an HP site.

What about the implementation team?

In-house implementation and evaluation.

Which other solutions did I evaluate?

There were no other solutions evaluated.

What other advice do I have?

StormRunner is one of the fastest-growing cloud based performance testing tools for running high concurrency tests very quickly.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user240087
CEO and Co-Founder with 51-200 employees
Vendor
We have managed to identify performance issues and when these issues are introduced.

Valuable Features

I can use it just as much as I need. It is quick and intuitive.

Improvements to My Organization

We are able to ensure very quickly and frequently that our product is ready for prime time. Using StormRunner, Codefresh has managed to identify performance issues and when these issues are introduced.

Room for Improvement

When you start a test, it takes times to initiate.

Use of Solution

We have been using the solution for 7 months.

Deployment Issues

No issues with deployment.

Stability Issues

No issues with stability.

Scalability Issues

We never encountered issues with scalability.

Customer Service and Technical Support

Customer Service: Excellent customer service. Technical Support: Excellent technical support.

Initial Setup

Very…

Valuable Features

I can use it just as much as I need. It is quick and intuitive.

Improvements to My Organization

We are able to ensure very quickly and frequently that our product is ready for prime time. Using StormRunner, Codefresh has managed to identify performance issues and when these issues are introduced.

Room for Improvement

When you start a test, it takes times to initiate.

Use of Solution

We have been using the solution for 7 months.

Deployment Issues

No issues with deployment.

Stability Issues

No issues with stability.

Scalability Issues

We never encountered issues with scalability.

Customer Service and Technical Support

Customer Service:

Excellent customer service.

Technical Support:

Excellent technical support.

Initial Setup

Very intuitive.

Implementation Team

In-house technical people.

Other Advice

StormRunner does the job in a very simple and cost effective way. HP has always lead this market.

Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
it_user54837
Sr. Director - Quality Assurance at a educational organization with 501-1,000 employees
Vendor
This software solution has allowed us to run larger, high concurrency performance testing cycles in a very repetitive manner.

What is most valuable?

Flexibility and speed to provision and start large (high concurrency) performance testing cycles.

How has it helped my organization?

This software solution has allowed us to run larger, high concurrency performance testing cycles in a very repetitive manner…a requirement for validating our application solutions and supporting infrastructure for customer facing web apps.

What needs improvement?

Out of the box reporting could be improved.

For how long have I used the solution?

We have been using the solution for 9 months.

What was my experience with deployment of the solution?

Some issues with UI but HP did provide high levels of support and the issue(s) were quickly fixed (within 3 days of initial product usage).

What do I think about the stability of the solution?

No issues with stability.

What do I think about the scalability of the solution?

No issues with scalability.

How are customer service and technical support?

Customer Service:

Customer service was an 8/10.

Technical Support:

Technical support 9/10.

Which solution did I use previously and why did I switch?

Utilized a different HP toolset (ALM/PC) for performance testing. SRL provided a better solution from a pricing and scaling perspective.

How was the initial setup?

Initial setup was straightforward…cloud based solution with a simple, flexible UI.

What about the implementation team?

Vendor team…expertise (10).

Which other solutions did I evaluate?

Looked briefly at Neotys…actively doing a POC of CloudTest for comparison purposes.

What other advice do I have?

You need to effectively plan anticipated usage of the toolset. This will enable you to purchase the appropriate amount of Vusers hours for your contractual period (in our case this was 12 months).

Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
it_user232248
Strategic Account Manager with 51-200 employees
Vendor
Benefit of StormRunner as opposed to other cloud-based Load testing tools is the ease of scripting. End users should understand this impact.

Valuable Features:

  • The ability to spin upload really fast.
  • The ability to create a script easily against an application.
  • The ability to provision the load so it comes from multiple geographies across the world. 

Room for Improvement:

The documentation could absolutely be improved and scripts. For example, during a performance test, the monitoring graph is not very clear as to what its representing. I wasn’t sure what the graphs represented so my guess is a lot of customers feel the same way.

Use of Solution:

One month - were doing a trial.

Deployment Issues:

No issues.

Stability Issues:

No, the only thing I remember seeing as a “bug” when you hover your mouse over the graph, it will jumps around and doesn’t follow the mouse movement, other than that I don’t remember seeing anything from a stability perspective.

Scalability Issues:

No, but haven’t really scaled it yet.

Initial Setup:

That's where the documentation could be really helpful, apparently I had an older version on my machine, which was creating conflict. If everything had gone as supposed to would have been simple but the documentation could have better and clearer.

Other Advice:

Customers should understand the amount of time it takes to build a script - benefit of StormRunner as opposed to other cloud based Load testing tools is the ease of scripting. End users should understand the impact of the time it takes to build scripts. If StormRunner can save you 50% of your scripting, you need to understand how that translates into value. Expertise of StormRunner and LoadRunner includes these elements.

Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor. The reviewer's company has a business relationship with this vendor other than being a customer: We are reseller.
it_user232020
Senior Test Analyst with 1,001-5,000 employees
Vendor
Valuable features include the fact that you don't have to worry about the infrastructure setup.

What is most valuable?

  • The fact that you don't have to worry about the infrastructure setup.
  • The fact that 1000's of virtual users can log on to the cloud without having to set up.
  • Provides a solution for new or existing customers.
  • Alternative to LoadRunner, for new or existing clients with agile.

What needs improvement?

At the moment, its a bit early for us to tell - does everything we need it to. HP is looking to expand to see what exactly it can be used for.

For how long have I used the solution?

We did a trial since it was launched.

What was my experience with deployment of the solution?

No issues with deployment.

What do I think about the stability of the solution?

No issues with stability.

What do I think about the scalability of the solution?

We were only running a small set of numbers, but no issues.

How are customer service and technical support?

Customer Service:

It's not something we've really had to worry about - we were able to crack it open and begin.

Technical Support:

The help files on the web are good and the video links/ guides are good.

Which solution did I use previously and why did I switch?

We use LoadRunner extensively for Performance testing.

How was the initial setup?

Setup was straightforward.

What about the implementation team?

In-house implementation.

What was our ROI?

I can't calculate the ROI right now but I would assume a lot considering savings on infrastructure cost and license costs. It will still take some time to run a performance test but setup saves a lot of time.

Which other solutions did I evaluate?

We evaluate a lot of testing tools. NeoLoad, LoadRunner, we prefer HP.

What other advice do I have?

If you're looking at an application that can be seen on the WAN and using agile approach, then its a tool that should be recommended and used.

Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.