What is most valuable?
- Load generator setup is not required so we can be running high concurrency tests in few minutes.
- Scenario setup for running tests was very easy.
- Distribution of the load from different locations was very easy.
- SiteScope and New Relic monitoring tools can be easily integrated with StormRunner Load.
- SLA configuration was available to monitor response times.
- Navigation breakdown report - to create a client navigation breakdown report.
- Real time graphs for monitoring results.
- Email reports.
- Benchmark- comparing previous test results with current tests.
- Anomalies detection.
How has it helped my organization?
Being able to setup and run 300,000 load tests using LoadRunner Performance Center takes around 1 to 2 hours when creating four scenarios of 75,000 each, including allocating/blocking load generators required for testing, initializing the fusers and running the test.
Running a 300,000 test was very easy using StormRunner. We just had to upload the script and create scenarios for 300,000 and run the test in 20 minutes.
Running back to back high concurrency test was very easy using StormRunner.
What needs improvement?
- Improved PDF reports and creating custom reports
- Real world- custom scenario setup
- Update existing scripts online without uploading them again
- Run time settings update in scenario
For how long have I used the solution?
I've used it for five months.
What was my experience with deployment of the solution?
None - there's no deployment required for running a load test.
What do I think about the stability of the solution?
During the initial days of high concurrency testing (300,000 and 600,000) we had issues with the StormRunner infrastructure e.g. graphs “disappeared” in the middle of a test, and HP had to give an online fix. Also, there were cases where tests appeared as “initializing” even though it actually started and HP again had to give an online fix. The StormRunner R&D team was very proactive in fixing the issues in very small turnaround time and supporting us running the high concurrency tests.
What do I think about the scalability of the solution?
Scalability issues were seen during initial days of testing but it was resolved by StormRunner R&D team.
How are customer service and technical support?
8/10. Technical Support
8/10 - their team was very proactive in resolving the issues and provided hotfixes to support us running performance tests whenever required.
Which solution did I use previously and why did I switch?
We were using Performance Center to run performance tests, so we had to switch to StormRunner to run high concurrency tests.
How was the initial setup?
As StormRunner Load is a web-based application, it's simple setup, just registering users through an HP site.
What about the implementation team?
In-house implementation and evaluation.
Which other solutions did I evaluate?
There were no other solutions evaluated.
What other advice do I have?
StormRunner is one of the fastest-growing cloud based performance testing tools for running high concurrency tests very quickly.