CA Runscope Review

A cloud-based environment for building and executing tests, allowing me to focus on building, running, and reviewing test results


What is our primary use case?

We use it for API regression tests and API production monitoring.

How has it helped my organization?

It has allowed us to go from no API tests to scheduled, automated tests integrated with PagerDuty in a few months. I was able to do this without having to hire a QA Developer. Existing manual QA testers were able to learn simple JavaScript and Runscope UI. Regression tests were executed in minutes instead of days.

What is most valuable?

It is a cloud-based environment for building and executing tests, which allows me to focus on building, running, and reviewing test results. I do not have to waste time managing development environments on local machines. Being cloud-based, I can check the status on the test results from anywhere on any device. I am not constrained by a physical machine where the tests are running.

What needs improvement?

File upload is a big part of the products that we test. The lack of file upload in Runscope requires us to still use UI (Selenium) automated tests for these scenarios. Having Runscope support file uploads is the biggest improvement that we would benefit from. I would be willing to pay more if file upload was supported.

For how long have I used the solution?

One to three years.

What do I think about the stability of the solution?

Stability has been good. It is not an issue for us.

What do I think about the scalability of the solution?

We have experienced issues with scalability so far. A few months back, I needed to do some load testing which was not what Runscope was designed to do. I was able to hack together a solution in Runscope which provided the data that we needed. This is a complement to its flexibility.

How is customer service and technical support?

I have only contacted tech support a couple of times. The experiences were positive.

Which solutions did we use previously?

Before Runscope, I only used UI (Selenium) testing solutions.

How was the initial setup?

The complexity of the setup was due more to the complexity of the API calls in the system that I was testing, not the complexity of Runscope. To make it easier to onboard new customers, CA may want to have example tests with a couple API calls demonstrating assertions and variables.

What's my experience with pricing, setup cost, and licensing?

The pricing plans that we have used so far have been selected purely on volume needed per month.

Which other solutions did I evaluate?

We evaluated Assertible and Postman.

What other advice do I have?

  • Prioritize tests.
  • Use environment variables.
  • Budget time to experiment with features before going full speed on test creation.
Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Add a Comment
Guest
Sign Up with Email