CA Runscope Review

Helps us verify that production deployments do not break the features our customers use

What is our primary use case?

We are using it for API end-to-end testing.

How has it helped my organization?

It helps us verify that production deployments do not break the features our customers use.

What is most valuable?

We have the ability to populate environment variables using scripts. This allows us to keep the configuration mostly dynamic and avoid a lot of manual updates when something changes. 

Schedules are a great way to have repetitive checks against the environment.

What needs improvement?

We would like to have an improved ability to share a set of tests across different environments (e.g., dev, stage, and prod). The shared environments feature is great in practice, but in reality, if you have two or three of them using the same set of tests, it gets complicated to figure out which environment is failing. 

Overall, the user interface (UI) requires a lot of clicking around. 

Waiting for asynchronous results requires a lot of "Pause" steps, which is quite difficult to get right. It would be great to have some type of "Waiting for Results" step, which would include success/failure conditions, then eventual timeout for retries.

For how long have I used the solution?

More than five years.

What do I think about the stability of the solution?

There have been no stability issues.

What do I think about the scalability of the solution?

There have been no scalability issues.

How is customer service and technical support?

We have found working with the technical support to be great.

Which solutions did we use previously?

We did not previously use another solution.

How was the initial setup?

The initial setup was straightforward, but it included a lot of repetitive manual work.

What's my experience with pricing, setup cost, and licensing?

Not applicable.

Which other solutions did I evaluate?

We evaluated AppNeta, New Relic Synthetics, and custom implementations.

What other advice do I have?

Create a script to populate tests from a config file. Store it in a source control and load/update it through the API. Doing things through the UI wastes a lot of time.

Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Add a Comment
Sign Up with Email