CA Runscope Reviews

Filter by:Reset all filters
industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
rating
Loading...
Filter Unavailable
Real User
Project Manager, QA Leader and Information Security Manager at a tech services company with 11-50 employees
Jun 06 2018

What is most valuable?

The schedule is definitely the most valuable. Also, being able to check the JSON attribute values is helpful. We always check the attributes. Finally, the notifications are helpful as well.

How has it helped my organization?

As we have them scheduled, we use the notifications. We know the exact moment it is breaking something that might impact the customer system, that might result in a feature not working. We know "in… more»

What needs improvement?

One thing that can be improved is the logging. Sometimes, when we have an error, we don't know how to find more information about the problem that has happened. Sometimes, it just returns something… more»

What's my experience with pricing, setup cost, and licensing?

I think it's a good price. Once I asked my boss to approve it, he approved it immediately.

If you previously used a different solution, which one did you use and why did you switch?

We did use Postman but not for scheduling; just to do a quick check, we ran it from there. I don't know too many tools in the market, so it's hard to make a comparison. I think Postman does… more»

What other advice do I have?

I would rate Runscope an eight out of 10 because it's still missing flexible scheduling, the exact time or the ability to sort the services the way we want them to run.
Real User
DevOps Engineer at a tech company with 1-10 employees
Jul 25 2018

What is most valuable?

API monitoring where specific tests are designed to emulate production data-flows through our pipelines. Tests are per-step configurable with script capabilities and support for token-based… more»

How has it helped my organization?

This solution has improved our responsiveness to issues in both production and non-production environments. Through integration with Slack, our development and DevOps teams are quickly aware of issues… more»

What needs improvement?

* Ability to configure Runscope behavior when a given test has its schedule modified. That is, in a non-production environment, we often need to pause a test (either by removing it from the schedule… more»

What's my experience with pricing, setup cost, and licensing?

Having the different plans organized by team size, buckets, and requests per billing period works for our needs. Larger companies may need to take a close look at these elements.

What other advice do I have?

Create a DevOps team story to evaluate Runscope, which includes setup for all tests/buckets which will be used in your production environment. Exercise all use cases and outcomes, including… more»

Which other solutions did I evaluate?

Some alternatives were evaluated prior to choosing Runscope, however, I do not have access to that information at this time.
CraigAndrew
Real User
Lead QA Engineer at a financial services firm with 51-200 employees
Aug 19 2018

What is most valuable?

* We love the fact that we can have our API tests run on a schedule as often as we need. * We also take advantage of being able to set up different environment settings so… more»

How has it helped my organization?

Not only does Runscope allow us to keep our tests organized and available to everyone on our team, but it also alerts us when tests fail, via integration with Slack, so… more»

What needs improvement?

Navigation can be a little tricky when changing run environments and switching between test run results and editing the tests. It would be nice if you could switch the… more»

What's my experience with pricing, setup cost, and licensing?

Pricing is very reasonable and is easy to pay when you are supporting a team of engineers.

If you previously used a different solution, which one did you use and why did you switch?

We have used Postman quite a bit, but Runscope makes it easier to group tests together in a logical way. In addition, being able to run tests on a schedule and make them… more»

What other advice do I have?

Tests can be brittle, so make sure you include checks to validate data and variables that you are using.

Which other solutions did I evaluate?

We did not evaluate other options, other than continuing to use Postman for our API testing.
Real User
Director of Engineering at a consultancy with 1-10 employees
Aug 23 2018

What is most valuable?

It uses JavaScript which is easy.

How has it helped my organization?

It helps make sure that the program running the production server hasn't crashed without anybody knowing. Runscope makes sure that it runs properly and, if it does crash, something happens. It sends an email. Then, with Slack, we can notify… more»

What needs improvement?

I've noticed, once or twice, when there were some updates pushed by Runscope, at times it didn't refresh the pages. I was running it, there were multiple tests running at the same time, and it didn't update the webpage I was looking at. So… more»

If you previously used a different solution, which one did you use and why did you switch?

Although I wasn't with the company when they chose Runscope, one of the main things I was told was that they chose this product is because it's cloud-based. The company has different data centers. You can run it off the cloud and against… more»

What other advice do I have?

It's a pretty good product, as long as the price is okay for you.
Brian Robison
Real User
Director, QA
Jul 25 2018

What is most valuable?

It is a cloud-based environment for building and executing tests, which allows me to focus on building, running, and reviewing test results. I do not have to waste time… more»

How has it helped my organization?

It has allowed us to go from no API tests to scheduled, automated tests integrated with PagerDuty in a few months. I was able to do this without having to hire a QA… more»

What needs improvement?

File upload is a big part of the products that we test. The lack of file upload in Runscope requires us to still use UI (Selenium) automated tests for these scenarios… more»

What's my experience with pricing, setup cost, and licensing?

The pricing plans that we have used so far have been selected purely on volume needed per month.

If you previously used a different solution, which one did you use and why did you switch?

Before Runscope, I only used UI (Selenium) testing solutions.

What other advice do I have?

* Prioritize tests. * Use environment variables. * Budget time to experiment with features before going full speed on test creation.

Which other solutions did I evaluate?

We evaluated Assertible and Postman.
Real User
Quality Assurance Engineer at a consultancy with 1,001-5,000 employees
Jun 06 2018

What is most valuable?

* Ability to schedule and compare the test suite results over time periods. * It has a relatively simple user experience. * Ability to copy tests suite between buckets. *… more»

How has it helped my organization?

Since we are working with Runscope, we have a tool which is shareable between all the team members in a uniform standard. Now, each of us can easily run all our tests in a… more»

What needs improvement?

* Ability to create code which runs before and after API tests. * The area of "initial script" is relatively small and incomplete related to code editors. * There are no… more»

What's my experience with pricing, setup cost, and licensing?

The price for the product is very reasonable for a team of more than four to five members.

If you previously used a different solution, which one did you use and why did you switch?

We were using editors to create scripts and run them locally. This was unstable, not shareable, or a uniform solution. We were not able to run big amount of tests… more»

What other advice do I have?

It is a relatively reliable tool for running big amounts of API tests, which you can schedule to run periodically, as the product is very good.

Which other solutions did I evaluate?

We did evaluate other solutions before choosing Runscope.
Real User
Quality Analyst at a tech vendor with 10,001+ employees
Jul 25 2018

What is most valuable?

* Being able to add tests as sub-tests in a parent test for regression testing * Notifications for tests that pass/fail * Being able to customize expected results with assertions * Being able to set… more»

How has it helped my organization?

We have been able to set up regression tests that cover everything we need to monitor. We then set them to run automatically and forget about them. We are notified whether they pass/fail, and have… more»

What needs improvement?

We would like to be able to select the actual time a test will run automatically. It appears that you can set up the automatic schedule, but the test runs based on the last time it ran and not a time… more»

What's my experience with pricing, setup cost, and licensing?

Pricing is fair.

If you previously used a different solution, which one did you use and why did you switch?

We used Postman. Runscope makes it much easier for those without a programming background to accomplish API testing.

What other advice do I have?

It's really easy to use and the documentation is extremely helpful if you get stuck on something. The product is flexible and allows you to set up variables and assertions in order to customize tests… more»
Real User
Director of Engineering at a tech services company with 11-50 employees
Jul 25 2018

What is most valuable?

We have the ability to populate environment variables using scripts. This allows us to keep the configuration mostly dynamic and avoid a lot of manual updates when… more»

How has it helped my organization?

It helps us verify that production deployments do not break the features our customers use.

What needs improvement?

We would like to have an improved ability to share a set of tests across different environments (e.g., dev, stage, and prod). The shared environments feature is great in… more»

What's my experience with pricing, setup cost, and licensing?

Not applicable.

If you previously used a different solution, which one did you use and why did you switch?

We did not previously use another solution.

What other advice do I have?

Create a script to populate tests from a config file. Store it in a source control and load/update it through the API. Doing things through the UI wastes a lot of time.

Which other solutions did I evaluate?

We evaluated AppNeta, New Relic Synthetics, and custom implementations.
See 3 More CA Runscope Reviews

Articles

User Assessments By Topic About CA Runscope

CA Runscope Questions

What is CA Runscope?

CA Runscope customers

Twilio, Microsoft, Adobe, Okta, Tesco, Charter Communications

Sign Up with Email