Broadcom Service Virtualization Reviews

The web services protocol for SOAP requests and responses is the most valuable feature. It is easy to set up the test for them without much coding, thus it is easy to train new testers to work on it in a graphical development environment where one can see the flow of information visually like the TIBCO product. It is easier than SoapUI to follow.

This product has introduced a simple-to-use middleware tool, which helped our team conduct web-services testing.

The most important thing is to have integration with Excel. It has a way to read the Excel sheet but not to write onto it. If I need to generate customized reports in LISA in Excel sheets for management purposes (which can show me number of test cases pass/fail/not started, etc.), I can't do this since you can't ask Service Virtualization to put specific values in particular cells with a particular color, etc. We were able to achieve it only using Java, which meant a lot of scripting, error debugging and rework, and this defeats the purpose of it offering a codeless environment to testers. A simple groovy scripting in SoapUI can achieve all of the above.

The Service Virtualization section needs to be more intelligent in offering a good solution to customers. For example, the virtual service created via the request response pairs needs spoon feeding of responses before the service starts performing. And I am not sure if it serves the purpose of virtual environment till the actual system becomes available. I have yet to see a proper implementation and successful usage of a virtual service through it.

CA should provide a free trial version of the tool (was not available till version 7.0.10) so that clients can try it out before making a purchase.

I've used it for two years.

There were no issues with the deployment.

The product is yet to become stable. I have seen instances when the tests have not been repeatable in LISA when scripting is done in Java. After removing and adding a character to the code, the test starts working again. Sometimes restarting it gets it working. Also, there are caching issues in LISA and it picks up the old values of the variables/parameters/properties in the tests even though the values have been updated and saved in the test before commencing the run. Thus, people prefer using more stable solutions like JMeter and SoapUI.

There were no issues with the scalability.

The customer support for LISA is average with the associates not being reachable all the time. Also, there were no specific communication protocols set up to interact with LISA support team and we were mostly relying on the Confluence forum to have our queries resolved. Also, the search results on Google do not provide answers or hints to the questions LISA testers face, unlike SoapUI and Ready API which have a lot of online support available. One can only rely on the user guide of LISA which does not cover all the scenarios.

The initial setup was straightforward, but the settings procedure, like providing the correct domain name, server name, username and password, did not work the first time they were provided from LISA.

I got it implemented through a vendor team. There are a couple of settings you have to play around with before it starts working.

Choosing this product was a management decision and we would prefer other solutions because in order to achieve a customized solution one would have to do scripting. This tool needs to offer a lot more options before it can be marketed as a codeless environment that can really help an organization chive results with such a testing team that is not good at coding.

QTP can achieve almost everything to do with scripting testing. Other tools like SoapUI/Jmeter being more popular become the obvious choice.

The two most valuable features for us are:

• Service Virtualization – It provides us with the ability to replace real downstream systems with virtual systems.
• Test Automation – It enables us to trigger events and automate test cases.

• It reduces cost when we can replace expensive downstream systems.
• It improves speed of testing when systems are not always available.

The biggest area of improvement is in the licensing. The licensing costs are very high.

I've used it for over four years.

There were no issues with the deployment.

There were no issues with the stability.

There have been no issues scaling it.

Technical support is excellent. I rate is a 9 out of 10.

We originally used Green Hat, but CA Service Virtualization provides greater flexibility.

The product is relatively simple to install and use. The complexities generally relate to system connectivity restrictions within the organization.

We developed using our existing contracting partners with support from the vendor.

Think “outside the box” when you think about how this product can be used. Don’t just think it can emulate systems because it can do much, much more.

The most valuable feature is that it's available 24/7. If I'm not able to procure all the infrastructure and I have to share it, I'm not afraid to share it. It doesn't depend on who else is using it because I know it's available.

The performance testing guys really depend heavily on us for their volume testing. The developers use Service Virtualization so that they can present a better product to QA. Sometimes by using it, we're able to implement something without even going to QA. We're bypassing QA.

For our current needs, it's doing what we need right now, although it should automatically generate tests to further make our development to implementation process more efficient.

I've used it for two or three years. My colleagues and I inherited it. It was purchased and sitting on the shelf somewhere until we started using it.

We had no issues deploying it.

It's highly stable.

It depend on how much you're willing to spend and how large your company is. We have 5 servers and it's scaled just fine for us. In a presentation I went to, American Airlines has 50 servers and it's scaled just fine for them.

As soon as you drop an email, someone responds.

It's easy to install as there's lots of documentation. Whether it's a small installation, a single server, or multiple servers, it's pretty easy. Again, American Airlines has it deployed on 60 servers. We have only 5.

We implemented it in-house ourselves.

The CAI aka Pathfinder is a great capability and will help the organization to achieve SV goals and expand the SV to different systems without many firewall changes or changes in the existing environment setup.

The financial industries have tight firewall security where introducing a new product among their existing infrastructure is not a hot cake or getting new MQs created for SV. At high-Level CAI looks like any agent-broker architecture but it does a lot in terms of Virtualization and Application Delivery Lifecycle where Dev or QA do not need to worry and do not need to depend on the environment teams to get the systems logs for triage or fix it.

Finally, last but not least CAI can also help the architects/designers to understand/generate the application flow diagram and analyse any abnormal behaviour.

The major ones for our organization are MQ and SOA Middleware.

We had a lot of dependency on MQ message, where CAI helped to get the MQ details and create VS out of it.

I feel a few areas can be improved, but I'm not sure if CA has already fixed these in the latest version:

1. Agents for different systems. I am not sure if existing agents can support all types of servers or systems.

2. Along with application details, can we capture network details to simulate network behavior?

3. Can we separate utility to monitor the CAI details rather than the portal, because Portal is accessible to all.

More Than 2 years now.

Yes, we had issues when we deployed into Oracle WebLogic server. .

I'd say 8 out of 10. Service is great in terms of community forms, pulse etc.

Technical documentation needs some more improvements by explaining live scenarios.

The technical support team we had for our engagement was really great, we are able to reach them and get everything done on time.

For SV no. We have not switched from any other solution, we started with LISA. The main factors why we decided to go with this tool was the capability towards virtualization and protocols supported.

It was not simple nor was is complex. To set up CA DevTest, you should know the tool and architecture very well along with his installation skills on the particular server you choose - like Unix or Linux. Firewalls and other areas are also very important.

We haven't yet measured ROI with the current implementation.

The tool was evaluated by an architect team. Yes, they had compared it with some competitive vendors such as HP and then purchased the CA license.

I'd like to see more features integrate with the CICD pipeline via APIs or command line etc.

I'd also like better ocumentation on live scenarios with more examples etc. as mentioned earlier.

Pathfinder, VS Easy

Ability to implement WSDL validations in the model using the built-in assertions.

The virtual services for Stress, Volume and Performance testing for applications which reduced the cost and also eliminated the dependencies of other applications and third party software.

Not all protocols are supported, but I've heard there will be a feature in version 8 with the help of which we can create models for those scenarios as well (surely a few limitations will exist).

3+

Sometimes, different versions of tools differ in behavior and new versions don't completely provide backwards compatibility.

Example: lack of ability to deploy a service built in LISA 7 to LISA 6 VSE using LISA 7 workstation.

When creating services with database steps, if the DB goes down the services will end and once the DB is up the services will be still be in an ended state, and a manual start will be required.

Triggering multiple requests sometimes increases the response time even though loadbalancers are used.

Customer service is very good.

Good.

We were using LISA 6, THEN upgraded to LISA 7.5 as the new features were beneficial for meeting business requirements.

Initial setup for Workstation is straightforward although setting an agent and broker for pathfinder to use recording is bit complex.

We implemented through a vendor team.

We have eliminated the complete dependency for all the backend systems for SVP (Stress,Volume and Performance). Testing using Service Virtualization saved huge amounts. Using service virtualization we have implemented a shift left concept to enable different teams to work in parallel instead of waiting for other teams to start the work, this also reduced the overall budget for the program.

A few other options were considered from Parasoft, HP and IBM.

The most valuable feature is the ability to create systems. We use TIBCO for our integration; the ability to create those systems when they are not available to us is very effective.

It has helped our organization in term of its availability and being able to create systems immediately. This happens in cases when our developers are held up or when we have to spend a lot of time switching between development and testing; that can really slow us down. This causes problems sometimes when we forget to disconnect and reconnect.

I would like to see improvements in the naming conventions. The current naming conventions of CA Service Virtualization are hard to use.

 I haven't seen any problems with stability. It looks pretty stable.

In terms of scalability, we're just getting started so we're just going through examples. CA is on site and they are taking us through the process. We haven't really gotten into the scale end too much. We are just taking small chunks and proving the concept.

CA came in and installed it. It seemed like it was a straightforward installation as long as you know what you're doing. I couldn't just do it out-of-the-box, but it seemed like it was pretty straightforward.

We're getting into the lean and agile process and one of the problems we have is the testing and delivery time frame. We're looking to improve the overall quality of the product. The code helped the efficiency of our programmers in the overall process.

I suggest just getting in and start using it. That is the best case scenario for us, to actually start proving the concept.

When selecting a vendor, I would look at support, the functionality of the product, and how well it performs. I would suggest starting small, take it in small chunks, and just start using it.

When we previously used CA SV we saw tremendous results, and went live with almost a 0% defect rate.

We use it for Service Visualization and to run tests to validate our services response with the database.

With CA SV it took 2.10 minutes as opposed to 210 minutes for a total validation per scenario. This reduced the development cycle time and provided a product with no defects from a QA perspective.

Performance and cost.

I used it a year ago and recently started using it again.

CA SV takes up lot of your system resources.

Test - service layer component testing automation)

Virtualize - the best mocking/stubbing tool I've ever seen, allows for easier data management

With Service Virtualization being a newer concept and market, there is a lack of high quality/experienced resources available across the board for any of the tools (as of mid 2013)...with that said, a good developer can come up to speed fairly quickly with this tool.

Can't wait to see the integration w/ Shunra, Nolio and Grid Tools in the near future!