What is our primary use case?
We are into performance engineering. There are a lot of back-end systems that are sometimes unavailable, or there are a lot of various dependencies, and we're trying to get rid of those. That's one of the reasons we got into DevTest.
And it's been helping us quite a bit now. We've been able to step out a lot of back-end systems. It's performed pretty well. There were a few hiccups, but we got in touch with the senior architects and they helped us out with fine tuning and, since then, it's been working out well.
What is most valuable?
The most valuable feature is that it supports so many protocols. We, being a large bank, have almost all the protocols, and it supports all of them, so that's one good thing.
How has it helped my organization?
I think it definitely helps us reduce time to market, and in terms of dollar savings it helps us quite a bit. There are many testing applications that reach out to third-party vendors where every transaction costs, and we are able to virtualize all of that and reduce the financial aspect.
What needs improvement?
One major feature I would like to see is on the user administration part. Right now, anybody can access any of the folders and any of the projects. If they could bring in someone amount of user administration, that would help us a lot.
What do I think about the stability of the solution?
Stability so far has been good. There have been no issues. There was a learning curve, about a year or so. It took us some time. It's not a plug and play tool. There was a lot of customization that had to be done, but with all of that done I think it is quite stable now.
What do I think about the scalability of the solution?
The hiccups we experienced were mostly to do with scalability.
There are multiple protocols that we actually work on, like HTTP, HTTPS and then MQ and TCP. HTTP and HTTPS were fairly basic, and we were able to get things done pretty efficiently. When it came to MQ and TCP, we came across a lot of challenges. That's when we got CA engaged and they helped us fine tune the tool itself so that we could reach the scale that we wanted to.
How are customer service and technical support?
I would say they're a four out of five. A lot of times what happens is, they come back with scripted answers and then we reach out to the account manager or somebody. Then they look into it and they help us out. That's why I'm taking out one star for them. Otherwise, they've always been good.
Which solution did I use previously and why did I switch?
We were on the lookout for a service virtualization tool.
How was the initial setup?
I would say it was complex.
The initial set up, if you look at the basic functions, it's quite straightforward. But for the scale that we were planning to implement it at, we did not get a lot of support from CA. So a lot of it had to be done in-house. We had to bring in our own architects and do a lot of environment setup and the like. We did not get a lot of support from CA. That's one area where I would say CA could have helped out better.
Which other solutions did I evaluate?
We did look at a couple of tools in the market. We did a PoC on DevTest. Back then it was called Lisa. Everything went well, so that's how we ended up getting into DevTest.
We didn't do a POC on the IBM product, I think it's called Green Hat - IBM's has a service virtualization tool - and we were actually contemplating that tool as well.
We went with CA because the PoC went well. There were a couple of guys who came in and actually helped us out with all the use case scenarios that we had, and we were able to implement it successfully. That was one of the reasons we went ahead with CA.
What other advice do I have?
When selecting a vendor, our criteria, what we are looking for are
- easy to use tools
- a good feature set
- and then awesome support.
I would say it's an eight out of 10. Eight because it has so much going on and there's a lot to learn, and there are a lot of things you can do with the tool. It requires quite a lot of learning effort. It's not plug and play. And a lot of customization has to go into it.
I would advise going with DevTest, definitely. It would depend on what you are looking to get out of the tool. If it's some of the basic protocols like HTTP, HTTPS, you could actually do it yourself. But if you're looking at a much larger scale, then definitely DevTest is the right tool to go ahead with.