Watch the full video review here
What is most valuable?
How has it helped my organization?
So some other benefits I actually think I made the tool do something it wasn’t really meant to do but it worked perfect for my environment. So we had a big project to tokenize credit cards. We wanted to get out of the business of hanging on to credit card numbers because of the inherit risk. So we had a massive project to come in tokenize those credit cards. One problem is the vendor much like any other third party comes back and says, well we don’t have a performance environment for this so… And if you performance test against our environment we’re gonna cut your access off because that’s gonna impact other people. So naturally the team turned to me and said, hey what do we have, can we virtualize this? I reviewed what they were doing and it was not a service call, it was actually a website form post. So I turned the call into rest call, slid it into the config file, told it to point to my service virtualization, and it worked just like an HTML post call and it worked perfectly. We were up and running probably within a day with this virtualized uh, endpoint. I couldn’t believe it ‘cause that’s not what the tool was made for, but it worked perfect. So that was one of those added benefits, undocumented features, right?
What needs improvement?
What do I think about the stability of the solution?
The stability of the product, it’s still a work in progress. There are sometimes that, just honestly I do have to go onto the system and, just the other day we had one service that had been running for months with no problem, and then for some reason it just stopped working. I could’ve reached out to support to get an answer. They’re usually pretty responsive and get it solved for me pretty quickly, but for the sake of expediency because we were blocking people I had to go in and remove and rebuild service virtual-endpoint. The good thing is it took me 30 seconds to do it, the bad thing is I had all my data that I had saved gone. I could get it back, it’s not a big deal but-so stability wise it’s not bulletproof but I haven’t had a major outage until the other day of that one service. That hasn’t caused any issues, there’s been no ripple effects because of that.
What do I think about the scalability of the solution?
I’ve had no problems with scale build. I’ve run, we’re doing a service gateway stress test right now in our production-like environment to understand if this gateway that we purchase is gonna be scalable for our big retail time period, fourth quarter time period. The service gateway said that they virtualize services which their definition of virtual services is different than a testing industry’s description of it. So I had to create my own endpoint, expose it through the gateway, execute stress test against it, and my bottleneck ended up being on the service gateway not on the virtual endpoint, and we were hitting well over 1,600 transactions per second. And my memory footprint on virtual machines. So that’s a virtual machine running service virtualization to was my CPU was probably right around 40 percent or so. So I still had way more scalability to go off this one instance.
How are customer service and technical support?
So on technical support, now this is where it gets a little tricky because I purchased this through a reseller. So my front line support does come from my reseller. I’ve had to have a few things escalated up to HP Support outside of that tier one level. They’re pretty responsive, they get back pretty quick, they’re very interested. I actually, I was kind of shocked when I was walking around HP Discover and I had a few of their solution architects go oh, we know you! You have success stories and you’ve had issues and I’ve looked into your issues. I’m like I don’t even know you, how do you know me? So it’s nice to know that they you know, that they have this, you know they definitely have a pulse on it They want to improve the product and be on top of things. They’re not looking to just kind of you know, fix it as it goes. You know, it’s just they want, they want to continuously improve it, so.
Which solution did I use previously and why did I switch?
Going into the purchase of HP Service Virtualization, prior to the purchase we actually ran into quite a few gotchas with third parties, with integrations, internal integrations where teams weren’t quite ready with their services. You know the consuming clients at a point to where they were held up, delayed in their delivery. So we watched their delivery cycles actually go from being in sprint, in lock step, in delivering you know with a, let’s say a three or four week cadence, three or four sprint cadence, it would bunch out. They’d wait ‘til the services team completed all their work, all the functional testing before they would start consuming the services and I looked at that and saw the inefficiencies of it. I kind of pulled the ROI together of getting a tool, started playing with all the various tools, open source tools, other big vendors, and it just turned out that this one worked best in our environment. The biggest savings we’ve had from it is by mocking out our third party calls. We don’t have those you know, the one cent, two cent per call hits anymore, bills coming back to us. We don’t have to worry about performance testing so we can mock it out with the service virtualization. And the biggest gain for me, that was cool but the biggest gain for me was actually having our internal customers be able to dev in sprint with the services team. We save so much time doing that.
How was the initial setup?
I didn’t really have any hiccups with the installation, it’s pretty straightforward. There are a few things, if you want to use some of the new features that are in that beta mode that you have to go in and you have to be very familiar with how to modify specific config files. I’m just naturally inquisitive so I just started digging through the config files on my own. I don’t recommend that unless you’re just that kind of person but other than that just out of the box set-up is so easy to stand up. Maybe the most challenging thing that we had was, overcoming an issue with SQL server implementation because I didn’t realize that each user that accesses service virtualization has to have their own development database instance inside SQL server. So we all shared the same instance and kept overriding everyone’s work. I found it in the documentation, it was there. I’m one of those guys that kind of implements first and reads documents later, and once I found it my temper level went down with the vendor. Even though, I think there’s something that, I think that’s something that can be improved upon. You should be able to share databases, but outside of that everything else went pretty smooth.
Which other solutions did I evaluate?
We did look at the IBM Solution, I looked at ITKO LISA (CA LISA). We also looked into Ready API by SmartBear which the SmartBear one was gonna be a little too complicated for some of the team members I had. It wasn’t bad, it just was a little too complicated. IBM was coming in kind of at the same price but I guess I kind of had a little HP bias. LISA was just outrageous for our budget. We could not afford LISA, and I even worked with some people that worked at LISA in the Dallas area and said you know, you can’t afford it. It’s not gonna work so when we brought it and did the proof of concept I was told by quite a few people in the industry that said here’s the deal, if you get it in and you can implement the first service that you execute and it fires right up and it works for you, it’s gonna work for everything in your group. I brought it, implement it, it took me probably no less than 30 seconds to stand up a virtual service and start consuming it in my performance environment, I was sold from that.
What other advice do I have?
Some of the criteria I looked at while looking to purchase one of these service virtualization tools was ease of use because I’m gonna have to give it to my org, the QA team owns it, not the developers. The developers, the whole shift left idea for development works partially but they are so slammed to meet tight deadlines that I can’t throw another tool on them. On the other note I don’t have on my team, I have QA analysts so they’re not all completely technical, they dabble a little on it. So I wanted something that they could pretty much do a drag and drop into it. I wanted something that would be easy to discuss with my ops teams to let them understand what I needed deployed to be successful in the environment. I was blown away with how fast the implementation was. I took the demo off the website, deployed it, ran with engineering team, probably spent maybe half a day getting everything I needed stood up, and then shortly thereafter we just all right we’re gonna buy this ‘cause it’s working fantastic. Bought the license, dumped it right on top of it and it still kept going, I mean it was no problem. So ease of use you know the ability to quickly stand something up, you know. When I mean quickly I mean someone’s calling me on the phone saying I’m blocked, I need this. The ability to stand up and test immediately is huge.
Rating - I’d probably put it closer to six or seven because it’s still maturing to be honest, just to be out there about it. I know that they went through a major change between I think the 2.0 release into this 3.0 release. I didn’t buy it prior to that but I’ve been reading up on it. Stability’s been all right, the implementation needs a little assistance like I talked about. I would probably say ease of use really puts my number up. When I was grading it between the other ones that was top notch for me was the ease of use. I’d like to see maybe some more documentation around API’s that are available to it so I don’t have to use other HP tools to integrate with it because I don’t have a whole suite of HP tools at my company. So like your automation for example is not UFT, it’s ran through homegrown automation solutions. So I would-I know the rest API available but it’s not documented and I know was that I can extract that information but I don’t want to do it that way ‘cause that’s a lot of work on my part. So I’ve talked to-talked to their solution architects and they said that yeah, this will become available just hold on, we’re baking it out. So I’m a little impatient I guess. I want-you know, moving into like a dev ops world you kind of want to get away from a lot of manual clicks and make it fully automated as much as possible, and that’s one of the components for me to make it automated.
So when evaluating these tools make sure that it works for the infrastructure that you have. I talked to a former colleague that was researching tools out there. He had alreadypurchased Greenhead, IBM Greenhead, only to find out that he inherited it and he found out that he couldn’t do anything with MQ series. Which that was a pretty big blow for him because he really needed MQ series out. So make sure that your technologies are definitely supported by the tool. Do a very thorough proof of concept on it. I said it was really easy to set up and it was. It was fantastic, but we ran it through different scenarios. I brought out some technical specialist not from HP but my reseller to sit there and show us how it works with TIBCO EMS, how it works with you know WCS services. How is it-how does it work with rest? Do a full soup du jour on it in our environment. That’s probably the biggest thing I would say is, don’t do it on the canned services or websites that always work in demo mode, do it on your stuff. If it works on your infrastructure then it’s probably you’re gonna be in good shape.