What is our primary use case?
We use Worksoft Certify for testing and non-testing.
- Under the testing umbrella, we use it for regression testing, disaster recovery, validations, and system refresh testing. This has been our main focus.
- Recently, we have started getting more requests for more RPA-light type of work, which is not testing. It is using the same skills, process knowledge, and tool sets to do work that would replace manual repetitive tasks with automation.
We use Worksoft Certify for some basic ServiceNow functionality or Workday releases, weekly and quarterly releases. For SAP Hybris and ECC, we are also using it for our internal security protocol testing. So, we test Office 365 and Windows 10 compatibility. We test some Excel functionality and file sharing, as part of our security protocols. The most in-depth end-to-end testing that we have is in SAP.
We have done a lot of manual testing. We still do a lot of manual testing for our projects. We've eliminated a significant amount of manual testing with our system refresh, automation, and for technical upgrades where changes are known. However, for projects with new enhancements and functionality, we are having a slower time penetrating into them. With regression testing, we have completely replaced it with automation.
Now, we are trying to shift as much as we can to start automating processes earlier in the project lifecycle, but it has still been a bit of a challenge. This is one of our stretch goals for this next year. The non-testing area is where we have had the most growth over the last six months.
How has it helped my organization?
The freshest example is some of the RPA-light activities that we have done. We have been able to save on a lot of manual work for some very highly skilled, expensive resources. This has been able to free up a significant amount of their time so they can spend more time on innovation and more creative, value-add activities. That's been one of the more rewarding things that we've done, and the most appreciated.
We run our tests and our full suites every week. We have them scheduled in batches so certain sections of our scripts run every day, then we run them through the whole suite every week. That is how we maintain them by running and repairing them.
To run them every week and make repairs takes us maybe two hours. Because they are scheduled on batches to run, it's lights out. They are pretty hardened at this point, so there are not a lot of repairs for data, etc. If there is a break, it is usually because there has been a change to a process that we were not aware of. Our automated scripts are the company's best business process documentation, as we don't have a business process management tool. Therefore, if anybody wants to know about our business processes, they come to us for something like training or new production support resources. They will come and watch our automation if they want to understand how the systems work.
What is most valuable?
The ability to run multiple processes at the same time remotely or on a schedule. So, we have some testing that we do every day, and it is pretty much lights out. It is unmanned. We have some virtual machines that run on a schedule. Therefore, it's out of mind testing unless there is an issue. They are very hardened tests. If there is an issue, it means that there is something that we needed to catch, so it is always a good catch. This has given us a lot of flexibility because now we can use those resources in other ways. Besides the basic automation capability, it has been great having the ability to test multiple applications and multiple processes at the same time and overnight, then just receive the results.
I have always appreciated the Capture tool. I'm excited about the new enhancements that have been made to it. I think this will make adoption a lot easier because the tool is a lot easier to use and has more capability. I'm excited, because this is a good time for us as we are expanding into more RPA-light space. It will be easier if we could have some of our more technical resources doing a lot of this capture work. Then if they do something wrong, it's very intuitive to stop, delete that step, and start over. It will make adoption a lot better, and we'll get better requirements and processes documented that we can then convert into automation.
The time it takes users to document, then for us to automate, depends. We have a lot of end-to-end business processes, but they have to go across functional teams. They tend to get passed around a bit. Worksoft definitely saves time. We were literally getting processes on napkins. We didn't care how we got them as long as we got them. They would walk us through really fast, and they would try to show us. We were doing screenshots and trying to record them in WebEx.
Since a lot of the business process owners are onshore and the developers for the automation are offshore. It was very challenging trying to find a time when they could get together. This product allows them to do the capture on their own time. It can be very quick. They just send it over, then they're done. They don't have to think about it anymore, and it's documented well enough that we then don't have to spend a lot of time coming back to them. It just makes the whole process more efficient.
What needs improvement?
I would like more reporting in analytics. There is a lot of manual work for us as program managers and test managers which has to do with supporting our value statements. E.g., if there is some way that we could systematically capture how long it is taking for automation processes to execute, then we could insert some notes as far as here is how long it took for them to do the manual capture. Then, we could calculate time saved and have a formula for savings.
If they have some templates that we could all partner, there are a lot of customers who have created their own, but with the new companies coming onboard. Do they create them on their own or do they try to leverage the best practice within our customer community? There is more we can do here.
What do I think about the stability of the solution?
There are no issues so far. We haven't had any problems with the tool not being available for us when we need it.
We had trouble with an upgrade once, but there was an immediate response on their side. We had a very technical resource who helped us get past it quickly. So, there has never been anything which has really stopped us from working.
What do I think about the scalability of the solution?
Because we started early with the tool, it took us a while. I feel like we almost glued things together as our needs became more. The capability that was delivered by Worksoft was there. Therefore, there hasn't been a time when they have been behind me when I have needed something that wasn't available.
I am expecting it will the same in the RPA space. We will grow together because of our close partnership, and if there is a gap, I can work with them to figure out what the best approach is to close it. I think we will be able to stay with the tool for a long time because of its scalability.
How are customer service and technical support?
I am aware that they have some challenges with some of their support resources, especially offshore which is very common. I don't think this is specifically a Worksoft issue. It always seems to be a software issue, and I know that Worksoft is aware of this and they are trying to make some improvements.
If ever I need to escalate something, I never feel like I'm stopped. I always feel like there is another level where I can go and get support. We have never had an issue which has gone unresolved for a long time. We try to follow the process, but since our team is so experienced with the tool, if we can't fix it and their support can't fix it either. then it gets escalated up through the chain, getting somebody whose pretty senior with the tools to help us.
Which solution did I use previously and why did I switch?
Our initial goal was regression testing. It was really expensive. It was throwaway work. We always had to outsource it. It overlapped other test cycles within a project. So, all the functional business folks were busy. It was something that if we left it up to the project resources, they didn't do a very good job with it. We would bring in manual testers almost literally off the street. They didn't know our processes. They ended up having to interact with our project resources anyway. It was just a mess. It was inefficient, clunky, expensive, and the quality was poor.
We knew that we had a lot of SAP implementations coming up because we had acquired several new companies. So, we made the decision at that time that we needed to automate regression testing. That was our first initial goal, and we've hit that. During our last major SAP implementation project. Our regression coverage was at 90 percent which is pretty much the top you can ever really expect. Now, we are looking at other use cases.
How was the initial setup?
The initial setup was a little rough in the beginning. It was so new for us. The whole idea of automation was new that trying to get the tool setup, internalizing all the best practice training and everything that came so quickly, was a lot to try to digest. Thus, we ended up asking if we could spread the mentoring out across a few months. This seemed to work better for us.
What about the implementation team?
We purchased the software, then we found a vendor on our own to help us with the development.
What was our ROI?
It have saved us significant time. I have an entire dashboard that I use to showcase to everybody the amount of manual hours that we have saved and how that equates to dollars saving.
On our last big SAP implementation project, we inserted an automation resource into the beginning of the project. Between automating regression processes, data staging, and using our automation to help repair cutover and conversion issues. We saved the project about $1,700,000.
Worksoft has paid for itself fives times over.
Which other solutions did I evaluate?
We evaluated Micro Focus UFT, Selenium, SAP CBTA, and Worksoft.
My main focus was an SAP automation solution, and Worksoft was really good in that space. They were an SAP partner, but I recognized that I wanted something to be more scalable across other applications, and that ruled out the SAP solution. We liked the price point of Selenium and some of the open source tools, but there are risks to something like that. You don't have as much control, and there are always security concerns. Our internal teams weren't excited about that, as they are not great with SAP. We already had a lot of pain points with UFT. It took way too long to develop processes with UFT. It required more coding.
What other advice do I have?
I would recommend taking a slower, organic approach to automation. It is hard to insert ourselves into the projects. The functional resources, business resources, and process owners don't have a lot of time for us. They don't see the value initially. It is overhead for them and more work. So, you have to bite off small chunks. Show the value, then build up the trust. If you try to be too aggressive and force something down everybody's throats, they will barf.
If you have super strong executive support and it's a top-down, e.g., the CIO says, "You will do this or else." You may be successful. However, in that scenario, your failures will be noticed and made very public. If you take a slow organic approach, where you're just trying to be really helpful and free up time, doing little favors here and there, you build up confidence. Then, people support you more for your success.
Start with the low hanging fruit for the value. Build it up. Once you get a bit more expertise, then start tackling the more complex processes.
Worksoft is a great supplier to work with. They have never pushed back when we have had issues or questions. They have always been available to help us. They put us in touch with other customers that have done something similar to what we were looking to do. They set up user groups by region so we could get together. They facilitate a lot of good discussions. That's why I mentioned we continue to grow together: customers and suppliers. It's just been a great relationship. We don't get that with every vendor. So, when we have it we appreciate it.
It has been very easy to use, but I don't think every automation tool is for everyone. I don't think just anybody off the street can come in and use it. Maybe for some basic stuff, but if you really want to maximize the use of the tool, you need some folks who are really experts in it.
We were able to really grow when we hit that inflection point: When we transitioned to a different vendor that we had doing our automation development. They were experts in the tool. That was when we started being able to deliver these creative solutions. That was when we were able to see the cost per automated script go down, because they were able to develop so much faster.
While it can be used by everyone on the surface level or to capture the business processes, to get more return on your investment, you have experienced resources using it.
Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Mar 12 2019