Worksoft Certify Reviews

We have done one POC (proof of concept) for Worksoft Certify, for one application, for one of our clients. The POC was successful, so we implemented the solution. We did it for a pharmaceutical product, a pharmaceutical domain.

What I found most valuable in Worksoft Certify is its identification feature. I also found its automation feature valuable.

What could be improved in Worksoft Certify is its integration with other tools, for example, test management tools such as Jira, ALM, or any other test management tools. That integration is missing.

I've used Worksoft Certify for two years, and I've used it in the last twelve months.

Worksoft Certify is a stable solution. I'm rating it a four out of five in terms of stability.

Worksoft Certify is scalable, and I'm giving its stability a rating of four out of five.

The technical support for Worksoft Certify was really impressive. It was really good. We needed to create new controls, and we got a good response from their team. On a scale of one to five, with one being the worst and five being the best, I'm rating them a five.

The initial setup for Worksoft Certify was very simple. On a scale of one to five, with five being the best and one being the worst, I'm giving the initial setup a five.

Deploying Worksoft Certify took just one hour after getting the license. The implementation was done by our in-house team.

I'm not aware of any licensing costs for Worksoft Certify.

Worksoft Certify is in hybrid mode in terms of deployment.

Currently, there are 50 users of Worksoft Certify in the company.

My rating for Worksoft Certify is ten out of ten.

Our primary use case is for SAP regression testing. We wanted to have our all SAP processes for regression to be optimized using Worksoft Certify. It took around one year for us to take that process and learn how it works, gradually improving it until now. 

We do regression testing on a weekly basis using Worksoft Certify. This does not include just SAP. We are also using the web-based tools when we are integrating end-to-end processes. When we are moving within one end-to-end process, we are shuffling between all the scripts and then going back to SAP. It works quite well.

At some point, we will have an RPA initiative to use this solution. For example, we need to obtain data from our development environment to check reports and verify them, then create transactions using Worksoft Certify.

We have more than 160 end-to-end processes. In our repository, we can have one transaction process or more than 40. There are big end-to-end processes and there are smaller ones. There are processes that can take hours to run and there are processes that can take a couple of minutes to run, but most of the end-to-end processes have multiple transactions. In weekly/monthly releases, we don't have any type of manual testing. We are just relying on Worksoft Certify for its test automation.

We also use Worksoft Certify for test automation in projects.

For scheduling, we are using an Execution Manager from Worksoft Certify.

Our business and IT users have been able to use the Capture feature, even without knowledge of testing tools. We have different IT people from each business area, like production planning or material managements. These people, without even having any knowledge of programming or Certify, just need to know how to use Capture. They are using it because it's really easy to use. You just have to initiate a Capture and do the same work that you are doing every day. Then, Capture will record everything.

You don't need to do any coding when creating end-to-end processes; it's a codeless because it is just recording. We don't need a specific person (or multiple persons) to work with the processes to do coding for the automation, which saves time. It increases the confidence of the process owner or IT person working on it because we provide them the capability to record their own process without doing any coding and just using Capture. They are confident that whatever they have recorded is their own process. It's not someone else recording their processes. The test automator can polish that process, but we are not recording it since they initiated the Capture. Overall, this saves time.

It has reduced our test maintenance time by more than 50 percent because we don't have to do manual test processes. We have saved over 150 man-hours monthly. It has increased our delivery times. We went from 200 man-hours down (three weeks work time frame) to approximately 40 man-hours (three days work time frame).

Using Worksoft Certify, we have found and repaired more bugs, though not that many. Users cannot manipulate information during the process. We schedule our tests at the night time and leave them working. Next morning, when we come into the office, we just check our system. These tests are pass and fail. For the failed test, we have screen captures where we can go and check the screenshot to know what the issue is. No one has to sit in front of the screen when you are running regression tests or scheduling them. I can schedule them and continue with my other daily routine work.

It is really good if you are using Capture in the new version of 11, where you can have a visual of the capture. The new version is much better. If I am using the Capture, I just initiate the capture and start working normally with my transactions. Capture will start recording all the steps that I'm working with. Before sending that one, I can even edit the capture. I can add more steps or remove them. This is one of the best features that Worksoft Certify has.

Worksoft Certify is really easy to use. You don't need any programming skills. If you just know a bit about programming, then you can easily use it because most things record by themselves. 

It's fully capable to automate testing for packaged applications, like SAP and Salesforce. You can do whatever you want and even create one specific process. For this process, you can create multiple processes. For the one transactional process that you have automated, you can utilize that process in multiple other processes too. 

They have different tools for different types of documentation, like Certify and Analyzer (for business processes). In Certify, from the repository, you can get really good, detailed documentation of what has been recorded, how it works, what steps have to be taken, and what data to use for every step. From a documentation point of view, it is valuable and can be used for future training.

Worksoft Certify's ability to build tests is really easy. You can build as many tests as you want with Capture. It is very reusable to apply one process to multiple processes. 

The scripting methodology is easy to learn. It is easy to maintain because it is presented in a simple, narrative way. You don't need to know programming.

When you are using an older version of Worksoft Certify, like version 10, Capture will be running under your status bar. Then, you don't know what transactions you are currently recording. In the old version, you don't have the visual of the capture.

It works well with other tools, but there is some integration required with Solution Manager. If you are using test repository in your solution manager, then there is no direct integration between Solution Manager and Worksoft Certify. 

There is an improvement needed in the reporting within Worksoft Certify. We have either a detailed report or a summary report. We don't have report that can be used for training purposes. A different tool from Worksoft has to be utilized for this.

Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify.

They have a lot of versions coming out every year, like four or five versions. They need to reduce this number. There should be one or two versions every year with add-ons, if necessary. Because if you want to upgrade your Worksoft Certify from an older version to a newer version, you almost need to reset the solution and we don't have that much time. We cannot dedicate four times a year to having the newest version of Worksoft Certify. Though, if I don't need the need changes or improvements, I can skip the most recent upgrade until the next version comes out.

Almost six years. We started launching it in 2014.

It is very stable; nothing happens.

Two people are responsible for maintenance, deployments, and updates: the test automation administrator and the test automator. One is mainly responsible for all the administration and maintenance of the scripts and everything that goes into the integration with SAP Solution Manager. The other guy mostly works with the execution of the script. 

The solution enabled us to scale up testing because we don't need to use manually testing. We are also using it for project testing. 

There are overall 12 to 13 users. We have three users identified for test automation. We have another 10 users who use different packages from models of SAP. They use it to capture their processes.

If you have any issues, cases, or discussion, you can go on the Community portal and ask from there. They have people and technical experts there who can provide you with solutions. It's one of the best things that Worksoft Certify is doing. You can discuss with others at different companies, or possibly the technical experts from Worksoft Certify if we have an issue. Then, we can open a case directly there. 

I would rate the technical support as a nine out of 10. They are very good and available at all the times. The technical support can assist through the Community portal. You can also email them directly. Their response time to contact you back is 24 hours.

Before Worksoft Certify, we needed almost two weeks or more of regression testing to move our changes from regression to production. We were spending more than 10,000 euros for that on top of our man-hours. Even then, it was not proper testing because the user could manipulate things because they were using manual testing. For example, if something was not working, they could try to use some other way to work it. After the implementation of Worksoft Certify, we have increased our scope for regression and have more than 162 processes from all the models. We have also increased the test coverage and have less issues.

We went from two weeks and 200 man-hours to three days of testing. Within three days, we can test for the monthly release. For the weekly release, we need maybe one day or less using Worksoft Certify to test our releases and move them to production. It has really improved the collaboration between teams and our dependencies between various processes. On top of that, it provides really quick reports and results for management. We are capable of providing quicker reports and results for regression, like, "What has happened? What has gone wrong? Is everything okay to move to production?" Management is also very happy with it because it is saving time and money.

I was not involved in the process.

We have seen ROI with its efficient reporting and ability to provide results to management. It has also increased collaboration between different teams, such as, purchasing and sales.

We have used Worksoft Certify as an RPA tool when we have needed to create the master data or upload some documents. This saves a lot of time. It can reduce 10,000 transactions from weeks down to a day and a half.

Purchasing and licensing are okay. Go for the perpetual licenses. In that way, you own a license, then you can purchase maintenance and support on top of that, so you don't have to pay every year for it. Even if you don't want it a contract with Worksoft Certify in the future, you will have your own license of it. Then, if your usage is not that much, you can have one or two perpetual licenses. However, if you want to run your processes, you will need more licenses, e.g., using the run-only licenses. They are really cheap compared to the full licensing.

There are multiple licenses available from Worksoft depending on your requirements. You can take any one of them, and they are really good. For example, if we are using five to 10 perpetual licenses where we need to update or create scripts and run parallelly, simultaneously multiple scripts with just one license, it will take months for us to run 162 processes. Therefore, we have run-only licenses that we use when it comes to executing our test scripts on releases.

It is integrated with SAP. It's one of the best tool to be used if you want to automize your testing for SAP. No other tool is as good as Worksoft Certify.

I have evaluated other tools, but I find Certify really tops them. Most of them are using the command line-based tools. With some of them, you need a expert who knows how to do programming. For Worksoft Certify, you don't need programming skills. For the SAP GUI, I haven't found another tool that can work as efficiently with SAP as Worksoft Certify does.

You just have to be clear about your processes and define them before going to capture one. If you capture some processes, you can just easily manipulate and merge them to the other processes. It is really easy to use Certify, but you do need to be focused. There is also integration with Execution Manager, which can be useful when you are using Worksoft Certify for scheduling and other things.

It is not that easy to work with SAP Fiori as compared to SAP's GUI.

It can be used with non-SAP platforms. 

Overall, the solution is a nine out of 10. For just the SAP GUI, it is a 10 out of 10. For the other tools, like the web part and Fiori, it is about an eight out of 10.

It is used for codeless automation. I am using its most recent version, and it is on-premises, but we are using it through some virtual machines. It is installed on some virtual machines, and by accessing those virtual machines, we're using Worksoft Certify.

It is codeless. Typically, you have to write code in Java, Python, or another language to automate web applications or ERP applications, but Worksoft Certify is completely different. You do not need to do any coding, and you do not need to know any coding language for automation. It is completely a techno-functional tool. You can just enable the application that you want and start automating. 

There are certain tools in the market for writing the code, but they are not supported with some of the ERP solutions available, such as SAP, Oracle, etc. Worksoft Certify is not like that. It supports each and every ERP solution available in the market.

An organization doesn't need to hire new people for automation. For example, if you already have people who are well versed in SAP, you can just train them on this particular tool, and they'll be able to do automation for you. You do not need to hire any new resources for this particular tool, and even if you are hiring, you would need a less number of people. It saves costs in terms of resources. 

Scheduling and lights-out testing are very easy. You can schedule any number of test cases on a recurring or on-demand basis. So, you can run any number of test cases. Test maintenance is very easy, and you can very easily manage all your tests, which also saves cost. So, it saves costs in terms of resources and maintenance.

The Worksoft Capture feature is most valuable. For example, if you are creating a sales order in SAP, you do not need to go to each field and do everything. You do not need to write code for each and every line. You can just turn on the Worksoft Capture feature and manually perform your actions. It will capture all manual actions, and it will give you the steps. It will write the steps for you. 

They have a scheduler in Execution Manager, but it is not customizable. Its UI needs a lot of improvement. The lights-out testing is a bit difficult with that particular tool, and it needs a lot of improvement. Of course, there are so many integration options with Worksoft for execution, but when it comes to Execution Manager, which is their own tool, there is a lot of scope for improvement.

The integration with mobile needs to be improved. Initially, they used to support certain applications, and now, they are supporting all the web applications, but with minimal knowledge, it's very difficult for any tester to automate web applications. That's where they need to improve a lot. They are already working on it. They have given additional features, and with the help of those features, you can easily automate, but they need to keep making it easier for business users who do not want to get into each and every technical aspect of it. They just want to capture the actions. It is working fine with SAP, but they have to concentrate on web applications. They also need to support Safari, as well as macOS, better.

If you keep on running it for a long time without removing unnecessary things, the load on the database increases, which impacts the performance of the tool. Sometimes, it hangs or is slow. We have faced this situation with the earlier versions, such as version 9, but in the current version, there is an improvement. The slowness has drastically reduced, but a lot of improvement is still required in this particular area.

Another negative of this tool is that its license is costly. It is a bit more expensive than other tools.

I have been using this solution for close to six years.

Its stability is very good. It is reliable.

Its scalability is very good. We are a very big organization. In my team, we have close to 14 automation engineers, but there are so many other projects where Worksoft is used. I don't know the exact number, but it would be very high. Its usage is not extensive because it is mostly being used for ERP applications. 

In my previous organization also, it was implemented for many projects.

Their support is excellent. When you raise a ticket, within a day, you get the solution. If they're not able to resolve the issue at their end, and it is on our side, they connect with us and show us what exactly is happening. They check everything. 

Its initial setup is very simple.

It was implemented in-house. We get some files that we can directly install, and that's it. We do not need to do anything other than double-clicking the given files, and it creates everything automatically.

It is costly. It is a bit more expensive than Tricentis Tosca and other tools. If they reduce its price going forward and have partnerships with other vendors, it will grow like anything.

Tricentis Tosca is a strong competitor. Worksoft Certify is more expensive than Tricentis Tosca. The solution that we use depends on a client's choice. 

In the long run, most people will prefer non-coding tools, where 0% coding is required. It is, for sure, a tool that would be useful, but there is only one hiccup. If you are executing tests multiple times, you'll have various results, such as pass, fail, etc. When you are running it for so long time, the file size on the database will keep on increasing. So, you have to keep on doing the maintenance. For example, if you need the results for 2018, you should download everything. You would need some kind of storage.

In many organizations, the managers think that if they hire one or two resources with Worksoft knowledge, they can train everyone in their team, and build a good team, but that's not going to happen in the short run. That's why I advise having two to three dedicated people to train other people and build a team. With only functional team members, it is very difficult to build a proper automation team. I have seen an automation team that wanted to train 20 people. Currently, we are training 100 people. There are three of us who are training 100 people, which doesn't work out very well. So, there should be a few dedicated people for training and implementation. 

I would rate Worksoft Certify an eight out of ten. It is a fantastic tool. Its maintenance is easy. Building test cases is easy, and planning is easy. Almost everything is easy, but the look-and-feel of the Execution Manager scheduler is not good, and currently, customization is not available, and product creation is not available.

We have been working on test automation for SAP-based platform for the last 15 years and leveraging Worksoft for SAP test automation in last 10 years. 

We have some fantastic use cases on how we can leverage Worksoft in large SAP transformation programs, especially for major global rollouts. If you look at some of the industries, like retail and logistics, we can see a lot of consolidation. When there is consolidation, we need to merge two SAP landscapes to a single landscape instance. It is a very complex transformation program. The testing is labor-intensive work in such a large scale IT landscape program. Worksoft has played a key role in delivering these complex integration programs on time, because we automated the core processes using Worksoft Certify. This helped us to automate the testing scenarios in new companies being integrated. At the same time, we test the existing company codes on the existing landscape to ensure that business as usual is not impacted.

If you look at the industry trend, the next six or seven years will be SAP S/4HANA migrations and adoptions of SAP cloud applications, whether it is SAP SuccessFactors, Ariba, or Fieldglass. We strongly believe that WorkSoft can play a pivotal role in delivering large-scale SAP S4 HANA programs since Worksoft can support the implementation of SAP cloud applications. We have delivered a couple of major SAP S/4HANA programs by leveraging Worksoft Certify. 

One of the key things that we have done is introduce test automation on day one in system integration testing (SIT). We don't want to wait until the program is over, then start the automation development. We start the automation right from the build phase, then we are able to deliver test automation on day one to the SIT.

If you look at the customer's experience, we have seen that if you take any testing engagement, whether it is large-scale system integration testing activities, UAT, or regression testing, and have seen that companies involve their business workforce to do lot of manual testing. Then, the business needs to take from the business critical activities and focus on testing. Worksoft significantly reduces the testing effort involved, so they can focus on their business critical activities. 

The company has been investing in a lot of new features. They are changing the trend in market demand, especially with their Worksoft Analyze features. These are important for customers when they have massive test case documentations. In addition, Worksoft is changing their direction towards Robotic Process Automation (RPA).

If you take any Robotic Process Automation, the key things you need are:

• The ability to automate SAP.
• The integrated software applications.
• Support for continuous testing or providing a platform to adopt DevOps.

I can see that Worksoft has been investing in these areas to make the product more relevant for SAP-centric platforms.

Another fantastic thing about Worksoft Certify is using it for end-to-end testing of packaged applications. We are able to automate, not just SAP, but the entire application ecosystem. If you take any company, SAP is the backbone, and if they use SAP ERP, then, there are multiple software applications, where some of them are SAP and some of them are non-SAP applications. Worksoft is one of the tools which can transcend across SAP and non-SAP applications. Non-SAP application include Java or .NET. Worksoft can seamlessly automate these applications.

If you take any ALM tool, Worksoft can be easily integrated, whether it is Micro Focus ALM or SAP Solution Manager. Another important thing is Worksoft's integration with SAP Solution Manager BPCA, this is helping customers to set up a risk-based testing platform. They can do impact assessment and identify test scripts to be tested based on the transports to be applied. Then, import that testing suite into Worksoft Certify and trigger the automation. We have seen that bi-directional integration between SAP solution manager and Worksoft Certify. This makes the product seamless, and if you take any company, they have their own ALM tool and the level of usage of Solution manager is varying. WIth the API that Worksoft provides, it can be easily integrated with the leading test management tools available in the market.

I would like Worksoft Certify to do automation at any layer (the UI layer, API layer, or database layer) and challenge competitors in the RPA industry, like UiPath and Automation Anywhere. 

A lot of customers are looking at testing, not just at the UI level, but testing the application or their ecosystem at the API layer. Worksoft could invest on testing on APIs. There are some open source tools available in market which do this, like SoapUI. 

The stability of the automation tool to run without any performance issues has improved with the use three-tier architecture. The three-tier architecture in Worksoft can help the customers access applications across globe. If global customers have geographically diverse teams, we recommend customers go with a three-tier architecture.

We have seen the customers starting their automation journey from zero test scripts to 8000 or 10,000 scripts. We have seen that Worksoft is able to scale up, then run thousands of test scripts concurrently and deliver value.

One of our customers has about 10 SAP applications with about 70 plus non-SAP applications. So, we automate approximately 80 SAP and non-SAP applications using Worksoft Certify.

A lot of our customers have used the technical support. Worksoft has a very good customer success team, as well as technical support team. When we started our first relationship with Worksoft, we had to build the integration between ALM and Worksoft Certify. So, Worksoft was agile enough to build that integration faster. We can see that the technical support team is fast enough to resolve some of the product bugs for any customer queries quickly. The team is open to build any type of new integrations to support the emerging tools available in the market.

As a partner, we help our customers to invest in the right tools and platform. So, we educate our customers to buy Worksoft Certify. We help them to build a business case or plan and do a joint PoC with Worksoft. We enable and empower our customers with enough details and help them to invest in the right automation tool for SAP, which is Worksoft Certify.

Some of our customers do not have any tools in their landscape. In that case, it is easy to position Worksoft Certify. Other customers already have other automation tools in their landscape in which they are experiencing a lot of pain points with their automation tool. They invested a lot of money in their automation tools and framework, but they did not realize the benefits. Therefore, we help those customers move away from their existing tools to Worksoft Certify. If they decide to build on their existing tool and use Worksoft Certify, it's a very complicated decision. We need to build a very strong business case and we also need to help the customers to migrate the automation test suite.

Worksoft installation is straightforward. They have a dedicated team to support the customers during the initial installation. It doesn't take more than couple of days to complete the installation. A lot of our customers are very happy with the way Worksoft completes the initial installations. 

Also, the professional services offered by Worksoft to help the customer to set up the test automation best practices using Worksoft has been helping the customers to roll out enterprise wide automation using Worksoft Certify.

Worksoft Capture 2.0 can help our customers to accelerate their automation development at least 40 percent faster than any other commercial tools available in the market.

We have seen that the initial Worksoft implementation has helped our customers reduce their testing cycle time by 50 percent. With further continuous improvement, we have seen cycle time reduced up to 75 percent. That is the level of productivity achieved using Worksoft Certify.

If I compare Worksoft Certify with other script-based automation tools, the test maintenance is relatively faster. We save about 60 to 70 percent of the maintenance effort by using it because it's model-based and scriptless. This helps the business team and the automation testers to quickly identify the changes needed to be introduced in the automation scripts.

The typical value realization takes about six to 12 months based on the number of scripts and how often the customer wants to execute.

Worksoft can help you to select the right automation platform, then deliver value quickly.

Our customers evaluate three or four major products along with Worksoft: Micro Focus UFT, Tricentis Tosca, Testimony from Basis Technologies, and then some of open source tools. Some of the customers evaluate all the tools, and some of the customers pick one of the other tools along with Worksoft Certify to make their decision.

There are a lot of tools available in the market. However, if you take any automation project, the lead time to implement an automation solution is typically, painfully longer. It may not give the right return on investments. 

We could have gone with any commercial tools available in the market or open source tools, but going with Worksoft gave us a few advantages. We were able to deliver and implement automation two and a half times faster than any other automation tools. Then, if you look at the total cost of effort to build, maintain, and run the scripts in the customer landscape, it is phenomenally low compared to other tools in the market.

Worksoft gives you return on investment faster than its competitors. It will make your QA organization more agile and nimble.

We recommend our customers be very serious about automation and not to experiment with too many tools. Start with a small PoC or pilot. Involve their business team to articulate the value of what Worksoft can deliver, not just within the IT department, but also the business. 

Worksoft Certify has ease of use, ease of maintenance, and value realization. Automation is not just completing testing faster, it's about reducing production support incidents, after go-live or ongoing production support incidents. It's a collective total cost of ownership and all about delivering value to the IT operations team, IT project team, and the business team.

With Capture 2.0, we have seen the product evolve. Worksoft Capture 2.0 is helping our customers to accelerate automation development. Introducing automation around day one is only possible because of Capture 2.0. When we are in the build phase, we can capture the important screens, whether it is in SAP Fiori apps, the SAP GUI, or SAP cloud apps. During the development phase, we can start building the automation scripts, then start leveraging automation on day one in SIT.

Worksoft can support modern UIs quite easily, such as SAP Fiori and Oracle Fusion. If you look at the modern UIs, it is all about the customer experience, and we have seen that Worksoft can evaluate that modern user interface and ensure that the customer experience is delivered as expected.

The primary use case is to automate their SAP and Web applications. We use Worksoft Certify for end-to-end regression and integration testing across our systems.

It helps us to implement automation testing as part of most projects, so the need for manual testing can be reduced. This really accelerates the testing process as a whole. Before, where it could take ten days to test a project, now it takes only one or two days to do the complete testing. So, it helps us to reduce our testing timelines.

We no longer need ten people sitting and manually testing something. We can just have one person running the entire regression automation testing suite, and this has saved us dollars.

I have found Capture 2.0 helpful compared to Capture 1.0. It allows you to have access to all the screenshots when you use it so you see what steps you are capturing, and if you are capturing it right or not. It is very user-friendly.

It is easy to use and learn the application. For example, I have an intern who joined me three months ago. Today, she delivers the same number of scripts as my experienced developers with great quality.

One feature that could be added to Capture 2.0 is generating a PDF file from your capture, so you can see your screenshots and steps. This will really help teams leverage the documents generated as part of requirement/training.

Right now, when we do regression testing, we manually have to generate all the reports and populate all the results in HP ALM. We really are looking for a solution to have send all the results to HP ALM once Execution Manager completes the execution, then automatically logs them.

Our offshore teams experience a lag/delay when using the Worksoft interface. As of now we use VPN and Remote Desktop to help us with this issues, it was be great to see how much Certify 11 has improved in terms to offshore accessibility.

Every time there is a new release of Worksoft, they present it in a conference. However, there is no training document nor one point of solution where I one see what new changes/feature have been implemented, like a portal. If I don't know how to use a feature, there is no training nor documentation available. When you reach out for support, it takes time for them to research it and get back to us.

I would like more use cases or at least a weekly email update to all the customers saying, "These are new features which have been included in the last week." That would really help.

Worksoft with SAP is pretty stable. 

When it comes to the web, you need a lot of extra effort in making sure that the tests are maintained, but that is the nature of the web application. You have stuff that is changing all the time, so you have to ensure that you maintain your tests regularly.

We have learned the scalability part over time. We were very successful with SAP, but not very successful with the web. Over the period of about two years, we were able to prove to our company that we can use Worksoft for different type of applications. While there is a learning curve, it is all about trying things out and failing few times before you get a success.

I would rate the technical support as a seven out of ten. There is a delay in time zones when we reach out to them. The response is not as quick as we expect it to be with the other solutions that we have. There is definitely a delay in timing, as the speed is lacking.

The initial setup was easy. We had all the information and the requirements that we needed to do the prep work before doing the implementation.

We have Worksoft help us each time that we do an upgrade or implementation. Our  upgrades have gone smoothly.

An example of saving time and dollars: We had project going on that used to require manual testing. The first time that they did manual regression testing, we had a group of about 15 testers who sat in our office for a period of two weeks to do the testing. Now, when we have to regression testing with automation scripts ready in our Worksoft Certify, we do it in a day or two. 

We looked at quite a few vendors and automation tools in the market. We chose Worksoft because of the ease of use and mentorship support they provide in making sure we were successful.

For a new customer who is to implement Worksoft Certify, I would suggest 'Start the right way'. Have a Worksoft mentor come in and help you with your automation journey specific to your organization so you can have expert support until you become successful with it. Once you are successful, you'll know what to do. E.g., we had a team of interns who got trained and they tried to work with it, but it did not work. Then, we had Worksoft help us (after two years), and it worked. 

My team and I do the regression testing. We are a team of three to four people. We are not working on just one project, we are working on five or six different projects.

What's next, well I hope we are able to present that next success story in next conference ;)

Worksoft Certify is being used to run automated weekly regression tests across some of our primary SAP systems in line with our Change and Release management strategy. These tests run every weekend without fail. The results are reviewed on every Monday morning to check for failures and to analyse if any failures are associated with the changes scheduled to be transported to the production environment that week. Failures (if any) are fixed and the tests re-run before transporting the associated changes into the Production environment.

We also utilize it for projects that need extensive business-user testing and functional testing. There can be testing requirements which come at short notice which can take three to four weeks of manual testing effort. By using Certify, we have been able to bring timescales down to a few hours of automated testing effort.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems which will demand extensive regression testing. 

We have never had any systematic regression testing regime in the organisation. This has helped in building an automation framework across our SAP application landscape, thereby introducing mandatory regression testing across all our key systems and improving the overall quality across our production systems.

From an audit perspective, results generated from Certify (BPP reports) provides detailed test evidence which is also being utilized for internal training purposes/training guides, etc. The BPP reports also provide details on failures along with screenshots.

We have a variety of complex systems in our landscape, one of them being the Openlink Endur which is a commodity trading and risk management system. We are currently building an automated regression test suite to support application testing for Endur.

Our weekend regression tests are performed in 'lights-out; mode. Tests are scheduled to run at a certain time over the weekend using the Execution manager functionality. Usage of Certify has also prevented some major defects going into Production and we have seen significant savings in all manual testing activities as the business users/functional teams are getting more time to perform 'value- adding' activities.

Post our recent upgrade to Solution Manager 7.2, we are currently in the process of implementing the Test suite functionality and the integration of the same with Certify. We expect Solution manager to be the single source of truth bringing out all the results from Certify which is going to be extremely beneficial from an audit perspective. We have already implemented the integration of Certify with HP ALM in our landscape.

Moreover, we have this reusable asset now which can be run frequently to support all our projects and change requests across our legacy SAP systems. Even last-minute testing requests are being accommodated by utilising the automated regression suite without any dependency on business users/functional users for their efforts. We use it across the multiple projects which need immediate assistance and for our weekly regression cycles. To give an example of a recent project which was a major platform migration from a Data Centre in Asia to Europe which needed extensive Disaster recovery testing and Functional testing/User acceptance testing. The initial testing estimate was approximately five to six weeks, however with the use of Certify we could do extensive testing in less than three hours saving many weeks of manual testing effort.

It is very user-friendly with an appealing UI, unlike a lot of other automation tools that we have evaluated. With sufficient training and adoption of best practices, the tool will certainly help organisations to successfully implement an automated testing framework and eliminate manual testing activities.

The fact that it can be used across SAP and non-SAP applications (including web-based apps like Web Dynpro) is a big advantage for us because we have a variety of SAP and non-SAP applications across the Johnson Matthey IT landscape. Being a 200-year-old organisation, our variety of legacy systems have a lot to benefit from the use of automated testing.

Certify has many interesting features, e.g.: 'PRIMO' which is the image recognition functionality is a life saver in instances where Certify standard functionality cannot identify and learn objects within certain legacy applications.

Regarding end-to-end testing of packaged applications, Certify is primarily used across our SAP application landscape and the Openlink Endur (commodity trading and risk management system). We hope to realise more benefits by implementing Certify across our wider application landscape over the next few months.

We have been using the Capture feature, although not the latest version, the initial version, for process captures was used to create our test designs. It has been a life saver in many instances, without the need to spend any extra effort to create test designs and captures. The test steps get captured in the background which generates an XML file which can be easily imported into Certify, creating the basic test structure which can be improvised/modified to make it a repeatable reusable test. In terms of the amount of time it takes users to create documentation automation using this feature, it is the same amount one would spend to do a manual test. While a person is performing a manual test, Captures are automatically generated in the background. We have used it extensively to build our test designs.

We have requested for some minor new features which Worksoft is considering.

The PRIMO image recognition functionality has room for improvement, especially around its ability to work with java interfaces, Execution manager scheduling, etc. as we have observed. As we explore more of our legacy systems, I am certain there will be a need to use more of the PRIMO features to learn the objects.

Overall from a SAP perspective, it works almost seamlessly.

One to three years.

The version of Certify that we are using has been mostly stable and we have rarely encountered any problems. Our weekend regression test failures are often associated with environmental/system performance issues and not related to the stability of Certify. I have been happy with the overall performance of Certify and how it has helped to optimise our tests.

I am confident that Certify can scale to fit our automation testing needs as we expand the current automation testing framework across the wider Johnson Matthey application landscape. We are also exploring options to identify potential areas where Certify can help support mass data uploads, etc. to benefit other teams in their day to day operations.

We have several concurrent users accessing Certify in our environment, primarily automation engineers, test engineers and tech managers.

Our interactions with technical support has not been the best always and there is room for improvement especially with respect to the time taken to respond to cases. However, with the right contacts and reasonable escalations we have always managed to get quick attention on our issues.

We have always been heavily reliant on manual testing and as a result, regression testing was not systematic and we could never think of implementing frequent weekly regression test cycles which was challenging. We decided to go ahead with automated testing and use Certify because:

1. Manual regression testing takes a lot of time and resourcing is always a challenge.
2. Regression testing not being systematic, the quality was very difficult to measure as we did not have a standard set of manual regression test scripts/sufficient documentation.
3. There was a delay in our time-to-market because all the testing was being done manually and there was no way we could accommodate frequent, weekly, regression test cycles. That meant high business risk, that we would have more defects in the production environment/ more associated costs.

We had all these challenges and we started exploring options to mitigate these risks and automation was identified as the way forward, nearly two years ago. We evaluated various automation tools in the market. It was critical that we had to identify a strategic tool which would cater to our SAP and non-SAP application landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

We went through a lot of initial challenges, mostly around internal resourcing issues. Looking back, I am happy to say that we could overcome these challenges and have managed to successfully implement an automation framework using Certify.

Early in 2017, we decided to go ahead with Worksoft Certify post evaluation of multiple automation tools. Our initial engagements with Worksoft consisted of several onsite workshops to explore the tool in detail along with technical feasibility assessments across our application landscape. These engagements were extremely beneficial and it gave us the overall confidence to adopt Worksoft Certify as our strategic test automation tool.  

We did a pilot implementation with Worksoft to see if we could take this ahead on a large scale before embarking on the major project to build the automated tests. Some key processes across our critical SAP systems were identified as candidates for this exercise. Test designs were created with support from the functional teams and taken ahead for automation build with Senior Worksoft consultants and our internal resources. This 7-week Automation Roadmap Engagement exercise was extremely successful and we learned a lot of lessons from it which helped us plan the next big phase of the automation roll out. It gave us overall confidence across the functional and management teams which subsequently led to securing the appropriate budget, etc.

One of the biggest lessons learned from this engagement was around the ways to structure our teams. This led to us going ahead with a Managed Services model with Worksoft. We have an offshore based Worksoft Automation Services Factory team who helps build our automated tests. The team can scale up/down based on our automation forecasts.

The automation deployment is still ongoing. The initial phase was completed across a five-month span. Currently we are rolling out the second phase of the automation build focusing primarily on our global Unify solution and the Openlink Endur application.

Regarding implementation strategy, we followed an agile two-week sprint approach. Our functional teams continuously created test designs and these were fed to the Automation Factory every two weeks, who in turn developed the automated tests. This was the most practical model, which worked well in our environment.

At its maximum capacity, we have had approx. 10 to 12 automation engineers in the Factory team. Our functional teams are spread across multiple global locations and we had between 3 to 6 resources working on test designs liaising with the business users as required.

From a script maintenance perspective, we spend an average of 4 to 5 hours every week with the current asset of nearly 800+ tests.

We have always worked directly with Worksoft, along with support from our internal resources. Worksoft has been delivering excellent services through their managed services model.

We have a res-usable re-runnable asset built which is saving a lot of time across the functional teams/business user community.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems, which will demand extensive regression testing. 

It is expensive compared to some of the other automation tools in the market. However, the benefits and ROI has proved that it has been a good investment.

We have concurrent licenses.

SAP TAO and Micro Focus UFT.

It was critical to identify a strategic tool which would cater to the testing requirements across our SAP and non-SAP application (including web based apps like Web Dynpro) landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

It is a great product and we have not seen anything which cannot be automated till date in our application landscape.

It is important to do sufficient technical feasibility assessments before deciding to go ahead with Certify and equally important to determine the best implementation approach which will work for your organisation. Functional teams/business users' buy in is critical as the test designs cannot be created without their continued support. Adoption of best practices around naming conventions/folder structures etc. will help in easy overall maintenance of the test assets, which will also help with the generation of development and execution dashboards/overall reporting.

I would rate Certify at eight out of ten. Worksoft has always been very supportive and responsive to our needs and this has certainly helped us achieve our initial milestones successfully. I am extremely proud of what has been achieved so far and looking forward to expanding the automation framework across our wider IT application landscape over the months ahead.

In my company, the solution is used for SAP automation.

The technical support of the product is an area of concern where certain improvements are required.

I have been using Worksoft Certify for more than three years.

I am satisfied with the tool's stability. There have been no issues with the product's stability.

Stability-wise, I rate the solution a seven to eight out of ten.

The tool's scalability features did meet my company's goals. Scalability-wise, I rate the solution a seven to eight out of ten.

The solution's technical support was pretty good. There is a ticketing system and a call system to contact the product's support team.

I rate the technical support a six to seven out of ten.

Neutral

For automation purposes, my company works with multiple open-source tools, like Selenium and Java, along with a few license-based tools, like Worksoft Certify, QuickTest Professional, HP ALM, LoadRunner, IBM Rational Performance Tester, and OpenText Silk Test.

The solution is deployed on an on-premises model.

There is a separate team in my company who has taken care of the product's deployment phase.

I chose the tool for my company since it suits the purpose for which I use it.

A specific feature that I found to be the most valuable in the solution for our company's work processes stems from the fact that it is useful as a low-code automation tool.

The tool is good for dealing with end-to-end testing scenarios in our company, and in my organization, we use it for SAP testing.

The tool is useful for functional testing, and it did add some value to my company.

The tool is useful to automate business workflows. My company used the product to automate the test cases for whichever systems were under testing phases.

I used the low code automation functionalities provided by the product. I did not use the scriptless test automation feature of the tool.

I rate the overall tool an eight out of ten.

Its primary function is test automation. We record the test procedures and execute them in an automatic way. It gives us reports, in a centralized way, of all the information that we need to manage test automation for the ERP system and a few other systems as well.

We moved from manual regression testing, once a month, to continuous testing every second day, with the same scope or even an increased scope now. We don't need to focus on the manual regression tests, so we've freed up hands for support activities and new developments. I don't think that it has changed how our organization works directly, but we have become more efficient and more flexible.

In our estimation now, we save about 160 hours of manpower every month because we use Certify. It's 80 hours of robots executing the tests, and they're doing it roughly three times faster than humans. To do the same manually, and keep up with the general pace of development, we would spend 240 hours on testing every month.

In terms of codeless, end-to-end process automation across packaged applications, technically speaking there is some programming or development, like you would learn in middle school. But you don't need to be a professional in any programming language to run it. That's how we've done it. We don't deploy any additional software packages with it. We haven't developed anything else, other than using Certify steps to run end-to-end test automation.

Nevertheless you still have to have a mindset of a developer or an engineer. You need to have understand cycles and other primitive functional blocks. Still marketing-wise, I would call it codeless due to its simplicity. 

SAP is our core and the people involved with it are not always developers. Some of them are analytics or system experts. They don't know how to write C++ or Python or Java. But they can do testing automation in Certify themselves without help from a centre of competence (CoC). The only time CoC will participate is when truly complicated logic required such as runtime variables staking.

In usual situations, the skill set to start doing test automation needed is much lower.

There are a few aspects that are valuable. First of all, there is the screen-grabbing where you can pick up objects and it automatically recognizes them on the screen. Worksoft call it Certify Capture or LiveTouch. It's automatic capturing properties of the UI objects on the screen or on a mobile device for use in test automation scripts.

Other important feature is the modularization of the tests. That's an important feature not only of Certify but of IT tools overall where you create and reuse the components. Our test scripts are done with a "Russian doll" approach. That's standard for modern IT, however sometimes you see IT products without such functionality. Worksoft Certify's ability to build tests and reuse them is done pretty well and balances simplicity with flexibility.

Due to it, the learning curve improved even further allowing non professional test developers to plug-and-play test scripts. Such approach allows you to have that nice, modular end-to-end test. Can it get better? Probably not without sacrificing some other nice features and simplicity.

In terms of the core functionalities important to us there are the actual coding, great test data management, execution and reporting. Those are the basis, which are shared across many test automation tools. 

We also value that the test execution and reporting are stored in a database. We can extract the results and get the data out more easily comparing to text log files. That's different compared to other tools on the market. In Certify, we have a structured test execution and overall structured data.

Last not least is multi interface support is important for our applications landscape. With certify we can test SAP, Net, Java, Web and Android applications with one tool.

Certify's web UI testing abilities for testing of modern applications like SAP Fiori was good when we started and they developed it to be even better. We all know that web applications can change objects in DOM quite fast and it breaks tests. To counter it Certify has made object recognition more flexible and generic, so we don't have any troubles.

For standard and UI stable applications like SAP ECC 6.0 we have no trouble at all due to predefined objects and rich library of methods. That means we don't need to spend time writing a library, which could happen in some of the open source frameworks.

One caveat is that if you start running models in different parts of the end-to-end process — when you really try to hit the sky and make everything automatic, to cover multiple supply chain tiers processes in one e2e test, or similar processes that are really complicated — then tool simplicity turns into disadvantage. On other hand it stops us from unnecessary over engineering the test automation framework.

Architecturally, because Worksoft specifically built a database-oriented application, you essentially store code in the database. Git and text files orientation is more traditional approach with boundless set of tools to control versions, manipulation and analysis. But at the same time, Worksoft supplies us with their own version control inside Certify that has sufficient functionality for now. 

When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us. 

None of these are showstoppers for our operations. Worksoft proved to delivery significant improvements in last 3 years and more we wait from 11.5 version. Overall, we are quite well covered with test automation related tools and nothing special is needed.

We've been using Worksoft Certify since 2017.

It's stable. Our approach is that we don't deploy the latest fresh version to production straight away. We wait for a few patches and follow with upgrade instructions. It doesn't matter if it's Certify, DB server, Linux or anything else.

Only case I could recall related to stability problem is a one mess-up in the database during over-few-versions upgrade about three years ago. It required some database cleanup with scripts provided by vendor support.

But looking at the overall stability, it doesn't go down or crash. SLA is 100 percent for us.

Because it is three times faster than manual testing, we can test more with fewer people. That gives us operational scalability already. Platform also has the Execution Manager tool if we would like to have technical scalability via a cluster of virtual machines. In addition Certify itself supports multi-agents, APIs, and integration with Jenkins and other applications including PowerShell. That means we can also develop scalable setup ourselves

As for overall scalability of Certify, licensing is the main bottleneck. You cannot scale more than you have licenses for. You cannot uplift your digital workforce just for one month and run 300 nodes strong cluster for a big upgrade tests. It is not different for majority of IT products, we don't see much of a problem for now.

Certify is our main tool for test automation for ERP, for PLM, and HR. It's covering two major applications that are at the core of any big machinery or manufacturing corporation. We continuously increase the scope, because systems are developed and extended.

We use their enterprise-level support and it pays out. We mostly use it for tricky situations. We often talk with 3^rd level, with architects and other experts behind the certify. It gives us good insights. Separate gratitude goes to active key account management on vendor side.

As for our improvement requests, we speak with the community and we speak with architects and provide necessary information. We don’t know do they take it into their backlog or is it just coincidence that Worksoft have provided most of things we asked for. In any case we get what we require for our testing, we in constant contant and we feel that we are listened to.

We did not have a previous technical solution.

The initial setup was pretty straight forward, in general. There is a database, there are clients to install, and you need a licenses server.

The key thing that you need to know is what you are doing, how you do test automation, in general. That's above the specifics of any tool. You must know how to slice system-under-test into functional components, make the right model for reuse. It is important to break down end-to-end testing into smaller reusable items. That's not trivial, and you faced such challenge with any tool. You have to know what you're doing.

The deployment included one of week training and then a proof of concept for a couple of months. We ran typical end-to-end process in the proof of concept and found our ways to deal with above mentioned slicing, development guidelines and how to establish roles. There are useful guidelines provided by vendor as well. Overall it took about one quarter from the GO decision to start running regression tests.

Our approach to testing is as a "safety net," so we don't need to have end-users who run it and hunt for new defects. It's just continuously testing given scope and raises a red flag if something goes wrong. Such approach secures an immediate feedback for the development team or for the quality assurance team.

End users are not involved into testing usually although we could run hybrid testing with a mix of automatic and exploratory tests.

We didn't work with any third-party. There was initial training from the Worksoft itself for one week. That was it.

The return on investment, when we count the 160 man-hours we save monthly, will take about seven years for our scale

License model is a traditional one. As with any other manufacturing company, the software testing is not our core business, it's not in our interest to invest capital into the licenses. It would be preferable to have a pay-per-use model. At the same time, it's a fair game, for now. 

Aside from their standard licensing fee, if you want to have enterprise support, there's an additional cost.

We have done a few comparisons and we're continuously looking at the market to see if there is anything better, cheaper and more suitable. Market is changing a lot, RPA solutions expanding to test automation market, Robot Framework ecosystem is thriving and some other established vendors are operating. We're keeping an eye on it for evaluation. So far, cost-wise, feature-wise, and in terms of the learning curve or skills requirements, we are good with Certify. From our comparisons now, we would pick Certify again.

As always, you have to know what you're trying to achieve. You also have to think about how do you model your system-under-test; how it is written, the quality requirements and standards. The key benefit comes from reusable test scripts. 

When it comes to Certify itself as a tool, keep your eye on objects. If you map something, keep a good inventory so you understand it and you don't put all the objects in one big basket.

For me, the biggest lesson from using Certify is that you can do quite interesting and complicated things with the codeless approach. You don't really need a complicated programming language to do testing. 

Overall, I would rate Worksoft Certify an 8 out of 10. Of course we would like it to be cheaper and more powerful and developed faster. But are there products that I would rate a 9 or 10? I don’t thinks so. It's at the top of the market from what we know about the market.