Worksoft Certify Review

Easily develop test automation


What is our primary use case?

SAP HANA S/4 HCM, FI, BI, Payroll, SuccessFactors, Fiori, other web HTML interface applications.

How has it helped my organization?

Our company automates a lot of functional tests for SAP. The automated tests are included in larger integration tests of SAP transaction, workflows and end-to-end processes. Successful implementation and execution of Worksoft Certify tests has been faster than execution of the tests manually. With other tools, it was hard to get successful technical execution right quickly. Automation is not easy. Worksoft Certify makes it easier yet flexible enough to handle some of the most complex automation tasks. Every time steps are automated, test execution times are reduced and application problems are identified faster. Every time that the automated script runs (many times if developed early enough and applied with each change of the application being tested - in the current environment and leveraged in another environment), the investment of work made to develop the script is realized. The more times that the test is executed, the risk of problems going undiscovered is reduced. Testing time is shortened. Avoiding the tedious tasks of manual testing easily repeatable steps saves money.

What is most valuable?

Worksoft Certify provides the ability to develop test scripts with a low bar of entry among automation tools. It’s very simple to develop. However, great care must be taken to use the tool as intended, otherwise efficiencies and effort savings will not be realized. Like all automation projects, well-designed automation, with thoughtful planning and application, is necessary or you will unable to get back the return after spending the cost in investing time and effort with the tool.

Worksoft Certify provided outstanding feature support for SAP test automation which is unrivaled by other tools in the market. If implementing and need to automate SAP, first pilot the tool with a representative scenario, and then evaluate that retesting the scenario multiple times will ultimately provide compelling overall value for SAP test automation effort.

What needs improvement?

Worksoft Certify can better identify web test objects by providing libraries through its Extensibility framework. It has improved with it's latest version through Extensibility Tools support and additional test object attributes. Object identification is a necessary barrier for the technical execution of a test script, aside from the script's needs to verify business rules. Automation engineers must affirmatively answer the question: Is the application automatable?

If Worksoft Certify or any other test automation tool can get object identification down consistently and flexibly so that maintenance of web object identification (and hence the web test script) is easy and done with no execution problems, it will allow more scripts to be developed and applied faster with the human test automation staff resources on hand.

A good part of the technical test automation work is to make the script run and affect script changes when the application under test changes. When done right, the business part of automating the test application functions remains the same - leveraging your test investment.

Certify may not the best solution for your needs, but among the commercial and open source offerings in the automation tools space, it should rank very high. It is also a leader in rapid SAP test automation, and can be used as general all-purpose test automation tool for varied clients like PC, SAP, and especially Web.

Only one or two tools I have experienced could be suitable in place of Worksoft Certify. However those tools also carry their own set of disadvantages.

For how long have I used the solution?

Three to five years.

What do I think about the stability of the solution?

In terms of stability, Worksoft Certify is a polished and mature commercial offering. We rarely encounter problems with the tool. When we do, Worksoft is very responsive to the issue.

We also find the new features in version 11 of the tool to be innovative. The more friendly and fast UI, greater support for test object Extensibility support, Capture 2.0 functionality and global search are all market-driven features that benefit the bottom-line in creating and maintaining Worksoft Certify automation script assets. The improvements are welcomed and we look forward to even more such improvements to make Certify a top-tier competitor in the automation market.

What do I think about the scalability of the solution?

In terms of scalability, we only have a few licenses. We manage and maintain the underlying database actively to avoid scalability issues. We are confident that Worksoft Certify can scale to fit our automation testing needs as we:

  • Scale-up with many more automated tests of varying length
  • Scale-out with unattended lights-out execution.

How are customer service and technical support?

I would give technical support a rating of 8/10, 10 being the highest. Worksoft is very quick to respond and address customer problems identified with Certify.

If you previously used a different solution, which one did you use and why did you switch?

My organization has been using Certify since 2007. We are committed and very happy with Worksoft. I picked up experience with Worksoft Certify when I joined my organization and leverage my existing automation skills to quickly become effective with the tool. I still use Selenium/Java/Python and was expert-level proficient several market leading enterprise QA automation tools.

How was the initial setup?

The setup was done prior to my joining the organization. The existing work helps make my drop-in experience very smooth. It would be harder from the ground up, since there is a database and application server that needs to be installed and administered, and procedures and policies to define so that development is standardized (establishing naming conventions, specific ways to do things when there are multiple options available, etc.)

What about the implementation team?

We implemented and developed in-house expertise. A vendor team would have accelerated and made adoption smoother.

What was our ROI?

We get at least a 2x return on investment (time, cost, labor) over manual efforts. When tests repeat execution the ROI can go higher. Tests often repeat when executed annually with planned annual system upgrades and additional time, labor and cost to maintain factored in. Including and reusing the tests in other testing activities such as sanity checks and regression cycles increases the overall ROI and investment value.

What's my experience with pricing, setup cost, and licensing?

Conduct a proof of concept (POC) so that you understand what needs to be invested with Worksoft Certify. You will need to understand what sort of time you need to initially set aside to learn and apply the tool, and how that time can be leveraged to shorten continuing development of Worksoft Certify test scripts (processes) as facility in the tool increases.

  • Someone needs to know to administer it.
  • Someone needs to know how to develop the tests.
  • Someone needs to develop a representative automated test script for a candidate application in your organization.
  • That person also needs to execute the scripts and then report results of the scripts.

That script needs to execute several times. Now a comparison needs to be done with the time it takes to execute the multiple test runs against the same time it would take to do the same thing manually. In this way, you can now quantify your savings.

Now apply the need over the instances in your organization and compare that to the pricing and licensing to see how many instances of this effort would give you a return on your investment.

You will find that you will have many instances that would justify the price and licensing costs.

Which other solutions did I evaluate?

We evaluated QTP and Selenium. Worksoft Certify was chosen because of:

  • The comprehensive coverage
  • Extensive support with SAP
  • Additional web test automation support
  • Ability to develop tests fast and execute them reliably
  • Technical support of the tool

The combination of all narrowed down our choice to Worksoft Certify. It was a great choice in 2007, and if evaluated today instead of 2007, it still would have to be Worksoft Certify since the importance of SAP support is priority and is significant over any of the other criteria and support provided by the other competitors.

What other advice do I have?

Perform a pilot. Apply it to your app under a test with representative workflow. Understand the process of developing Worksoft Certify tests by doing it. Execute the develop tests multiple times. Can you live with developing the tests for Worksoft Certify? Is there value in the execution over multiple times quickly? Can you justify this effort over the cost of your current practices? If you can, then this incremental step can be leveraged and then take you further into more automation successes and favorable outcomes with Worksoft Certify.

Our organization usually tests on beta and production releases of Worksoft Certify so we are privileged to be able to work with the tool ahead of the general market. In that, we experience more issues that would not get out to production releases. Also even after release we work closely with Worksoft Certify support to identify and resolve Worksoft Certify functionality and may receive such priority support than a normal Worksoft Certify client would experience since we have such intimate technical knowledge, experience and relationship with Worksoft and the Worksoft Certify automation tool.

Because we are an active participant in a robust enterprise commercial process to help improve the stability of Worksoft Certify (staged releases to select customers to improve quality prior to some production release), I would say we’re contributing towards a good job of keeping up the stability of the automation tool by testing it first hand in our complex situation and environments and responding back with realistic and practical feedback encountered. We accomplish our organization's mission to develop and run diverse, varied and large amounts of Worksoft Certify processes automation and also assist Worksoft mission to develop and improve the Worksoft Certify automation software.

When speaking on stability, I refer to the stability of the Worksoft Certify software tool. Execution stability of the developed test scripts (Worksoft Certify processes) is a whole separate concern. Even so, Worksoft Certify also shines here in comparison with other tools. To really address this properly and clearly, you need to automate a small test scenario in an application that is to be tested and compare the tool script execution with another tool to see that Worksoft Certify achieves a high-level of repeatability and resiliency in script execution.

Even so, we do work with an occasional problem that comes out in production releases and experienced that the interaction and speed to which the problems are addressed and resolved by Worksoft Technical Support to be extremely satisfactory. I am not sure if this is representative of all of Worksoft’s clients, but we are very happy with their speed and service with which they handle our issues. This contrasts with the unsatisfactory paid subscription support experienced with commercial HP QTP/UFT and IBM Rational tools, and the lack of any support (other than searching the internet and relying on the user and development community) for open-source tools like Selenium and JMeter.

Without the interactive and engaging support, some of these problems would be extremely difficult to solve on our own. However Worksoft solutions were technical in nature beyond what a normal client user could identify and resolve on their own. Using the available and comprehensive technical support and educational services increases the likelihood of success and a positive outcome and benefit with Worksoft Certify. Successes with Worksoft Certify will come from meaningful collaboration to address automation issues rather than handling problems through independent or isolated effort (e.g. the problem could have been addressed with another customer and the resolution could have been worked out already. Worksoft support would be in the best position to know the possibility that a solution exists). 

This is typical of automation tools and the experience of other tools in the automation tool trade. The knowledge and experience is specific to the tools. As such, going to the source who has the most experience is the recommended strategy.


Disclosure: My company has a business relationship with this vendor other than being a customer: Our organization is a long-time customer of Worksoft Certify. We have a close customer relationship with them participating in Americas SAP User Group (ASUG), SAP Sapphire with them and attending their Customer Users Conference regularly.
2 visitors found this review helpful
Add a Comment
Guest
Sign Up with Email