SmartBear TestComplete Review

The automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers.


What is most valuable?

  • SoapUI/ReadyAPI integration
  • Selenium integration
  • We run all of tests using TestExecute, this is just the executer for all TestComplete tests – command line version so can be integrated with any CI tools
  • We used to these tests seamlessly on Jenkins (TestComplete has a plugin for Jenkins)
  • The best part is TestComplete gives choice of selecting any of the six languages it supports
  • List of checkpoints available in TestComplete is one of the strongest available in the market
  • We also make heavy use of version control integration and JIRA integration
  • Data driven testing feature of this tool is one of the best

How has it helped my organization?

We did a POC for number of tools in past and eventually decided on TestComplete. We started with one tester and one test which we used to trigger manually. As the product matured, it added cross browser, mobile testing functionality, integration with Selenium/ SoapUI and Jenkins plugins, so we started using it more and more. Currently, the automated test team size is 20 and we run hundreds of tests automatically at the end of build process without any human intervention. For us, the automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers. The ROI has improved significantly and all the regression tests for our product are automated. We have plans of using TestComplete in the test driven development approach where developers can make use of our tests as part of their unit testing and I am sure we are going to achieve a lot more from this tool in coming years.

What needs improvement?

  • Error handling features in the tool are a little limited
  • The tool currently is Windows only, would like to see it being implemented for Mac and Linux
  • Once the tests are created in one languages, you can’t change it use other languages

What was my experience with deployment of the solution?

There have been no issues with the deployment.

What do I think about the stability of the solution?

The tool sometimes seems a little unstable and crashes sometimes on Windows 10.

What do I think about the scalability of the solution?

There are no issues with the scalability.

How is customer service and technical support?

Customer Service:

Average

Technical Support:

Good

Which solutions did we use previously?

No. We did a POC on multiple tools and TestComplete was our first selection

How was the initial setup?

StraightForward

What about the implementation team?

In-house

Which other solutions did I evaluate?

QTP, TOSCA, RATIONAL, RANOREX

What other advice do I have?

The tool has great capability but also has great potential to demoralize if you implement it in incorrectly. Automated tests complement manual testing and doesn’t remove manual tests completely so I would suggest keep your manual tests, they will come handy as reference when the automated test fails or needs an update. I would suggest following before you start implementing the tool in you automated test environment

  • Choose the language to be used and stick to it. TestComplete supports six languages (VBScript, JScript, Python, DelphiScript, C++Script, C#Script)
  • Choose the naming convention for objects in the name-map and keyword tests so that you can make use of these tests in several projects
  • Always try to reuse the tests with parameterised values wherever possible rather than duplicating the tests
  • Version control the tests so that you can maintain multiple versions for multiple branches of your tests
  • Define and document a standard process for automation and communication
  • Execute the tests frequently so that you can get the best out of the tests and defects are identified as early as possible
Disclosure: I am a real user, and this review is based on my own experience and opinions.
1 visitor found this review helpful
Add a Comment
Guest
Sign Up with Email