SmartBear TestComplete Review

It reduced the regression timeline for a complex AUT, but there's an issue object mapping related to the need to modularize the test script code.


Valuable Features:

TestComplete is used for testing Windows desktop products; specifically the Embarcadero VCL interfaces created by Delphi/C++. All debug flags must be enabled during compilation that generates an output file required by TestComplete to identify and interact with the UI objects in the application.

An important feature of TestComplete is the ability to modularize testing. A lot of effort has been put into breaking the test script into reusable functions/methods that can be called from any test. A number of function libraries were created. This enabled reuse of code and kept the projects and project suites small. This is important because the size of these test artifacts have an impact on project loading time.

Improvements to My Organization:

Test automation with TestComplete significantly (estimated 80%) reduced the regression timeline for a complex AUT with a large number of test cases. Automation with TestComplete has significantly shortened the feedback loop and the timeline to get a release production ready. A secondary benefit is that manual testers have begun thinking more technically about writing tests cases.

Room for Improvement:

There are two major areas for improvement:

  1. Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
  2. There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.

Use of Solution:

I have used TestComplete for 20 months. It was chosen as an incumbent toolset could not interact with the product to be tested.

Deployment Issues:

Integration with third-party products; specifically Microsoft Team Foundation Server and HP ALM could not be overcome. A custom integration to HP ALM was written using the HP ALM OTA API.

Stability Issues:

TestComplete has been a stable product.

Customer Service:

Customer Service can be slow to respond to electronic forms of communication and they do not have a way for a customer to speak directly to customer support. You create a ticket online, and request a phone call.

The team seems to be very knowledgeable when communication is established.

Previous Solutions:

TestComplete was added to the toolset. HP UFT is used for automated tests for other products.

Initial Setup:

Setup is straightforward unless third-party tool integration is required. Integration with Microsoft Team Foundation Server is a little complex for initial configuration. Once it is understood the process is repeatable.

Implementation Team:

Implemented in-house. Implementation is not difficult to implement or write tests especially if you have experience with other test automation tools.

ROI:

We have not done a ROI calculation. However, automated testing with TestComplete has cut regression test time by months.

Other Solutions Considered:

HP UFT was tried but the object recognition did not work with the implementation technology.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment
Guest

Sign Up with Email