What is most valuable?
TestComplete is used for testing Windows desktop products; specifically the Embarcadero VCL interfaces created by Delphi/C++. All debug flags must be enabled during compilation that generates an output file required by TestComplete to identify and interact with the UI objects in the application.
An important feature of TestComplete is the ability to modularize testing. A lot of effort has been put into breaking the test script into reusable functions/methods that can be called from any test. A number of function libraries were created. This enabled reuse of code and kept the projects and project suites small. This is important because the size of these test artifacts have an impact on project loading time.
How has it helped my organization?
Test automation with TestComplete significantly (estimated 80%) reduced the regression timeline for a complex AUT with a large number of test cases. Automation with TestComplete has significantly shortened the feedback loop and the timeline to get a release production ready. A secondary benefit is that manual testers have begun thinking more technically about writing tests cases.
What needs improvement?
There are two major areas for improvement:
- Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
- There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.
For how long have I used the solution?
I have used TestComplete for 20 months. It was chosen as an incumbent toolset could not interact with the product to be tested.
What was my experience with deployment of the solution?
Integration with third-party products; specifically Microsoft Team Foundation Server and HP ALM could not be overcome. A custom integration to HP ALM was written using the HP ALM OTA API.
What do I think about the stability of the solution?
TestComplete has been a stable product.
How are customer service and technical support?
Customer Service can be slow to respond to electronic forms of communication and they do not have a way for a customer to speak directly to customer support. You create a ticket online, and request a phone call.
The team seems to be very knowledgeable when communication is established.
Which solution did I use previously and why did I switch?
TestComplete was added to the toolset. HP UFT is used for automated tests for other products.
How was the initial setup?
Setup is straightforward unless third-party tool integration is required. Integration with Microsoft Team Foundation Server is a little complex for initial configuration. Once it is understood the process is repeatable.
What about the implementation team?
Implemented in-house. Implementation is not difficult to implement or write tests especially if you have experience with other test automation tools.
What was our ROI?
We have not done a ROI calculation. However, automated testing with TestComplete has cut regression test time by months.
Which other solutions did I evaluate?
HP UFT was tried but the object recognition did not work with the implementation technology.