SmartBear TestComplete Review
Its environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.


Valuable Features

Object-based recording, and data-driven testing.

Separation of data into Excel files made tests modifiable by QA personnel with limited development experience, and object-based recording kept maintenance to a minimum.

Improvements to My Organization

TestComplete's environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

This allowed us to schedule execution of lengthy tests for nighttime and non-core hours, and to synchronize tests with Jenkins build releases.

This freed up QA assets to perform more specialized testing and reduced redundancy.

Room for Improvement

Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

Use of Solution

I've used it for approximately eight months.

Customer Service and Technical Support

Customer Service:

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

Technical Support:

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

Initial Setup

The initial set-up was exceedingly simple. There is a silent mode option available during set-up, which is very convenient when deploying to multiple machines or remotely.

Implementation Team

We implemented it through an in-house team.

ROI

We did not maximize our ROI until we put somebody full-time on our TestComplete endeavours. The tool does have a learning curve, and it wasn't until we had an in-house expert on it that we began to see the benefits of automated testing over traditional QA roles.

Pricing, Setup Cost and Licensing

The licensing options for TestComplete both running a licensing server. This prevents users from running more sessions simultaneously than purchased keys. This can prove problematic if you want multiple developers writing or running tests at once, and prevents you from using your key while a distributed test is running.

For pricing, carefully consider how many machines you want running the software, rather than the number of developers.

Other Solutions Considered

We also evaluated another SmartBear product called SoapUI. The change to TestComplete occurred because we changed our target from web applications to desktop.

Other Advice

My advice in regards to implementation would be to choose carefully which tests to automate, specifically focusing on lengthy procedures, tasks that require looping, or places where you want to test against multiple data sets.

Additionally, I found it beneficial to prefix my keyword tests with a character and number to provide logical ordering instead of alphabetic.

I also found it beneficial to record "undo" steps with each keyword test; this allows each test to be more stand-alone and prevents your test from being dependent on the state the previous test left the application in.

Finally, I would suggesting limiting the number of test applications per test suite to prevent bloated name-mapping schemes.

Disclosure: I am a real user, and this review is based on my own experience and opinions.

3 Comments

it_user336978Real User

I used Test Complete long before for Load Testing (before it was separated to Load Complete) and I never used it as a multi project.

All I can say is that, live UI and after test reporting features were fine, but biggest gap in the tool in that time was timing issue. There was no time based testing, it was only running one time, that’s we could not get a reasonable answer for that.. and that’s why we were switched to Visual Studio at that time. But I don’t know the situation in Load Complete now, I did not use it and I don’t do performance testing any more.

Thanks,

Rasim.

16 November 15
Deva VeluchamyReal UserTOP 5POPULAR

Comparatively, Testcomplete reporting is better and customizable. It could have multiple projects in a single project suite and reporting for the entire suite can be obtained which should solve the purpose of having different projects in a single suite. Only improvement from automation developer perspective I would expect is their editor which has very basic features comparing some open-source code editors.

19 November 15
it_user336978Real User

Test Complete provides detailed reports since every profit software should do, since people pay for its reporting functionality also. On the other hand, some big and expensive products stil has the reporting gaps, for example there are third party reporting tools for HP QC.

19 November 15
Guest
Why do you like it?

Sign Up with Email