SmartBear TestComplete Reviews

The name mapping function to manage the objects across different web pages and the support for different web browsers are the most valuable features for us.

We use TestComplete to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%. This helps to reduce manual effort dramatically, and by using the same script, we can run the test on both IE and Google Chrome.

The scripts are saved as binary files on disk. This makes it troublesome to perform version control and merge it with the work that is done by other team members. I am not sure what the reason is that SmartBear chose to save the files as binary, but if they can change to plain text for the scripts in the future, that would be helpful.

I've used it for one year.

Some of the objects/parent objects are dynamically created in the web page. During the test, TestComplete will have problem to identify these kind of objects. This will cause a lot of false rejection and block the following test cases. Thus it requires some rework when running all the test cases as a batch.

I have not used any customer service/tech support yet. I cannot provide more comments on that.

TestComplete is the first tool that I use for automation on UI and I have not used any other tools yet.

It is pretty straightforward. Just by following the instructions given in the document and do the corresponding configuration on the web browsers that you want to test. However, we also encounter some problems such as the Chrome blocking users from proceeding because of an unsafe SSL connection, but we found a work around eventually.

We implement the automation by ourselves, not through any other vendor team. My suggestion is that you need to decide how to divide the automation work into corresponding components carefully. Otherwise, it will be hard for you to share the common scripts with other team members. Also, the code architecture design is very important as well.

I am not responsible with the licensing quotation/renewal, but the benefit that TestComplete brings to us is, apparently, acknowledged.

Object-based recording, and data-driven testing.

Separation of data into Excel files made tests modifiable by QA personnel with limited development experience, and object-based recording kept maintenance to a minimum.

TestComplete's environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

This allowed us to schedule execution of lengthy tests for nighttime and non-core hours, and to synchronize tests with Jenkins build releases.

This freed up QA assets to perform more specialized testing and reduced redundancy.

Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

I've used it for approximately eight months.

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

The initial set-up was exceedingly simple. There is a silent mode option available during set-up, which is very convenient when deploying to multiple machines or remotely.

We implemented it through an in-house team.

We did not maximize our ROI until we put somebody full-time on our TestComplete endeavours. The tool does have a learning curve, and it wasn't until we had an in-house expert on it that we began to see the benefits of automated testing over traditional QA roles.

The licensing options for TestComplete both running a licensing server. This prevents users from running more sessions simultaneously than purchased keys. This can prove problematic if you want multiple developers writing or running tests at once, and prevents you from using your key while a distributed test is running.

For pricing, carefully consider how many machines you want running the software, rather than the number of developers.

We also evaluated another SmartBear product called SoapUI. The change to TestComplete occurred because we changed our target from web applications to desktop.

My advice in regards to implementation would be to choose carefully which tests to automate, specifically focusing on lengthy procedures, tasks that require looping, or places where you want to test against multiple data sets.

Additionally, I found it beneficial to prefix my keyword tests with a character and number to provide logical ordering instead of alphabetic.

I also found it beneficial to record "undo" steps with each keyword test; this allows each test to be more stand-alone and prevents your test from being dependent on the state the previous test left the application in.

Finally, I would suggesting limiting the number of test applications per test suite to prevent bloated name-mapping schemes.

• Initially support for flex/flash technologies, but since that project ended my primary use is Functional Automation against a web front-end built on AngularJS.
• I use VBScript for a majority of my tests; it is one of many language choices available when writing scripts.
• Out of the box support for desktop, mobile and web.

I am the only test engineer on the project, so time is limited in terms of automation development. Using TestComplete I was able to immediately (after installation) begin coding regression tests for available functionality.

My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

• Number of tests = 75
• Automated Time to execute = 15 minutes
• Manual Time to execute = 360 minutes (approx.)
• Automation time saved = 345 minutes (5+ hours)
• 2300% improvement

I have substantially reduced the amount of time it takes for regression and build viability activities.

I pay for support and maintenance; having used the “support” functions through online forums, I can say there is room for improvement. I would like to have access directly to TestComplete developers as opposed to “users” who have some level of expertise with the platform.

I've used it for two years.

No issues encountered.

No issues encountered.

No issues encountered.

5/10, there are knowledgeable users in the forums, but I would like to go straight to the TestComplete staff with solution questions. That said, I have had very little need for product or development specific support.

Other tools were evaluated which supported Flex/Flash web apps, but we chose TestComplete for its rich feature set as well as Smartbear’s available suite of tools outside of Automated Testing.

It was very straightforward! You just need to install and begin, no special plugins or out of the ordinary system requirements were necessary.

We implemented it in-house. As with any automation platform, carefully planning the intended usage, understanding the technology under test and determining which features will be used are critical to success.

Given the cost was minimal, the best representation of ROI from my perspective is time saved using automated tests vs. manually execution.

My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

• Number of tests = 75
• Automated Time to execute = 15 minutes
• Manual Time to execute = 360 minutes (approx.)
• Automation time saved = 345 minutes (5+ hours)
• 2300% improvement

Our recent purchase was a single node locked license for one user which was very inexpensive. That fit our project needs and the automation strategy. The solution resides on inexpensive laptop for development and execution.

The most valuable feature is the ability for a non-developer to develop intelligent, robust, data-driven tests.

TestComplete has greatly improved our organization functions by allowing us to develop, manage, and execute many forms of testing in one central repository. Being able to develop, manage, and execute JUnit, NUnit, PHPUnit, PyUnit, Selenium, functional, and nonfunctional tests has given our company great insight as to the health of our testing efforts.

I have used this solution for four years.

No issues.

No issues.

The level of technical support leaves a lot to be desired. The only way to get support is via email, therefore problems that could take a matter of minutes to solve by phone or web conference can sometimes take several days.

We previously used IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, HPE Mercury Quick Test Professional, and IBM Rational Robot.

We switched because of the ability for a non-developer to develop intelligent, robust, data-driven tests.

The initial setup was very straightforward and took very little time.

No advice. I did not deal with the licensing and pricing.

I did compare the tool to IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, and HPE Mercury Quick Test Professional.

TestComplete is a great product. It is a perfect fit for an organization that has a hard time finding testers with a development background and want to centralize the development, management, and execution of their tests.

I found the Keyword Test feature very valuable. Even with my limited programming knowledge, I was able to create automation tests using the Keyword Test feature. Also, the ability to create image based tests was extremely helpful, especially when automating tests for mobile devices.

TestComplete has enabled us to automate a lot of our tests for Android devices.

I've used it for three months.

I would rate it 10/10. Every time I had a question or issue I couldn’t figure out, they were right there to help.

Set-up was pretty straightforward.

We implemented it through an in-house team.

The two most valuable features are the manual script library (projects) and Test Execute.

I set up a manual test for new functions and hand it off to a tester (who uses Test Execute). Once the new function test is completed, it is added to regression for all future versions. This has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

From my understanding, the product has changed significantly since v5 was released. I don’t know that it has necessarily improved, for my needs. If I worked in a different shop, then I might need an entirely different feature set compared to what I use now.

We've used it since May 2007.

No issues encountered.

No issues encountered.

No issues encountered.

I can't recall the last time I needed support.

I can't recall the last time I needed support.

Typically, I used my own method for automation (scripting tool, test data stored in SQL, results stored in SQL, result comparison to baseline with custom script). At the time I purchased TestComplete, the script recording tool wasn’t that great and I didn’t have the skills to write my own scripts. Over time, I used a different automation program and picked up the syntax for its scripting language. The product is currently called WinAutomation, though it had a different name at the time, that I can’t recall.

The initial setup was very straightforward. I installed the main app on my local workstation and test execute on five workstations available for testers. We use a shared network drive to store tests.

I installed everything myself. It was very straightforward. The real key is knowing what you need for your environment and to adapt the available tools.

It’s hard to sell testing tools, especially in a company that is owned by someone who thinks of software as a mule to pull his cart. It was a very tough sell and it is hard to quantify in dollars the money saved. However, I can say, with confidence, that there have been over one hundred bugs caught through regression since the software was purchased, which is a great track record for a team this size.

The most valuable features are the desktop and mobile modules.

The licensing costs are a little bit high and should be reduced.

I would like the ability to automatically distribute web applications to all of the concurrent users.

We have been using TestComplete for the past two years. 

The stability is good.

The technical support is quite good.

The initial setup is not too straightforward and not too complex. I would say that it's okay.

The price varies depending on the plan that you choose. The option we chose was around $2,000 USD.

I am currently investigating the differences between TestComplete and TestLeft.

TestComplete is a good product compared to other options that are on the market.

I would rate this solution a seven out of ten.

To me, the Object Browser has proven to be the most valuable feature of this product. This tab (and its associated tools) allows the test developer to locate objects within the application to be tested, and retrieve properties and methods that can be used in scripting. This saves a significant amount of time over referring to the application source code for such information. It also makes use of the recording feature periodically to rough out scripts for new tests. The recorded scripts are very fragile, but they can provide a good base for writing stable scripts.

While our test automation initiative is still in its early stages, TestComplete is allowing us to script and run complex business process tests against our desktop applications. Being able to create smaller modules that can be referenced by other scripts or keyword tests allows for a flexible and easy to maintain test suite.

A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

The product has been in use at the company for about two years and I have been actively using it for about 8 months.

I haven’t experienced issues that I would attribute directly to the application. There have been some issues with timing and other common problems, but many of those have been corrected through modifications to the tests and the environment.

My experience with their technical support has been less than satisfactory, but all of my issues were resolved. Of the technicians I have worked with, two of them were very knowledgeable and one of them did little more than email knowledge base articles that had been ruled out in the original issue report. The latter tech had the additional problem of being about 12 hours out of sync so a full day was lost with every email exchange. I received prompt responses and quick resolutions from both of the knowledgeable technicians I have dealt with although one of them was rather gruff and left me feeling like I was bothering her. Overall, SmartBear needs to work on their customer service and technical support.

This application did not replace a previous automation effort.

When deploying new installations of TestComplete, I ran into multiple issues with the licensing engine. The first time I experienced the problem, I contacted support and was directed to a knowledge base article (which I had been unable to find while searching for the error message). I have since bookmarked the article and reference each time I need to do a new install. Once the licenser has installed properly, the configuration and licensing goes smoothly.

The implementation was performed in-house, and was quite easy despite the issues with the licenser. The best advice I can offer would be to plan out your testing environment and requirements before you begin. If you plan to use virtual machines for your test systems, make sure you also have a physical box to run the licenser that is dedicated to the lab. Do not attempt to use someone’s PC as it may not always be available when the test machine are trying to run.

We have not performed an ROI analysis.

SmartBear’s product licenses are reasonably priced and generally in line with similar companies and products.

I reviewed several products before deciding to continue our automation efforts using TestComplete. Among the products reviewed were Telerik TestStudio, Testing Anywhere, and Ranorex.