We just raised a $30M Series A: Read our story

SmartBear TestComplete OverviewUNIXBusinessApplication

SmartBear TestComplete is #3 ranked solution in top Test Automation Tools, #4 ranked solution in top Functional Testing Tools, and #4 ranked solution in top Regression Testing Tools. IT Central Station users give SmartBear TestComplete an average rating of 8 out of 10. SmartBear TestComplete is most commonly compared to Tricentis Tosca: SmartBear TestComplete vs Tricentis Tosca. The top industry researching this solution is Computer Software Company, accounting for 30% of all views.
What is SmartBear TestComplete?

TestComplete is a powerful and robust automated testing tool for mobileweb and desktop  applications. Quickly and easily create accurate and repeatable tests across multiple devices, platforms and environments – regardless of experience level. It supports multiple languages, modern control sets and integrates with open source frameworks and tools like Selenium, SoapUI and Jenkins.

SmartBear TestComplete Buyer's Guide

Download the SmartBear TestComplete Buyer's Guide including reviews and more. Updated: October 2021

SmartBear TestComplete Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.

SmartBear TestComplete Video

Archived SmartBear TestComplete Reviews (more than two years old)

Filter by:
Filter Reviews
Industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
Rating
Loading...
Filter Unavailable
Considered
Loading...
Filter Unavailable
Order by:
Loading...
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Search:
Showingreviews based on the current filters. Reset all filters
RN
Consultant at a tech services company with 1-10 employees
Consultant
Stable and Integrates well with Azure DevOps

Pros and Cons

  • "The most valuable feature is the integration with Azure DevOps."
  • "What is currently missing from this solution is better support for mobile testing."

What is our primary use case?

We use this solution for test automation. A few months ago we began working on a project, and this solution is part of an evaluation phase. We are trying to include the whole suite from SmartBear including HipTest and SoapUI Pro.

Our primary use case is a web solution that we are trying to get working on the mobile solution, based on Flutter. All of our apps are built on Flutter, which is still very new for us. There is no support right now, so we are working closely with SmartBear TestComplete to integrate it.

We have an on-premises deployment.

What is most valuable?

The most valuable feature is the integration with Azure DevOps. That has been, by far, the best thing for us. We can use it to integrate everything that we are running in the test automation tool, which is linked directly to the test cases. This is the only tool on the marked at actually has this functionality, and it has only come about since 14.2, the latest version.

What needs improvement?

What is currently missing from this solution is better support for mobile testing. This is what we are currently negotiating with them for.

Specifically, we would like to see Flutter support in the next release of this solution.

For how long have I used the solution?

We have been using this solution for a few months.

What do I think about the stability of the solution?

From my experience at this point, I would say that this solution is stable. Even with the earlier versions, as long as you set it up properly, it was stable.

What do I think about the scalability of the solution?

In terms of scalability, I think that it works pretty well if you have agents.

How are customer service and technical support?

We have not spoken directly with technical support. Currently, we are talking to the product manager.

How was the initial setup?

We are using the SmartBear suite, which meant that setting up TestComplete was pretty straightforward. As we are still evaluating, it has not been fully deployed.

What other advice do I have?

We haven't fully evaluated this solution, although I know that they have made a lot of updates since I worked on it the last time. The results of this evaluation will set the standard for other projects. We are creating the basis for everything. As things progress, it will be me creating the test, but it will be executed on several machines.

I would rate this solution an eight out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Deva Veluchamy
Test Automation Lead/Consultant at Aspire Systems
Consultant
It supports various and different versions of browsers in web testing, but it's slightly unstable with longer executions of automation scripts.

Pros and Cons

  • "Test items, project variables helps in managing automation suite and scheduling execution."
  • "Increased performance with less memory and CPU usage."

What is our primary use case?

Evaluated for automating regression suite for windows desktop application and found testcomplete suits well in the aspect of identifying most of the objects and in few other parameters.

How has it helped my organization?

Good test coverage through automation and provides unique solutions to the most of automation challenges (e.g. comparison of images).

What is most valuable?

- Object spy eases the object identification method.

- TextObject recognition feature of this product exposes most of the objects to perform the actions.

- Test items, project variables helps in managing automation suite and scheduling execution.

- Jenkins integration to schedule executions.

- Integrated with selenium webdriver.

- Conversion of JScript project to Javascript project

- Integrated to Environment manager

- Support to read text from PDF

What needs improvement?

- Increased performance with less memory and cpu usage.

- Supports various and different versions of browsers in web testing.

- Support to Windows mobile application. (Native iOS and Android apps are supported currently.)

For how long have I used the solution?

More than five years.

How is customer service and technical support?

Customer Service:

Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

Technical Support:

Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

What about the implementation team?

In-house team is used for implementing framework for automation through TestComplete. Tool must be explored completely and knowing its unique features is more important before implementation as it has lot of inbuilt features.

What's my experience with pricing, setup cost, and licensing?

TestComplete now have come up with three modules (Web, Desktop & Mobile), so based on the type of product for automation, it is adequate to purchase the required module. Node-locked and floating license are available which could be decided based on the team composition and work strategy. Current license pricing is available at http://smartbear.com/product/testcomplete/pricing/

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: October 2021.
543,936 professionals have used our research since 2012.
WB
Test Automation Specialist at a tech vendor with 201-500 employees
Real User
The tool has the ability for a non-developer to develop intelligent, robust, data-driven tests

What is most valuable?

The most valuable feature is the ability for a non-developer to develop intelligent, robust, data-driven tests.

How has it helped my organization?

TestComplete has greatly improved our organization functions by allowing us to develop, manage, and execute many forms of testing in one central repository. Being able to develop, manage, and execute JUnit, NUnit, PHPUnit, PyUnit, Selenium, functional, and nonfunctional tests has given our company great insight as to the health of our testing efforts.

For how long have I used the solution?

I have used this solution for four years.

What do I think about the stability of the solution?

No issues.

What do I think about the scalability of the solution?

No issues.

How are customer service and technical support?

The level of technical support leaves a lot to be desired. The only way to get support is via email, therefore problems that could take a matter of minutes to solve by phone or web conference can sometimes take several days.

Which solution did I use previously and why did I switch?

We previously used IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, HPE Mercury Quick Test Professional, and IBM Rational Robot.

We switched because of the ability for a non-developer to develop intelligent, robust, data-driven tests.

How was the initial setup?

The initial setup was very straightforward and took very little time.

What's my experience with pricing, setup cost, and licensing?

No advice. I did not deal with the licensing and pricing.

Which other solutions did I evaluate?

I did compare the tool to IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, and HPE Mercury Quick Test Professional.

What other advice do I have?

TestComplete is a great product. It is a perfect fit for an organization that has a hard time finding testers with a development background and want to centralize the development, management, and execution of their tests.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user744705
SQA Team Lead at a tech vendor with 51-200 employees
Vendor
Advantages include Name Mapping and finding objects at run time, although it has memory leaks

What is most valuable?

Name Mapping and finding objects at run time. I have no issue because its IDE is simple to use.

How has it helped my organization?

  • License is less expensive than the other products available in the market.
  • I have automated the regression test suite of my company's product.

What needs improvement?

It has memory leaks. I have seven complete test projects, all of them loaded in one project suite. I don't know why memory increases when I just write the code and check-in/out files from the VSTS.

For how long have I used the solution?

For last five years.

What do I think about the stability of the solution?

Often crashes with large projects. File check-in takes too much time in TFS.

What do I think about the scalability of the solution?

No idea.

How are customer service and technical support?

Community is very helpful.

Which solution did I use previously and why did I switch?

I didn't switch from another solution.

How was the initial setup?

I would say straightforward, and really easy to work on TestComplete with its simple IDE.

What's my experience with pricing, setup cost, and licensing?

They are changing the licensing structure which is really bad. TestComplete 11 supports all TestComplete modules in one license. TestComplete 12 has separate licenses for each module, Web, Desktop, etc. I hate the new licensing structure.

Which other solutions did I evaluate?

I have no idea.

What other advice do I have?

It's a really good product with minimal issues.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Analista de TI Sênior/Teste de Software at a tech vendor with 1,001-5,000 employees
Vendor
Provides easy recording, understanding, and use without too much programming

What is most valuable?

It has easy recording, understanding, and use without too much programming.

What needs improvement?

Should improve parts for desktop products.

For how long have I used the solution?

I have used it for four years and still use it today.

What do I think about the scalability of the solution?

Yes, some.

How are customer service and technical support?

Good, I only needed it once.

Which solution did I use previously and why did I switch?

Yes, I had already used it. I changed because I did not have any solutions as close to the real thing as this. Free tools mostly do not resolve the problem.

How was the initial setup?

The first was difficult only in learning how to do it. Going forward, it has been easy and intuitive. We scale projects the best way for the company as we find that the simpler way to do it.

What's my experience with pricing, setup cost, and licensing?

It is expensive, and it is difficult to acquire living in Brazil. They should facilitate this. There are few representatives for this solution with little interest in selling it here.

Which other solutions did I evaluate?

Yes, JMeter.

What other advice do I have?

It is a great product. Just get advice before installation and everything will be much easier for you. I strongly recommend it.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user372528
Senior QA & Test Manager, Head of Test Automation at a tech services company with 501-1,000 employees
Consultant
The automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers.

Pros and Cons

  • "Selenium integration."
  • "Error handling features in the tool are a little limited."

What is most valuable?

  • SoapUI/ReadyAPI integration
  • Selenium integration
  • We run all of tests using TestExecute, this is just the executer for all TestComplete tests – command line version so can be integrated with any CI tools
  • We used to these tests seamlessly on Jenkins (TestComplete has a plugin for Jenkins)
  • The best part is TestComplete gives choice of selecting any of the six languages it supports
  • List of checkpoints available in TestComplete is one of the strongest available in the market
  • We also make heavy use of version control integration and JIRA integration
  • Data driven testing feature of this tool is one of the best

How has it helped my organization?

We did a POC for number of tools in past and eventually decided on TestComplete. We started with one tester and one test which we used to trigger manually. As the product matured, it added cross browser, mobile testing functionality, integration with Selenium/ SoapUI and Jenkins plugins, so we started using it more and more. Currently, the automated test team size is 20 and we run hundreds of tests automatically at the end of build process without any human intervention. For us, the automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers. The ROI has improved significantly and all the regression tests for our product are automated. We have plans of using TestComplete in the test driven development approach where developers can make use of our tests as part of their unit testing and I am sure we are going to achieve a lot more from this tool in coming years.

What needs improvement?

  • Error handling features in the tool are a little limited
  • The tool currently is Windows only, would like to see it being implemented for Mac and Linux
  • Once the tests are created in one languages, you can’t change it use other languages

What was my experience with deployment of the solution?

There have been no issues with the deployment.

What do I think about the stability of the solution?

The tool sometimes seems a little unstable and crashes sometimes on Windows 10.

What do I think about the scalability of the solution?

There are no issues with the scalability.

How are customer service and technical support?

Customer Service:

Average

Technical Support:

Good

Which solution did I use previously and why did I switch?

No. We did a POC on multiple tools and TestComplete was our first selection

How was the initial setup?

StraightForward

What about the implementation team?

In-house

Which other solutions did I evaluate?

QTP, TOSCA, RATIONAL, RANOREX

What other advice do I have?

The tool has great capability but also has great potential to demoralize if you implement it in incorrectly. Automated tests complement manual testing and doesn’t remove manual tests completely so I would suggest keep your manual tests, they will come handy as reference when the automated test fails or needs an update. I would suggest following before you start implementing the tool in you automated test environment

  • Choose the language to be used and stick to it. TestComplete supports six languages (VBScript, JScript, Python, DelphiScript, C++Script, C#Script)
  • Choose the naming convention for objects in the name-map and keyword tests so that you can make use of these tests in several projects
  • Always try to reuse the tests with parameterised values wherever possible rather than duplicating the tests
  • Version control the tests so that you can maintain multiple versions for multiple branches of your tests
  • Define and document a standard process for automation and communication
  • Execute the tests frequently so that you can get the best out of the tests and defects are identified as early as possible
Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user292632
QA Automation Lead at a tech services company with 501-1,000 employees
Consultant
Our automation engineers can create tests more easily, but the code editor gives poorly formatted code.

Pros and Cons

  • "It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems."
  • "The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools."

What is most valuable?

It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems.

How has it helped my organization?

TestComplete replaced QTP as the preferred choice of tool for the organization. It is much faster, works better across technologies (esp. Flex based UI) and is better compatible with newer technologies directly out of the box. We could have our automation engineers create tests more easily. Also, we were able to set-up lab machines to enable distributed runs for more applications in a shared environment.

What needs improvement?

The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools like Notepad++. Performance is another aspect which can be improved.

For how long have I used the solution?

I've used it for one and a half years.

What do I think about the scalability of the solution?

TestComplete was fairly easy to scale once we had the licenses in place for the organization.

How are customer service and technical support?

We used tech support for some specific third-party grids we had to automate. The support was average and we ended up creating our own automation solution for that piece of automation.

Which solution did I use previously and why did I switch?

We previously used QTP, and later moved to TestComplete as it provided more out of the box support for newer technologies like Flex and AIR.

How was the initial setup?

Initial setup was very straightforward. We did face some hiccups in license procurement however, once licenses were procured the process from there was quite smooth.

What about the implementation team?

We had a mixed team. The implementation was smooth overall and requires a few skilled automation experts to oversee the transition/initial implementation.

What was our ROI?

We achieved ROI in eight months from the start of implementation. Get a good automation architect to implement a good ROI directed framework. It is very easy to lose direction during a mmigration.

Which other solutions did I evaluate?

  • QTP
  • Selenium
Disclosure: I am a real user, and this review is based on my own experience and opinions.
LB
Software Developer at a retailer with 1,001-5,000 employees
Real User
Our biggest problem was the timing between input and responses, but a whole suite of tests could be run automatically.

Pros and Cons

  • "The ability to run a whole suite of tests automatically (which we did overnight)."
  • "We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses."

What is most valuable?

The features of TestComplete that were most valuable to me were:

  • The ability to run a whole suite of tests automatically (which we did overnight)
  • The ability to create test scripts that were easy to modify;
  • The ability to easily review a log of software errors, if any

How has it helped my organization?

It would have been a huge task to have to run the test scripts manually, probably 12 hours straight. We were able to run tests on daily promotions of software so all software was tested every night, making any bugs instantly visible the next morning. It gave us quick confirmation that software changes worked, and without breaking any existing software.

What needs improvement?

We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses. This was because sometimes the Windows PC feeding the scanner script data was faster than the scanner could process it, so we had to constantly tweek the wait times so the script wouldn’t bomb out. I have since used software with a “wait for” function that would wait for a response, but if TestComplete did, we didn’t know about it.

One of the main reasons I was hired was to use TestComplete with green screen applications, which was possible, but not easy. You had to know the exact column and row position of every text line on the screen so you could verify the string being tested and analyze exactly where and what the response would be.

For how long have I used the solution?

I used TestComplete as a QA Automation consultant for about six months.

What was my experience with deployment of the solution?

No

What do I think about the stability of the solution?

No

What do I think about the scalability of the solution?

No

How are customer service and technical support?

Customer Service:

We never needed customer service.

Technical Support:

We never needed technical support.

Which solution did I use previously and why did I switch?

Nothing previous to TestComplete.

How was the initial setup?

That happened before I was hired, so I don't know.

What about the implementation team?

TestComplete was implemented with the help of contract software developers who created the test scripts, which was invaluable to be able to use it for testing, and to use them as templates so we could copy and modify to make new scripts.

What was our ROI?

unknown.

What's my experience with pricing, setup cost, and licensing?

unknown.

Which other solutions did I evaluate?

TestComplete was installed before I was hired, so I know nothing about this.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Julio De Lima
Principal QA Engineer at Capco
Real User
Supports Desktop and application testing and enables centralizing all scripts in one tool

Pros and Cons

  • "TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications."
  • "TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services."

What is most valuable?

Desktop and web application support. TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications. It's an amazing feature for companies that want to automate UI tests on each application built in-house.

How has it helped my organization?

Before using TestComplete on our projects, we used to use a lot of tools to automate our applications, such as Sikuli or Java Robot to automate desktop apps, and Selenium WebDriver for web apps. After starting to use TestComplete, we were able to centralize all scripts in only one tool and technology.

What needs improvement?

TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services. In a microservice world, this is a big flaw. Another thing is that the cross-browser support has a lot of different traits between browsers. It should be improved.

For how long have I used the solution?

Five years.

What do I think about the stability of the solution?

Sometimes the app crashes during test execution based on the amount of code that is running. It's recommended that you modularize scripts, but that is not a justification for not being more stable.

What do I think about the scalability of the solution?

TestComplete has a test executor app that can be used in a distributed test execution environment. The problem is that this is a paid product. Thus, it was pretty expensive to scale this architecture.

How are customer service and technical support?

Eight out of 10.

Which solution did I use previously and why did I switch?

I used to use HPE QTP but TestComplete has the same features, low price, and support for handling a lot of stacks. In addition, I don't need to use hundreds of plugins.

How was the initial setup?

To start working with TestComplete, we only needed to install (next, next, finish flow) and then start using it. There are some configurations to do to help increase efficiency, but I do not consider that more than a nice-to-have.

What's my experience with pricing, setup cost, and licensing?

Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day. If they will each use it all the time, buy licences for all of them.

Which other solutions did I evaluate?

No, I did a PoC and discovered the pros and cons.

What other advice do I have?

Do a PoC and try to understand if TestComplete fits your context and requirements. Use the script-driven approach instead keyword-driven, because the former is more efficient.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user711891
Test Analyst
Vendor
It has increased effective testing processes and reduced the cycle time. I would like to see recovery scenarios.

What is most valuable?

One of the valuable features is object identification. It provides the different options to users so they can uniquely identify the objects.

How has it helped my organization?

By automating the regression test cases, it has increased effective testing process and reduced the cycle time.

What needs improvement?

I would like to see recovery scenarios.

For how long have I used the solution?

I have been using this solution for over four years.

What do I think about the stability of the solution?

I did not encounter any issues with stability.

What do I think about the scalability of the solution?

I did not encounter any issues with scalability.

How are customer service and technical support?

I would give technical support a rating of 8/10.

Which solution did I use previously and why did I switch?

When we started using this tool, only TestComplete provided the support of the automation for WPF objects.

How was the initial setup?

The initial setup was easy.

What's my experience with pricing, setup cost, and licensing?

This is cheaper compared to other testing tools.

Which other solutions did I evaluate?

We evaluated other options QTP.

What other advice do I have?

This is the best tool for GUI and Web Automation.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user710514
Test Automation Engineer at a energy/utilities company with 10,001+ employees
Vendor
Calls different operations through batch files. The product is not stable enough and it crashes often.

Pros and Cons

  • "Runs in different remote machines. We have multiple versions of the software being tested."
  • "Product is not stable enough and it crashes often."

What is most valuable?

  • Tests can be execute and run separately.
  • Runs in different remote machines. We have multiple versions of the software being tested.
  • Calls different operations through batch files.
  • Has the option for combining data and is keyword driven. It helps people with less knowledge in programming to work with it.

How has it helped my organization?

We already have the UI smoke test and have integrated to our build system, which runs each day for multiple version of the product. This saves us a lot of time.

What needs improvement?

* Product is not stable enough and it crashes often

The application under test is a complex scientific application developed in C++ & C# and use different technologies. So when I try to do Name mapping, it sometimes hangs and have to restart Test complete

* Checking from TestComplete to TFS has issues.

I mostly try to checkin the changes with Visual studio as TestComplete hangs while you try to check in with many changes directly to TFS from Testcomplete.

* Possibility to run a part of keyword tests through TestExecute

Consider you have a keyword test with 10 individual tests. By using test execute, you can only run the main one, not the individual ones.
So I like to have the possibility of running from the 3rd test or just running the 5th one.The option is not present in TestExecute (you can do it in Test complete).


What do I think about the stability of the solution?

It would help if it were more stable as it sometimes hangs and crashes.

What do I think about the scalability of the solution?

We have different version of the product and it framework/project was easily scalable and used by other projects.

How are customer service and technical support?

We had multiple technical training sessions which were helpful and almost all the answers are in the community.

Which solution did I use previously and why did I switch?

When I started at this company, they were using TestComplete.

How was the initial setup?

Getting used to product did take some time.

What's my experience with pricing, setup cost, and licensing?

The price is less, compared to other products, such as QTP.

Which other solutions did I evaluate?

We had a choice with Coded UI as we developed it using C#. However, TestComplete is the preferred choice one over coded UI, as it needed to grow more as a test tool.

What other advice do I have?

Get training and decide on a framework that suited for your application. It always depends on what you want to do with the tool.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
HM
QA Manager with 201-500 employees
Real User
I like the cross browser compatibility. The way objects are added and used when utilizing descriptive programming could be improved.

Pros and Cons

  • "I like the cross browser compatibility. It saves a lot of time re-writing scripts to accommodate different browsers."
  • "The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT."

What is most valuable?

I like the cross browser compatibility. It saves a lot of time re-writing scripts to accommodate different browsers.

How has it helped my organization?

We are consultants. So we simply provide an automated solution to a client, then move on. We don’t use the product in our day to day work.

What needs improvement?

The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT.

In UFT, using descriptive programming for a web page you can use.

Browser(description).WebList(description).Select anything.

Regardless of how many panes, frames, panels etc are in the hierarchy before the Weblist object.

But in Smartbear you have to store every frame, panel etc.

So that if you didn’t use the ‘Alias’ functionality you would have an object description miles long.

But even having to use the Alias, you still have to add each and every frame, panel, etc. whereas in UFT you can just use page.object and it will find the object on the page (as long as you’re using unique descriptions!!) without worrying about frames, panes, etc.


For how long have I used the solution?

We have used this solution for about two years.

What do I think about the stability of the solution?

I did not encounter any issues with stability.

What do I think about the scalability of the solution?

I did not encounter any issues with scalability.

How are customer service and technical support?

I didn’t require any technical support.


Which solution did I use previously and why did I switch?

I routinely use SmartBear, UFT, and SilkTest. I fit the application used to whatever my client requires.

How was the initial setup?

The initial setup was very straightforward. Even the mobile testing side was easy to setup

What's my experience with pricing, setup cost, and licensing?

My advice so far, is that while it’s not quite as powerful and easy to use as UFT, its price tag more than makes up for it. It makes it an excellent cost saving alternative.

Which other solutions did I evaluate?

I use several tools.

What other advice do I have?

As with all tools, verify that it will do what you need for a reasonable price.

Disclosure: My company has a business relationship with this vendor other than being a customer: We are in the process of setting up a partner arrangement with SmartBear.
it_user710532
Architecture Analyst at a tech company with 5,001-10,000 employees
Real User
Automated testing was used to validate security protocols for web applications

What is most valuable?

Automated testing was used to validate security protocols for web applications.

How has it helped my organization?

It is easier to store and execute test cases, track test results, and save time with automation.

What needs improvement?

LoadComplete could be a little more user-friendly, but is still better than LoginVSI. Network issues sometimes made for poor connections.

For how long have I used the solution?

We have used this solution for one year.

What do I think about the stability of the solution?

I did not encounter any issues with stability. The product was available within expected measures.

What do I think about the scalability of the solution?

I did not encounter any issues with scalability.

How are customer service and technical support?

I would rate the level of technical support as excellent.

Which solution did I use previously and why did I switch?

We switched from HPE QC/ALM. It did not offer automation on the same scale.

How was the initial setup?

The initial setup was made easier with professional services. Learning to code was harder.

What's my experience with pricing, setup cost, and licensing?

I tell people it is cheaper than ALM with the same features and better UI.

Which other solutions did I evaluate?

We did evaluate other options, but I can’t recall which ones.

What other advice do I have?

Make sure you have a large volume of testing to transfer. Otherwise, it is not worth the money.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
JM
Senior Automation Test Analyst with 10,001+ employees
Vendor
Allows us to test both desktop and web applications

Pros and Cons

  • "It allows us to test both desktop and web applications."
  • "It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing)."

What is most valuable?

It allows us to test both desktop and web applications. This is a very important feature for system integration tests, as we develop desktop applications, web applications, and a Web API using the same database.

What needs improvement?

It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing).

In fact, this is an issue regarding how to read the test log in MHT format generated by TestComplete.

We know a test log is always generated automatically by test tool after executing a test. TestComplete provides a standard IDE (Integrated Development Environment) interface for user to review the test log. We don’t have issue reviewing the test log within TestComplete regardless how big the test log is (Our test log record the test running for more than two days without stopping. The test log extension name is .tcLogs and the size of total test log is very big over 2.5 GB).

TestComplete also provides a function for the user to export the log into MHT format file via manually selecting each test log after complete a test or dynamically exporting the test log during test executing via command in test script. The MHT log file can be opened by IE. However, if the MHT file is a little big, such as >200M, the user could experience obvious performance issues. It takes a very long time for IE to open this file because IE loads all the contents into one page which could finally eat up all your system memory. I tried to find another tool to open the big MHT log file, but no luck. Even executing a small test (running for one to two hours), the MHT log file size can easily reach to 200M, if a screenshot is included, for example. Normally, each one round of our regression test runs for 30-48 hours. It is impossible for us to open the big MHT log file as the system runs out of memory and an error appears definitely after hours of waiting to just open this file by IE.

TestExecute is a small tool to just execute the test developed by TestComplete. But TestExecute only generates MHT log file rather than TCLOGS file. Obviously, we cannot open the MHT log file if it is very big. So we gave up using TestExectue and export MHT log file function in TestCompelete at all.

We have to buy six individual full license of TestComplete to run and review the test.

For how long have I used the solution?

We have used this for nine years.

What do I think about the stability of the solution?

Overall, TestComplete is quite stable. But the only issue affecting me is if TestComplete is terminated unexpectedly while test is still running (such as, terminating TestComplete without stopping the test first, window restarting, or shutting down unexpectedly), user could lose all the logs.

What do I think about the scalability of the solution?

There were no issues with scalability, but it may have a performance issue with expansion.

How is customer service and technical support?

Technical support is excellent.

How was the initial setup?

The initial setup was straightforward.

What's my experience with pricing, setup cost, and licensing?

The product is becoming more and more expensive. There are two types of licenses: locked and float. Locked license save more, but can only be used in a physical machine. Float licenses can be run in virtual machines.

Which other solutions did I evaluate?

QTP, was very expensive when compared to TestComplete, many years ago.

What other advice do I have?

This tool is very easy to use and very powerful. It has a short test development circle and good technical support. There is an expensive license, but it could save more, as testers may spend more time achieving a certain test goal if using an Open Source tool. You need to consider the testers' script experience and the company’s budget to choose the right tool.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Senior Consultant at a tech services company with 10,001+ employees
Consultant
Using this solution I am able to process whole pages at once rather than doing everything one by one

What is most valuable?

Valuable for us is the ability to identify objects by using Find methods. I am able to process whole pages at once rather than doing everything one by one.

What needs improvement?

Using object spy can be slow sometimes and seems to require a lot of resources.

For how long have I used the solution?

We have been using the solution for six months.

What do I think about the stability of the solution?

The solution freezes sometimes, but not very often. Sometimes it doesn’t find any objects from the browser and needs to restart the browser or the whole machine. This happens with Internet Explorer.

What do I think about the scalability of the solution?

It is very difficult to have multiple developers when using Name Mapping. You can’t merge all the files and this creates conflicts.

It scales better for multiple users when using pure script, which reduces the amount of files that cannot be merged.

How are customer service and technical support?

Support calls were helpful at the beginning, and I haven’t used the support since then.

Which solution did I use previously and why did I switch?

We used TestPlant eggPlant and are still using it for thick clients where objects are not available. The development speed for TestComplete is faster and more reliable when objects can be seen.

How was the initial setup?

It is easy to install and use.

Which other solutions did I evaluate?

We went through quite a few of them. This product seemed best for usability and recording features.

What other advice do I have?

Test the trial, then go for it. Keyword tests seem viable for straightforward test cases, but to create dynamic architecture you might want to stick only with scripts and drop Name Mapping.

Keyword tests are easy and fast to record, but adding complex logic for them can be tricky and time consuming.

Disclosure: My company has a business relationship with this vendor other than being a customer: We are partners with SmartBear.
it_user288375
Quality Assurance Engineer with 501-1,000 employees
Vendor
We've now automated 90% of the testing we used to perform manually, but I would have preferred Python support in earlier versions.

Valuable Features

It's easy to work with and doesn’t take much to get it setup to start working with it.

Improvements to My Organization

In my previous three positions, there was not a test automation solution in place. I was given the opportunity to explore options. Once I chose a solution, we were able to implement TestComplete and were able to automate about 90% of the manual testing that was done prior to implementing TestComplete.

Room for Improvement

This product continually improves and in v11, they now have Python support. This was something that I wanted and they provided it the latest version.

Use of Solution

I've used it for 11 years.

Deployment Issues

I have not run into any issues with deployment.

Stability Issues

I have not run into any issues with stability.

Scalability Issues

I have not run into any issues with scalability.

Customer Service and Technical Support

Customer Service:

Another one of the many reasons why I chose TestComplete. The level of customer service and technical support can’t be beat. SmartBear always answers my questions within 24-48 hours.

Technical Support:

They have a great website, help and forums that also help in finding the answers I need in a timely manner.

Initial Setup

It was very straightforward which was another reason why I went with TestComplete. I was able to use the demo version (when I was searching for options) to create tests easily, so when I had to demonstrate the product to my management teams, they could easily see why TestComplete was the product for us.

Implementation Team

The implementation was done by me. The best recommendation is to read the help guide, especially if you are using the product in different ways, like floating licenses. This is where the license server is on one machine and people have to access TestComplete that are not local. However, if you run into any issues, the customer support department is there to help in any way they can.

ROI

My ROI has been the fact that it takes less time to run all of the tests that were done previously. Prior to TestComplete, it would take over three weeks to run all of the tests that needed to be run for a release. After TestComplete, we have been able to reduce that time to less than one week.

Pricing, Setup Cost and Licensing

One of the main reasons I went with TestComplete, besides the information that I already provided, was the cost for TestComplete and TestExecute. They make it very easy for large to small companies to implement without large costs. The licenses are broken down the amount of users that need to use it and they also have the option of floating licenses.

Other Solutions Considered

I looked at various options like QuickTest Professional, Rational Functional Tester, and SilkTest.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Software Quality Analyst I at Bentley Systems Incorporated
Consultant
This application is helping a lot in regression testing. If something in the application breaks, we get to know about it quickly, and this saves us time.

What is most valuable?

Object Browser is one of very powerful tool TestComplete provide which can see what processes are running in the memory.

How has it helped my organization?

This application is helping a lot in regression testing. If something in the application breaks, we get to know about it quickly, and this saves us time.

What needs improvement?

There

For how long have I used the solution?

My team has been using it for three years, and I've been using it for one-and-a-half years.

What was my experience with deployment of the solution?

Not at all. The framework was already in place with my team.

How are customer service and technical support?

Customer Service: It's very good. Technical Support: It's very good.

Which solution did I use previously and why did

What is most valuable?

Object Browser is one of very powerful tool TestComplete provide which can see what processes are running in the memory.

How has it helped my organization?

This application is helping a lot in regression testing. If something in the application breaks, we get to know about it quickly, and this saves us time.

What needs improvement?

There

For how long have I used the solution?

My team has been using it for three years, and I've been using it for one-and-a-half years.

What was my experience with deployment of the solution?

Not at all. The framework was already in place with my team.

How are customer service and technical support?

Customer Service:

It's very good.

Technical Support:

It's very good.

Which solution did I use previously and why did I switch?

That is my first product of this type.

How was the initial setup?

Initial set-up was pretty straightforward, but that was basic. Complexity grows when you go deep in testing.

What about the implementation team?

We do get some training from SmartBear, but we implement tests on our own.

What was our ROI?

The ROI makes it worth buying this product.

What other advice do I have?

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Systems Engineer at a tech services company with 51-200 employees
Consultant
It is similar to other tools like Selenium IDE where you can record and play test scripts. There is no support for mobile testing automation.

Valuable Features:

For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It is a great tool and hard to learn and use. 
 
Pros:
- It is similar to other tools like Selenium IDE you can record and play test scripts 

- Developer needed to create test scripts using VB Script, Jscript, Delphiscript etc - 
- QA perspective, helps test results logs helps in finding precisely the issue where test failed
- Tests can be recorded in keyword driven or screen capture modes on some screens.
- Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on and it is good to use it for Cross browser testing.
 
Cons:
- There is no support for mobile testing automation.

- QA need to depend upon developer to correct the scripts any change happens. Automated

Improvements to My Organization:

Recording the scripts and running the scripts should be available to all users.

As a QA, I would like to use it by myself using the application recording and replaying the tests.

Room for Improvement:

On some of the UI application pop up screens, Closing and opening screens etc.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Systems Test Manager at a transportation company with 1,001-5,000 employees
Real User
The latest version can be used for mobile testing, but more effective object mapping for Citrix systems is needed.

What is most valuable?

Keywords-Driven frameworks and Data-Driven Testing (DDT) methodologies used for building effective frameworks.

How has it helped my organization?

The fact that the latest version can be used for mobile testing.

What needs improvement?

A more effective way in Objects Mapping for applications built on domains/platforms such as Sliverlight, and Citrix is needed.

For how long have I used the solution?

I've used it for three years.

How are customer service and technical support?

Technical support can be improved by being more efficient in responding to questions.

Which solution did I use previously and why did I switch?

Used some other products, but this was found to be the most suited for business requirements.

How was the initial setup?

The initial step was straightforward which was made by carrying out a POC and the decision was to go for this solution.

What about the implementation team?

Following initial training by the vendor, the implementations are done in-house.

What's my experience with pricing, setup cost, and licensing?

It is value for money, and different licensing options are available.

What other advice do I have?

Fig 1: Building of Automation Testing framework.

Fig 2: Automation Testing Results

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Project Lead at Aspire Systems
Consultant
A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. It is tedious to monitor the progress since the logs are available only in the tool.
Authored by Sabhari Murugan and Ujwal Unni from Aspire Systems Testing Community. Overview Software Quality is dependent on Test Automation for its ability to provide vast coverage with least possible resource. Having said that, automation has its own overhead and maintenance which requires additional effort. To reduce such additional effort, besides ensuring stability, robustness and increase the ROI - there are certain considerations to be focussed at different stages of the project. Here is an insight to such areas of considerations and how it can be dealt with ‘TestComplete’ – a potential test automation tool. What is TestComplete? Started as a tool that is preferred for Test Automation of Desktop applications, today TestComplete is one of the top…

Authored by Sabhari Murugan and Ujwal Unni from Aspire Systems Testing Community.

Overview

Software Quality is dependent on Test Automation for its ability to provide vast coverage with least possible resource. Having said that, automation has its own overhead and maintenance which requires additional effort. To reduce such additional effort, besides ensuring stability, robustness and increase the ROI - there are certain considerations to be focussed at different stages of the project. Here is an insight to such areas of considerations and how it can be dealt with ‘TestComplete’ – a potential test automation tool.

What is TestComplete?

Started as a tool that is preferred for Test Automation of Desktop applications, today TestComplete is one of the top competitors and a noticeable performer in the commercial market. The recent versions of the tool support Desktop, Web and mobile applications to a great extent. The tool gained its popularity due to its

  • Robust and stable execution engine
  • Rich in-built libraries
  • Support for multiple Scripting Language
  • Minimal knowledge required for beginners
  • Access to System level operations like system restart, Process management and monitoring
  • Cost effective, when compared with similar tools in the market
  • Events (Decide what to do at - Beginning of Test, End of Test, Error encountered, etc...)

Doing it right:

Test Automation is not an area with constant or limited needs and expectations. New technologies, frequent changes, emerging software systems and new feature additions create constant challenges for tools and testers. There are certain considerations in each phase of Test Automation solution to build a suite that could provide high ROI, with least effort and maintenance. Here is a brief about some of these challenges / requirements that are more likely in Test Automation solutions that use TestComplete.

1. Frequent Changes

“Change is good” – in many ways, for instance it makes way for new features and enhances user experience. Besides, who likes to use software that has no updates? For any product to be successful, it should have the ability to change and adapt to the cutting edge technology.

Implementation stage: Framework design.

Solution:

1. Name Mapping – an object repository used to store the instance of controls. By doing so any change in the Properties of controls such as size, colour or caption can be isolated leaving the automation scripts unaffected.

2. Modularity Framework- a Test Automation framework in which all the common functions such as entering text in text box, selecting items in a combo box, clicking buttons are developed as reusable functions and used in Test Case Script

Doing so, the Test Case Script will not have any code that directly interacts with the product, the controls are referred through Name Mapping and actions are performed through the re-usable functions. Hence Test Case Script is isolated and no change will be required unless there is a change in the workflow of the product.

2. Testing Multi-Language Products

When testing applications that are released in different language versions it is crucial to test all the language versions to ensure quality.

Implementation stage: Framework design.

Solution:

1. Using an XML or ini file. The file contains a common ID for each caption and their equivalent value in the respective language. For example, below is a representation of the xml file content for the caption of Ok button in three languages.

<caption>
                <id=’YesButtonCaption’>                
                <English=’Yes’>
                <German=’Ja’>
                <Finnish=’Kylla’>
</caption>

In the Test Case Script or in the reusable functions instead of directly using the caption, use a re-usable function that fetches the appropriate language text from the XML when the id and the language are passed as parameters. For example: getCaption(“YesButtonCaption”, “German”)

3. Different Levels of Execution

Test Suites may vary from less than hundred cases to more than 20,000 cases. It is not always the case that the entire test suite has to be executed; hence cases should be grouped based on features or workflows. For example automation suite of a banking application can be grouped based on features like money transfer, account creation, payments and so on to enable execution at three levels – Case level, Selected Group level and suite level.

Implementation stage: Framework design.

Solution:

1. Driver Script – Instead of matching each test Item to a test case, group them based on features and create a driver script for each feature. A driver script is just a function that calls all the test cases under the respective feature one after the other.

2. Further enhancement can be done by using XLS file to have list of Cases and a column with Yes or No values that determine if the particular case has to be executed or not.

4. Windows event log monitoring

There are scenarios where the application may seem to work fine, even when there are some issues while updating data in the database or storing data to a file. Monitoring the Windows Event log gives a better insight into such situations and helps to identify them earlier.

Implementation stage: At the time of requirement.

Solution:

1. TestComplete provides full read access to Windows event log, leveraging this feature at suite level can capture all the events logged related to the application under test during execution.

2. TestComplete also provides access to real-time resource usage of any process such as CPU usage and Memory usage, logging these details will help identifying memory leakages.

5. Memory issues due to large suites

Scalability is one essential aspect to be considered in Test Automation. TestComplete is designed to handle any number of Test Cases which means almost any Lines of Code (LOC). But having too many script units will have a huge memory foot print which accumulates over execution, which eventually leads to tool crash or other performance issues.

Implementation stage: At the time of requirement.

Solution:

1. If there are way too many Test Scripts in an Automation suite, the suite can be split into multiple smaller suites. All the smaller suites are independent of each other. In this case, when a whole suite execution is required Batch Files can be used to continue the execution from one suite to another.

6. Continuous Long-time Execution

Having around a few thousand Test Cases in a Test Suite means executions can go on continuously for days. This makes the system vulnerable for performance and memory issues.

Implementation stage: At the time of requirement.

Solution:

1. The application under test can be closed and restarted in certain intervals to free memory and have better performance

2. It is not just the application but also the tool that is in running state for too long, hence it makes a difference if the tool is restarted in some required intervals approximately once a day. This can be made possible by two ways- TestComplete’s in-built function “RebootandContinue”,which restarts the system and continues further execution or using batch file to close TestComplete after sometime into execution, let the system be idle for a while, then start TestComplete and continue the execution.

7. Automatic Report generation and mail notifications

When execution continues for days together it is tedious to monitor the progress since the logs are available only in the tool, this means either the execution has to be paused or we will have to wait till it completes to view the report or log

Implementation stage: At the time of requirement.

Solution:

1. The in-built function to Save Logs can be used at regular intervals to automatically export logs to the specified location as and when required.

2. TestComplete also supports automatic mail notifications with the logs as attachments. Once configured results will be mailed automatically as per the configuration

8. Scheduling Tests in Test Execute

SmartBear has an execution tool that is relatively less expensive than TestComplete that can be used only for placing executions. Unfortunately, TestExecute does not enable us to select the Tests that has to be executed. So one way or another TestComplete is required to modify the Tests to be executed.

Implementation stage: At the time of requirement.

Solution:

1. The .PJS and .MDS files in a Test Suite are the two XML files that stores the data about the project and Test items to be executed. Developing a simple tool to read and edit these XML files will completely eradicate the need of TestComplete in placing Executions.

Other Best Practices:

  • Always ensure each test script works independently.
  • Avoid test data dependencies between test scripts.
  • Test data is always externalized.
  • Write the test scripts from scratch instead of using record and playback.
  • Always ensured test automation scripts sequence are configurable for execution.
  • Frequent catch-up with Dev & Design team to understand the impact on automation scripts due to upcoming releases / changes.
  • Tool & Environment upgrades should be taken up as early as possible to avoid surprises at neck of the moment.

A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. Automation Framework is the basic building block of a robust automation suite and framework design is the most important phase of an automation project. This is the phase where all the above considerations should be predicted in order to design a framework that meets all the requirements of the project and accommodate further improvements and new requirements that may arise over the course of the project. Such Anticipation, preparation and worked arounds are the way to ’Doing it right, with TestComplete’.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user347607
Technical Leader at a tech services company with 10,001+ employees
Consultant
I was part of a systems architecture team and we were looking into various kinds of automation tools. We found this solution to be the most complete tool.

Valuable Features

It allows me to invoking an application and then to browse the application. I've done some good research with TestComplete along with Google Earth.

Improvements to My Organization

I was part of a systems architecture team and we were looking into various kinds of automation tools. We found this solution to be the most complete tool.

Room for Improvement

More APIs could be added.

Use of Solution

I used it for six months.

Stability Issues

I didn't encounter any stability issues in my short tenure, although I remember invoking the application twice although there is was an instance already open.

Customer Service and Technical Support

Customer Service: I haven't had to use customer service. Technical Support: I haven't had to use technical support. …

Valuable Features

It allows me to invoking an application and then to browse the application. I've done some good research with TestComplete along with Google Earth.

Improvements to My Organization

I was part of a systems architecture team and we were looking into various kinds of automation tools. We found this solution to be the most complete tool.

Room for Improvement

More APIs could be added.

Use of Solution

I used it for six months.

Stability Issues

I didn't encounter any stability issues in my short tenure, although I remember invoking the application twice although there is was an instance already open.

Customer Service and Technical Support

Customer Service:

I haven't had to use customer service.

Technical Support:

I haven't had to use technical support.

Initial Setup

Initial setup was easy, but implementation was a little tricky initially. I could not find any sample projects that I could use as a guide.

Pricing, Setup Cost and Licensing

It's open source, so it's free.

Other Solutions Considered

We looked at other solutions, but this was the most complete.

Other Advice

It's a great tool that covers many features and is straightforward with logs.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Mobile QA Tester at a comms service provider with 1,001-5,000 employees
Vendor
It's enabled us to automate a lot of our tests for Android devices.

Valuable Features

I found the Keyword Test feature very valuable. Even with my limited programming knowledge, I was able to create automation tests using the Keyword Test feature. Also, the ability to create image based tests was extremely helpful, especially when automating tests for mobile devices.

Improvements to My Organization

TestComplete has enabled us to automate a lot of our tests for Android devices.

Use of Solution

I've used it for three months.

Customer Service and Technical Support

I would rate it 10/10. Every time I had a question or issue I couldn’t figure out, they were right there to help.

Initial Setup

Set-up was pretty straightforward.

Implementation Team

We implemented it through an in-house team.

Valuable Features

I found the Keyword Test feature very valuable. Even with my limited programming knowledge, I was able to create automation tests using the Keyword Test feature. Also, the ability to create image based tests was extremely helpful, especially when automating tests for mobile devices.

Improvements to My Organization

TestComplete has enabled us to automate a lot of our tests for Android devices.

Use of Solution

I've used it for three months.

Customer Service and Technical Support

I would rate it 10/10. Every time I had a question or issue I couldn’t figure out, they were right there to help.

Initial Setup

Set-up was pretty straightforward.

Implementation Team

We implemented it through an in-house team.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user340986
Senior QA Analyst at a financial services firm with 501-1,000 employees
Vendor
It helps to organize test runs using the Test Items page of the project editor.

Valuable Features

Data driven testing using Microsoft Excel Viewing properties of objects using Object Spy Organizing test runs using the Test Items page of the project editor

Use of Solution

For versions 8.X to 9.20.2460.7, I've been using TestComplete for about four and a half years.

Stability Issues

It seems to crash once every two to three months.

Customer Service and Technical Support

Customer Service: 7/10 Technical Support: 7/10

Initial Setup

For a node-locked license, it's pretty straightforward. For floating licenses, it's a little more complex because you have to set-up the license manager and do a little bit of configuration on the computers that use the floating licenses.

Valuable Features

  • Data driven testing using Microsoft Excel
  • Viewing properties of objects using Object Spy
  • Organizing test runs using the Test Items page of the project editor

Use of Solution

For versions 8.X to 9.20.2460.7, I've been using TestComplete for about four and a half years.

Stability Issues

It seems to crash once every two to three months.

Customer Service and Technical Support

Customer Service:

7/10

Technical Support:

7/10

Initial Setup

For a node-locked license, it's pretty straightforward. For floating licenses, it's a little more complex because you have to set-up the license manager and do a little bit of configuration on the computers that use the floating licenses.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user340989
Quality Assurance Specialist at a tech vendor with 201-500 employees
Vendor
It has saved time by eliminating the need for manual testing. It's also led to the discovery of bugs that wouldn't have been found otherwise.

What is most valuable?

With TestComplete, I can discover bugs faster and easier than testing manually. It eliminates the need for regression testing sessions as TestComplete can be set up to test automatically overnight against each day's build from programmers.

How has it helped my organization?

It has saved time by eliminating the need for manual testing. It's also led to the discovery of bugs that wouldn't have been found otherwise.

What needs improvement?

I'm completely satisfied with TestComplete 10, but looking forward to seeing what's in store for version 11!

For how long have I used the solution?

I've used it for about two years.

What was my experience with deployment of the solution?

The initial deployment of TestComplete was difficult, but with the assistance of SmartBear technical support, I was able to get it up and running.

How are customer service and technical support?

SmartBear's technical support is excellent. However, due to time zone differences, support can be a little slow.

Which solution did I use previously and why did I switch?

Previously, testing was performed manually. TestComplete was a great solution to automate the process.

How was the initial setup?

Installing TestComplete was easy, however "floating license" configuration was challenging with the use of VMs.

What about the implementation team?

I implemented TestComplete independently. Their "How-To" videos, forums, and support team are incredibly valuable!

What's my experience with pricing, setup cost, and licensing?

If you have any questions about licensing, Smart Bear's staff is very helpful!

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user340992
Test Automation Specialist at a tech services company with 501-1,000 employees
Consultant
It gives us the option to work in the same suite without changing from one project to another.

What is most valuable?

  • Object spy and object recognition are working very well, giving us a lot of possibilities.
  • Option to work in the same suite without changing from one project to another, which is better than HP QTP/UTF.
  • USEINIT feature is really nice (+/- like importing in Java)

How has it helped my organization?

It didn’t really change the way my organization works.

What needs improvement?

A test case design (schema) module would be nice.

For how long have I used the solution?

I've been using it for nine months, and my customer has been using it for three years.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

I just contacted them once. They responded very quickly, which is good, but the problem was on our side, so I cannot really evaluate.

Which solution did I use previously and why did I switch?

TestComplete was already in place when I arrived, but I know that they chose it because of the debugger and the fact that TestComplete recognizes more objects than QTP/UFT.

What about the implementation team?

It was done in-house.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Haiping Gong
QA Automation Manager at a comms service provider with 501-1,000 employees
Vendor
We use it to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%.

What is most valuable?

The name mapping function to manage the objects across different web pages and the support for different web browsers are the most valuable features for us.

How has it helped my organization?

We use TestComplete to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%. This helps to reduce manual effort dramatically, and by using the same script, we can run the test on both IE and Google Chrome.

What needs improvement?

The scripts are saved as binary files on disk. This makes it troublesome to perform version control and merge it with the work that is done by other team members. I am not sure what the reason is that SmartBear chose to save the files as binary, but if they can change to plain text for the scripts in the future, that would be helpful.

For how long have I used the solution?

I've used it for one year.

What do I think about the stability of the solution?

Some of the objects/parent objects are dynamically created in the web page. During the test, TestComplete will have problem to identify these kind of objects. This will cause a lot of false rejection and block the following test cases. Thus it requires some rework when running all the test cases as a batch.

How are customer service and technical support?

I have not used any customer service/tech support yet. I cannot provide more comments on that.

Which solution did I use previously and why did I switch?

TestComplete is the first tool that I use for automation on UI and I have not used any other tools yet.

How was the initial setup?

It is pretty straightforward. Just by following the instructions given in the document and do the corresponding configuration on the web browsers that you want to test. However, we also encounter some problems such as the Chrome blocking users from proceeding because of an unsafe SSL connection, but we found a work around eventually.

What about the implementation team?

We implement the automation by ourselves, not through any other vendor team. My suggestion is that you need to decide how to divide the automation work into corresponding components carefully. Otherwise, it will be hard for you to share the common scripts with other team members. Also, the code architecture design is very important as well.

What was our ROI?

I am not responsible with the licensing quotation/renewal, but the benefit that TestComplete brings to us is, apparently, acknowledged.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user341001
Junior ASP/.Net Developer at a construction company with 501-1,000 employees
Vendor
Its environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

Valuable Features

Object-based recording, and data-driven testing.

Separation of data into Excel files made tests modifiable by QA personnel with limited development experience, and object-based recording kept maintenance to a minimum.

Improvements to My Organization

TestComplete's environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

This allowed us to schedule execution of lengthy tests for nighttime and non-core hours, and to synchronize tests with Jenkins build releases.

This freed up QA assets to perform more specialized testing and reduced redundancy.

Room for Improvement

Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

Use of Solution

I've used it for approximately eight months.

Customer Service and Technical Support

Customer Service:

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

Technical Support:

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

Initial Setup

The initial set-up was exceedingly simple. There is a silent mode option available during set-up, which is very convenient when deploying to multiple machines or remotely.

Implementation Team

We implemented it through an in-house team.

ROI

We did not maximize our ROI until we put somebody full-time on our TestComplete endeavours. The tool does have a learning curve, and it wasn't until we had an in-house expert on it that we began to see the benefits of automated testing over traditional QA roles.

Pricing, Setup Cost and Licensing

The licensing options for TestComplete both running a licensing server. This prevents users from running more sessions simultaneously than purchased keys. This can prove problematic if you want multiple developers writing or running tests at once, and prevents you from using your key while a distributed test is running.

For pricing, carefully consider how many machines you want running the software, rather than the number of developers.

Other Solutions Considered

We also evaluated another SmartBear product called SoapUI. The change to TestComplete occurred because we changed our target from web applications to desktop.

Other Advice

My advice in regards to implementation would be to choose carefully which tests to automate, specifically focusing on lengthy procedures, tasks that require looping, or places where you want to test against multiple data sets.

Additionally, I found it beneficial to prefix my keyword tests with a character and number to provide logical ordering instead of alphabetic.

I also found it beneficial to record "undo" steps with each keyword test; this allows each test to be more stand-alone and prevents your test from being dependent on the state the previous test left the application in.

Finally, I would suggesting limiting the number of test applications per test suite to prevent bloated name-mapping schemes.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Systems Engineer at a tech services company with 51-200 employees
Consultant
The automation module helped us save time in regression tests, but we needed to perform adjustments on our systems to get it to fit in.

Valuable Features

The automation has improved over time, allowing it to integrate with other external tools for defect tracking.

Improvements to My Organization

The automation module helped us save time in regression tests.

Room for Improvement

It works by recording and playing a script for some part of development. However, there's no way of knowing where it failed. The developers needed help to correct these errors to script them in order to work. I noticed many changes made from version to version.

This tool is also very expensive compared to the similar tools in the market. Price needs to be affordable.

Use of Solution

I've used it since 2012.

Deployment Issues

We faced lots of code breaking, and needed to perform adjustments on our systems to get Test Complete to fit in.

Stability Issues

Sometimes it runs fast, and sometimes it runs slow.

Scalability Issues

No issues encountered.

Customer Service and Technical Support

Customer Service:

They used to respond in less than 24 hours, and compared to SpiraTest, it's not that great.

Technical Support:

SpiraTest took a minute to respond to any technical support issues, but Test Complete takes longer.

Initial Setup

It's not complex.

Implementation Team

It was already in place when I joined the company.

ROI

6/10.

Pricing, Setup Cost and Licensing

It's priced high, and they should look into it to make it more competitive.

Other Solutions Considered

We also looked at Selenium IDE, and chose it because it's cheaper.

Other Advice

Check the price and compare to other available tools in the market and decide select the one best fits the needs.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user69072
QA Manager at Airpush India Pvt Ltd
Vendor
Automation testing using TestComplete
For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It's a paid automation tool. It's an little tricky to learn and use but quite similar to Selenium. You can use xpath, record and play to script test cases. Pros: - Similar to Selenium IDE, you can record and play test scripts - Similar to Selenium webdriver, you can create test scripts in VBScript, JScript, DelphiScript, C++Script or C#Script. (personally, I liked VBScript) - Test results logs helps in finding precisely the issue where test failed - Tests can be recorded in keyword driven or screen capture modes - Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on - Can easily do…

For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It's a paid automation tool.

It's an little tricky to learn and use but quite similar to Selenium. You can use xpath, record and play to script test cases.

Pros:
- Similar to Selenium IDE, you can record and play test scripts
- Similar to Selenium webdriver, you can create test scripts in VBScript, JScript, DelphiScript, C++Script or C#Script. (personally, I liked VBScript)
- Test results logs helps in finding precisely the issue where test failed
- Tests can be recorded in keyword driven or screen capture modes
- Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on
- Can easily do cross-browser testing

Cons:
- Automated tests can not be run on Mac OSX
- No support for mobile testing automation (though there is a package 'seetest' which can be integrated with TC)
- It can not read xpath values for browsers other than IE, Firefox, chrome, Safari.

Also, you can learn about TestComplete by this book(written by Genndiy Alpaev and reviewed by me):

https://www.packtpub.com/application-development/testcomplete-cookbook 

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user327474
QA Automation Engineer at a computer software company with 501-1,000 employees
Vendor
We can test our applications manually for a full 48 hours continuously, but it lacks its own log viewer.

What is most valuable?

The opportunity to work with DevExpress and WPF objects.

How has it helped my organization?

  • Approximately, we get new build once per week. So I need to provide smoke testing for it. In case of manual tests execution, it is needed 48 hours (six working days) to smoke it. With TestComplete, we need just eight hours (or even less - if we're using multiple machines)
  • Stability and Stress testing. It is impossible to test our applications manually for 48 hours (two full days) continuously. With TestComplete, it is possible

What needs improvement?

  • Better stability, as sometimes, TestComplete crashes when attempting to delete over 10 logs
  • Support for the latest versions of DevExpress
  • More comfortable XML editor (like in Notepad++)
  • A better script editor. I will be happy, if the TestComplete editor would contain a design like the one in Visual Studio or Eclipse - errors and unusable variables being highlighte, and refactoring opporunities
  • Own log viewer in TestExecute. For now, it is possible to open it only within a browser, which is not very good, because if the log is 2GB or more, it will take 10-20 minuts to open with Internet Explorer

For how long have I used the solution?

I've used it for three years, including, v7, & v9 with TestExecute v9, and it's been in use on the project since 2010.

What was my experience with deployment of the solution?

When we migrated from v7 to v9, we encountered a problem with using recursive code; in DelphiScript recursion was completely broken.

What do I think about the stability of the solution?

Sometimes, TestComplete crashes when attempting to delete over 10 logs.

How are customer service and technical support?

Customer Service:

It's acceptable.

Technical Support:

It's acceptable.

Which solution did I use previously and why did I switch?

I tried Visual Studio 2010 (Coded UI Tests feature). The tool was changed, because TC is cheaper and more acceptable for those apps testing. But for now, this project doesn't use QA automation and I am working in another project

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Sr. Software Quality Consultant at a tech services company with 501-1,000 employees
Consultant
We're able to perform NPAPI test runs on Chrome, but the ExtJS extension and reading of JQuery standard modal screens is still not so good.

Valuable Features

Test Runs on Chrome. Test Runs are faster than the previosu version.

Improvements to My Organization

After NPAPI was unsupported by Chrome, our test runs were halted on Chrome for a previous version of TestComplete. Started running again with version 11.11.

Room for Improvement

ExtJS extension and reading of JQuery standard modal screens is still not so good. For Automating ExtJS based systems you have to do record and playback to extract methods and no substantial documentation has been provided.

Deployment Issues

None.

Stability Issues

The product is relatively stable.

Customer Service and Technical Support

Customer Service:

It is quick.

Technical Support:

Not so helpful most of the times. They always fail to produce a work around.

Initial Setup

It was easy.

Implementation Team

In-House.

Pricing, Setup Cost and Licensing

Do not buy all the modules at once, if your primary system under test is for example Web based then only buy the web module.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user323649
QA Team Lead at a tech vendor with 10,001+ employees
Vendor
It supports .Net, but it needs to have templates of tests from scratch.

What is most valuable?

Good IDE Support of third party controls Flexible in coding Supports .Net assemblies Good community

How has it helped my organization?

It's decreased the time needed for the regression testing cycle.

What needs improvement?

Memory consumption Speed of interaction with controls Sets of prepared libraries like asserts Templates of tests from scratch

For how long have I used the solution?

I've used TestComplete for three years.

What do I think about the stability of the solution?

It's stable.

How are customer service and technical support?

It's good, I like the support.

Which solution did I use previously and why did I switch?

Previously was QTP. TestComplete is much more flexible light,and easy to use.

How was the initial setup?

It's…

What is most valuable?

  • Good IDE
  • Support of third party controls
  • Flexible in coding
  • Supports .Net assemblies
  • Good community

How has it helped my organization?

It's decreased the time needed for the regression testing cycle.

What needs improvement?

  • Memory consumption
  • Speed of interaction with controls
  • Sets of prepared libraries like asserts
  • Templates of tests from scratch

For how long have I used the solution?

I've used TestComplete for three years.

What do I think about the stability of the solution?

It's stable.

How are customer service and technical support?

It's good, I like the support.

Which solution did I use previously and why did I switch?

Previously was QTP. TestComplete is much more flexible light,and easy to use.

How was the initial setup?

It's quite easy to start using it. There were no problems from the beginning.

What about the implementation team?

Use JScript for quick wins. Start creating tests as small as possible.

What was our ROI?

I would say that we have now automated about 100 tests, so it saves us three days every month..

What's my experience with pricing, setup cost, and licensing?

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user324366
QA Engineer at a computer software company with 51-200 employees
Vendor
It’s got flexible logging capabilities, but needs implementation of screen recording during testing.

Valuable Features

  • Its Multi browser testing capabilities
  • Record-and playback
  • Network suite testing (remote testing) capabilities
  • Key word driven testing with ability to finetune tests using scripting languages
  • Its flexible logging capabilities
  • High level of support from Smartbear
  • Support of other users in large userbase

Improvements to My Organization

Using TestComplete, the application tests are executed at night unmanned, so that test engineers can focus on developing new tests instead of executing existing tests.

Room for Improvement

Implementation of screen recording during testing (video recording).

Use of Solution

I've used it for approximately four years.

Stability Issues

No issues encountered,

Scalability Issues

No issues encountered,

Customer Service and Technical Support

Customer Service:

Very professional and adequate. I would rate this on a 9.5/10.

Technical Support:

Very professional and adequate. I would rate this on a 9.5/10.

Initial Setup

It's straightforward.

Implementation Team

I implemented it myself, using instructions from the vendor’s website.

Pricing, Setup Cost and Licensing

Start with evaluation version of the product, which can be downloaded from Smartbear's website, and after the initial evaluation, decide which licensing method you need.

Other Solutions Considered

I have evaluated other solutions, the cost model, the level of support of legacy software, the easy of use and easy implementation approach were main reasons to choose this product.

Other Advice

Focus on implementing a test automation framework. Start by downloading a free evaluation trial version of the product, and see if the product works well with the application(s) you want to design an automated test frame for.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user323895
Team Lead at a tech vendor with 501-1,000 employees
Vendor
It has a unified way of finding and manipulating controls from different vendors on different platforms, though it lacks support for different web UI frameworks and V8.

What is most valuable?

It supports JScript and it has unified way of finding and manipulating controls from different vendors on different platforms. As a key feature for beginners - It can record user actions and reproduce them.

How has it helped my organization?

First two companies began to rely on results of automated regression.

What needs improvement?

There is no qualified support for different web UI frameworks and no V8 support as well. The other thing that must be improved is their licensing policy with Virtual Machines.

For how long have I used the solution?

I've used it for six years.

What was my experience with deployment of the solution?

There were some issues.

How are customer service and technical support?

Customer Service:

5/5.

Technical Support:

5/5.

Which solution did I use previously and why did I switch?

I used different solution afterwards and I can say there is no other product with such great flexibility.

How was the initial setup?

The initial setup is as simple as any product with a Windows installer.

What was our ROI?

ROI is simple. We've spent three months on development and automated one month of regular regression tests done manually for each release. Now we did it for each build.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user323742
Test Automation Engineer at a tech vendor with 1,001-5,000 employees
MSP
It has helped significantly improve and speed up regression testing, although it could be improved with cross-platform testing (Linux, MacOS).

What is most valuable?

Ability to test all types of application (desktop, mobile, and web).

How has it helped my organization?

In all projects where I participated it helped significantly improve and speed up regression testing, which is usually bottleneck before release.

What needs improvement?

Since v11 of TestComplete it has all the features I was dreaming about during these years. Probably the only one great thing could be cross-platform testing (Linux, MacOS).

For how long have I used the solution?

I've been using it since 2006.

What was my experience with deployment of the solution?

No, I have never had any difficulties with TestComplete. All possible issues are also covered in TestComplete's help system and available online.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

It is impressive! What I like the most is the fact that support team helps solving problems not only for their registered customers, but everyone else as well.

Which solution did I use previously and why did I switch?

In last 12 years I also used SilkTest and Squish. They have some benefits which TestComplete doesn't have, but in general TestComplete overcomes them.

How was the initial setup?

Was it difficult to set up the application first time? No. Or was it difficult to start using it? No, it was easy thanks to its tutorials.

What about the implementation team?

I did everything by myself. There is nothing difficult in it, TestComplete is very user-friendly.

Disclosure: My company has a business relationship with this vendor other than being a customer: My company doesn't, but I wrote a book on TestComplete and sometime run online courses for students in Russian.
it_user323700
Assistant Lecturer at a university with 1,001-5,000 employees
Vendor
We're able to create test inputs with tens of thousands of fields, but it crashes or hangs at times.

Valuable Features

  • Front-end click and record
  • Ability to read in from and output to text files

Improvements to My Organization

  • Creating test input with tens of thousands of combinations of fields
  • Evolution of regression suite to sanity check new builds

Room for Improvement

  • Product ungracefully crashes or hangs sometimes
  • Sometimes gets caught out by messages appearing which confuses what is in focus

Use of Solution

I used it for five years.

Deployment Issues

No issues encountered.

Stability Issues

No issues encountered.

Scalability Issues

No issues encountered.

Customer Service and Technical Support

Customer Service:

It's good.

Technical Support:

It's good.

Initial Setup

It's a simple installation, plus a license key.

Implementation Team

  • Some dev input required to develop app for playback
  • Products own script playback facility is unreliable and we needed something that could kill and relaunch TestComplete

Pricing, Setup Cost and Licensing

This was all handled by management.

Other Solutions Considered

An evaluation of Microsoft competition found it to not be suitable for front-end testing (it was more suitable to integration testing).

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user323592
Software Engineer at a manufacturing company with 51-200 employees
Vendor
It has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

What is most valuable?

The two most valuable features are the manual script library (projects) and Test Execute.

How has it helped my organization?

I set up a manual test for new functions and hand it off to a tester (who uses Test Execute). Once the new function test is completed, it is added to regression for all future versions. This has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

What needs improvement?

From my understanding, the product has changed significantly since v5 was released. I don’t know that it has necessarily improved, for my needs. If I worked in a different shop, then I might need an entirely different feature set compared to what I use now.

For how long have I used the solution?

We've used it since May 2007.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

Customer Service:

I can't recall the last time I needed support.

Technical Support:

I can't recall the last time I needed support.

Which solution did I use previously and why did I switch?

Typically, I used my own method for automation (scripting tool, test data stored in SQL, results stored in SQL, result comparison to baseline with custom script). At the time I purchased TestComplete, the script recording tool wasn’t that great and I didn’t have the skills to write my own scripts. Over time, I used a different automation program and picked up the syntax for its scripting language. The product is currently called WinAutomation, though it had a different name at the time, that I can’t recall.

How was the initial setup?

The initial setup was very straightforward. I installed the main app on my local workstation and test execute on five workstations available for testers. We use a shared network drive to store tests.

What about the implementation team?

I installed everything myself. It was very straightforward. The real key is knowing what you need for your environment and to adapt the available tools.

What was our ROI?

It’s hard to sell testing tools, especially in a company that is owned by someone who thinks of software as a mule to pull his cart. It was a very tough sell and it is hard to quantify in dollars the money saved. However, I can say, with confidence, that there have been over one hundred bugs caught through regression since the software was purchased, which is a great track record for a team this size.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Quality Assurance Practice Leader with 51-200 employees
Vendor
Top 20
It reduced the regression timeline for a complex AUT, but there's an issue object mapping related to the need to modularize the test script code.

What is most valuable?

TestComplete is used for testing Windows desktop products; specifically the Embarcadero VCL interfaces created by Delphi/C++. All debug flags must be enabled during compilation that generates an output file required by TestComplete to identify and interact with the UI objects in the application.

An important feature of TestComplete is the ability to modularize testing. A lot of effort has been put into breaking the test script into reusable functions/methods that can be called from any test. A number of function libraries were created. This enabled reuse of code and kept the projects and project suites small. This is important because the size of these test artifacts have an impact on project loading time.

How has it helped my organization?

Test automation with TestComplete significantly (estimated 80%) reduced the regression timeline for a complex AUT with a large number of test cases. Automation with TestComplete has significantly shortened the feedback loop and the timeline to get a release production ready. A secondary benefit is that manual testers have begun thinking more technically about writing tests cases.

What needs improvement?

There are two major areas for improvement:

  1. Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
  2. There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.

For how long have I used the solution?

I have used TestComplete for 20 months. It was chosen as an incumbent toolset could not interact with the product to be tested.

What was my experience with deployment of the solution?

Integration with third-party products; specifically Microsoft Team Foundation Server and HP ALM could not be overcome. A custom integration to HP ALM was written using the HP ALM OTA API.

What do I think about the stability of the solution?

TestComplete has been a stable product.

How are customer service and technical support?

Customer Service can be slow to respond to electronic forms of communication and they do not have a way for a customer to speak directly to customer support. You create a ticket online, and request a phone call.

The team seems to be very knowledgeable when communication is established.

Which solution did I use previously and why did I switch?

TestComplete was added to the toolset. HP UFT is used for automated tests for other products.

How was the initial setup?

Setup is straightforward unless third-party tool integration is required. Integration with Microsoft Team Foundation Server is a little complex for initial configuration. Once it is understood the process is repeatable.

What about the implementation team?

Implemented in-house. Implementation is not difficult to implement or write tests especially if you have experience with other test automation tools.

What was our ROI?

We have not done a ROI calculation. However, automated testing with TestComplete has cut regression test time by months.

Which other solutions did I evaluate?

HP UFT was tried but the object recognition did not work with the implementation technology.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user303108
Application Support Specialist at a logistics company with 1,001-5,000 employees
Vendor
It provides for automated testing, helps to install plugins and extensions, and we use it as a data-driven testing and debugging tool, but I've only used the free-trial period so far.

What is most valuable?

Possibility of the automated testing, installing plugins and extensions, data-driven testing & de-bugger.

How has it helped my organization?

I’ve used it only as a free 30-days trial and I'm not sure of how our company is using it.

What needs improvement?

I’m not a specialized tester and I’ve used it just for some private tests and have found in it what I was looking for.

For how long have I used the solution?

I've used it for 3 years.

What was my experience with deployment of the solution?

I've not had issues with deploying it.

What do I think about the stability of the solution?

I've had no issues with stability.

What do I think about the scalability of the solution?

I've had no issues with scalability.

How are customer service and

What is most valuable?

Possibility of the automated testing, installing plugins and extensions, data-driven testing & de-bugger.

How has it helped my organization?

I’ve used it only as a free 30-days trial and I'm not sure of how our company is using it.

What needs improvement?

I’m not a specialized tester and I’ve used it just for some private tests and have found in it what I was looking for.

For how long have I used the solution?

I've used it for 3 years.

What was my experience with deployment of the solution?

I've not had issues with deploying it.

What do I think about the stability of the solution?

I've had no issues with stability.

What do I think about the scalability of the solution?

I've had no issues with scalability.

How are customer service and technical support?

Customer Service:

I didn't need customer service, so I have no experience with it.

Technical Support:

I didn't need technical support, so I have no experience with it.

Which solution did I use previously and why did I switch?

We did not use a different solution.

How was the initial setup?

It was neither straightforward nor complex.

What about the implementation team?

I'm not sure whether we did it in-house or through an outside team.

What was our ROI?

I'm not sure.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user295749
Quality Assurance Automation Development Engineer at a manufacturing company with 1,001-5,000 employees
Vendor
I would like to see improved language support, but it allows me to input parameters without having to write a quick VB script around it to try it.

What is most valuable?

The test tree makes for quick work of choosing which tests you want to run at any time. Whether it's all or nothing, I can re-run parts of a test suite without having to start over again.

The Object Spy has surpassed any other "spy" tools I have ever used. It allows me to input parameters without having to write a quick Visual Basic script around it to try it. Diving deep into object trees can be made a breeze with the search function.

What needs improvement?

I would like to see improved language support, with Python being my

first choice. Visual Basic is outdated, where C# has been its

replacement for years.

For how long have I used the solution?

This solution has been in use for almost three years, since I started working at this company. It has been mostly stable with static test cases, and only minor maintenance. The updates typically are changing search criteria. We upgraded from v9 about one year ago.

How are customer service and technical support?

Technical support has been good. There was an issue where certain installations of TestComplete would give an ambiguous error on startup. After teleconferencing with one of their support specialists, we were able to solve it. Forum support on their community site has also been good.

Which solution did I use previously and why did I switch?

No previous solution I know of was implemented prior to my employment here. Attempting to implement the same solution in UFT would have been nearly impossible.

How was the initial setup?

The initial setup was straightforward. Licensing was simple, and getting the initial object mapping was painless. Only a high-level base set of object mapping was done.

What about the implementation team?

The implementation was done entirely in-house. The projects were made from scratch, where function libraries & test suites were made from existing manual test cases.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Assistant Manager - Quality Control at a tech services company with 1,001-5,000 employees
Consultant
I'm able to automate functional test cases which I previously did manually, but I've come across difficulties in recognizing dynamically-generated runtime objects.

Valuable Features:

Dynamic Find methods FindChild FindAllChildren FindAll Keys method Project variables Time delay methods

Improvements to My Organization:

It helped to automate all the possible functional test cases which I previously used to do by manual testing.

Room for Improvement:

I have used it for Web Application automation and sometimes find difficulties while recognizing dynamically-generated runtime object.

Valuable Features:

  • Dynamic Find methods
    • FindChild
    • FindAllChildren
    • FindAll
  • Keys method
  • Project variables
  • Time delay methods

Improvements to My Organization:

It helped to automate all the possible functional test cases which I previously used to do by manual testing.

Room for Improvement:

I have used it for Web Application automation and sometimes find difficulties while recognizing dynamically-generated runtime object.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user294939
Test Engineering Team Leader at a consultancy with 501-1,000 employees
Consultant
After installation, I was able to immediately begin coding regression test for available functionality.

What is most valuable?

  • Initially support for flex/flash technologies, but since that project ended my primary use is Functional Automation against a web front-end built on AngularJS.
  • I use VBScript for a majority of my tests; it is one of many language choices available when writing scripts.
  • Out of the box support for desktop, mobile and web.

How has it helped my organization?

I am the only test engineer on the project, so time is limited in terms of automation development. Using TestComplete I was able to immediately (after installation) begin coding regression tests for available functionality.

My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

  • Number of tests = 75
  • Automated Time to execute = 15 minutes
  • Manual Time to execute = 360 minutes (approx.)
  • Automation time saved = 345 minutes (5+ hours)
  • 2300% improvement

I have substantially reduced the amount of time it takes for regression and build viability activities.

What needs improvement?

I pay for support and maintenance; having used the “support” functions through online forums, I can say there is room for improvement. I would like to have access directly to TestComplete developers as opposed to “users” who have some level of expertise with the platform.

For how long have I used the solution?

I've used it for two years.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

5/10, there are knowledgeable users in the forums, but I would like to go straight to the TestComplete staff with solution questions. That said, I have had very little need for product or development specific support.

Which solution did I use previously and why did I switch?

Other tools were evaluated which supported Flex/Flash web apps, but we chose TestComplete for its rich feature set as well as Smartbear’s available suite of tools outside of Automated Testing.

How was the initial setup?

It was very straightforward! You just need to install and begin, no special plugins or out of the ordinary system requirements were necessary.

What about the implementation team?

We implemented it in-house. As with any automation platform, carefully planning the intended usage, understanding the technology under test and determining which features will be used are critical to success.

What was our ROI?

Given the cost was minimal, the best representation of ROI from my perspective is time saved using automated tests vs. manually execution.

My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

  • Number of tests = 75
  • Automated Time to execute = 15 minutes
  • Manual Time to execute = 360 minutes (approx.)
  • Automation time saved = 345 minutes (5+ hours)
  • 2300% improvement

What's my experience with pricing, setup cost, and licensing?

Our recent purchase was a single node locked license for one user which was very inexpensive. That fit our project needs and the automation strategy. The solution resides on inexpensive laptop for development and execution.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Mobile QA Developer at a tech vendor
Vendor
It provides direct access to object internal properties and methods, but scripting languages should be extended to support more popular programming languages.

Valuable Features

  • High UI technologies coverage
  • Direct access to object internal properties and methods

Improvements to My Organization

Faster feedback on any product changes and uncovered problems before full regression starts. That can save a lot of time.

Room for Improvement

Scripting languages should be extended to support more popular programming languages like Java, C#, Ruby, Python. Many other vendors have such support or moved into such a direction. e.g. SilkTest has OpenAgent since 2008 with support of Java, C#. HP recently published LeanFT which integrates with popular development environments (Eclipse, Visual Studio). IBM solutions were initially designed for Java. Squish supports multiple languages. So, TestComplete should have support of such programming languages apart from VBScript and JScript modifications.

Use of Solution

About three years onwards.

Stability Issues

TestComplete had problems during long hours runs. In some cases it could crash without leaving any logs. Also, if there were some unhanded exception it could drop the error message waiting for user input. This is not really acceptable for nightly runs and required human assistance. But this problem is rather language-specific

Customer Service and Technical Support

Customer Service:

I never had to speak to them.

Technical Support:

I never had to speak to them.

Initial Setup

The setup is pretty easy.

Implementation Team

We used a vendor team. My advice for implementation is only one, follow the best practices (some of them are available on the vendor site). They were formulated based on real experiences.

ROI

ROI was never calculated explicitly, but normally the ROI point was expected to be reached after three to five months of the project.

Other Solutions Considered

Mainly, when we did an evaluation and comparison, TestComplete had two big advantages -

  • Good technology support
  • Relatively low price in comparison to other vendor tools
Disclosure: I am a real user, and this review is based on my own experience and opinions.
it_user293901
Principal Software Engineer - Automation at a tech services company with 10,001+ employees
Consultant
Our testing team can spend more time testing new features, but there often are issues with web test execution.

What is most valuable?

  1. Object Browser is the most powerful feature. This has helped me tear apart every single component of the application under test. It's an awesome feature.
  2. Test Editor and Debugger: This has really made my life easy as I can trace memory and stack values very easily.
  3. Data driven testing has been made very easy. I am comfortable using any sort of backend to support data driven testing such as Excel/databases/etc.
  4. It supports using third party DLLs has given me an edge on several occasions.

How has it helped my organization?

Previously, we had to wait for the end-of-sprint to make a stable release. Now, rolling out a release is not a problem. As soon as a build is deployed, the scheduler quickly executes User Acceptance Tests, and verifies that the build is fit to be delivered to the clients.

What needs improvement?

  1. Support for working with COM and ActiveX objects should be more powerful. Currently we have to drill down a lot to find solutions for certain tasks.
  2. Cross browser testing is a bit complex when it comes to scripting. Developers have to take care with it care, or else issues may arise at deployment time.

For how long have I used the solution?

Overall, I have used this for more than two years, including v1040. Currently I have been using it hands on for the last year.

What do I think about the stability of the solution?

There often are issues with web test execution. Web pages often do not load on the first attempt.

How are customer service and technical support?

Customer Service:

Customer server is absolutely robust. The team respond within 12 hours.

Technical Support:

I never needed to use paid technical support as their forum is extremely helpful. The community members respond to issues within 24 hours. It’s really amazing that within the past twelve months, I have never had a technical question I asked remain unanswered.

Which solution did I use previously and why did I switch?

I am an expert on the following:

  • Selenium RC and Servers/Java - but this has multiple limitations
  • IBM Rational Functional Tester - it's expensive, and not as powerful as TestComplete
  • AppPerfect - is not in competition with any of the above, it’s pathetic

How was the initial setup?

It was straightforward. Both the node locked and floating licenses were easily deployed, and now the whole team is using it.

At one point, we had to update the hardware of the servers where the licenses were deployed, and this was very easy. We just disabled the licenses from the account and deployed them onto the upgraded servers.

What about the implementation team?

I did it myself in-house. It’s very simple. No rocket science needed. Just read the manuals and you can do it.

What was our ROI?

We are saving a lot in terms of client satisfaction. We are not yet in a phase where we have savings in terms of a reduced QA team, but our clients are happy that they get bug free software. Moreover, our testing team spends more time testing new features, and we are confident enough at build roll out time that all previously developed modules are stable.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
QA Automation Engineer with 501-1,000 employees
Vendor
A versatile tool for developing and executing automated test suites.

What is most valuable?

To me, the Object Browser has proven to be the most valuable feature of this product. This tab (and its associated tools) allows the test developer to locate objects within the application to be tested, and retrieve properties and methods that can be used in scripting. This saves a significant amount of time over referring to the application source code for such information. It also makes use of the recording feature periodically to rough out scripts for new tests. The recorded scripts are very fragile, but they can provide a good base for writing stable scripts.

How has it helped my organization?

While our test automation initiative is still in its early stages, TestComplete is allowing us to script and run complex business process tests against our desktop applications. Being able to create smaller modules that can be referenced by other scripts or keyword tests allows for a flexible and easy to maintain test suite.

What needs improvement?

A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

For how long have I used the solution?

The product has been in use at the company for about two years and I have been actively using it for about 8 months.

What do I think about the stability of the solution?

I haven’t experienced issues that I would attribute directly to the application. There have been some issues with timing and other common problems, but many of those have been corrected through modifications to the tests and the environment.

How are customer service and technical support?

My experience with their technical support has been less than satisfactory, but all of my issues were resolved. Of the technicians I have worked with, two of them were very knowledgeable and one of them did little more than email knowledge base articles that had been ruled out in the original issue report. The latter tech had the additional problem of being about 12 hours out of sync so a full day was lost with every email exchange. I received prompt responses and quick resolutions from both of the knowledgeable technicians I have dealt with although one of them was rather gruff and left me feeling like I was bothering her. Overall, SmartBear needs to work on their customer service and technical support.

Which solution did I use previously and why did I switch?

This application did not replace a previous automation effort.

How was the initial setup?

When deploying new installations of TestComplete, I ran into multiple issues with the licensing engine. The first time I experienced the problem, I contacted support and was directed to a knowledge base article (which I had been unable to find while searching for the error message). I have since bookmarked the article and reference each time I need to do a new install. Once the licenser has installed properly, the configuration and licensing goes smoothly.

What about the implementation team?

The implementation was performed in-house, and was quite easy despite the issues with the licenser. The best advice I can offer would be to plan out your testing environment and requirements before you begin. If you plan to use virtual machines for your test systems, make sure you also have a physical box to run the licenser that is dedicated to the lab. Do not attempt to use someone’s PC as it may not always be available when the test machine are trying to run.

What was our ROI?

We have not performed an ROI analysis.

What's my experience with pricing, setup cost, and licensing?

SmartBear’s product licenses are reasonably priced and generally in line with similar companies and products.

Which other solutions did I evaluate?

I reviewed several products before deciding to continue our automation efforts using TestComplete. Among the products reviewed were Telerik TestStudio, Testing Anywhere, and Ranorex.

What other advice do I have?

Disclosure: I am a real user, and this review is based on my own experience and opinions.
ITCS user
Sr. Software Quality Consultant at a tech services company with 501-1,000 employees
Consultant
Whereas we did regressions and Smoke manually in the past, we now do automated testing. However, Intellisense demos do not work on methods in the classes.

What is most valuable?

  • Dynamic Find methods - FindChild, FindAllChildren, and FindAll
  • Keys method
  • Project variables
  • Time delay methods
  • aqString methods
  • Intelisense

How has it helped my organization?

Regressions and Smoke was done manually in the past, which has been replaced by Automated Testing.

What needs improvement?

The web testing framework of TestComplete is not very helpful for an Automation Engineer. It requires the same effort as Selenium, and. in most cases, Selenium proves to be a better testing tool for web-based testing.

Web Application Testing has the following issues:

  • Events do not work on Modal Dialog
  • Dynamic Object Capturing is too hard and takes too much time in finding the correct object
  • Intellisense demos do not work on methods in the classes

Similarly, the mobile testing framework of TestComplete is not of much use either. I do not recommend anyone use this tool for web or mobile testing.

For how long have I used the solution?

I have been a user of this tool for five years, including previous versions, on desktop, mobile and the web. I have completed a few major projects with thsi tool and have enhanced numerous test suites created with this tool.

What do I think about the stability of the solution?

There have been quite a few issues. Delay methods are not so dynamic in TestComplete, so if a screen hasn't loaded, and the delay is a static delay, then the test moves forward in script, but the previous operation has not been done, resulting in failure of the whole test one after another.

This also means that TestComplete and TestExecute work in a not so similar manner, causing TestExecute to be practically useless. This happens in the case of HTML5 based web application.

How are customer service and technical support?

Customer Service: Level of customer services is satisfactory. Replies are swift and prompt.
Technical Support: Solutions are scarce. If there is no solution to a problem, the support people will tell you there is no way around it, and then you are stuck. The issues are not resolved in the next releases, mainly because these are not escalated on the backend.

Which solution did I use previously and why did I switch?

SilkTest was being used previously with my previous employer. It was abandoned because it was hard to script and advanced features were not being added.

How was the initial setup?

The previous solution I used was SilkTest, which was being used by my previous employer. It was abandoned because it was hard to script, and advanced features were not being added.

What about the implementation team?

It was done through a vendor. It's better to get it done through vendors, as they are then responsible in the future.

What's my experience with pricing, setup cost, and licensing?

It is not a good idea to choose this tool only if it is being used by others. In our case it is an expensive tool, and we should have exerted the same effort by using Selenium, and would have had better results. It appears that if you have a bif web application built in HTML5 and Jquery, and uses Divs and Modal dialogs, then avoid using TestComplete.

Disclosure: I am a real user, and this review is based on my own experience and opinions.