At the moment, our focus is automation and API contract validation. We are looking for tools that focus on delivering the aforementioned functionalities rather than more generic ones.

In the future releases of SmartBear TestComplete, I think the integration towards Jira could be useful. Reporting and working with test managers is important to me without getting into too specific details. A solution needs to generate and shift through data results while improving reporting and automated reporting of errors to make it simpler. I want the solution to be more usable since someone with much to do can't afford to take very long on reporting. SmartBear products generally have a weak link when it comes to integration with other test management tools like Inflectra.

One notable drawback is the absence of native integration with Git. Additionally, navigating various functions within the TestComplete suite may not be as user-friendly. I've encountered specific challenges, such as playback issues with recorded scripts and inconsistent failure messages when running the same script.

The current price of SmartBear TestComplete is an area of concern. The price of SmartBear TestComplete needs to be lowered in the future since competition in the market is very high. Some companies also provide automation tools for free, so SmartBear TestComplete should consider improving its areas attached to costing or pricing while incorporating the latest technologies. I could not connect to BrowserStack with SmartBear TestComplete, whereas I could connect to BrowserStack with TestProject.

The pricing is the constraint. 

They don't have any smart features for data-driven testing. The test data-driven framework should be more explanatory, and it should be more advanced.

For example, if I want to evaluate some of the web applications, which have a lot of test data entries, and I want to put my data from Excel or that kind of integration. Maybe it has that feature but not a smart feature. In my previous experience, I have used some smart tools with very smooth features. If they could add a data-driven testing feature, it would be ideal.

At times, identifying or locating an element can be somewhat challenging. However, in a recent test update, they introduced the Optical Character Recognition (OCR) capability. This introduction has reduced the challenges to some extent, as we can now utilize OCR if the normal method doesn't work. Nevertheless, there is still significant potential for improvement in TestComplete's ability to identify various object elements.

I don't have any specific concerns to mention. I have observed significant improvements in TestComplete over the past few years, especially in its support for highly dynamic object elements used in products like Salesforce Dynamics 365. In earlier versions, there were numerous challenges, but the current version is far superior to its predecessors.

When we try to automate and keep the same script in a few cases, it fails due to control changes. You must have a session open when running in TestComplete, otherwise, your script will fail.

When compared to the competitors, it is a little more complicated.

During the distribution of our regression test cases, the control IDs are not always recognized correctly.

Our frequency control IDs are getting changed. For example, if I develop a script in one mission and then try to deploy the same script in another mission, the script will not be executed properly.

Selenium is more efficient than TestComplete at the point of execution, in my experience.

In terms of execution, attempting to run a script from multiple missions will result in the script being deployed and failing to work in some of the missions.

It would be beneficial if they provided a new open-source library that could be integrated with Selenium. Currently, we must create Java bridge libraries in order to integrate with Selenium. It would be preferable to have a simple APA for consuming the Selenium feature console.

This solution could be improved by making it easier to visualize where there is a failure without having to look at it in detail. In comparison, Leapwork allows us that visibility and showed us exactly where it failed more easily.

Right now, the product only supports Windows. We'd like to see it work with other operating systems as well. They should definitely be working with iOS, for example. That should just be a given.

Using object spy can be slow sometimes and seems to require a lot of resources.

1. Support for working with COM and ActiveX objects should be more powerful. Currently we have to drill down a lot to find solutions for certain tasks.
2. Cross browser testing is a bit complex when it comes to scripting. Developers have to take care with it care, or else issues may arise at deployment time.

There could be API interfaces with this tool.

There is a problem with usability because the speed decreases, which could be an issue with scalability because of too many hits on the site. It depends on how distributed their systems are, and how well they can handle multiple connections.

The artificial intelligence needs to be improved.

Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work.

* Product is not stable enough and it crashes often

The application under test is a complex scientific application developed in C++ & C# and use different technologies. So when I try to do Name mapping, it sometimes hangs and have to restart Test complete

* Checking from TestComplete to TFS has issues.

I mostly try to checkin the changes with Visual studio as TestComplete hangs while you try to check in with many changes directly to TFS from Testcomplete.

* Possibility to run a part of keyword tests through TestExecute

Consider you have a keyword test with 10 individual tests. By using test execute, you can only run the main one, not the individual ones.
So I like to have the possibility of running from the 3rd test or just running the 5th one.The option is not present in TestExecute (you can do it in Test complete).

The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT.

In UFT, using descriptive programming for a web page you can use.

Browser(description).WebList(description).Select anything.

Regardless of how many panes, frames, panels etc are in the hierarchy before the Weblist object.

But in Smartbear you have to store every frame, panel etc.

So that if you didn’t use the ‘Alias’ functionality you would have an object description miles long.

But even having to use the Alias, you still have to add each and every frame, panel, etc. whereas in UFT you can just use page.object and it will find the object on the page (as long as you’re using unique descriptions!!) without worrying about frames, panes, etc.

The integration tools could be better. It would be useful if we could use it with other test management tools.

We'd like to see the solution add a few more features to the offering.

They're working on many features. Of course, the roadmap is not news to me, but yes, of course, they are working on different features.

I was going to say load and performance but I really didn't get a chance to get into the load and performance part of TestComplete.

I can sense there's still a little bit more room for improvement in terms of making the whole testing process easier. Because it was desktop first, it kind of focused on COTS-based testing and API's because everyone's using it at the moment.

To bring it up to a 10, I would be looking for the addition of some key functional API testing. What I mean by key is like the video streaming or a consistent flow of data being used and then how much can it handle and how much API testing can it do because that's where the load and performance comes in. The API testing is where I would see a lot more improvement.

TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services. In a microservice world, this is a big flaw. Another thing is that the cross-browser support has a lot of different traits between browsers. It should be improved.

• Error handling features in the tool are a little limited
• The tool currently is Windows only, would like to see it being implemented for Mac and Linux
• Once the tests are created in one languages, you can’t change it use other languages

There are two major areas for improvement:

1. Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
2. There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.

In SmartBear TestComplete the integration with Jenkins could be easier. Additionally, some of the controls could have better customization options. For example, if a grid is used and it contains multiple controls within it, it can be a checkbox, radio button, or any kind of control, the way the Object Spy is operating currently there is a lot of room for improvement.

There are a lot of improvements that are required on the Object Spy, such as how we find the locators. That is a feature which I would love to see. 

It is currently very tightly coupled to the SmartBear ID. There should be a way that they could support the integration with external libraries and make it flexible to integrate with open-source frameworks. This would be a good option to add.

It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing).

In fact, this is an issue regarding how to read the test log in MHT format generated by TestComplete.

We know a test log is always generated automatically by test tool after executing a test. TestComplete provides a standard IDE (Integrated Development Environment) interface for user to review the test log. We don’t have issue reviewing the test log within TestComplete regardless how big the test log is (Our test log record the test running for more than two days without stopping. The test log extension name is .tcLogs and the size of total test log is very big over 2.5 GB).

TestComplete also provides a function for the user to export the log into MHT format file via manually selecting each test log after complete a test or dynamically exporting the test log during test executing via command in test script. The MHT log file can be opened by IE. However, if the MHT file is a little big, such as >200M, the user could experience obvious performance issues. It takes a very long time for IE to open this file because IE loads all the contents into one page which could finally eat up all your system memory. I tried to find another tool to open the big MHT log file, but no luck. Even executing a small test (running for one to two hours), the MHT log file size can easily reach to 200M, if a screenshot is included, for example. Normally, each one round of our regression test runs for 30-48 hours. It is impossible for us to open the big MHT log file as the system runs out of memory and an error appears definitely after hours of waiting to just open this file by IE.

TestExecute is a small tool to just execute the test developed by TestComplete. But TestExecute only generates MHT log file rather than TCLOGS file. Obviously, we cannot open the MHT log file if it is very big. So we gave up using TestExectue and export MHT log file function in TestCompelete at all.

We have to buy six individual full license of TestComplete to run and review the test.

The scripts are saved as binary files on disk. This makes it troublesome to perform version control and merge it with the work that is done by other team members. I am not sure what the reason is that SmartBear chose to save the files as binary, but if they can change to plain text for the scripts in the future, that would be helpful.

Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

I pay for support and maintenance; having used the “support” functions through online forums, I can say there is room for improvement. I would like to have access directly to TestComplete developers as opposed to “users” who have some level of expertise with the platform.

From my understanding, the product has changed significantly since v5 was released. I don’t know that it has necessarily improved, for my needs. If I worked in a different shop, then I might need an entirely different feature set compared to what I use now.

A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

We have something called Name Mapping in TestComplete it is the only Name Mapping that is available for TestComplete. If they allowed two Name Mappings, or run Name Mapping in run time, that would be ideal.

TestComplete has its own XPath Identifiers. It auto-identifies the XPath of the application. If that engine could better identify more XPaths automatically and make the process more flexible, that would be better. 

Code analysis would be helpful. If they could give us a code analysis feature, similar to something called Ranorex Coach (which will give us auto-suggestions of how to use certain things), that would be ideal. In Tricentis, they have a feature whenever we identify an element or automate an element, they give a unique identification element that the tool notes when we map. Here, we don't have a feature that can mention the unique object identification kind of stuff. We need to highlight it on screen and then we need to check it. 

The one issue we have is that the tool was installed in a local machine. Now, it's more popular when you use tools that work in online environments. It would be better if it was available on the cloud. We'd like to access it on remote desktops. I don't want to have to install it on every machine.

It would be ideal if the solution could offer one subscription for time and one for life. Right now, when you buy the solution, you need to pay for one solution. You receive one set up and you install it and it's just in that one machine. It would be ideal if they could offer one subscription where you can connect to different machines with a group subscription. Or maybe they could offer different pricing. They need to be more flexible.

The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools like Notepad++. Performance is another aspect which can be improved.

This product continually improves and in v11, they now have Python support. This was something that I wanted and they provided it the latest version.

I would like to see improved language support, with Python being my

first choice. Visual Basic is outdated, where C# has been its

replacement for years.

Scripting languages should be extended to support more popular programming languages like Java, C#, Ruby, Python. Many other vendors have such support or moved into such a direction. e.g. SilkTest has OpenAgent since 2008 with support of Java, C#. HP recently published LeanFT which integrates with popular development environments (Eclipse, Visual Studio). IBM solutions were initially designed for Java. Squish supports multiple languages. So, TestComplete should have support of such programming languages apart from VBScript and JScript modifications.

The solution needs to extend the possibilities so that we can test on other operating systems, platforms and publications for Android as well as iOS.

Right now, Complete can test only on native Android and native iOS applications.

• Better stability, as sometimes, TestComplete crashes when attempting to delete over 10 logs
• Support for the latest versions of DevExpress
• More comfortable XML editor (like in Notepad++)
• A better script editor. I will be happy, if the TestComplete editor would contain a design like the one in Visual Studio or Eclipse - errors and unusable variables being highlighte, and refactoring opporunities
• Own log viewer in TestExecute. For now, it is possible to open it only within a browser, which is not very good, because if the log is 2GB or more, it will take 10-20 minuts to open with Internet Explorer

I didn't use it very heavily. One issue that I found was that there wasn't a quick way or a button to move Visual Basic scripts to TestComplete. We have a lot of such scripts in our organization, and it would be very useful to have some option to easily move these scripts. It is currently possible to convert these scripts to TestComplete, but it is not easy. I have to write some code, but everything is not available immediately.

We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses. This was because sometimes the Windows PC feeding the scanner script data was faster than the scanner could process it, so we had to constantly tweek the wait times so the script wouldn’t bomb out. I have since used software with a “wait for” function that would wait for a response, but if TestComplete did, we didn’t know about it.

One of the main reasons I was hired was to use TestComplete with green screen applications, which was possible, but not easy. You had to know the exact column and row position of every text line on the screen so you could verify the string being tested and analyze exactly where and what the response would be.

The web testing framework of TestComplete is not very helpful for an Automation Engineer. It requires the same effort as Selenium, and. in most cases, Selenium proves to be a better testing tool for web-based testing.

Web Application Testing has the following issues:

• Events do not work on Modal Dialog
• Dynamic Object Capturing is too hard and takes too much time in finding the correct object
• Intellisense demos do not work on methods in the classes

Similarly, the mobile testing framework of TestComplete is not of much use either. I do not recommend anyone use this tool for web or mobile testing.

The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS.

The licensing costs are a little bit high and should be reduced.

I would like the ability to automatically distribute web applications to all of the concurrent users.

- Increased performance with less memory and cpu usage.

- Supports various and different versions of browsers in web testing.

- Support to Windows mobile application. (Native iOS and Android apps are supported currently.)

It works by recording and playing a script for some part of development. However, there's no way of knowing where it failed. The developers needed help to correct these errors to script them in order to work. I noticed many changes made from version to version.

This tool is also very expensive compared to the similar tools in the market. Price needs to be affordable.

The solution needs more training manuals or some form of online forum for learning. It needs more documentation. 

Some sort of troubleshooting portal should be available for users.

The solution could be faster. It would be good to increase the reusability of the code so that rerun time can be saved. It would be ideal to be able to run multiple scenarios at the same time.

Right now, in terms of recording and scripting the solution is not user-friendly. They need to improve on this.

A test case design (schema) module would be nice.

• Memory consumption
• Speed of interaction with controls
• Sets of prepared libraries like asserts
• Templates of tests from scratch

The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools.

On some of the UI application pop up screens, Closing and opening screens etc.

A more effective way in Objects Mapping for applications built on domains/platforms such as Sliverlight, and Citrix is needed.

More APIs could be added.

ExtJS extension and reading of JQuery standard modal screens is still not so good. For Automating ExtJS based systems you have to do record and playback to extract methods and no substantial documentation has been provided.

It has memory leaks. I have seven complete test projects, all of them loaded in one project suite. I don't know why memory increases when I just write the code and check-in/out files from the VSTS.

Should improve parts for desktop products.

I would like to see recovery scenarios.

Since v11 of TestComplete it has all the features I was dreaming about during these years. Probably the only one great thing could be cross-platform testing (Linux, MacOS).

Implementation of screen recording during testing (video recording).

• Product ungracefully crashes or hangs sometimes
• Sometimes gets caught out by messages appearing which confuses what is in focus

I have used it for Web Application automation and sometimes find difficulties while recognizing dynamically-generated runtime object.

What is currently missing from this solution is better support for mobile testing. This is what we are currently negotiating with them for.

Specifically, we would like to see Flutter support in the next release of this solution.

I’m not a specialized tester and I’ve used it just for some private tests and have found in it what I was looking for.

In the cross-browser domain, it has a few snags with Microsoft Edge and Chrome; although, these problems are not critical. Overall, The internal customer feedback is above average.

There

LoadComplete could be a little more user-friendly, but is still better than LoginVSI. Network issues sometimes made for poor connections.

I'm completely satisfied with TestComplete 10, but looking forward to seeing what's in store for version 11!

There is no qualified support for different web UI frameworks and no V8 support as well. The other thing that must be improved is their licensing policy with Virtual Machines.