SmartBear TestComplete vs TestRail comparison

Cancel
You must select at least 2 products to compare!
SmartBear Logo
7,138 views|5,012 comparisons
85% willing to recommend
IDERA Logo
3,564 views|1,637 comparisons
90% willing to recommend
Comparison Buyer's Guide
Executive Summary

We performed a comparison between SmartBear TestComplete and TestRail based on real PeerSpot user reviews.

Find out in this report how the two Test Automation Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.
To learn more, read our detailed SmartBear TestComplete vs. TestRail Report (Updated: April 2023).
767,847 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"This company offers end-to-end capabilities for test suite creation and execution. One feature that I particularly appreciate is the tagging system. Tags are highly valuable, as they allow you to assign tags to your test cases. When there's an impact in a specific area, you can search for and run all test cases associated with that tag. I find this functionality very useful.""TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications.""It allows us to test both desktop and web applications.""When compared to other tools, it is very simple.""The solution has a very nice interface.""The ease-of-use and quality of the overall product are above average.""The reporting is ready to use and doesn't require any setup.""TestComplete is simple, it's a very easy-to-use tool."

More SmartBear TestComplete Pros →

"I was on the lookout for automation testing on the browser and I believe this tool is very interesting in that matter. The solution is useful for UI testing. You just need to add the URL that is to be checked.""Integration with Confluence and JIRA.""The product’s most valuable feature is the UI. The structure of test cases is easy to understand.""The API to support integration of the homemade automated testing tool.""From a testing perspective, the management is awesome. I am able to do testing and then add the reporting and the evidence. It is fair in terms of the price that you're paying. You get what you're paying for.""Reliable and stable. It is important that TestRail be up and running 24/7 as we have users around the world using it.""Most valuable features are the ease of organizing test cases and a great API for sending results from automated test run results into the database.""I haven't faced any stability issues using the solution. Stability-wise, I rate the solution a ten out of ten."

More TestRail Pros →

Cons
"SmartBear products generally have a weak link when it comes to integration with other test management tools like Inflectra.""I didn't use it very heavily. One issue that I found was that there wasn't a quick way or a button to move Visual Basic scripts to TestComplete. We have a lot of such scripts in our organization, and it would be very useful to have some option to easily move these scripts. It is currently possible to convert these scripts to TestComplete, but it is not easy. I have to write some code, but everything is not available immediately.""It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing).""During the distribution of our regression test cases, the control IDs are not always recognized correctly.""Error handling features in the tool are a little limited.""Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work.""The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS.""This solution could be improved by making it easier to visualize where there is a failure without having to look at it in detail."

More SmartBear TestComplete Cons →

"Reporting could be more flexible regarding repeating reports.""TestRail by Gurock could improve by adding a defect management module tool. It would add a lot of value if I want to install it and I don't have Jira or an isolating team. For example, if I am providing a service it's separated from the development team, it then would be better to have defect management included with the test management. However, as it is now I need to be integrated with Jira or another defect management tool to complete the testing process.""There are a number of improvements that have been requested. While I don't have a list of these requests available, many can be found on Gurock's forum.""I do see room for lots of improvement in it. In terms of usability, duplication with test cases and constant creation of projects isn't easy. There is also too much API integration into automation tools, which is not there in ALM with UFT. Instead of setting it up as a project and using it, we set it up as a system for usability. It also lacks in the traceability aspect. For traceability, you need to use the JIRA plugin and drag traceability on JIRA, but the functionality is still quite limited. The biggest gap is mainframe testing. It would be good if I could start with mainframe testing. Manual granting of access is another issue. There is no API that I could use with another system where it is automated. There is an API for loading somebody to a project but not for adding to the application.""It's not easy to create a custom report. It's not straightforward. A good improvement would be if there was a way to report and create a custom report without using a plugin or scripting language.""With TestRail, the APIs are there, but they may not be able to easily integrate with the Jenkins.""I have faced some issues with the integration between TestRail and Jira, which haven't been permanently resolved yet.""The platform needs improvement regarding performance and creating links."

More TestRail Cons →

Pricing and Cost Advice
  • "The product is becoming more and more expensive."
  • "My advice so far, is that while it’s not quite as powerful and easy to use as UFT, its price tag more than makes up for it."
  • "The price is less, compared to other products, such as QTP."
  • "Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day."
  • "TestComplete now have come up with three modules (Web, Desktop & Mobile), so based on the type of product for automation, it is adequate to purchase the required module."
  • "This is a pay-per-use service that is not expensive, and cost-efficient if you have a small team."
  • "The option we chose was around $2,000 USD."
  • "The licensing costs are in the range of $1,000 to $3,000."
  • More SmartBear TestComplete Pricing and Cost Advice →

  • "Use TestRail Cloud (online TR hosted server) and don't worry about maintenance or scalability. It saves a lot of cash and time."
  • "My advice to others is to shop around for the best deal. Some options out there are free in cyberspace."
  • "Pricing for small teams seems correct with respect to competitors."
  • "Negotiate the best deal you can."
  • "Its price is definitely not more. If they introduce automation, they can charge more."
  • "The price of the solution is based on how many users you have per year. When you grow, it is segmented, For example, 10 to 25, you have a price, and more than 50, or 100, you need to take the enterprise license. I don't think we will reach this point."
  • "I give the price a five out of ten."
  • "The product is not much expensive."
  • More TestRail Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
    767,847 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:This company offers end-to-end capabilities for test suite creation and execution. One feature that I particularly appreciate is the tagging system. Tags are highly valuable, as they allow you to… more »
    Top Answer:There are certain challenges related to the license management system in place. It comes with a high cost. An annual price is around four thousand five hundred plus per user, whereas UiPath is only… more »
    Top Answer:At times, identifying or locating an element can be somewhat challenging. However, in a recent test update, they introduced the Optical Character Recognition (OCR) capability. This introduction has… more »
    Top Answer:I use the solution for test management.
    Top Answer:The product has a reasonable price in terms of the features.
    Ranking
    8th
    Views
    7,138
    Comparisons
    5,012
    Reviews
    7
    Average Words per Review
    503
    Rating
    7.1
    3rd
    Views
    3,564
    Comparisons
    1,637
    Reviews
    8
    Average Words per Review
    309
    Rating
    8.4
    Comparisons
    Also Known As
    TestRail by Gurock
    Learn More
    Overview

    What is SmartBear TestComplete?

    TestComplete is a reliable, sturdy automated testing platform created by SmartBear Software. SmartBear Software is a worldwide technological leader known for developing quality enterprise-class development and testing solutions.

    TestComplete simplifies the process of creating tests for numerous applications, including, but not limited to; Desktop, Android, IOS, Web browsers, and Windows. Application tests can be manual, scripted, and even recorded by using keyword-driven or data-driven functionality. There are even additional options for error reporting and automated playback. The object repository is extremely accurate and is fully customizable. TestComplete can easily be used by experienced developers and even by manual novice testers to develop quality automated UI tests quickly.

    TestComplete offers three different testing scenarios:

    • Desktop: Users can easily and quickly automate UI tests using today’s most popular desktop applications, such as; Windows, Java, Python,.Net, VBScript, and more.

    • Web: Users can effortlessly create renewable tests for all of today’s popular web applications, including JavaScript frameworks on 2000+ trusted browser and platform integrations.

    • Mobile: Users can safely build and automate serviceable UI tests on actual or virtual android or IOS devices, locally or in the cloud. Users can create code or codeless tests. TestComplete seamlessly integrates with many of today’s popular frameworks.

    Key Features

    • Easily create automated UI tests: TestComplete offers scriptless Record and Replay or simple keyword-driven tests to quickly develop any type of UI test users may require. Tests can be recorded once, then replayed when needed across various applications on mobile, web, or desktop environments. TestComplete integrates with many different languages, such as Java, Python, C+, and more.

    • Keyword driven tests: Users can easily divide testing steps, actions, objects, and data with an integrated keyword-driven test structure. This makes it easy for every user to participate in the test automation process; there is no programming experience needed. Everything is made simple with easy-to-use point-and-click options.

    • Data driven tests: Easily distinguish data from test commands to keep administrative efforts simple. Users can improve overall coverage by running various automated mobile, desktop, or web UI tests.

    • Record and Replay: Users can reuse created automated tests across every environment as often as desired. This helps to expand overall test coverage and represents a huge cost and time savings.

    Reviews from Real Users

    Sandhiya T S., Sr Solutions Engineer at Lexington Soft, relates, “The record and replay aspects of the solution are quite useful for people. With them, you don't have to write any scripts. Basically, you can record your actions and play them back later. The initial setup is also very easy.”

    Sai S R., Staff Test Architect at a tech services company, says, "The most valuable features of the SmartBear TestComplete are self-healing, they reduce the maintenance required. The different languages SmartBear TestComplete supports are good because some of our libraries are written in Python, JavaScript, and C#. It's very easy to put them all under one project and use them."

    TestRail helps you manage and track your software testing efforts and organize your QA department. Its intuitive web-based user interface makes it easy to create test cases, manage test runs and coordinate your entire testing process.

    Easily track and follow the status of individual tests, milestones and projects with dashboards and activity reports. Get real-time insights into your testing progress and boost productivity with personalized todo lists, filters and email notifications. Efficient test management, get started today!

    Sample Customers
    Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
    Apple, Microsoft, Boeing, Intel, NASA, Amazon, HP, Samsung
    Top Industries
    REVIEWERS
    Computer Software Company31%
    Manufacturing Company13%
    Financial Services Firm9%
    Comms Service Provider6%
    VISITORS READING REVIEWS
    Computer Software Company19%
    Manufacturing Company11%
    Financial Services Firm11%
    Government7%
    REVIEWERS
    Computer Software Company50%
    Recruiting/Hr Firm13%
    University13%
    Financial Services Firm13%
    VISITORS READING REVIEWS
    Computer Software Company23%
    Manufacturing Company10%
    Financial Services Firm10%
    Healthcare Company7%
    Company Size
    REVIEWERS
    Small Business28%
    Midsize Enterprise28%
    Large Enterprise45%
    VISITORS READING REVIEWS
    Small Business23%
    Midsize Enterprise15%
    Large Enterprise62%
    REVIEWERS
    Small Business61%
    Midsize Enterprise17%
    Large Enterprise22%
    VISITORS READING REVIEWS
    Small Business29%
    Midsize Enterprise15%
    Large Enterprise56%
    Buyer's Guide
    SmartBear TestComplete vs. TestRail
    April 2023
    Find out what your peers are saying about SmartBear TestComplete vs. TestRail and other solutions. Updated: April 2023.
    767,847 professionals have used our research since 2012.

    SmartBear TestComplete is ranked 8th in Test Automation Tools with 70 reviews while TestRail is ranked 3rd in Test Management Tools with 21 reviews. SmartBear TestComplete is rated 7.6, while TestRail is rated 8.0. The top reviewer of SmartBear TestComplete writes "A stable product that needs to improve its integration capabilities with other test management tools". On the other hand, the top reviewer of TestRail writes "A tool that provides effective test management and real-time reporting capabilities". SmartBear TestComplete is most compared with Tricentis Tosca, Katalon Studio, Ranorex Studio, OpenText UFT One and froglogic Squish, whereas TestRail is most compared with Zephyr Enterprise, TFS, Tricentis qTest, Tricentis Tosca and IBM Rational Quality Manager. See our SmartBear TestComplete vs. TestRail report.

    We monitor all Test Automation Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.