Tricentis Tosca Review

It's an enterprise solution that approaches automated software testing from a business perspective, although it doesn't allow distribution of test cases to different machines for parallel execution.


Valuable Features:

For me there are three outstanding features:

The first one is the test case design feature. When using the feature in your project - I definitely recommend using it – it forces you to think about your test approach, and your overall test strategy. Using this feature you have to define valid and invalid states and conditions of your SUT from a business perspective (e.g. defining a valid age or a valid combination of two business properties). The tool helps you here by generating all necessary combinations automatically out of your prior definitions. You link them to a test case template and all the necessary combinations of your test cases are generated automatically. In most cases you need to adapt the generated test cases for the specific test case, but the fundamental work is done by the tool.

The second one is the simplicity of creating test cases and the underlying model based approach. The process of creating a test case is pretty straightforward. In case of a GUI application you “scan” the current screen of the SUT first. Afterwards all UI controls are displayed and you can choose the relevant one for your test. A representation of the screen – a module – is created and stored in a module folder. You can then simply drag and drop this module into your test case. In the test case, you select, for each UI control, an action e.g. “click” or a specific value. Afterwards, you can arrange the test cases in different execution lists, which are the representations of a test run from a business perspective (e.g. smoke test, regression test, etc.).

Overall, everything is reusable simply via drag and drop, and you will always have a single point of maintenance (e.g. in a future release of your SUT a button was added, you just need adapt the module. Linked test cases and execution lists are updated automatically).

The third one is the support of multiple UI technologies as well as non-UI technologies. In addition, there are (extra payable) features like mobile testing support and service virtualization capabilities.

Improvements to My Organization:

The tool is meant to be an enterprise solution for automating software tests. The vendor has its own test philosophy and wants you to follow it. It tries to hide all technical information of the SUT and forces you to always look at your SUT from a business perspective.

The target audience are business domain experts and software testers without technical knowledge. This can be confusing for testers with a technical background who are used to scripting their way through the SUT.

Introducing the Tricentis Tosca Testsuite in a company goes ideally along with changing the whole business process of software test. I would say the company needs to adapt to the tool, and not the other way round, which is not always bad.

Room for Improvement:

Though the tool claims to be an enterprise solution, it lacks some enterprise features. I would like to see a central management capability for admins, where you define initial workspaces, parametrize settings or update all clients centrally. Furthermore, the tool needs a real versioning functionality.

The out-of-the-box support for third party ALM tools must be improved. Embedding it in an existing application landscape often goes along with developing an adapter for third party components. In terms of extending the product from a technical viewpoint, the documentation for APIs is rare and incomplete. It definitely needs more code examples and a platform for developers.

The tool always gives the user one specific view where you create your test cases. This works perfectly for GUI applications, but not always for non-GUI applications. For instance, when defining test cases for web services, you have exactly the same layout as creating a test case for a Java application. Technology specific characteristics are somehow mapped into the existing view, which increases the complexity. Furthermore, the abstraction at non-GUI tests is not given as in GUI tests. You really need to have the technical understanding of web services or XML to create test cases, this runs counter to the target audience.

The test case design functionality is really complex, and tends to be rocket science, so this needs to be simplified. It goes along with the reporting functionality, when you want to create your own layout and customize the report. I didn’t really get it to work.

It would be nice to distribute test cases to different machines for parallel execution. Especially when it comes to mobile test or multi-browser test where the test case keeps the same but the underlying technical base changes.

Use of Solution:

I ‘ve been working with the Tricentis Tosca Testsuite for more than four years in the context of software test consultancy. I have a decent overview how the tool is used in different companies over several projects. I have gained experience in all functional areas of the tool. Furthermore, I have developed a lot of technical extensions to enhance the tools capabilities.

Customer Service:

Overall I would rate the customer service as excellent. The support is available via email and phone 24/5 and is located in the headquarters of Tricentis. This means that the support staff is sitting next to the technical experts which, nowadays, is rare. You get a quick response that is often the solution for your problem. In case it is a more difficult inquiry, and is not solvable with first level support, you’ll be informed that your inquiry will be passed to an expert. Depending on the complexity, you can count on having a solution within one or two working days, which is pretty fair.

Furthermore the support team offers remote sessions, together with technical experts, to find a solution directly on your machine. The support is available in German and English.

On the downside, the community outside the Tosca support portal seems to be pretty small, or is nearly nonexistent. You hardly find solutions, or best practices, for a specific problem on the web. Other vendors have a more active community, and you can often simply Google a solution to the problem. In most cases with Tricentis, you need the support team or you have to contract on-site consultancy.

Initial Setup:

It depends on your environment and your company policy. Basically, you have three components: a license server, a database and the desktop application which needs to be installed on the same machine where the SUTis located. If you want to use the mobile and/or service virtualization feature you need additional components and preparations.

Cost and Licensing Advice:

You will not find an official pricing list. You have to contact the local sales team.

Other Advice:

The tool is a desktop application and comes with all advantages and disadvantages of a desktop application in terms of deployment and scalability.

There is a cloud version (TaaS), but this version is only suitable for manual tests and web tests. Web applications need to be publicly accessible within the cloud environment (so it is not suitable for internal testing environments). It would be a nice feature if the cloud version of Tricentis Tosca Testsuite is able to drive your locally installed application.

Do not buy this tool if you simply need a driver for your automation, as open source solutions (e.g. Selenium), or technology specific solutions are more suitable here. Also, if you are looking for a complete ALM solution, there are more capable tools out there. If you have a testing department full of techies, they will feel too restricted with the tool. It’s also not suitable for manual testing only, as there are better solutions for this purpose out there. But, if you have the plan to test multiple technologies, and your testing departments consists mainly of business domain experts or software tester with few technical background, and you are planning to introduce this tool as an enterprise solution and/or want to replace existing fragmented solutions, then it is worth evaluating it.

You definitely want to have a product training first prior to implementation, and you definitely want to have on-site consultancy while making your first steps with the tool. In most cases, you also need technical consultancy in cases the tool does not recognize UI controls.
Focus on a small prototype project first together with on-site consultancy. Take the learnings from that project, and do the next one on your own.

The company is a lot smaller than the top dogs, but they are compensating it with passion and a lot of new features in every release.

Disclosure: My company has a business relationship with this vendor other than being a customer: I wrote this review in Sept. 2015 before I began working at Tricentis.
2 visitors found this review helpful
3 Comments
Stefan SteinbauerVendor

Hey Ronny, Have you had a look at the distributed execution in 9.0? You will like it! http://www.tricentis.com/tricentis-tosca-testsuite/what-is-new/9-0/

16 November 15
HenryReal UserTOP REVIEWERTOP 5

Really cool review

21 November 15
Product Owner at TricentisVendor

Hi Stefan! Thanks for the hint. This is really a cool feature, which was on my wishlist. Great job!

16 December 15
Guest

Sign Up with Email