Tricentis Tosca Review

Very rich automation/test management framework but consider the coding needed to integrate with your application


Recently, I was asked a set of questions regarding my experience with TOSCA. Here are the questions and my answers:


General info:

Is there any need for coding or scripting? Based on my research, I expect that the module is created when scanning has been finished. But some users commented that coding or scripting is needed.

- Depends on the application and technology you are trying to automate. TOSCA has a lot of engines (HTML, .NET, Java, Webservices, etc.) with control library's that can be used out of the box. For something that is not covered by these engines (like a custom java widget) you will need to code for steering.

Are control statements like if/then/else and loops supported?

- Loops are not directly supported. If/then/else are supported for templates (data driven test cases).

Test case organization:

Is it possible to group test steps and according to our needs include/exclude particular test groups?

- Yes

Is it possible to rearrange test steps once they are created?

- Yes

Is it possible to perform multiple execution for test steps and test groups?

- Yes

Is there a possibility to create some variables whose values can be adjusted in some of the test steps?

- Yes. In TOSCA variables are called "buffers"

Is possible to add references in a test to some other test?

- It's possible to organize test steps in the library and reuse them in TC's. For other kinds of references you are free to create custom fields (to store path to other TC for example).

Is it possible to modify verdicts, for example, if a test fails that is declared as passed?

- Yes

Is a test progress view supported? Is it possible to stop/start a test on an error or some other event?

- Yes. You can define different behavior on an error (for example requesting a manual action to continue).

Is it possible to schedule tests? This is useful to have when night runs are executed

- You can trigger TOSCA execution using the TOSCA API from cmd line using windows scheduled jobs.

When some test step fails in some test group, how is that reflected to the corresponding test group? Does the parent test group also fail?

- The parent test groups are organized as "Business execution lists". If one TC fails, Business execution list will also fail.

Logging:

Is it possible to filter results?

- Yes

How is logged data is stored?

- Execution results stored in a TOSCA repository (local or shared (DB))

Is it possible to export logs/results?

- Yes. You can print repository content to various formats (excel, PDF, jpg, etc.)

Other:

How does TOSCA work with the command line?

- The TOSCA API offers all operations in the cmd as the TOSCA Commander shell. Run TC's, Create TC's, print report, everything...

Is it possible to configure test speed? For example, to insert some delay when GUI is slow.

- Yes, there are multiple settings and options to play with the speed.

Is there any possibility to explore a linux system with TOSCA? Maybe via putty or WinSCP?

- I don't have experience with TOSCA and LInux. Theoretically TOSCA can run external applications. We have created integration of TOSCA and JUnit tests for our needs.

When a test is created on one system, can it be easily transferred to another system? For example, when I create a test, I want to deliver it to another colleague who is working on another machine. Or does he have to create it on his own machine?

- You can use multi-user repository, which has checkin/checkout operations like SVN or CVS. You can also export your TC's or other TOSCA objects to file and import to other local/shared repository.

In the end, what is your opinion/experience about TOSCA?

- Overall impression about TOSCA is quite positive - it's a very rich automation/test management framework. In the right hands it can be used for creation of a really powerful and complex solution. There are many more features that are useful for automation but there can really be a need for some coding effort to integrate with your application - this needs to be considered.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
12 visitors found this review helpful
4 Comments
UserUser

Agreed with some of your points. We also use TOSCA extensively for large programs. It is needs to get mature. There is a lot of need of coding. its not simple scanning. A little bit hyped.

The worst part I feel is the customer service from tosca. We have been facing an issue of TOSCA JIRA integration since last 4 months but its not yet resolved. Nor they are ready to raise the priority/severity of ticket.

21 May 15
Oliver AllabauerVendor

Hi Vivek,

I am sorry to hear that the experience with our team was not matching your expectations in this case. I understand that it can be frustating from your point of view having to fight for higher prioritization due to felt impact of an issue on your side. Though our team needs to ensure that the call prioritization matches the corresponding contractual definitions. We aim to reach a mutual agreement where applicable, unfortunately this is not always possible.

As I was going through the issue related incidents it seems that the "out of the box" JIRA integration Tricentis provides is not capable of handling the exact multi-project aproach that you want to go for on the bugtracking side. This aproach is of course doable, but requires some customer-specific adaption related to the JIRA integration component, unfortunately this is something that the Customer Service Team is not able to directly provide you with.

I would really like to get more first hand information on your felt pain points with the services and tools Tricentis provides, so I just offered you some direct contact options for in depth discussion with me. I did this via opened call in our incident management system. I would be happy to hear from you on that.

Best regards
Oliver

27 May 15
User with 201-500 employeesVendor

Great review!!!

09 July 15
Senior Manager of QA at a tech services company with 501-1,000 employeesReal UserPOPULAR

My review was done for the older version of TOSCA.
Currently we work with version 8.4.0 and this tool has advanced a lot.
Tricentis added many useful features and TOSCA has become once of most mature automation tools I worked with.
If/then/else and loops supported, plugin for Continuous Integration exists now, JUnit test results can be imported into TOSCA, test data service allows to setup distributed execution, etc.

03 September 15
Guest
Sign Up with Email