Tricentis Tosca Benefits

AVP, Testing Service Owner at a insurance company with 10,001+ employees
We have five large business units: financial, property and casualty, enterprise applications, marketing, and emerging business. And then we have a shared-services organization. Across this, we have more than 200 Agile teams that do build work. A lot of the time, these agile teams focus on developing and testing the work that has been handed to them. When Tosca came in, one of the things we started thinking about was how Tosca could help us facilitate some end-to-end testing. When I say end-to-end, that doesn't mean in one particular business solution area or in one particular department, but rather, how do we go across departments? If we have to create a retirement plan, the work is not just in the retirement area. It has to flow from a lot of different applications and different business units, and that facilitates the end-to-end. The way we are organized right now is by department. Initially, our scope was to see if we could do end-to-end, but we reduced the scope because that would have meant that many people had to be ready at the same time to consume the work. So what we said is that if we can use Tosca to do end-to-end for an application, then we can use orchestrator tools like Jenkins or Concourse to create an end-to-end flow from a business perspective. To give you some numbers, we will be harvesting a saving of almost $1 million, in the first area that we tried this. We had 36 manual testers and we were able to go down to 14 quality engineers, so we are seeing some savings. That was the biggest. In other areas, we are seeing savings from reducing by three or four people. In terms of cases covered by testing automation with Tosca, it's very difficult to put a number on that. Where Tosca has really made a difference is where we had manual testing only and the percentage was zero. In the area that I just mentioned, where we went from 36 to 14 testers, they were at zero percent automation and they're already at 40 percent. The goal is to be 80 percent by mid next year. Out of our 200 teams, we have not finished all the waves yet for Tosca. Around 50 percent of them are already using it, if not more. In January of 2018, the number of associates we had doing manual testing — I'm not even talking about contractors — was around 370. Our projection is that by the end of next year, we are hoping to go down to 230 associates. That's about a 37 percent decrease in the test-analyst workforce. Most of that is going to be enabled via automation using Tosca. We have seen lead-time change impacts as well. We are seeing some defect numbers dropping down and we are still operationalizing. The biggest is Speedplay, where we were doing testing manually. Now, with automation, we are able to execute some of those regression tests sooner. Tosca enables us to run the entire regression test suite immaterial of where changes have been made. We now have much more confidence in the areas where we've implemented it. We have a high level of confidence in the changes that we are making because we know we have a regression suite that we can run. We have seen workflow improve. Before, we wouldn't have thought off coordinating among different applications. For example, application A does their testing and tells application B, "Here is what we need." Application B goes into their cycle and they do their work and say, "Okay, we're ready." That might have taken a day, two days, five days, or a week. Now, we have examples where we have been able to directly call the database for application B and retrieve the information. What used to take six days, we are able to do in six hours. One more example where Tosca helped a lot recently was when there was an issue where our retirement plan holders were seeing incorrect information on their PDF statements. There were 32,000 PDF statements that needed to be validated after the fix was done and it would have taken 60,000 hours for one person. So we had a couple of our folks create scripts. It took them two days and they executed all the 32,000 PDF validations in one day. Tosca didn't only help us in terms of time, because it was more an issue of cost-avoidance, but it helped us gain the trust of business because business signed off on it. Business was the one which had said, "We need to validate each and every PDF manually," and that's where that estimate had come from. We are seeing a lot of these success stories. We expect, once we have the full implementation done next year, that we'll see many more examples. Overall, by next year, we will be looking at a total reduction in testing costs of about $14 million, across the span of three years. That includes both build and run. Build is very difficult for us to harvest because if we reduce the money in build — if we take away two people — the demand grows and they add two developers. But overall, from a build and run perspective, we are looking at forecast savings of $14 million. This year alone, we have proved that we have reduced the overall spend by $3.5 million. Another big use case which where we have been helped a lot is through Tosca BI, which helps with large-volume validation from a data perspective. For a lot of our data lines, we have an automation framework, but it doesn't do as well when it comes to comparing huge volumes of data from source to target. That's a place where Tosca BI has helped us because it can do those large file and data comparisons in a very short time. View full review »
Global QA Manager at a financial services firm with 1,001-5,000 employees
We're currently going through an upgrade of a system here, in-house, and Tosca has allowed us to do more testing compared to what we were previously able to do. In the past, it would just take too much time for humans to do. This additional testing that we're doing, which provides additional test coverage for the upgrade, would take a couple of individuals three weeks to do, and we're able to do it within 24 hours with Tosca. Also, the solution enables us to run the entire regression test suite, immaterial of where a change has been made. We're able to run all our regression tests at one time. We do not have CI/CD set up yet, so it still has to be manually kicked off. For example, for the in-house upgrade I mentioned, most of what they're building is regression test cases because we want to make our upgrades faster and we want to do them more frequently. So as the users are building out the test cases, they're also creating the regression execution list. They can build upon that and then run that when they need to. And then all they do is do the analysis of the execution. Tosca is also slowly starting to remove redundant test cases. We're working with the users to build better test cases and remove those redundancies as well. This really helps because we have the central repository which allows us to share everybody's test cases and to reuse the test cases globally. We have delivery teams in Edinburgh and in the Netherlands as well as here in the US. It really allows for that single collaboration and for reduction and reuse of test cases. We're still analyzing how this has affected our testing efficiency. That's one of the reasons we're upgrading to 12.3. We want the Tricentis Analytics to help paint that picture a little bit better for us. We are starting to see the test speed increase a little bit. Looking, again, at that in-house upgrade's tests, if we had manual testers doing it, it would take a couple of weeks, and now we can execute test cases in 24 hours. Another team just completed some test automation test cases that take two minutes to run. When they first ran them they ran into errors and issues which were legitimate issues. The developers fixed them and they ran the tests again and found some more errors. They went back to developers. The whole time it took to resolve things was about 20 minutes. That is a huge improvement over how things worked before. It would have taken hours if it was manually tested, verified, etc. Finally, we have BAs who have been trained on Tosca, people who have picked it up fairly quickly with little background in development or coding, and they've been able to get up and running on it fairly quickly. We would like to get to the business to be able to help control testing as well, but we're not there yet. We are using exploratory testing, but we're not 100 percent using that quite yet. View full review »
Continuous Delivery Lead at a tech consulting company with 1,001-5,000 employees
- Brought a good environment of test automation - Allowed a significant increase in Continuous Integration View full review »
Find out what your peers are saying about Tricentis, Micro Focus, SeleniumHQ and others in Functional Testing Tools. Updated: February 2020.
397,983 professionals have used our research since 2012.
Manish Kum@r
Senior Technical Automation Specialist at a consultancy with 51-200 employees
Tricentis Tosca has evolved from being a functional test automation tool to becoming a Continuous Testing platform. It has helped teams within our organization become more aware of the testing requirements in terms of risk and priority. It has also enabled teams to optimize risk coverage and enabled them to test more wisely. View full review »
Robert Foerg
Project Manager at a logistics company with 5,001-10,000 employees
We build the test cases, then we just need run it by one person to get feedback right away. Otherwise, it would require planning with certain testers. View full review »
Ulrich Becker
QA at a tech services company with 11-50 employees
Compared to the old tools, WinRunner and TestDirector by Mercury Interactive, it accelerated building, performing, and maintaining regression tests by a factor of approximately ten times. View full review »
Vaishali Mane
Practice head for Automation at a tech services company with 10,001+ employees
Helps decrease execution time. View full review »
Daniel Kämena
Consultant im Testingbereich
It did not improve because the GUI was not static, so it was not possible to write tests for a GUI that was in progress. View full review »
Find out what your peers are saying about Tricentis, Micro Focus, SeleniumHQ and others in Functional Testing Tools. Updated: February 2020.
397,983 professionals have used our research since 2012.