If you were talking to someone whose organization is considering Tricentis Tosca, what would you say?
How would you rate it and why? Any other tips or advice?
We use the on-premises deployment model. My advice to others is to get training by a professional. This will help anyone use the product. Tosca allows users, even if they don't have some IT knowledge, to quickly start and use Tosca after about two weeks. You can take any business person, and just offer them good training. After about two weeks or a month, they could be independent and will be able to automate some complex scenarios. You don't have to be an IT specialist to handle the product. I'd rate the solution eight out of ten.
Before looking at a tool, it's not just about implementing the tool. It's about having the right mindset to be able to implement a tool. With Tosca, we didn't go and say, "We want to use Tosca. Go." We have a whole training where we talk about what needs to shift, how you need to shift, why we are doing it, and making people understand the intent behind using some of these tools. Otherwise it does not work. We have seen enough failures with other implementations to know that if we don't have the right mindset, these things don't work. Also, having the right partnership and setting expectations with the vendor is very important. If they have good best-practices and if they know what they're doing, that will help. Having a long-term vision is very important, instead of just saying, "Okay, I'm going to implement Tosca. Go," and not knowing where you want to go or what you want the future to look like. Those are things that helped us a lot in our implementations of Tosca. One lesson, as I said, is that working with the tool vendor helps. Not that Accenture didn't do a great job. They did. They helped us a lot, especially from a process and best-practices perspective. But having Tricentis in there would have been good. Tosca was relatively new in the US when we started using it. The roadmap was not fully baked. I would love to work with them on their cloud roadmap. I still don't have a great answer when it comes to cloud roadmap for Tosca, from Tricentis. That's something we're still pushing them on. Tricentis also has a great relationship with Salesforce and that has helped us a lot. We're trying to push them to have a similar kind of relationship with Guidewire, with Guidewire 10 coming and the cloud option. That's another thing which we will be using Tosca for. We have been asking Tricentis to have some kind of partnership with Guidewire, and they have been asking us if we can introduce them. We're working through that. We have a user group for that with other Tosca users that are in Guidewire and who are moving to Guidewire 10. It would be great if Tricentis could take a look ahead at some of these big vendors, packaged applications, and form some relationships with them. That would help companies like us who are heavy users of these packaged applications. We would like to see something similar to what they have done with SAP and with Salesforce. Guidewire is big right now. That's an area they have not done anything in yet. We were an IBM shop. We used to use Rational Team Concert and Rational Requirements Composer. We're shifting to JIRA for requirements. We're still looking at what we will use for persistent requirements, but right now it's JIRA. Even if users want to use Tosca for requirements, and there are teams who are doing it because it helps us get good coverage, we still use JIRA and then we try to integrate from JIRA to Tosca, but we have not done that yet. Tosca has a risk-based functionality, which we are not using, which they added after we started with Tosca. It can take a look at the data and say, "Hey, you have 200 test cases and you are getting 50 percent coverage based on the requirements that you have in Tosca. You can get the same coverage using 100 test cases." The reason we didn't use the Tosca's risk-based testing is that risk-based testing is applicable irrespective of the tool. Since Tosca is not something that is the only standard at our company — we have multiple tools — we had to choose a tool that we could use across the enterprise and not be dependent on the Tosca licenses. So we use another tool for risk-based testing. When you talk about redundancy, there are two aspects. One is: We have been using Tosca for three years; we have 2,000 scripts. Even though Tosca has all the functionality by being model-based with reusability, not everyone understands that. In these past three years, new people might have come in and added test scripts and test cases without knowing that something already existed. They may have created duplication and redundancy. Can Tosca go and help us with that? I don't think so. We are actually looking for a tool that can help us do that. We were looking at Saffron AI Suite, which is an Intel product, but Intel decided not to support it. We're still looking for a tool that can help us with duplication. But what we are doing is from a data perspective is the following: If I say, "Hey, here is my data, here is my test case, here are the data elements that are needed. Tell me what is the minimum number of scripts I need to get maximum coverage?" Tosca can do that. But as I said, we didn't want to depend on Tosca because we're not using Tosca across the enterprise. We're using another tool called Hexawise to help us do that and it's something that we're implementing across the company. It is much more cost-effective for us than having Tosca licenses for everyone. Tosca is expensive. We're trying to use the output from Hexawise to create test cases in Tosca to help us get that minimum number of test cases we need to get the maximum coverage. That coupling has been working well for us. Maintenance is better with Tosca. The way Tosca is structured, it tells you where your tests are failing and the like. If I have nine quality engineers for an application and they are spread across three build teams, if they have the continuous integration implemented, whatever issues or errors come up, we expect them to keep maintaining things on a go-forward basis. Maintenance is absolutely easier with Tosca, provided it has been implemented the right way. We're not differentiating between people who are doing build work versus people who are doing maintenance. It's the same people. We expect them to build the test cases and maintain them as well. I would rate Tosca at about eight out of ten. We're pretty happy with the results we've seen with Tosca.
If you're going to have somebody come in and help set up, don't go with a third-party vendor, but have Tricentis consultants come in, do training, and help set up. It's worth the extra money to have them come in and do right, versus a third-party that might not know everything. That was our experience. We tried the third-party route to help us, and we immediately saw that it was not the right way to go. That's when we went to Tricentis themselves to come in and help us. The biggest lesson I've learned is more connected to the tool's adoption. If people have been doing the same thing for over ten years, it tends to be a little bit hard for them to switch over because they want to do things the way they've always been doing them. But the tool itself is fairly simple. It's a pretty solid tool. There are a few ways to overcome the resistance to new technology. But it's really about creating urgency around why the tool is important to the company and why people need to adopt it. We've done lunch-and-learns to help people understand it. We also champion any success stories with the tool, through newsletters that go out. Sometimes, it's just about bringing in new people with the right mindset. We've slowly been increasing our rate of testing automation using Tosca. We did a digital transformation with SAFe Agile, and then with adopting test automation. So the tool-adoption piece has taken a little bit longer. We're not quite where we want to be yet. We're still building on it. Tosca covers about 50 percent of our test cases at this point. The solution hasn't reduced our cost of testing yet. We're still a little immature in this process. In terms of delivering more features for release, we are watching features but we're not there yet. We're just not mature enough to have those hard facts to help us state that this is helping complete more features. From a defects standpoint, we're trying to get a better handle on how defects are captured here, because before there wasn't a concise, approved process on defects. Everybody did their own thing and I'm pretty sure things fell through the cracks. People weren't identifying defects as defects. That's another one of the things we're trying to get better at. Tricentis keeps building on BI and Tosca itself, and they're just getting better and better every time. A lot of the things they're focusing on are the right things.
For organizations who want to aim for a unified continuous testing platform, and are willing to pay for it, Tricentis Tosca can be a decent investment.
Figure out your requirements and environment. We had the impression Tosca would be the perfect fit for us. We did a proof of concept test with all our applications, Tosca was able to work with them and drive the proof of concept goal. Finally, we made a decision a year ago to work with Tricentis Tosca. We are quite happy with the product.
My most important criteria when selecting a vendor are its technology and is it stable enough.
Take the online training on Udemy before you confirm the intent to buy it.