What is our primary use case?
My company is quite large. My team supports the various different IT teams around the business who then work with their business partners. For example, if you think about our research and development division, our clinical trials division, and our manufacturing division, each one of those business groups has their own IT teams who build tools that the business team needs to do its jobs. My team provides the tools that the IT teams would need to build the tools for their teams. So, we are two or three layers removed from the broad side of the business.
We are the team who provides this solution to other groups. However, in terms of our usage, because the company has done things by paper signatures, official paper, and hand documentation for a long time, there has not been a whole lot of progress yet on the automated testing sector. Therefore, our usage is very small compared to other businesses of our size.
We have 25 concurrent VM licenses for Sauce Labs today. That provision amount has not changed in my time here because it has never exceeded that capacity. So, it is a slow, upward trend, but it is very slow right now.
How has it helped my organization?
In January of 2021, there were 515 automated tests that ran through Sauce Labs in the company. By October, we were just shy of 3,000 tests. That is a 600% increase.
In general, I know that there are different pockets of the business that build apps for different cases. For example, we have an iOS dev team in one part of the business. We have a Windows team in a different part of the business, and they are able to use the same platform without us having to try to support multiple different things. In that sense, the ability to test on additional browser, OS, and mobile device combinations has been a great help.
All our automated tests run in parallel. This helps us to demonstrate value to other parts of the business who have not yet adopted the solution. For example, when there are groups who use Sauce Labs and we try to share learnings, that team presents, and says, "I have saved this many specific hours. Or, I can run 225 tests (or however many tests) in parallel. It saves us this much time and this many dollars to have our application go out that much faster."
What is most valuable?
Because there is not super high usage for our team, the cross-browser functionality is a big feature for us. Our machines are mostly Windows. Being able to test with Safari, on a Mac, and other types of browser pieces without having to manage all the infrastructure is the biggest feature that our team enjoys.
Sauce Labs is excellent in terms of the number of browser/OS combinations, mobile emulators and simulators, and real mobile devices that it offers. I have not heard any complaints from my customer teams about any lack of selection or particular combination that they are going after that Sauce Labs doesn't already have.
Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. Their API for inserting test results is fairly robust. This was an important factor for us because our adoption is still kind of low. There are not a lot of teams who know how to use it right out the gate. Now that our adoption is slowly growing and teams are figuring out how to use it, this has proven to be a big win.
Sauce Labs provides access to automated functional testing. We have been very pleased with that. We make use of the automated piece and the manual piece. There are some teams who like to just poke around with a different browser to see how their app behaves by doing one-off tests.
For how long have I used the solution?
I have been with the company for three years. I have been supporting it all the time that I have been here. The company had the solution for roughly a year or so before I joined the team.
Buyer's Guide
Sauce Labs
April 2024
Learn what your peers think about Sauce Labs. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
769,789 professionals have used our research since 2012.
What do I think about the stability of the solution?
The stability is great. When I first took over, I was trying to find any general maintenance Windows, etc. so our team could be aware. I remember posting the first few outage notices internally to our customers, saying, "Sauce Lab is going to be down for such and such." Probably because of our lack of adoption, any kind of instability concerns have been really insulated for us.
Latency has not been a concern. Once the pipeline gets set up, latency doesn't make too much of a difference because our development teams are attending their meetings or performing other work while the pipeline runs. Therefore, they are not overly concerned about the speed of what Sauce Labs can provide in terms of latency. At the same time, because the company is so archaic by modern standards with our automated testing footprint, any time savings over somebody going through and clicking the buttons by hand in an application is going to be greatly realized.
What do I think about the scalability of the solution?
So far, scalability has been great for what we need. It is hard to tell its true scalability since we are only running 25 VMs concurrently.
There is definitely room for growth. Across all our separate divisions, people are realizing that automated testing is a major opportunity for improvement. As these individual groups continue to learn from each other and ramp up their own use of automated testing in solutions like Sauce Labs, that will really drive our growth.
How are customer service and support?
We have made use of Sauce Labs’ technical expertise to help us integrate automated testing into our CI/CD pipeline and DevOps toolchain. We have done that to great success with their support teams. I have also had the Sauce Labs team out onsite to the company at one point before the COVID pandemic hit. So, we were able to bring in several people from the company who were interested in seeing how to shape some of it for themselves. Sauce Labs was able to present and provide direct guidance to those teams, and it was a really successful event.
The technical help wasn't overly helpful for me because I am just a really technical guy. I try to figure all that stuff out for myself. However, it was extremely helpful for our team at the company. A lot of the time, most of them don't like to try to figure things out for themselves. They want a dedicated instruction set, rule book, or whatever to say, "Here is how you go and do this." For Sauce Labs to come in and be able to actually show how it was done, I think that provided a lot of value for them.
The technical support team has been great for any kind of issue that my team can't resolve on our own. We were always comfortable opening tickets with the support team. They have always been super responsive and educational when helping us to understand the cause behind the problem, not just how to fix it. I would rate them as nine out of 10 because there have been one or two occasions where it hasn't been absolutely perfect, but it has been really stellar overall.
Sometimes, my customers find a way to go to Sauce Lab support directly, and either they are unable to communicate to Sauce Labs support what their issue is or Sauce Labs is not able to understand them. It is one of the two. Usually, my team plays that middleman and we can facilitate pretty well, providing context to any potential problems or issues within the company as well. For example, if there are specific company systems or things interacting with Sauce Labs that our customer teams may not know about.
On a rare occasion, I will come into a ticket where a customer will have reached out to me after reaching out to Sauce Labs, saying, "Sauce Labs doesn't understand what I am going through. They are not being very helpful." So, I try to do clean up there. Outside of those extremely rare occasions, I have only had one or two of those support issues.
Sauce Labs doesn't know what your test results are. This is a super application specific thing, but it is something that the company struggles with understanding. The app is just the endpoint for the execution of your commands. It can't understand whether the test that you are trying to execute is passing or failing. So, you need to be sure to take that extra step and let Sauce Labs know the results of the test if you want to allow them to assist you in debugging what may have been wrong.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
It provides access to automated functional testing, visual regression testing, and front-end performance, as well as browser/OS combinations, mobile emulators and simulators, and real mobile devices, making it an all-in-one testing suite, which is extremely important to us. This was the primary driver for why we went with Sauce Labs before I joined the company. Having this around is definitely huge.
How was the initial setup?
The solution was adopted before I joined the company.
What was our ROI?
In general, our test execution speed, time to results, and time to deployment have dropped significantly (by approximately 10%), which is huge.
What other advice do I have?
For the teams who have it set up, Sauce Labs runs with every code commit and provides our developers with immediate feedback. We don't have that as an organization-wide thing right now because we are trying to increase our adoption and execution across the userbase. We are trying to champion this and educate individual teams on how to use it, but they have to develop their own process first. Also, we have regulations in place that prevent us from moving as fast as other companies would like us to.
Going forward, as we continue to adopt not only the function of automated testing, but also the methodologies, best practices, etc., having all their features will be really powerful.
I would rate the solution as nine out of 10. This is mostly due to our company's lack of engagement and being able to fully understand the product and usability for us.
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.