What is our primary use case?
My company uses Sauce Labs to run all smoke and regression tests for our application.
We primarily run our tests on Chrome 83, but we occasionally run on other versions (81 or 84) to check to compatibility. Our smoke suite runs on a nightly build, and regression on a bi-weekly basis.
For building out new automation features, we have to have a passing Sauce Lab run before creating a pull request. Doing this ensures that our new automation features will run on other environments and not just locally.
How has it helped my organization?
The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later.
The visual data is helpful for all levels within the organization from the QE engineer level up to the director level.
The ability to change the browser version has also ensured that our application stays compatible with previous and updated browser versions.
What is most valuable?
One major feature that I like about Sauce Labs is the recording feature. I love the option to watch a video playback and command and view log when I'm trying to figure out why my test was passing locally but may fail on Sauce Labs. It's helped to make sure of my automation.
Another feature I find valuable is the ability to choose browser versions. This feature allows us to test on our minimum browser version but also check/verify that our application is working on the most recent browser version as well.
The insights tab is also very valuable, as I’ve discussed early; it provides a great deal of data and determines trends in our automation sooner.
What needs improvement?
From a company use standpoint - I have no complaints. The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended. Personally speaking - I like to maintain my automation framework and would like to occasionally test that it continues to run on Sauce Labs after releases without stealing from Sauce Labs.
For how long have I used the solution?
I've been using Sauce Labs consistently for a year now since I've started with my company. After being introduced to Sauce Labs, I've incorporated it into my framework that I maintain.
Which solution did I use previously and why did I switch?
A previous solution I had used was running a VM from a NAS server to run our automation scripts.
What's my experience with pricing, setup cost, and licensing?
Be sure to plan out your solution before starting the free trial; 14-days goes by rather quickly.
What other advice do I have?
Sauce Labs was already integrated into our framework and SDLC process when I started with the company. After using it for a year now, I would highly recommend this to anyone.