What is our primary use case?
We use SauceLabs for Mobile, Browser and Backend testing.
Our QA requirement is to test our apps against all major browser platforms including Safari, Firefox, Chrome and Internet Explorer.
Apart of visual and functional testing of the application components, we also collect network traffic produced by the apps for further analysis. With latest update, accessing har files got supported natively by the SauceLabs.
For mobile testing we leverage Sauce Connect tunnels as we need to connect the application under test to the corporate network and collect the produced traffic as well.
How has it helped my organization?
1. Achieved better cross platform test coverage for both Browser and Mobile testing
2. Frontend and Backend testing are now more connected
3. With Sauce Analytics we now use resources in more optimal way and too extensive cloud resources usage is no longer a concern
4. We no longer need to support a huge farm of mobile devices - SauceLabs Real Device Cloud works pretty well for us.
5. In some cases, like peak real device usage, SauceLabs Real Device Cloud doesn't work as well due to the delay required to release devices after testing sessions end
What is most valuable?
1. Easy scaling to multiple web platforms -> SauceLabs was the cheapest option for us
2. Support of tunneling traffic from the cloud to the enterprise network -> that is just essential for our testing
3. Video recording of tests execution process -> shortened root cause analysis time for use by 30%
4. User groups management and cloud resources consumption control -> now each group of users under our organization has dedicated pool of devices/emulators and concurrent resources access is no longer an issue
What needs improvement?
1. They still have very slow support and ask many excessive questions before starting to fix the issue.
When we submit tickets in their reporting system, we expect them to start tracking down the problem as soon as possible. Instead, it looks like they ask standard questions just to delay work to a few days. E.g.
1st day: "Could you confirm you use the latest version" - "Yes, we do"
2nd day: "Ok. Can you then try to run that with extra flag Y and grab the log for us" - "Yes, please, take the log"
<2 days of silence>
5th day: ... - "Guys, is there any progress on that?"
and so on...
Sometimes questions are really unrelated and take time like "Have you read the Getting Started guide?"
2. They still have platform-specific issues, especially on Mobile platforms. They wouldn't affect trivial tests though.
E.g. they have issues related to SauceConnect traffic routing specific to iOS 9 and Android 4.
This affect tests that want to sniff the traffic only though. Please, let me know if you want more technical details.
3. Sometimes they change configuration options (e.g. rename...
E.g. recently they changed desired capability for iPhone Simulator in a way like "iPhone 6 simulator" > "iPhone 6"
That affected multiple tests suddenly. They haven't made any notification about that though.
For how long have I used the solution?
I believe, for more than 7 years already