What is our primary use case?
We use Sauce Labs for mostly our automated testing on cross-browser and emulator devices. Our team mainly focuses on web product testing so using a third party vendor to help with the external resouces is a must. There are many cases that we need to run our test on multiple browsers like Chrome, Firefox, IE, Edge etc. Sauce Labs has all the options for us. Of course, because our products are mostly web based, we need to ensure the cross-browser testing for every release cycle. Instead of doing it manually on many real devices, we use Sauce Labs since they provide many emulator devices.
How has it helped my organization?
Sauce Labs has been helping us a lot in reducing the number of manual testing. By moving most of our test to the cloud and not running on the EC2 instance, we not only save time for the service but also the maintenance for the external servers. Sauce Labs is the best out there. They beat their competitors because of its price, customer support, and an amazing forum where you can ask and get answered really quickly. They never stop expanding. Whenever we have a new update for any browser, devices, etc. they make sure it will be in their devices and browsers farm so users can use it immediately.
What is most valuable?
Live device testing. As we all know, It's really hard and challenging to find/purchase many real devices to test because it will be costly and not all the team can be able to purchase all of the devices out there. We used to have a lot of real devices under our labs. However, it is really time-consuming to maintain those devices and make sure they are up to date with the testing requirements. Sauce Labs knows that and already has a solution for their live device testing which allows us to test our app on the actual devices out there and see the real result.
What needs improvement?
As a web product QA team, we sometimes need to spot check some new child site on multiple browsers and OS(es). It was a little time consuming for us since we need to click on each of the browser/OS combinations and start a new session to test. Every sprint, with new features and child pages being added, we mostly need to do the same steps over and over again. We would like to see if Sauce Labs can be able to open multiple different browsers on the same website at the same time, just by 1 click. It will reduce a lot of time for manually click and change the devices/OS combination.
For how long have I used the solution?
What do I think about the stability of the solution?
We rarely had any downtime with Saucelabs, except sometimes our tests cannot connect through the tunnel. It's not always but once in a while and with some random tests only. Their support is really great, they always help us to diagnose and find the root cause of the problem.
What do I think about the scalability of the solution?
The performance is great, we never had any slowness or trouble with running our tests.
How are customer service and technical support?
Amazing! They are very quick in response and very throughout. They even go way beyond to help troubleshoot our code and give the coding example to help with all the questions that we had.
Which solution did I use previously and why did I switch?
We did use another third party vendor for cloud testing. However, their price increased a lot, and their support was a little slower than Sauce Labs. That's why we switched.
How was the initial setup?
It was very easy. They have a wiki and a GitHub repo with all of the codes for every programming language. We can either use their template page or inherit it to our use cases.
What about the implementation team?
We developed our test framework based on Sauce Labs in-house. There was no other vendor team.
What's my experience with pricing, setup cost, and licensing?
We have a team that manages all the licenses, and they share the pool with the rest of us. However, we haven't had any problem of running out of concurrent devices/connection because we agree to not running many test in parallel at the same time.
Which other solutions did I evaluate?
There are only two big companies in this field and since we were moving away from the other one, we obviously did not consider any other options.
What other advice do I have?
If you are looking for a cloud devices testing provider, look no further. They will beat their competitors on price, support, and a lot more.
Which deployment model are you using for this solution?
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)