When evaluating Test Automation Services, I believe that the most important aspect to look for is a proven track record of successful implementations and a focus on aligning automation efforts with the specific needs and goals of your organization. It's essential to choose a provider with a history of delivering a positive test automation ROI. Also, I recommend checking out the article at https://www.cleveroad.com/blog.... It provides valuable insights into understanding test automation ROI, offering practical guidance on how to calculate it effectively.
Search for a product comparison in Test Automation Services
Experience automating different application types and staffing the project to be a success. You also need to ask if your SMEs have the capacity to help with test strategy. You need to first see what type of test you are trying to automate and how the applications are architected. For instance is it a thin-client, thick-client, hosted through a non-persistent VDI, mainframe, does it use unfriendly HTML tags (i.e. canvas), SAP, etc. You need to see if the company providing testing as a service has knowledge in this area. You need to make sure they are doing a test tool evaluation before picking an automation framework. Often times this may require multiple automation frameworks based on application types.
At the end of the day you need to see to it that the project is staffed correctly. You need to realize that Automation isn't some "easy" button that allows you to realize a positive ROI within a year. From my experience for any automation project to be a success you need the following: automation experience, test strategy knowledge, and functional understanding of the application under test. Since we are talking testing as a service we can probably assume they have little knowledge of the application under test (unless they wrote it or it's a COTS product) which is your job to provide, without it low value test cases will be delivered and the entire effort and investment will be wasted.
When evaluating Test Automation Services, I believe that the most important aspect to look for is a proven track record of successful implementations and a focus on aligning automation efforts with the specific needs and goals of your organization. It's essential to choose a provider with a history of delivering a positive test automation ROI. Also, I recommend checking out the article at https://www.cleveroad.com/blog.... It provides valuable insights into understanding test automation ROI, offering practical guidance on how to calculate it effectively.
Experience automating different application types and staffing the project to be a success. You also need to ask if your SMEs have the capacity to help with test strategy. You need to first see what type of test you are trying to automate and how the applications are architected. For instance is it a thin-client, thick-client, hosted through a non-persistent VDI, mainframe, does it use unfriendly HTML tags (i.e. canvas), SAP, etc. You need to see if the company providing testing as a service has knowledge in this area. You need to make sure they are doing a test tool evaluation before picking an automation framework. Often times this may require multiple automation frameworks based on application types.
At the end of the day you need to see to it that the project is staffed correctly. You need to realize that Automation isn't some "easy" button that allows you to realize a positive ROI within a year. From my experience for any automation project to be a success you need the following: automation experience, test strategy knowledge, and functional understanding of the application under test. Since we are talking testing as a service we can probably assume they have little knowledge of the application under test (unless they wrote it or it's a COTS product) which is your job to provide, without it low value test cases will be delivered and the entire effort and investment will be wasted.