2018-07-22T09:27:00Z

What's the best way to trial functional testing tools?


We all know that it's important to conduct a trial and/or proof of concept as part of the buying process. 

Do you have any advice for the community about the best way to conduct a trial or PoC? How do you conduct a trial effectively? 

What mistakes should be avoided?

Guest
55 Answers

author avatar
ExpertTop 5Consultant

Based on my 10 plus years of experience in automated functional testing, my advice is the following.
1) First, determine what the most important need is for acquiring an automated testing tool.
2) Write out your company’s core systems, applications, and processes that are in use that you believe have a beneficial need for an automated functional testing tool. Based on this, research which functional testing tools are compatible and supported for the key systems and applications that the testing tool will be executing on. For example, some popular open source functional testing tools will not work with .NET or SAP, so you may have to go with an Enterprise functional tool. Conversely, if you have a very small company and all the applications are Web-based, you might want to choose an open source tool.
3) Determine if you plan on having a current employee be the primary person developing automated scripts or if you will need to bring in an outside resource. With this in mind, factor in the labor costs.
4) If your company is going to purchase an Enterprise Software tool, initially have the vendor come onsite and give a live demo. Have your subject matter experts and stakeholders present and be sure to ask questions about any limitations the vendor’s testing tool might have with the company’s systems and applications.
5) Very Important** If you believe that your decision makers are leaning towards purchasing a certain testing tool then also do the following. Prepare an actual written test case with requirements that covers the critical functionality of one of your company’s applications. Have the vendor agree to send one of its test automation engineers onsite for a specified day. Next, allocate a computer for the vendor’s test automation engineer to use and have them install the vendor’s software on the computer. Next, give the vendor’s test automation engineer the written functional test case, requirements, and test data. Ask the automation engineer to develop a working automated script based on the documentation you gave them against the (AUT) Application Under Test. Monitor how long it took for them to develop the automated script.
6) Finally, set-up a date within 1-2 days for the vendor’s test automation engineer to give a POC (Proof of Concept) live demonstration of the script they developed, including executing the script and displaying the tests results to all the key stakeholders. Have everyone present take notes.
7) The next day or so, have a meeting with everyone present at the POC demo discuss both the Pros and Cons of the testing tool. After careful deliberation, then decide to go with the vendor’s functional testing tool or explore other options.

2020-03-23T22:17:01Z
author avatar
Vendor

Some tips in conducting PoC.

1. Create a decision matrix. Try Pugh Selection matrix. Here, we identify specific criteria in selecting our tool (Ex. support for a specific technology, market acceptance of the tool, skills needed to setup and maintain the tool, etc). Then rate the tools for each criterium. We will then consider the tools with the highest points, and remove the tool with the lowest points from our selection list.

2. Compare tools using Strategic Positioning Matrix.
This is a good way in having a general comparison of the different tools. In this matrix, we identify just two variables(ex. Price and Ease of Use). We use these variables as the X and Y axis. We then plot the tools according to our rating. Using this, we will get an idea of the strength and weaknesses of the tools. For example, we will see that high priced tools are easier to use while cheaper tools are harder to use. Though for Selenium, it's easy to use and free. Reference: https://goo.gl/d3Krz2. We can use as many matrices as we like, but each matrix will only need two variables at a time.

*When I do the PoC, I present both the Decision Matrix and the Strategic Positioning Matrix to the client, and will get their concurrence and approval in selecting the tool. After the presentation, I get to know which specific tool is desired.

3. Finally, once we know what we need, we can try out 1-2 tools, or more, depending on the deadline and our available resources. We will use the free trial version offered by most tools which last for only a month. If the performance proved satisfactory, then we proceed with acquisition and full implementation of pilot runs.

2018-08-20T15:07:25Z
author avatar
Vendor

In the fast-paced agile and DevOps world Test Automation tools play a major role. The tools you select should enable Continuous Testing with all the essential components right from day one.

Please read through the below links for more details on how to select the right tools for Continuous testing in Agile and DevOps.

https://testsigma.com/blog/test-automation-tool-ready-for-shift-left/
https://testsigma.com/blog/why-unified-automated-testing-software-for-continuous-testing-in-agile-and-devops/
https://testsigma.com/blog/essential-elements-of-test-automation/

2018-08-23T13:33:25Z
author avatar
Real User

1. Find the application/technology landscape of your company irrespective of your project?
2.Identify end to end scenario covering all the applications technologies
3.Identify the scenarios of the application where the custom build objects are used
4.Identify the tools which are supporting the technologies and the support availability with the vendor as well as in Internet.
5.Conduct POC with the tools for the identified requirements
6.Select the tool which automated all your requirements and which can be used easily to customize the automation approach according to the customer and developer needs without any tool vendor support.

2018-08-21T12:09:30Z
author avatar
User

Hi Rhea Rapps,
As per Miguel Jerone Ormita tips are very useful. This is a very effective way
Make a Decision/Feature matrix(Tools features,support cost, Tool Ease of use as a initial evaluation). what i was observed is there are good tools which can not pay for adds in google(Example: Simplify QA tools which is a script less and very easy to use -www.simplify3x.com https://www.linkedin.com/company/simplify3x-software-private-limited).
I will recommend request for demo from each vendor to validate the features instead of going to read all the documentation.

Conduct Trail effectively tips
1) Under stand the Needs of your organisation( Only need automation tool or Defect management tool included etc)
2)run the trail versions and check the compatibility of your system under test application.

What mistakes should be avoided for POC?

1) Identification of the Test case for POC( Pick the complex end to end test should be considered as you came to know the tool efficiency and will be useful for your project deliverable's within out slipping the time ).
2) Team understanding on the tool confirmation.

Thanks

2018-08-21T03:22:38Z
Find out what your peers are saying about Tricentis, Worksoft, Micro Focus and others in Functional Testing Tools. Updated: September 2020.
441,726 professionals have used our research since 2012.