2014-06-08 06:20:00 UTC

When evaluating Load Testing Tools, what aspect do you think is the most important to look for?


Let the community know what you think. Share your opinions now!
Guest
3737 Answers

author avatar
Consultant

My Criteria on Tool Selection:

1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)

2015-10-13 04:55:36 UTC
author avatar
Vendor

It really depends on what you test. In mobile app/game testing, it really is the coverage, capability to automate across devices and get parallel test runs done on those, easy adoption of test automation frameworks, cross-platformity between different OS platforms, and speed.

2015-08-14 11:50:58 UTC
author avatar
Vendor

Addressing the need. I'll initially define what I expect a tool to do and, perhaps, how I think it should work in my project. The research process will be based on finding tools that meet this need (or set of needs).

Underlying this, during my research, I may reject options base on usability for its purpose. Even if a tool technically meets the needs, if a product looks difficult usability, I will reject it. I'd rather have a tool that people will adopt quickly and promote, than something that sits on the shelf, or worse, gets in the way.

Other key factors include:
- implementation costs: time, training, price/maintenance, resources required for operation
- integration with the rest of the tool chain

2015-01-14 15:45:08 UTC
author avatar
Consultant

handles large loads of data
Test execution
Time
Easy to use

2015-11-18 13:35:37 UTC
author avatar
Vendor

I'm primarily concerned with the tool's ability to work right "out of the box" with my software under test... with a realistic expectation that we may use several different tools at once to accomplish our testing goals.

2015-10-13 17:43:43 UTC
author avatar
Vendor

Simplistic and elegant - ease of use. The tools need to fit what we do and not force us to change and fit the tool. ROI is a big factor as far as time investment (price is secondary, unless extreme) .
My evaluation many time consists of "direct use" In other words install it and try to use it. If I have to refer to the docs for simple "surface" usage - I might drop it there and then, because that indicates problems as you try to do more complex actions

2015-10-13 17:14:02 UTC
author avatar
Vendor

There no unique set of requirments/Criteria,
those depent by Size of the Company, current Testing Maturity level, available FTE who can be assign.

than i do agree with Sailaja Kamineni, the good crateria to assess the tool after having correctly and honestly agree with CIO and CFO on the above mentions dependencies are:

1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)

2015-10-13 09:54:04 UTC
author avatar
Vendor

My general criteria are:
Longevity/history to the tools vendor,
Test environment compatibility (interface, power source, intended performance requirements),
Cost (initial costs as well as maintenance costs),
Support/Training,
Ease of Use,
Reliability of the test platform,
Ability to trace results to requirements and design.

2015-10-08 18:47:36 UTC
author avatar
Vendor

UX and ease of use (really easy of maintenance of test-related content). The testing tool needs to reduce burden on the testers and not make their job more difficult than not having a tool.

2015-10-05 21:28:27 UTC
author avatar
Consultant

I will answer and positioning myself as a consultant and a tester,

As my experience, for consultant viewpoint, these point is the main concern:
1. Easy to understand for the tester, even with the less experience testers.
2. Easy to integrating with existing client systems, included but not limited to their Operating Systems, hardware configurations, etc.
3. Easy to be agile, can be adopt any changes in client environments.
4. Can attract the top level management attention, especially with the capability from tools itself.
5. Suitable when aligning with client IT strategy

There, for tester viewpoints, below its the concern:
1. Easily to be understand
2. 24/7 Support
3. Less coding,
4. Auto/manual testing capabilities

2015-09-10 09:43:27 UTC
author avatar
Top 20User

Firstly, the tool MUST meet the requirement of the project. Secondly, easy to use and reliable. Thirdly, provide full support for a wide range of platforms including OS, web browsers, etc.

2015-09-09 03:40:52 UTC
author avatar
Vendor

Ease of use, covers all vendor products, flexible for all business solutions

2015-08-25 14:14:54 UTC
author avatar
Vendor

Ease of use. I want something I can delegate to Computer Science undergrads. Flexible and able to handle changes to the interface without breaking huge numbers of tests.

2015-08-24 21:16:07 UTC
author avatar
Consultant

Easy to use, Less customization and compatible with all the Operating systems ! It should give a comprehensive reports at the end of every iteration.

2015-08-24 17:40:06 UTC
author avatar
Consultant

Before we reach to a decision of accepting any tool, we need to first see the Application under test, type of testing involved(manually), technologies, no. of functionalities that needs to be automated and development approach. Then we should go for POC, define the POC steps, comparison criteria, select tools for comparison, execute your test, compare your test result and draw your conclusion.

2015-08-21 09:23:42 UTC
author avatar
Vendor

The test tool has to work well with the apps we are testing.

2015-07-16 21:50:48 UTC
author avatar
Vendor

End to End testing should be possible truth every single technology in UI and nonUI.
Non developers should have the possibility to build up and maintenance test cases
Easy to use
Should have a test data management possibility
Should support manually testing also
Should support the test team to find with data combinatorik the right test cases

2015-07-09 15:19:00 UTC
author avatar
Vendor

Ease of use, flexibility - can I expand its basic usage to fit our customized needs, does it work with our current architecture, will it work with our future architecture, does it work across platforms, can I run multiple copies (allow developers to run local copies), does it incorporate good regression testing, does it have good code coverage/metrics that are easily understandable for my developers and business users, does it address secure coding practices.

2015-07-07 12:00:19 UTC
author avatar
Vendor

Nedd to be easy to use. Preferable in the cloud to avoid infrastructure high costs as well licensing.

2015-07-06 15:27:19 UTC
author avatar
Consultant

Be able to see the thing.. testing as a service, without the hassle of installation, setup, configurations and so on is the main key.. after that the features and how they are designed for use..

2015-06-03 10:58:04 UTC
author avatar
Consultant

When speaking about test execution and management tools one major criteria would be how well it fits with the life cycle tools and processes that are used within the company or teams to deliver software or services, because as testing does not exist on its own (in most cases) it is necessary to go beyond the usability, easiness of maintenance, learning curve, price and overall availability of testing funcionalities in order to select a facilitator tool that fits the whole of the company and not only the testing needs.

2015-05-20 18:02:04 UTC
author avatar
Vendor

As less Coding as possible..Ability to create tests as fast as possible

2015-05-13 06:30:27 UTC
author avatar
Consultant

Capacity of program and how many platforms is possible to automate in this tool!

2015-04-14 18:59:13 UTC
author avatar
Vendor

Easy automation, less maintenance, exhaustive reporting, end to end tracking of test cases and defects.

2015-02-12 04:21:18 UTC
author avatar
Vendor

Find one that will test native app on mobile devices.

2015-01-19 17:27:18 UTC
author avatar
Top 5Real User

Testing aforesaid is super important either automate or manual. There are lots of factors influences as all the experts’ comments on this topic. It is important to know how efficient for the project needs to accomplish. The tool that should have innumerable dependencies which demonstrates the tester not to rely on one environment such as understanding the coding and scripting invoke and support different languages and services. The security integrated functional testing tool advances the tester to learn the knowledge of how important is the security and vulnerability as a normal tester.

2014-12-26 13:22:59 UTC
author avatar
Vendor

The key is to make sure you have defined WHY you need a tool first. A tool is never the solution. A tool is the way you have to support your solution. Make sure that you have determined the root of your problem to solve and then how the tool will support your response to it.

2014-12-20 07:05:39 UTC
author avatar
Vendor

Licensing cost, Capabilities and use of easy

2014-12-12 09:45:23 UTC
author avatar
Vendor

Quick Take-off for first time automation for any application
Less maintenace due to changes in application and quickly ready to use for testing
Esay GUI for end users for creation, maintenace and reporting
Suppport the wider range of technologies adapted by end user orgnisation

2014-11-28 08:47:36 UTC
author avatar
Consultant

Need to know what my project needs to accomplish,
- Does the tool have relevant features?
- Installation Requirements (memory storage space, os etc)
- Can the tool be integrated with other project apps/ tools?
- Licensing cost?

2014-11-10 11:24:35 UTC
author avatar
Consultant

Value for money. Effective usage across SDLC phases.

2014-09-26 07:47:21 UTC
author avatar
Vendor

Ease of use throughout - building the script, execution, analyzing the results, help features, support from the tool administration and lastly cost factor.

2014-08-27 19:44:55 UTC
author avatar
Consultant

Depending on your needs of the tool when looking for or selecting a tool, you can usually look at the benefits of the tool;

1. Performing a needs finding exercise to determine your environment, this depends on the context of the environment either functional or automation based and or the methodology being used.

2. After defining your needs does the chosen tool or tools being researched apply you can methodically go through thousands of tools to get to the end result of a pool of 2-5 tools for example for a POC (Proof of Concept).

3. Understanding your staff what other tools have your staff used in other environments this helps natrow the list further, as when selecting a tool you have to think about the end user to enhance the capability.

4. Budget, you can narrow down from thousands of tools to a top 10 list based on you budget needs as a tool could cost x to implement and y to be continually used.

5. Product how the tool will interact with the product as you may be told a certain performance or automation tool may not work, however you can leverage the needs finding exercise to determing further research criterion.

6. Trial or play with a tool, when researching a tool or wanting to gain exposure down load a tool and trial it to see what makes it tick, as you can learn alot more by experience than by the sales information presented

Above all, if trying to research sometimes the classification of the tool helps in the relevant context, i.e. You can narrow the research field by understanding a tool better as some can be under a generalist category like automation or performance you may find a tool could be a specialist web automation tool and this will assis your research efforts even further.

2014-08-10 22:04:14 UTC
author avatar
Vendor

Does the tool meet my needs and does my team have the talent to use the tool.

2014-07-31 15:06:10 UTC
author avatar
Vendor

Specifically for test automation tools:
1. That is can automate the software under test. This is very important since I have evaluated many tools and, as we all may agree, every test automation tool has its strengths and weaknesses but the most expensive tool is nothing compared to an open source if the former cannot do critical requirements.
2. How easy it is to find the answers. There are tools out there that are great and seems to be easy to use but when you need to use the help feature, it is so hard to find what you are looking for and once you find it, following it seems to have some ambiguity.
3. As Rusty and Aruna mentioned, ease of use.
4. Developers can also leverage the tool for their own tests. If the tools will be easy for testers but can frustrate the developers, it can be an issue because sometimes it is best to go to your developers first before sending a support ticket to the tool vendor.
5. Integration features - it should be able to integrate with Issue management tools such as JIRA or whatever is already used in the organization. The test result should also be able to be exported and displayed via Confluence or Sharepoint so that stakeholders do not need to go to the one who has the license to check the test results.

2014-07-11 01:33:18 UTC
author avatar
Vendor

How is the usability?
How does it fit in our IT landscape?
Do I have interfaces or options to extend functions? (Rest API?)
Which technologies do I need to test? (one for all or specialized)
How many and what kind of personnel will I need to operate the tool?

2014-06-20 08:10:24 UTC
author avatar
Vendor

Easy to use, less coding needed, easily costomizable and UI based, so that a less technical person could also be comfirtable to use the same.

Cheers!!

2014-06-12 04:54:36 UTC
Find out what your peers are saying about Micro Focus, Apache, Neotys and others in Performance Testing Tools. Updated: March 2020.
406,607 professionals have used our research since 2012.