2014-06-08 06:20:00 UTC

When evaluating Load Testing Tools, what aspect do you think is the most important to look for?


Let the community know what you think. Share your opinions now!
Guest
3737 Answers
Real User

My Criteria on Tool Selection:

1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)

2015-10-13 04:55:36 UTC13 October 15
Vendor

It really depends on what you test. In mobile app/game testing, it really is the coverage, capability to automate across devices and get parallel test runs done on those, easy adoption of test automation frameworks, cross-platformity between different OS platforms, and speed.

2015-08-14 11:50:58 UTC14 August 15
Real User

Addressing the need. I'll initially define what I expect a tool to do and, perhaps, how I think it should work in my project. The research process will be based on finding tools that meet this need (or set of needs).

Underlying this, during my research, I may reject options base on usability for its purpose. Even if a tool technically meets the needs, if a product looks difficult usability, I will reject it. I'd rather have a tool that people will adopt quickly and promote, than something that sits on the shelf, or worse, gets in the way.

Other key factors include:
- implementation costs: time, training, price/maintenance, resources required for operation
- integration with the rest of the tool chain

2015-01-14 15:45:08 UTC14 January 15
Real UserTOP 10

handles large loads of data
Test execution
Time
Easy to use

2015-11-18 13:35:37 UTC18 November 15
Vendor

I'm primarily concerned with the tool's ability to work right "out of the box" with my software under test... with a realistic expectation that we may use several different tools at once to accomplish our testing goals.

2015-10-13 17:43:43 UTC13 October 15
Real UserTOP 10

Simplistic and elegant - ease of use. The tools need to fit what we do and not force us to change and fit the tool. ROI is a big factor as far as time investment (price is secondary, unless extreme) .
My evaluation many time consists of "direct use" In other words install it and try to use it. If I have to refer to the docs for simple "surface" usage - I might drop it there and then, because that indicates problems as you try to do more complex actions

2015-10-13 17:14:02 UTC13 October 15
Real UserTOP 10

There no unique set of requirments/Criteria,
those depent by Size of the Company, current Testing Maturity level, available FTE who can be assign.

than i do agree with Sailaja Kamineni, the good crateria to assess the tool after having correctly and honestly agree with CIO and CFO on the above mentions dependencies are:

1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)

2015-10-13 09:54:04 UTC13 October 15
Consultant

My general criteria are:
Longevity/history to the tools vendor,
Test environment compatibility (interface, power source, intended performance requirements),
Cost (initial costs as well as maintenance costs),
Support/Training,
Ease of Use,
Reliability of the test platform,
Ability to trace results to requirements and design.

2015-10-08 18:47:36 UTC08 October 15
User

UX and ease of use (really easy of maintenance of test-related content). The testing tool needs to reduce burden on the testers and not make their job more difficult than not having a tool.

2015-10-05 21:28:27 UTC05 October 15
Consultant

I will answer and positioning myself as a consultant and a tester,

As my experience, for consultant viewpoint, these point is the main concern:
1. Easy to understand for the tester, even with the less experience testers.
2. Easy to integrating with existing client systems, included but not limited to their Operating Systems, hardware configurations, etc.
3. Easy to be agile, can be adopt any changes in client environments.
4. Can attract the top level management attention, especially with the capability from tools itself.
5. Suitable when aligning with client IT strategy

There, for tester viewpoints, below its the concern:
1. Easily to be understand
2. 24/7 Support
3. Less coding,
4. Auto/manual testing capabilities

2015-09-10 09:43:27 UTC10 September 15
UserTOP 20

Firstly, the tool MUST meet the requirement of the project. Secondly, easy to use and reliable. Thirdly, provide full support for a wide range of platforms including OS, web browsers, etc.

2015-09-09 03:40:52 UTC09 September 15
Vendor

Ease of use, covers all vendor products, flexible for all business solutions

2015-08-25 14:14:54 UTC25 August 15
Real User

Ease of use. I want something I can delegate to Computer Science undergrads. Flexible and able to handle changes to the interface without breaking huge numbers of tests.

2015-08-24 21:16:07 UTC24 August 15
Consultant

Easy to use, Less customization and compatible with all the Operating systems ! It should give a comprehensive reports at the end of every iteration.

2015-08-24 17:40:06 UTC24 August 15
Real User

Before we reach to a decision of accepting any tool, we need to first see the Application under test, type of testing involved(manually), technologies, no. of functionalities that needs to be automated and development approach. Then we should go for POC, define the POC steps, comparison criteria, select tools for comparison, execute your test, compare your test result and draw your conclusion.

2015-08-21 09:23:42 UTC21 August 15
Real User

The test tool has to work well with the apps we are testing.

2015-07-16 21:50:48 UTC16 July 15
Vendor

End to End testing should be possible truth every single technology in UI and nonUI.
Non developers should have the possibility to build up and maintenance test cases
Easy to use
Should have a test data management possibility
Should support manually testing also
Should support the test team to find with data combinatorik the right test cases

2015-07-09 15:19:00 UTC09 July 15
Real User

Ease of use, flexibility - can I expand its basic usage to fit our customized needs, does it work with our current architecture, will it work with our future architecture, does it work across platforms, can I run multiple copies (allow developers to run local copies), does it incorporate good regression testing, does it have good code coverage/metrics that are easily understandable for my developers and business users, does it address secure coding practices.

2015-07-07 12:00:19 UTC07 July 15
ConsultantTOP 5

Nedd to be easy to use. Preferable in the cloud to avoid infrastructure high costs as well licensing.

2015-07-06 15:27:19 UTC06 July 15
Consultant

Be able to see the thing.. testing as a service, without the hassle of installation, setup, configurations and so on is the main key.. after that the features and how they are designed for use..

2015-06-03 10:58:04 UTC03 June 15
ConsultantTOP 10

When speaking about test execution and management tools one major criteria would be how well it fits with the life cycle tools and processes that are used within the company or teams to deliver software or services, because as testing does not exist on its own (in most cases) it is necessary to go beyond the usability, easiness of maintenance, learning curve, price and overall availability of testing funcionalities in order to select a facilitator tool that fits the whole of the company and not only the testing needs.

2015-05-20 18:02:04 UTC20 May 15
User

As less Coding as possible..Ability to create tests as fast as possible

2015-05-13 06:30:27 UTC13 May 15
Consultant

Capacity of program and how many platforms is possible to automate in this tool!

2015-04-14 18:59:13 UTC14 April 15
User

Easy automation, less maintenance, exhaustive reporting, end to end tracking of test cases and defects.

2015-02-12 04:21:18 UTC12 February 15
User

Find one that will test native app on mobile devices.

2015-01-19 17:27:18 UTC19 January 15
Real UserTOP 5

Testing aforesaid is super important either automate or manual. There are lots of factors influences as all the experts’ comments on this topic. It is important to know how efficient for the project needs to accomplish. The tool that should have innumerable dependencies which demonstrates the tester not to rely on one environment such as understanding the coding and scripting invoke and support different languages and services. The security integrated functional testing tool advances the tester to learn the knowledge of how important is the security and vulnerability as a normal tester.

2014-12-26 13:22:59 UTC26 December 14
Real User

The key is to make sure you have defined WHY you need a tool first. A tool is never the solution. A tool is the way you have to support your solution. Make sure that you have determined the root of your problem to solve and then how the tool will support your response to it.

2014-12-20 07:05:39 UTC20 December 14
User

Licensing cost, Capabilities and use of easy

2014-12-12 09:45:23 UTC12 December 14
User

Quick Take-off for first time automation for any application
Less maintenace due to changes in application and quickly ready to use for testing
Esay GUI for end users for creation, maintenace and reporting
Suppport the wider range of technologies adapted by end user orgnisation

2014-11-28 08:47:36 UTC28 November 14
Real User

Need to know what my project needs to accomplish,
- Does the tool have relevant features?
- Installation Requirements (memory storage space, os etc)
- Can the tool be integrated with other project apps/ tools?
- Licensing cost?

2014-11-10 11:24:35 UTC10 November 14
Consultant

Value for money. Effective usage across SDLC phases.

2014-09-26 07:47:21 UTC26 September 14
User

Ease of use throughout - building the script, execution, analyzing the results, help features, support from the tool administration and lastly cost factor.

2014-08-27 19:44:55 UTC27 August 14
Consultant

Depending on your needs of the tool when looking for or selecting a tool, you can usually look at the benefits of the tool;

1. Performing a needs finding exercise to determine your environment, this depends on the context of the environment either functional or automation based and or the methodology being used.

2. After defining your needs does the chosen tool or tools being researched apply you can methodically go through thousands of tools to get to the end result of a pool of 2-5 tools for example for a POC (Proof of Concept).

3. Understanding your staff what other tools have your staff used in other environments this helps natrow the list further, as when selecting a tool you have to think about the end user to enhance the capability.

4. Budget, you can narrow down from thousands of tools to a top 10 list based on you budget needs as a tool could cost x to implement and y to be continually used.

5. Product how the tool will interact with the product as you may be told a certain performance or automation tool may not work, however you can leverage the needs finding exercise to determing further research criterion.

6. Trial or play with a tool, when researching a tool or wanting to gain exposure down load a tool and trial it to see what makes it tick, as you can learn alot more by experience than by the sales information presented

Above all, if trying to research sometimes the classification of the tool helps in the relevant context, i.e. You can narrow the research field by understanding a tool better as some can be under a generalist category like automation or performance you may find a tool could be a specialist web automation tool and this will assis your research efforts even further.

2014-08-10 22:04:14 UTC10 August 14
Real UserTOP 10

Does the tool meet my needs and does my team have the talent to use the tool.

2014-07-31 15:06:10 UTC31 July 14
Real User

Specifically for test automation tools:
1. That is can automate the software under test. This is very important since I have evaluated many tools and, as we all may agree, every test automation tool has its strengths and weaknesses but the most expensive tool is nothing compared to an open source if the former cannot do critical requirements.
2. How easy it is to find the answers. There are tools out there that are great and seems to be easy to use but when you need to use the help feature, it is so hard to find what you are looking for and once you find it, following it seems to have some ambiguity.
3. As Rusty and Aruna mentioned, ease of use.
4. Developers can also leverage the tool for their own tests. If the tools will be easy for testers but can frustrate the developers, it can be an issue because sometimes it is best to go to your developers first before sending a support ticket to the tool vendor.
5. Integration features - it should be able to integrate with Issue management tools such as JIRA or whatever is already used in the organization. The test result should also be able to be exported and displayed via Confluence or Sharepoint so that stakeholders do not need to go to the one who has the license to check the test results.

2014-07-11 01:33:18 UTC11 July 14
Vendor

How is the usability?
How does it fit in our IT landscape?
Do I have interfaces or options to extend functions? (Rest API?)
Which technologies do I need to test? (one for all or specialized)
How many and what kind of personnel will I need to operate the tool?

2014-06-20 08:10:24 UTC20 June 14
Consultant

Easy to use, less coding needed, easily costomizable and UI based, so that a less technical person could also be comfirtable to use the same.

Cheers!!

2014-06-12 04:54:36 UTC12 June 14
Find out what your peers are saying about Micro Focus, Apache, Neotys and others in Performance Testing Tools. Updated: September 2019.
365,257 professionals have used our research since 2012.
Sign Up with Email