What is most valuable?
- Templates: Allows us to standardize fields, workflows throughout hundreds of HPE ALM projects.
- Customizable Events drive workflow: Saves a few seconds of time when we set default values, customizable dependent lists. When we translate this to hundreds of projects and hundreds of concurrent users, the productivity gains are immeasurable.
- Traceability: Traceability from Requirement -> Test Plan -> Test Lab -> Test Runs -> Defects.
- Rich sets of permission settings for different roles.
- Business Views: It is easier to pull reports for novice users.
- OTA, REST API: Utility for adding users and massaging data.
How has it helped my organization?
It has improved collaboration between our test teams.
What needs improvement?
- Client-side ActiveX with patch upgrades
- Support TDD/BDD
- Adding features like Kanban, project plans, resource utilization, and JIRA’s big picture.
- Traceability reporting: If HPE can generate a traceability matrix - Traceability from requirement -> Test Plan -> Test Lab -> Test Runs -> Defects.
- Defects aging report: A look at how long each defect is from the time it is created and how long it has remained at a particular status.
- Reports to build information: Currently, HPE ALM does not support the copy and paste of test instances.
How can we duplicate a test set within a project?
- How do we copy a test set from one project to another, and also copy the associated test plan along with it?
- Reports on automation:
- How do we capture the number of automation test cases as some automation test cases have multiple test cases in each automated test?
How do we calculate the ROI of automation?
How do we determine which test cases should be automated, because sometimes the effort of automation does not have a good ROI?
How do we calculate the ROI of HPE Sprinter if (1) Automated Script generation is used? (2) Mirroring is used?
How do we know which testcase is created using Sprinter?
- Requirements coverage reports.
- Cumulative trending reports for test execution and defects outstanding.
- Auto-generation of test summary report.
- Inbuild best practices for fields, such as root cause category.
- Copy of test sets in the test lab.
- Upload of test execution results.
- Offline test defects to third parties and sync upon checking in.
For how long have I used the solution?
We have been using the service for 18 years.
What do I think about the stability of the solution?
The solution is very stable, if you have the right person to manage it.
What do I think about the scalability of the solution?
The solution is very scalable, if you have the right person to manage it.
How is customer service and technical support?
The technical support has been deteriorating since the Mercury days. I would give technical support a rating of six out of 10.
Which solutions did we use previously?
We were looking at other solutions, such as JIRA, due to all the issues I have raised.
How was the initial setup?
The setup was straightforward.
What's my experience with pricing, setup cost, and licensing?
HPE ALM has been sold to Micro-Focus. I am not sure if Micro-Focus will be flexible.
If no flexibility is provided, you can easily move out in weeks, if you have the right resources.
Which other solutions did I evaluate?
We evaluated Zephyr, QASymphony, XQual, Perforce, and Rational Quality Manager.
For integration purposes, we evaluated Tasktop, Orasi, HP Synchronizer, and ConnectALL.
For automation purposes, we evaluated UFT, Selenium, and Robot.
What other advice do I have?
Below is a checklist for others considering a test management solution:
- CIO sponsorship
- Number of projects
- Number of concurrent users
- Standardization needs
- Ease of control and management
- Access rights for individual roles
- Event driven workflow customization
- Extending beyond test management to TDD, BDD, test data management, test environment, and CI/CD tools.
- Focus on where you want to solve each problem and identify relevant tools for each of these.
- Availability of skilled resources
- Hiring the right resources to manage: There are easily millions of test management users, but it is not easy to find a true-bred expert.
- Keep in touch with what’s happening in the industry. You need to be focused and not swayed easily.
- Know your stuff.
- Support all your users and make life easier for them.
- Integration with automation tools, performance tools, security tools, and Jenkins/Bamboo/Team City.
- Define the test process that ties in with your test management too.l
- Form a training team to constantly train users.
- Open API for customization
- Export of info to MS Excel.
- Ease of migration.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Jul 23 2017