What is most valuable?
The Open Test Architecture (OTA) and development of the REST API. The OTA is a published set of functions that administrators and users can use to interact with HP ALM programmatically. The most common example HP ALM users would recognize is the Microsoft Excel upload template, which allows users to upload test scripts to HP ALM projects directly from an Excel worksheet.
The REST API sneaked into HP ALM with little fanfare. The REST API has no application overhead and is fast. HP extended the API through patches in v11.0. Please check your current version and patch level to see which functionality is now included in the REST API.
How has it helped my organization?
On a recent multi-year project, the average defect resolution time for all defects was over twenty-two days. My goal was to reduce this number by 20%. It was an easy goal to reach because no one realized that 80% of 22 days was still a number out of bounds for defect resolution. I used custom fields, defect workflow and custom reports to move defects through their lifecycle. Within thirty days the defect resolution time was reduced to 3.1 days and averaged 1.1 days over the next eighteen (18) months.
What needs improvement?
The graphical user interface has the most room for improvement. Not all screens within the integrated suite refresh the same, some screens or activities are self-refreshing and some are not.
I would also like to see the “Disable Quick Runs” added back as a site parameter or built as an internal function within a project.
For how long have I used the solution?
I've used it for seventeen years (1998 to present). This product was initially developed by Mercury-Interactive and released as Test Director. My first enterprise installation and administration experience with Test Director 2.0 was in 1998.
What was my experience with deployment of the solution?
Over a seventeen year period, yes. The key to maintaining a site today is in patch management. Keep the patches up to date.
What do I think about the stability of the solution?
When the patches lag too much, it may be safer to build a new site and port the data than to try and patch an existing site in place.
How are customer service and technical support?
Customer support for this product is with the vendor, not necessarily HP. After a site has been up and running a few years and all the original implementers are gone, it can take some time to even determine the vendor. My satisfaction level with vendors range from acceptable to excellent. Technical Support
My personal satisfaction level with HP service and support website is low. I get the majority of my technical information from colleagues or third party discussion forums.
Which solution did I use previously and why did I switch?
In the Application Lifecycle Management space, HP ALM and IBM Rational are the two big players. I recently participated in an evaluation of the IBM Rational Jazz Platform. The client had been using IBM’s ClearCase and ClearQuest for many years. During the evaluation, an unrelated IBM audit detected a long dormant pack of five ClearCase licenses on an active server. The cost associated with this incident ended our evaluation of the IBM solution.
Historically, most people considered this to be a defect tracking only tool. In that domain, tools are plentiful. Over the years I’ve used VI editor on UNIX, Microsoft Excel worksheets Microsoft Access databases, Bugzilla and Notepad for defect management.
How was the initial setup?
The installation and configuration of an HP ALM site is straightforward for those with enterprise software deployment experience. An installation requires at minimum, a dedicated server with an operating system and database connection. The most typical, physical or virtual, hardware configuration I encountered over the years was a single Microsoft Windows server running web, license and application software servers. Both Oracle and MS SQL Server databases respond adequately, and when given a choice now select a schema based on DBA agreeability.
The installation of an HP ALM site establishes a service endpoint for communicating with other applications via Web Services. I believe the configuration and management of these services is the most complex part of a site installation and requires substantial planning to map fields and permissions across multiple applications.
What about the implementation team?
HP ALM resellers typically perform the initial set up and configuration of the HP ALM site and user projects. In some cases, larger testing firms are also resellers and provide the tool as part of the project. I fundamentally disagree with buying a tool from its eventual user.
I advise clients to do the upfront planning and limit users with access to the site administration console to three or less. The planning required for a successful implementation requires much more time and effort than the deployment itself. Deployments are typically scripted while planning requires humans. Access to the HP ALM site console is separate from project access. I have seen sites with twenty or more registered site administrators. I believe this occurs more as a symptom of long term neglect than an implementation issue.
What's my experience with pricing, setup cost, and licensing?
I encourage clients to use the built in service accounts and APIs where practical.