Helix ALM Review

It gave us the ability to define and manage a set of test cases and test sets to be run, but automation was hard to maintain for complex scripts with multiple subtest calls and I/O.


Valuable Features

The test management component (ability to define and manage a set of test cases and test sets to be run), and the test metrics were often used by me.

Improvements to My Organization

During that time the tool was in use by me, the test organization coalesced into a pyramid (centralized/organized) with Test Manager on top of the pyramid, Test Leads (for each shift) below the Test Manager, and then the Test Engineers creating tests, mapping requirements, and executing the tests.

It improved bilateral communications. Instantaneous progress metrics were viewable by the entire team. At the time, The Configuration Control Board partly relied upon TestTrack reports to judge the efficacy testing and to help determine the next set of fixes required to be placed into the release baseline.

Use of Solution

I've used the TestTrack test case management portion for approximately two and a half years during the life of the project in a medical device manufacturer’s R&D organization.

Deployment Issues

There was an effort to automate the scripts so that signals, including GUI key strokes and message traffic could be sent to the device without human intervention, but would capture and report results in the defined format.

The test cases and the automation was hard to maintain for complex scripts with multiple subtest calls and I/O. It was judged that more and better technical support was required to fix broken scripts and monitor modifications to subtest cases. In other words, we were missing needed hands-on management and dedicated Subject Matter Experts (SMAs). Should we have found the appropriate resources, and given appropriate planning and capital, the test environment could have been expanded throughout the R&D department for Verification and Validation.

Customer Service and Technical Support

The Seapine representatives and the reps that helped us to land the test server were very responsive, helpful, and knowledgeable. We required training and integration of the tool within our test environment. They understood our needs and sold us exactly what was required.

It should be noted that we had the DOORS requirement tool with several hundred requirements that was considered the record of origin for all of the test case development. So there was a need to keep the DOORS tool and add a method to synchronize requirements between DOORS and the TestTrack environment. They helped us tailor their tools suite and they wrote the original export/import routines for the DOORS requirements interface.

Initial Setup

The client server setup required a savvy IT person to help set up the server and permissions. I would say that our configuration was average in complexity. Each client required some very minimal setup.

Implementation Team

The implementation was in-house with some vendor training. However, recall that the vendor provided us with a utility to transfer and synchronize our DOORS requirements to the TestTrack tool. I would advise at least one-day training course for any that use the tool including managers and test engineers.

ROI

ROI was traceability, real-time metrics, and test environment integrity. A graphically remote test manager could evaluate information on his/her computer dashboard to identify progress (e.g. how many tests created, approved, test sets, test runs, created/approved status, test cases populated within specific test sets, test cases passed, ID of test step failures, requirements without tests, tester ID, dates of when results were achieved and more).

Pricing, Setup Cost and Licensing

Their current pricing strategy allows for economy at three levels:

  • Perpetual for $595.00 (upfront purchase and yearly maintenance cost)
  • Hosted $30/seat (10 seat minimum; free support and upgrades)
  • Subscription $30/seat (unlimited users; free support and upgrade)
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment
Guest

Sign Up with Email