What is most valuable?
Test management and reporting. Those are the two most important things. I tell my customers that the two main reasons they have ALM:
- Quality management, project management from a QA perspective - testing, defect management, how testing relates back to requirements.
- And reporting to make good business decisions in the future.
How has it helped my organization?
When I was a customer, it improved my organization because I was able to manage, to enforce standards on building tests, executing tests, and manage centralized reporting.
Now, I translate that over to my customers from various levels of the spectrum from complete, "We have no idea what to do to, we're doing stuff but we know we need to change," to "We've got some stuff and we just want to tweak what we're doing now."
What needs improvement?
I'd like to see the idea of users being flushed out more, so not just, "This defect is now assigned to a particular person," or "This person is assigned to execute a test."
I want to see the users expanded out to teams where you have five users and the sixth user is the manager, so the manager can roll the idea of somebody being responsible and accountable. The idea of things being assigned to a team of users and users belonging to that team. There are ways of getting around this in the tool because it's very customizable, but I'd like to see that separate from the idea of using security groups, which is one way of getting around that.
I'd like to see the concept of teams put into it.
What do I think about the stability of the solution?
ALM has gotten more stable over the years. It's a stable app. Like any other large, complex application, you run into things every now and again. We have a system to report things and get them taken care of.
What do I think about the scalability of the solution?
I have customers that are small and customers that are enterprise-wide. So I'm able to deploy it in both kinds of environments and customize the tool, depending on size and level of maturity, for any kind of customer. Also within any vertical as well.
How are customer service and technical support?
I have used tech support. Mostly because I'm with a consulting company and we also support ALM. We have our own internal support organization that people can get into.
In terms of Micro Focus support, because I'm a more advanced user - I've been using this tool since version 7 - I typically don't get a whole lot from first-level support. I tend to want to go right up to second, third, or even directly with the development organization. So I'm more the outlier, edge-case kind of person compared to most customers out there.
Once I get to the people that are at the level that I know I need to deal with, they're good. I'm also dealing with the people on the other side of the ocean, working directly with people who may have actually coded ALM to begin with.
Which solution did I use previously and why did I switch?
When I became a customer in 2000/2001, when I first started, I was involved in the decision to purchase the solution. Now, as a professional services consultant, that decision has been made and I'm going in there to either deploy, upgrade, or help them use ALM to best suit their needs. In some cases I help them figure out what it is they need to have ALM do for them or how to customize it best.
When I was a customer, we were not using another solution. We were completely manual and I was a department of one. I was the QA organization for a small development company and the two company owners said to me, "We want to invest in this, go look and see what's out there and show us what our options are and what you think the best option is."
What caused us to switch to this solution was the customizability. The fact that we could make it give us the information that we needed to get out of it. The support organization seemed very top-notch. I actually learned a lot from the support organization when I was getting started in it. And I found it more intuitive then the Rational solution.
How was the initial setup?
I've deployed it in many organizations because I'm a consultant. I've deployed it, upgraded it, customized it, in various ways for different customers.
In terms of complexity, it really depends on the needs of the customer.
When I was a customer in a small development organization that only had 20 people in the entire company, I deployed it, I did the customization - that was way back in the day.
Now, I have customers along the entire spectrum from small to large enterprise. Some customers are okay with near vanilla, out of the box. And some customers have very complex sets of business logic that they feel, for whatever reason, need to be enforced as far as how their defect management lifecycle is going to go. How their test construction, test execution lifecycle is going to go, how they want to manage requirements, and that can require significant customization.
Some of my customers have compliance concerns, they have digital signatures and they have FDA 21 CFR Part 11 compliance. They have all of these rules that they have to follow and some of them are subject to interpretation, so with one particular rule I have one customer who says, "This is how we interpret the rule," and they have me customize it one way; and I have another customer who says, "No, we're not going to interpret it that, way we interpret this way," and it's a completely different set of customizations.
Which other solutions did I evaluate?
Back then it was Mercury Test Director, which is now ALM. We were also looking at the Silk products, and we were looking at the Rational, now IBM, products.
What other advice do I have?
When selecting a vendor to work with, I want to see that the technical people are really knowledgeable of what they're talking about. I want to know that the tool can give me what I need, not just, this is a standard proof of concept. I want to see what I need to see, and I want to know that, down the road, I'll be able to either get out of it what I need or be able to learn or have somebody come in to help me get out of it what I need. Because if I'm not getting out of it what I need, then I've wasted my money.
I give it a nine because nothing is perfect, there's always room for improvement, especially when you're talking about an app system as large as ALM is. I've been using it for so long it's kind of second nature for me to think about where its strengths are, and know that if I can't get something done one way there's always another way around it. Or I can integrate something into it or build work flow to make the UI behave the way I want it to.
Regarding advice to a colleague about ALM, remember that your process and your methodology should be driving what you need out of their tool and not the other way around. Tools can do some really cool stuff. You may look at it and say, "Okay, maybe we could get some value out of this feature that we're not doing today." But don't make that the driving force. It really needs to be able to support what you're doing and force the things that you want to get out of it. Because there's a truism in reporting: If you don't capture the data you can't build a report that's meaningful. So make sure it can get you what you need.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
A really good breakdown of the ALM story.