What is our primary use case?
We have a relatively splintered tool set and a number of tools which could not connect all of those things together. Therefore, the use case for ALM Octane was that we were trying to create a single version of the truth. A single source of everything to change within the IT department.
I work in the programs management department.
We are using the latest version of the product because we are cloud-based. We receive all the deployments as they are released.
How has it helped my organization?
Octane has definitely improved the capability that we have for visibility within our tool set. The ability to report and see the current status on change, defect, and test runs on a spring by spring basis within our programs. Previous to this, change management was done in one system and testing was done in another system. Defects were in one of those systems, but they were like forgotten children and weren't really linked to anything.
Octane has made everything a lot more visible. It's ability to relate everything together and create spider diagrams of change, the lifecycle of that change, defects, and the test status. These have made a massive difference to the visibility and our ability to trace back to the origin of a change, where it started, and see how it finished.
It's beginning to improve our processes as well. We are seeing some real improvements in the way we do things. We are becoming more agile in the way we do it because of that and in a way that stories are managed. Stories are given lifecycles as opposed to just being entities within a tool.
The visibility that we received from the ALM tool is that we can see a change through from its early requirements all the way through to development check-ins to the pipeline release then to the point that it's deployed. We can see the full lifecycle of the change within the ALM tool including integrations that we never before had in a change management tool. It's almost revolutionary for some people here to see check-in information appear against a user story in an ALM tool.
What is most valuable?
- Octane is built on the SAFe framework, which is the agile methodology that we are currently following.
- The capability that it has for so many out-of-the-box integrations is a fantastic feature.
- The very clean, easy to use UI that it promotes.
What needs improvement?
The reporting side of ALM Octane could do with a few areas of improvement. There is not enough flexibility in the way that we can cut up the data to report on certain things. For instance, with test information, we can't split that up by team, so it's quite difficult to see what coverage each team is currently working on. Some tech managers and scrum managers want to see the testing which going on within their team, but it is difficult to see. We only get a more holistic overviews of that.
I come from a testing background, and think the testing could be improved.
For how long have I used the solution?
Less than one year.
What do I think about the stability of the solution?
It is a very stable platform. We occasionally see areas that come up with a more client side, so they're not blanket across everyone. Sometimes people use the wrong browser. The product clearly states that it doesn't support IE, but then who would support IE, as it is end of life.
We've only had a few stability issues. Generally, we have issues following any deployment they do, so if they do a deployment on a Sunday, then we may have a couple of issues on a Monday or Tuesday. However, their support and willingness to react and resolve issues for us has been second to none. They've been low impact to the point where it has not damaged anyone's perception at all.
What do I think about the scalability of the solution?
We have 220 users on it. I have spoken to clients who have 3000 users on it. It's relatively scalable. We haven't seen any performance issues at all as we have ramped up the amount of data that we have on it or the amount of users. Our users include scrum masters, developers, testers, product turners, subject-matter experts, and business intelligence analysts.
In terms of usage going forward, we will rollout to our operations department. We'll get Ops using the same platform, so that should be another 60 to 70 users. The benefits of this would be that we would have more work concentrated all in the same place. Therefore, we can have a lot of crossover between other departments which aren't currently on ALM Octane that we can get onto Octane. This would make it work better and make it easier to manage because it would be a single place for work to be referred between teams, as opposed to having to go to a different tool if someone needs something hardware or software related to be created.
How are customer service and technical support?
For a company of Micro Focus' size and delivering this large of a tool, their engagement with me has been unbelievable. It has been to a point where I have never experience engagement like this from a software house. I speak to developers and architects. I speak to people who actually care about the issues that they are speaking about. I don't just get someone in a call center who is logging a ticket, and says, "Someone is looking into this." Then, the ticket disappears into the abyss for three months. It's really nice to see and have intimate feedback on your suggestions or queries. That relationship has been almost as valuable as the tool.
The technical support, help desk, or service desk where you log a ticket on their service platform has the ability to turn around an issue quickly and is very reactive.
I logged an issue on Monday afternoon, and within 12 hours, it was fixed. They did a deployment on Sunday, where they made changes to the history area of every ticket. Then, on the Monday, that history had vanished. We noticed the history had disappeared. The history for every single change that we had in ALM Octane was gone. I logged a ticket with them late in the afternoon on a Monday, then by 9:00 in the morning the next day, our history had been restored. Whenever they do a deployment, if we have issues, it takes them no longer than three or four days to resolve that issue and deploy a fix for us.
One of the biggest strengths of the community that developed Octane is they are so willing to listen to their customers and learn, then improve the tool that they have delivered. They try and make it fit for the customers who are using it.
Which solution did I use previously and why did I switch?
We were using Enterprise Tester and Rally (CA Agile) for change management. We switched from Rally to ALM Octane because of the lack of integrations and lack of drive to see Rally improve (from the company) because Rally is now owned by CA Technologies, and technically called CA Agile. CA has multiple other products that do the same thing as Rally. They have sort of acquired Rally, and it almost gives the impression that they will end of life Rally at some point, then take the user base and put them onto the tools that CA have.
Also, Rally's age is a factor. Rally was one of the first scrum-based agile tools. It did a lot of things very well in its early life. It's been overtaken by newer agile tools now. The last reason was because Rally was not our choice. It was a tool that was pushed onto us by a third-party integrator when we brought them onboard to help us deliver a large program, so we just ended up with it. When you don't bring in a tool yourself and don't integrate it yourself, it ends up being a little bit of a mess on the administration side. There was a lot of stuff in it that had no home, no direction, nor desire to ever be completed, and had not been managed correctly. Thus, the administration to cover the tool was enormous.
We switched because outgrew the Rally tool with our process. It had gone beyond the capabilities of Rally.
People are generally happy with the position that they are in now in comparison to the position that we were in when we were on Rally. The administration is certainly a lot better now that we're on ALM Octane purely because people have a desire to not want to end up in the same situation, thus people are more conscientious of what they're doing.
How was the initial setup?
The initial set up was very simple. The tool from getting our license to starting to use it, there is not a lot to do. We have evolved with the tool, as the tool has gone on, but we started using it straightaway. There was nothing that we needed to do to make that tool work. We have taken a very step based approach. We started using it, then we developed some changes in the way the workflow flowed. We have added additional fields here and there, where we decided we needed to do so. Then, we added additional bits of functionality through other bits of integrations as we've been seeing the need or when we know we've embedded it in processes with other things. We've rolled things out slowly. It seems slowly to us, but it's actually not taken that long.
There is not a deployment. It's literally they give you access. They go, "Your licenses are ready," and you login. That's it, then you start using the tool.
The planning phase for me was a year long project, getting everyone on the system and all the data migrated. Initially, it was about creating a need, because no one knew they needed a new tool until someone looked into it. I identified the need and problem, did the analysis, made the recommendations, presented the options, made the recommendations, and collected the requirements. There were a lot of requirements. Then, I went out and engaged with our InfoSec Department and our procurement process. I officially got sponsorship from the directors in about March for the project who saw it and put some money aside to be able to do it. It was a fairly smooth process from start to finish, but it was hard for me because initially there was no need for it. I created the need for it, then from that point on, it was a very smooth process.
I was the single person driving that process, but then it was a member of staff from procurement. I touched base with multiple areas of the business that would have been using it to gather requirements, so nine scrum masters for half an hour each. Architects were all advisory. Contract specialists/managers to do the contracts. We had our legal team. I was the single resource that drove the process, created the documentation, and found the supplies.
I am the person now maintaining the system. It shouldn't take more than me, but it probably won't be me forever. The only reason it requires maintenance at the moment is because of misuse, so it's not like things go wrong with it all the time. It's more of a case of that it's self-sufficient and I can go through and review the work that people do, ensuring they are using the tool and populating it as we would like them to, thus we can get quality data out of it.
What about the implementation team?
We purchased it via a supplier. Octane is being supplied by EOH Europe for us. I have worked with them in the past. They were happy to put us in touch with Micro Focus. We already have a couple of other tools through EOH, so we already had an existing relationship with them.
Working with EOH Europe was fantastic. My contact at EOH was very helpful. He has always been there to help with the multiple questions that I ask all the time about various different things, not necessarily related to Octane, but about anything that they supply us.
My biggest challenge as the integrator has been about changing culture. The tool does what it does. That is all. It has received a very positive reception by the majority of the people that use it.
Changing the culture means improving the way that we do things, our processes, and the way that we do this is by having communities. We have a community to address concerns, misunderstandings, and conversation points that people bring, then we try and improve the practices that we do by trying to get everyone aligned to the same practices.
What was our ROI?
It's a bit early on to see improvements in times and deliveries. Our entire company has only been using the product since December of last year, so we don't have enough trend data.
We will see ROI once we have the automation suite connected up to Octane. We will then have the ability to report on automated testing versus manual testing and the ability to see those tests automatically parsing with the tool. When Octane shows us when our CI process fails and shows us what the story that failed, we will have return on investment. Because we will have not the overhead of having to do an investigation of having to find out what the change was, because Octane will tell us all of that information.
What's my experience with pricing, setup cost, and licensing?
For what it does, it's very reasonably priced. I like the licensing model as well, because it's very flexible. You can scale licenses up and down for short periods of time.
In terms of pricing, it's comparable to what we had previously. It's not priced at the higher end of the scale by any means. It's priced nicely, in the middle of the market. For what you're getting, it's a very good tool.
Which other solutions did I evaluate?
It went through official procurement process where we went out to tender with seven different suppliers. We had responses from five of those suppliers. We had demos from five of those suppliers. We followed three more through, then we eventually selected Micro Focus ALM Octane. At which point, we started demoing Octane and ran it through 2018 whilst we were doing contract negotiations and signing contracts, which was probably the single hardest part of the entire thing.
Four of the seven vendors that we looked at were Micro Focus, CA Agile (incumbent), VersionOne, and Jama.
We went with ALM Octane because of its functionality and it is presented very cleanly and simply. You are never more than three clicks away from where you need to be in Octane. Another reason that we went for ALM Octane as a tool is because of our relationship with Micro Focus as a company.
ALM Octane has a cleaner version than VersionOne, which is a little busier.
What other advice do I have?
If you're looking for a tool which will complement a CI or DevOps process, where you want to have a single point of visibility or a single version of the truth, and see all of the stuff that happens through that journey, Octane is the tool which will to give you that.
The biggest lessons learned: When you start focusing on a new tool that prides itself on having a very tight process to make things visible, you learn how other people don't necessarily follow its processes as tightly as you would expect them to.
Using the SAFe framework helps our workflow patterns. We have been using SAFe for about four to five years, and we've actually been using it properly for maybe two and a half to three years. We're still not perfect by any means, but we are definitely pushing forward in the right direction to become more focused on delivering the true version of that methodology. Although ALM Octane doesn't do every element of that methodology yet, they are endeavoring to clean up a lot of those areas. They are trying to mop of some of the methodology that SAFe works on adding in things. We have seen quite a lot of new features recently that have been specifically focused towards SAFe, which has been really positive for us.
ALM Octane has improved our use of agile, but we still do some waterfall stuff. We will always carry on doing some Waterfall stuff until certain systems fall out of use because we have old systems and those old systems don't lend themselves to agile.
ALM Octane has presented us the opportunity to push forward with a true CI/CD approach, which is where we want to get to.