What is our primary use case?
I work for a section of our company where what we do is host enterprise tools that our consulting projects can use. Potentially, as we get more and more users, we can have hundreds of projects at a time. We're not a typical use case where we have one way that we're using the tool. The tool is being used on various consulting projects.
Our use cases vary drastically. We have some people who have told us they just use it for testing, there are some people who just use it for defect management. People are familiar with other tools, like JIRA and ALM and even AGM. Octane is new, so some people are trying to take baby steps into adopting it.
Day-to-day, how we typically use it, and how we're promoting it should be used, is for Agile project management with manual testing, including release management, sprint management; end-to-end type of use. We use it to manage our releases in sprints. Other teams within my group also use it for testing and defect management, and that's how we promote it, and train our consulting project teams to use it.
How has it helped my organization?
For our use case, it's brought our entire team into a single tool. We're all looking at the same real-time data. Our project management office has been able to set up dashboards for individual teams, and do comparisons by teams, of integration, and cross-team integration, burn-up, burn-down, and cumulative flow types of things. So from a PMO perspective, there is a really good overview, from how we've set up our dashboards, to know where each team is and how they're progressing and how much work they have that integrates with other teams. That's really helpful.
The feedback that we've gotten is that the way testing is closely tied into the product Backlog has made it more intuitive, or easier to manage the relationship between building out an application and testing it. In other tools, that is more segregated. The way it's designed in Octane, people have said it makes more sense to them, and that it's easier for them to understand their data and to maintain and test their solutions.
What is most valuable?
Very generally, the feedback that we've received is that people really like the user interface overall. It's intuitive to use, it's easy to learn, people like the usability features, the user experience.
Another thing that people really like about Octane is how easy it is to customize. In some previous tools, that has been very limited, or you had to know how to write code to do some of the customizations, or it was very confusing. Going back to the user interface, they've made the customization of the tool, the workspace settings, very easy for people to figure out and use. We've gotten a lot of good feedback on that.
In general, I think people really like the Team Backlog and the capacity bucket for each individual team member. They like that ability to track capacity and progress very easily that way, by individuals.
What needs improvement?
I work pretty closely with Micro Focus, particularly on ALM Octane. Right now we have a backlog of some 60 or 70 enhancement requests, varying in priority from very high to low.
In general, there's a trend in our requests to have the ability to export data, en masse, out of Octane. There are capabilities within Octane to export data, but there are specifics around test suites and requirements and relations, as well as certain attributes, that we would like to be able to export easily out of Octane and into a database or Excel.
One of the things that a lot of our project teams have complained about is the simplicity of reporting that's available in Octane, and that they have to export data out of it in order to create the types of reports that their PMO or their client wants to see. Octane provides solutions around OData, and integration into reporting tools, but what people really want is smart and good reporting, advanced reporting, within the tool. They don't want to have to go out to another tool for reporting.
In general, we also have some requests to beef up the manual testing abilities and the ability to report on testing progress. All the basics are there, but there's an issue of maintainability. For example, one thing that we brought up to them recently was: Once you plan a test and it creates a run, more particularly a suite run, you can't edit the suite run afterward. It locks you in, and we're saying that that is not realistic with how people work. Mistakes are made and people are humans and we change our minds about things. So the tool needs to allow for a bit more flexibility in that testing area, as well as some better widgets to report on progress.
For how long have I used the solution?
One to three years.
What do I think about the stability of the solution?
Overall, we haven't had any issues with stability. There are two things that do come up.
It seems like we have issues with Elastic, the integration to it. Intermittently we have these issues where global search isn't working, or widgets aren't populating, so there's something a little bit unstable with that integration. It could be on our end, or it could be something with the setup, I'm not sure.
We're also having performance issues. It's not really stability, but we do see some slowness in the system and in our performance testing, so we're working with Micro Focus on that to figure out how to resolve those issues.
What do I think about the scalability of the solution?
The performance issues that we have come about when we have load on the system. We're trying to figure out if the source of those issues is the environment, and what the optimized settings would be for the environment: memory size, number of disks, things that we're doing in our performance testing, etc. But we're also looking at the software to see if there are any issues there.
We are working with about 180 to 200 concurrent users, which isn't a terribly high number. We're looking at all sorts of angles but we're currently in the middle of it, so it's hard to tell what the source of the issue is.
Micro Focus has definitely been good about helping us out with all of that, giving us advice, hardware related, on what our settings should be. Maybe we're not sized exactly correctly. According to Micro Focus - they also, of course, do their own performance testing - and they haven't seen the results that we have.
How are customer service and technical support?
Because we're a partner and we've been working with them for years, we'll have quick calls twice a month, at least for Octane, to talk about new enhancement requests that are coming up, providing them with feedback on the tool as we're hearing it from our project users, and to review our highest priority requests and their statuses and if they have been included in their release planning for upcoming releases.
We have a pretty good dialogue going back and forth with them, so we know when to anticipate the functionality that we're looking for is going to get delivered. That helps us when we're making decisions around which upgrades to take.
Tech support does a pretty good job. Sometimes it's a little frustrating because they're the Level 1 support, the helpdesk support. They often are trying to rush us to close out tickets. I understand that they've got metrics, but that part is a little bit frustrating.
When we get it escalated, when we're working with their research and development or with the customer service contacts that we have, they're much more amenable to our requests. They like listening to what we need and the type of support that we're requesting. They take us pretty seriously when we escalate and have high priority issues, and they really try to get us resolutions as fast as they can. We definitely appreciate that.
Which solution did I use previously and why did I switch?
We have ALM implemented and we're still using AGM. We are making the switch to Octane because we implemented AGM with integration to ALM so that we could have Agile project management and a manual testing tool for our project teams. The nice thing about Octane is that it doesn't require integration. Integration always introduces a potential for points of failure. If you can house those capabilities within a single tool, why not go in that direction? It provides ease of use, less maintenance, etc.
Also, this is the direction the vendor is going in. Several years ago, our organization made the decision to go with HPE, now Micro Focus, for the majority of our suite of enterprise tools. We're following the direction that the vendor is going, but also recognizing that there are advantages to the tool that has good capabilities. We're not blindly following them, we're doing our assessments and saying, "Hey, this looks like a good thing for us." Of course, if it requires fewer tools, less maintenance, less setup, we'll go in that direction. That's how we made the decision to go with Octane.
There are other things that we haven't deployed yet, but the advantages of the direction they're going in for integration into the DevOps world to support CI and CD, that's a direction we want to go in. I'm on the Agile solution team, but we have the testing solution team, so they're very interested in those types of capabilities as well. Octane is opening that door for us to get more and more functionality hubbed in a single tool.
How was the initial setup?
I don't do the software installation or that side of things, but in terms of our implementation strategy, we have four environments in which there are seven servers. In our lower environments, our base environment, we have one server that gets installed.
We don't have any integration that we support currently, so it's a standalone environment. We do integrate into an Elasticsearch farm, as well as to LDAP for user creation, password validation, etc. We have those basic types of integration setup, but we don't have integration to other tools such as DevOps tools, yet. We are currently working on integration to PPM, and that's going to be deployed in the next couple of weeks.
Once we get up to our stage and production environments there are multiple servers on a load-balancer, so that adds an extra degree of complexity to the setup. They're also externally exposed to the internet so that our clients and external users can have access to the tools.
What about the implementation team?
It was just us and Micro Focus.
Which other solutions did I evaluate?
We did not evaluate other options before choosing Octane. At that point, we were in pretty deep with HPE. But before we chose HPE as our vendor for the bulk of our enterprise tools, we did an evaluation of different vendors, different suites of enterprise tools that we could possibly host, before we made that decision to go with ALM and AGM and UFT.
What other advice do I have?
The way that we approach it is that we don't rush into a decision and say, "This is the tool that we have to use." One thing that's nice is that there's always an option for a SaaS trial for 30 days or 60 days. Micro Focus has been very kind to us and given us extensions on our trial versions so that we would have enough time to evaluate the tool and the SaaS version before we make a full, educated decision about how we want to move forward. That's a good place to start: Plan on getting a trial version and plan out your assessment, what your objectives are, what your requirements are for the tool, and then just get in there and start using it.
I use Octane in my day-to-day work, but I'm mostly an administrator of the tool's usage on our consulting projects.
With respect to how tools and processes are evolving to adapt to the change from traditional Waterfall, one of the things our organization is finding is that it's not a switch that you turn on - that you're "traditional" one day and you're "Agile" the next. So, having tools that are flexible enough to accept variability, and that are flexible enough to adjust to project teams transitioning and becoming more Agile as they go along, is important. Octane, because of some of the additional features that are there and that are not in some of the other Agile tools we've looked at - like the Quality module, the quality story, the ability to customize workflow and business rules, and also having the Requirements module - lets you still be a little bit traditional when you need to be, while you're learning to become more Agile. There's some transitioning that the flexibility in Octane lets you do, where other tools might be more rigid in enforcing pure Agile project management.
As for lessons we've learned about adapting tools and processes for Agile, I feel that's very similar to what I just said. It's this journey that people are on. Where we started was with very traditional project management tools and, as Agile became more the trend, we recognized the need to add more tools into our landscape that would support it better.
The way that we work is that, while we host all of these enterprise tools, we don't enforce that these are the tools that are to be used on projects. We have to be a bit more flexible than that. Recognizing the need to have enterprise tools available for project teams that couldn't find their own tools, or clients that didn't have their own tools, that's where we brought in AGM and then, eventually, Octane when it came onto the market.
The other thing that's helpful to recognize with this transition, is that you can't become Agile on day one, once you make the decision that's the direction you want to go. It's very good that we have the ability to integrate our more traditional project management tool with our Agile tool. Currently what we support for project teams that are doing a bit of both, what we used to call "hybrid," is their integrating of their Agile project management tool, like AGM or Octane into a traditional workplan tool like PPM so that they can see the full breadth of their project progress across both more traditional tasks and Agile tasks in a single place. We're bridging that gap by using multiple tools and integration.
In general, ALM tools help in the transition from Waterfall to Agile because you have a tool enforces some processes, and provides a little bit more rigor than you would have otherwise. Having those ALM tools available has helped us enforce some consistency and adherence to Agile processes.
To date, we've had 136 projects, that's 136 workspaces, and about 1,000 users.
In terms of increasing usage of Octane, we deployed AGM and ALM four or five years ago. The problem in our organization - and this is another thing we've talked to Micro Focus about, and they're hearing similar feedback from other places - is that people are used to what they know. If people have used AGM or ALM on a previous project, they're just going to go with that.
We do have some early adopters. People have been keen, they've heard about this new tool Octane, checked it out, and those early-adopter types were on the bandwagon pretty soon. There are some people that are lagging behind and kind of skeptical. We're dealing with the psychology of that. Part of that is knowing there is not really a great reason for us to continue supporting two tools that do very similar things. AGM and Octane have a lot of overlapping capabilities. We're looking at our strategy for how long we want to continue to maintain and support two tools that do the same thing.
We're trying to encourage people who are used to using AGM, or are leaning more that way, that they should come over onto the Octane side, because that is the direction that the vendor is going in. That's where the investment is going, and that's where all the new functionality is coming out. We're trying to increase adoption in a variety of ways to get those people onto the Octane side.
We have an assessment planned early next year to strategize when we might scale down AGM, and maybe even cut off provisioning new projects, but we don't know the timeframe of that yet.
In terms of maintenance of Octane, their roles are project manager-types, people who do the server administration, and DBAs. There's also a QA group and a PMT group that we enlist on a very short, annual basis to do our performance testing.
I would rate Octane at seven out of ten. There's definitely some functionality that I think could make our lives a lot easier, especially around the extraction of data and the reporting. Those things would really help us out. I'm conservative on rating things.