What is our primary use case?
It's hard to say what our primary use case is. There are so many new changes and options. From the perspective of the general organization, it's the shared space/Workspace concept that is of major interest. From the architecture side, with Elastic and Oracle storage, it gives us fast filtering and it's fast at finding information. On the process side, it gives us the option to control or organize workflow by user stories and tasks, within the different teams.
How has it helped my organization?
The most important thing is that we have, 8,000 users, more or less, and if you could save each user who's working with our current ALM tool, just in opening drop-down menus and searching for information, just ten minutes a day, it's a huge improvement. The concept of Octane is to have the most information and the most important functions available with one or two clicks. This is a big point of savings, in time and money.
The powerful widgets, the Dashboard module, the ability to drill down to the information you need and the ability to configure it to your needs, are big advantages as well, and one of the biggest pain points in ALM. In the latter, you would need to install different tools. In our company, everybody just exported everything to Excel and reported everything in Excel. That was a pain. The possibilities of the Dashboard module will save another couple of thousand Euros, because of all of the time and the small developments that were required around ALM will not be necessary anymore because everything is supported by one tool.
The general capabilities of Octane, setting up rules instead of programming VBA scripts for controlling the workflow, and making life easier for the users with the template functionalities, are very big benefits.
In the old-world ALM, it was hard to adapt the tool as a central point, into a complete toolchain. The requirements of our industry, the automotive industry, of a tool like ALM or Octane are a bit more complex rather than just software development. We are using a complete toolchain. We are using a toolchain of about 70 interfaces around the old ALM. With Octane, we can kick out a lot of those tools, although there will still be some 25 to 30 tools which are connecting different databases, loading information into Octane or from Octane, etc. The openness and the possibilities of the REST API, from an operations and maintenance perspective, are also a big plus.
What is most valuable?
It's the tool with the best UI and lots of options. I could continue for hours.
What needs improvement?
We are the lucky guys as we have a type of "design partnership" with Micro Focus so I do have a call once a week with R&D, discussing problems from a user's perspective and from an operating maintenance perspective. We do have a close collaboration on improving Octane for the needs of the automotive industry. We are contributing a lot of features, thoughts, and ideas, and Micro Focus is very good at implementing many of them and improving their tools. They are getting feedback from a couple of hundred users working in the early phase of transition now: How users are thinking, how they are using or abusing Octane, and what could be improved for the benefit of both companies.
An example of one of the features we have requested is inheriting information from a test suite into a suite run and into a menu run, so the user does not have to add that information, update it manually.
Other examples of areas in which we're looking for improvement are the UX, how it looks and feels, new widgets which make life easier. There are various things, all over the tool. We're also looking for REST API enhancements because our IT guys are trying to add this and that by REST and it takes a long time and the response is low. Also, the error messages that might pop up if you're doing something by REST, they could be better.
Micro Focus is paying attention and working on things actively. I've never seen a design partnership like this between companies. For both sides, it's very intense, it's very interesting, and it's a huge benefit for both sides.
For how long have I used the solution?
One to three years.
What do I think about the stability of the solution?
The stability is very good. It's surprising, even with so many new features and functionalities built-in, every three to four months. There are nearly no error messages popping up, and if one does, it's mainly because of our own configuration of the rules and permissions. It's surprisingly infrequent that you get error messages because something in the tool, in the background, was not working.
What do I think about the scalability of the solution?
The scalability is pretty good. We did some load tests in advance, before making the decision, pro or con, on Octane. With Octane we are in a very good position. We expect to have some 5,000 users by the end of 2019/beginning of 2020 working on Octane. I believe it can go up to 2,000 users working on Octane in parallel. We hope to be powerful enough, with the architecture and everything else we set up, to meet users' expectations of performance.
How are customer service and technical support?
In general, tech support has improved a lot in the past couple of years. Since we do have a close collaboration, it's very special being in contact with a couple of key developers, or the architects of the tool, by e-mail, so it's a very short process. I am part of a small circle of people who are allowed to email problems directly, without using any tools or going through first, second, or third-level support. We are allowed to write emails directly to the developers to discuss problems and ask for enhancement requests.
But even the general process to report bugs or request changes has improved a lot during the past couple of years. They're much faster, more responsive.
Which solution did I use previously and why did I switch?
We are on ALM.NET version 12.53. All of the cars that are in development in our company and which are due to be launched within the next two years, are still being worked on - and will continue to be worked on - in ALM, because the data migration scenarios are too complex in our organization to do a big-bang migration from ALM to Octane.
Our strategy here is to start with a new Service Pack (that is a type of operating system for the car). For new models, which will hit the market in two-and-a-half to three years with Octane, it will be a smooth transition. We are onboarding the users who will start working on that new operating system in two months. We will make plans at the end of 2019 or the beginning of 2020 for which projects in ALM need to be migrated to Octane, to be able to shut down ALM a little bit faster. Otherwise, we would need to run this ALM.NET instance for an additional five to six years, until all the existing developments platforms are finished.
We knew we needed to go with Octane for a couple of reasons. From the business side, there are some requests which ALM could not cover, in terms of data storage, etc. Since we have so many test cases, and given how Versioning and base-lining work in ALM, it would require so much storage that we are not using the Versioning in ALM right now. This was one of the biggest pain points, from an ISO perspective in terms of testing. Also, operation maintenance is hard. I am running the biggest ALM.NET instance in the world, according to Micro Focus. We have the most users, the most data, and the most complex VBA workflow code in a single instance of ALM. And this needs to be migrated smoothly to Octane.
In addition, Internet Explorer, which is not the finest tool, will be removed from our company's IT at the beginning of 2020. So there are a lot of smaller reasons which lead to the need to change to a different tool, to a more flexible tool, to a more powerful tool. For example, the Autonomous Driving guys will add tons of test cases and automated tests, which would cause ALM, in the configuration we are in right now, to crash in two to three years. That's clear, and it was clear to our management as well. To change to a different system, we have only a small time window: when starting with a new Service Pack, like the operating system. That's the phase we're in right now.
How was the initial setup?
For us, it's a very complex setup right now. It's not only setting up a server, installing Octane, and doing configurations. Our plan, within the next two to three months, is to have a shared Workspace concept with six or seven Workspaces. We do have a major challenge to do all the configuration stuff, defining methods and processes. We also have to connect at least ten major tools or databases, which are synchronizing information into Octane, or which are used for the special methods of test planning and test automation; pulling information from Octane and running them on our test benches in semi-automated cars.
That's a very complex process but we're making progress fighting some small problems and some bigger problems. We've found solutions for all problems.
Because we have some 70 to 80 suppliers which have an automated defects exchange, our development, our testers, are reporting defects and those defects are exchanged with those 70 to 80 suppliers. So it's a very complex situation we are in.
What's my experience with pricing, setup cost, and licensing?
I am not part of the cost negotiations. But the purchasing guys confirmed that Mirco Focus offered the best pricing to us.
Which other solutions did I evaluate?
We started analyzing which tool we should use for the future two years ago or so, and we started digging into Octane very early. I work on Octane from a test perspective: What the tool is capable of doing, and how we can adapt our processes to the new options we have with Octane.
Options we considered include JIRA, which is used by a lot of small teams. There is a standard toolchain around JIRA, but JIRA, with the amount of data and information we need to run, was very easy to strike from the list. We dug into IBM and PTC Tool Chain. We looked into ALM Octane vs codeBeamer from Intland. We did some comparisons but ended up with Octane.
With JIRA and the toolchain, you cannot run a business like we are doing with car development, using a lot of plugins to get the needed functionality into one tool. You have so many small companies providing those plugins. What if the company providing one of the tools or plugins you're using collapses or doesn't support a new JIRA version? So this was not an option.
PTC is also using different tools to support the full functionality of requirements, test cases, team management, the backlogs, tests and defects, and so on.
The biggest competitor to Octane was codeBeamer, but from our perspective, Octane did much better in performance. Octane is not able to do as many relations between the data objects as codeBeamer, but performance was the key factor for us.
We are now setting the foundation for the next eight to ten years. We had to have that in mind, as well as the increase in data, the increase in users, the increase in data objects and rules, and the complexity of development which is dividing into different pieces. To cover all this in one single tool led us to Octane. Cost and license fees were also a big issue. The two solutions, codeBeamer and Octane, weren't that far apart but, here as well, Octane was first.
Finally, with the benefit of this design partnership, which is unique - and we are very proud of this relationship with Micro Focus - it's a plus for us to be able to influence the tool and get features, especially within the timeline that we need them. That's one of the biggest advantages.
What other advice do I have?
Do a quick scan of tools in the market and dig into your needs. Especially for a project with a lot of users with different styles of working together, Octane is the best tool because for the shared space/Workspace concept. Management is able to get a total overview of all the projects or workspaces and the teams are able to operate in their particular styles. That would be my advice.
For small teams, there might be different solutions that are cheaper, for example, JIRA, and which are more flexible. But if you need to run bigger businesses, Octane is the best because it's replacing a whole toolchain.
We are in the midst of transitioning from ALM to Octane for a all users working on the new model This includes now all parts of the company. We startet with a pilot user group of about 50 to 75 people which has already been in production on ALM Octane since June 2018. Now, March 2019, we do have 1700+ users working on Octane. My role in the company is as Product VP on the business side, so it includes defining the new working processes, how the users should work in ALM Octane, and defining all the transition stuff, etc.
We're transitioning to implementing Agile methodology in DevOps. It's a little more complex when trying to build cars because you have these little rubber and metal parts which constantly refuse to be agile and digital. But from the whole methodology, we recognize we have to be much faster than in the past and we are trying to combine both worlds. We are in a transition to what we call "Water Scrums." We cannot let go of all old Waterfall processes but we are trying to adapt as many Agile processes as possible. It will take two, three, four years to come to a stable, new process of for working with a more Agile methodology. We are digging into that and trying to adapt as fast as possible.
For example, we have a control unit which runs the software which is built into cars, and producing a control unit takes time. It requires planning and we have different teams contributing software which is running on that single processor. To combine them, it makes for more complex planning, rather than just software development. Think about the fact that you have functions in the car for the customer, like a navigation system for example, and that system is getting information from the engine, from the drivetrain, etc., so you have the complexity that different teams are working on different parts of the software on that control unit.
In terms of lessons learned in this process, the benefit is that we try to be much faster and work in smaller iterations, delivering new functionalities faster. But we always have the limitation of the physical materials which are not agile: as an example, the welding and producing of the machinery which produces the outside of a door into which you can then plug in a window, and the window needs to be of made from glass and rubber, etc. The metalwork would take 25 to 30 weeks until you get the new mold or form for the metal sheet, while software can be delivered once or twice a day. We have to combine those worlds and that is the hard part. But there is also a big chance to make a lot of partial improvements.
As for ALM tools helping with transitioning to Agile, Octane is the much better tool and we need to find a good way, supported by Micro Focus, to move data such as test cases, requirements, runs, and so on from the old ALM to Octane; to change to Octane to get the most benefit and the fastest benefit from the whole tool. Car development is a process of two or three years at least, and sometimes as much as five, for a model to hit the market, and for data migration that's a big issue. We need to dig into that, and the plans are not finalized yet.
For deployment and maintenance of the tool, there was a major team of experienced IT guys and process guys from our side, about 25 people, supported by about 60 other people just for the special processes of the different development departments. We call them "key users." They are collecting information and reporting it to the core team. For maintenance, it's a team of six people who are implementing changes requested by the core team. Depending on the workload, on average, maintenance is done by three people. There are numerous software developers working on the interface tools, perhaps some 30 IT guys working on the different tools we need to launch with Octane.
After deployment, I expect we will need two to three people to maintain it, depending on how good the tools are. For example, we use a known tool where users can request accounts on Octane, the roles they want to have, and there are guys from the business side who approve the requests. If this tool is working, you can do the onboarding, get the users' credentials into Octane, by a script, so there won't be any work for the IT guys. Right now, we need one person for at least two hours a day to add users to the different tests and integration instances and to the production servers.
Right now, Octane is at about an eight out of ten because, from our perspective, there are a couple of functions still missing. As mentioned, we are in close contact with R&D and they will implement those functions within the next year. Then, I would say, we will be at a nine, close to ten.
One of the biggest benefits is not having to wait another one-and-a-half to two years until the next major release of ALM. Even selling the weak points to the users is much easier because we can say they will be fixed in half a year or in three-quarters of a year, instead of two years, if that.