Automic Continuous Delivery Director Review

Enables us to easily design a pipeline to promote applications from development to production

What is our primary use case?

Application software delivery is the primary use case.

What is most valuable?

There are many valuable features. For me, it's the fact that you can easily design a pipeline to promote applications from a development environment up to a production environment, and the team can become autonomous in designing those pipelines. In an Agile organization where you are searching for such autonomy, having that kind of capability is very valuable.

The second valuable aspect is its capability to drive external systems like deployment automation engines or to integrate with Agile Central.

What needs improvement?

What we would like to know is how to use CDD in Agile, at scale, in our organization. We have rolled out the SAFe model, but what we would like to have is better integration with Agile Central, for instance, or at least at the plugin level, where we would select only certain stories, instead of many stories, in the sprint.

We would like to have a more user-friendly interface. It is already very friendly, but as soon as you start to have many applications with many tasks, the applications should be easier to manipulate on the screen.

We would like to have export capabilities to paper because, for auditing purposes, we need to keep them.

We would like to add some documentation inside the tool as well.

This is information we have already communicated to the development team of CDD and they will work on it. 

In addition, there is the testing aspect. It's a strong need now in the market: Not only to deploy/promote an application into whatever environment, but also to be able to execute testing and report on the quality of the application.

For how long have I used the solution?

One to three years.

What do I think about the stability of the solution?

Astonishingly, it's quite stable. The quality of the application is pretty good. We haven't had bugs or crashes. It seems to sustain the load we give it. It's working. We haven't had any problems with it.

What do I think about the scalability of the solution?

Scalability is a question for me. You can put more load on it, but the real question is not about the tool itself, it's about the all the new practices in continuous delivery in an Agile, at-scale organization where the basic principles are not yet embedded in the organization. So tomorrow I could select a tool from a competitive company, for instance, XL Relieve or XL Deploy, but I will have exactly the same problems: "How do I define what I put in my delivery/release pipeline?"

There is no secret recipe for that. You have to work with your customer and adapt their way of working to benefit from the tool. Some teams are more agile and are more autonomous than the others and fit perfectly well inside CDD. But if you take mainframe applications, most of the time they are big, monolithic applications, and it is the applications which are a problem to fit inside CDD. In that case, you need to be able to work with those users and find solutions so that continuous delivery brings value. Otherwise, you just have deployment automation which is of minimal value.

So the tool can scale, but it cannot scale faster than the way you change your organization.

How is customer service and technical support?

I like technical support very much because I've been in contact with the development team from the very beginning. I'm in a somewhat special situation, but they have been helpful from the very start. They were eager to listen to our change requests.

But I also think that the development of the tool should go faster. The market is moving very fast, and I think it's very important that CA realize what its customers are asking. I need to push CA to get my new requests or improvements because my own customers are requesting them. 

They're eager to listen, to propose, to understand. They deliver it. But now it has to go faster. Otherwise, people will say, "Okay fine, the tool is there, but we want to see it evolving."

It might be that due to the recent acquisition by Broadcom it might have slowed down. Sometimes my users sometimes are wondering will the tool stay or will they have to change tools? They see other tools coming on the market, so the competition is starting to be there. There will be more pressure on the CDD team to deliver newer versions. They do release multiple times a year. That's very fine, but I think they should release even more often. That's probably why they are going to the SaaS version as well.

How was the initial setup?

It's very straightforward, technically, to deploy the tool. To use it at a small scale it works. The problem is when you go Agile at scale, you encounter problems with who owns a release. Do we give autonomy to squads or should it be the tribes that have autonomy? Then the question is, what do you put inside those pipelines? Do you want to work in project mode, or do you want to work in a more product-oriented mode?

When you introduce a tool like CDD, technically it's easy to run, very easy, even if you run with containers or run the SaaS version. It's very easy to integrate it. But it's the cultural shift which is much more difficult to achieve, and this is proportional to the size of an organization and the way it works. I'm in a context here where we have a very large organization and our Agile process is not that mature. Therefore, we still work in a pure ITIL way, very Waterfall.

When you introduce a tool like CDD, you tend to shift left. You tend to give more agility and, therefore, people need to be trained. All of that has nothing to do with the tool itself. It is more about the cultural shift, as well as achieving continuous delivery in an organization that was strictly controlled by a release process which was extremely manual. You encounter a lot of resistance because you replace jobs with automation. It provides greater flexibility for certain teams, but you are really changing the organization. 

It brings a lot of value when you look at the organization completely; what it can change and make faster and better. But the people have to change their way of working, their culture. They have to trust more, they have to change their roles as well. This is the most difficult part when you use a tool like CDD. You could use whatever other release automation tools or continuous delivery tools and you will encounter exactly the same problems. The problems are not related to the tool itself but more to the concept behind the tool.

In terms of deploying CDD, they provide a containerized version or you run it on your own application server. In our case, within a month we had the environments provisioned: Dev, QA, and production, and everything was set up and integrated.

Our implementation strategy was, first of all, to do a PoC to check if what we were expecting from this type of tool - meaning integration, its capability to provide autonomy and visibility to teams - was as expected. Then we did a deployment in a Dev and QA environment, where we re-integrated it with external systems in the organization and then we delivered it to production, into the test centers. First of all, we were working in the Active-Passive mode, but now we are moving to an Active-Active mode in production.

We had coaches coaching squads to use the tool. What we see now is that we have to re-engineer the release process to stop working like before, where teams were only delivering up to QA. To make things more autonomous, delivery should be up to production. That means enabling a DevOps strategy, which is something that has to be taken into account in an Agile strategy as well.

That's where we are right now. Some teams have become autonomous and use the tool without our supervision. They roll it up to production. They see benefits because they no longer have to follow "run books." Run Books were the set of stuff they had to execute which were contained in Excel sheets. Everything is tracked inside the tool. And when they introduce automation behind the scene, what we see is that everything is reproducible from very environment - Dev, QA, and Production. And it happens just by clicking the button on the screen. That works. It fosters collaboration between the operational team and the development team which, before, was broken or hidden behind a wall.

To deploy CDD we requested our infrastructure, we got unique servers, we got databases, etc. We had two employees delivering CDD, and now those two people are running the platform. But they do many things in addition. They are DevOps engineers.

What about the implementation team?

I am from a consulting company and I provide the integration and the rollout services for CDD, Agile Central, and Release Automation.

Which other solutions did I evaluate?

I looked at other solutions from other companies. At that time, CA was not ready with its solution. Luckily, we were stuck in the decision-making process with budgeting, etc. Then, CA came out with the early CDD. It was quite promising so they put a lot of effort into being on time. They came out with a first version which fit our initial purposes. That's why we went to CDD. 

It was very lightweight to install. Other tools needed a lot of expertise and infrastructure. But now, other tools are known in the market. I think we made the right choice but those competitors also make strong arguments to be present in big enterprises. Again, it is not the tool which makes the Agile way of working work, it is the people using the tools.

We looked at solutions from IBM, Microsoft and, recently, solutions from the company that created XL Release. It is interesting, but for me, it is still the same as CDD.

What other advice do I have?

As I said above, a tool is just a tool. What you need to do is more along the lines of thinking about how to roll out usage of the tool inside your organization. Rolling it out for a couple of teams is very easy. Rolling it out for people who are eager to change their way of working is very easy as well. If you work in an organization that does not have a long history with hundreds and hundreds of applications and rules and policies to follow, it usually means the employees are free to work in their own way. We benefit a lot from that. But as soon as you need to go Agile and respect regulations, when you have a huge legacy behind you, the tool will be there and will be used, but you first have to break that big monolith and change the way of working. Then you will benefit from using the tool.

For continuous testing, we are implementing a solution based on CDD and external tools because we want to deploy an application inside an environment, test it, and automatically promote it to the next one if the quality is met. Unfortunately, today in CDD, we don't have that capacity to integrate the testing yet. I know there is a Dataset Repository coming, but I have never used it.

We use CDD for the continuous delivery pipeline and we integrate it with a continuous integration platform like Jenkins. It's needed because we want to be able to deliver faster and with better quality. We want the developers to know the changes they are making in the applications or in the product that they're running in production. They should know all the changes and how to test it with the testers. In a DevOps organization, you want everything on the same page and, in the pipeline, you want to have visibility into what's going on, what is delivered where, and what the quality is. That way, when you deploy up to production, it's kind of a non-event. You are sure that it will work if you have previously deployed everything properly in QA.

In terms of tracking user stories and fixes within the release pipeline, we have demonstrated these capabilities in a PoC and in some test environments, but not yet in a production environment. To have it properly tracked, your organization needs to have enough maturity in its Agile processes. We are currently running out "Agilification" at scale, and therefore we need to have the proper squad and tribe organization. We need to properly size and design each feature and story and have the proper state. Currently, for instance, people are not in agreement on what an accepted state for a story is, or what an accepted state for a feature is. Therefore, we have the capability in the tool - we demonstrated it - but today the business still has difficulties tracking the progress of stories and features in Agile Central. Therefore, providing automation in CDD, where we update both systems with CDD, seems too early. But our aim is to do it. We have demonstrated it, we have run it in PoC and it works. But the question is, how can we scale the Agile way of working to an organization of 2,000 people? This is the pain point today.

Getting there would be the most important business value because then, everyone - even the business - could track the delivery of business value. That's the plus of the tool. If it could integrate the testing, and by that I mean the quality aspect, it would mean that the product owners and business could release autonomously and have a complete view into the quality of what they release. Then, CDD would become an orchestrator, just executing the delivery of a pipeline. But the goal is to reduce all manual intervention to the minimum. Basically, we would shift-left all activities and control to the business, so that they could track and act on the delivery of that business value.

We are delivering something like 140 applications and that needs to increase to more than 350. We have 25 groups or squads of users. Each squad can have between four and eight people, sometimes twelve. So we have something like 150 users. But we don't manage users individually, we manage groups of users.

The majority of the teams are DevOps teams, but we still have transversal teams which need to become more DevOps. For that, we have to break the way they work, to split those teams and move them into squads. But then the question comes up about the ownership of what previously belonged to the transversal team. It has to move to the business squad or the product squad. The definition of those products goes alongside the change in the organization. We support both models and we roll out CDD at the same time we roll out the Agilification.

We even use CDD to deploy its own plugin. We have multiple instances of CDD, one in Dev, one in QA, and one in Production. We use the Production instance to deploy plugins that are a component of CDD, into our own CDD. In our squad, we also have other applications and we used the same CDD to deploy those same applications.

From a customer point of view, the customer comes with a new application. We look at the technology, we automate the technology to deploy that application. It could be a WebSphere application or it could be an Oracle database, etc. We train them on how to use CDD and give them the autonomy to deliver their application, given that we have automated the deployment task and integrated it with the test automation team. That team then provides the test automation capabilities, such as running a performance test or running a load test. That is not our specialty. Our specialty is orchestrating the delivery and deployment and to trigger tests.

I know other organizations that have deployed the solution here in Belgium, and we often meet to discuss our progress. Because, like I said, deploying the tool is easy, but using it properly to benefit an Agile-at-scale organization is difficult. It's difficult to "Agile-ify" at scale. We discuss how to do that properly, and how to benefit from the tools, whether it's CDD or Release Automation. They often go together. In addition, customers might have JIRA or Agile Central. I'm lucky, I'm using the three: Agile Central, CDD, and Release Automation. They are quite well integrated.

Compared to what I have seen on the market, I would round it up to a nine out of ten. It's the simplicity. It's very simple to use and deploy and deliver. And it works.

The reason it doesn't get a ten is that you need knowledge about what you are doing, about the shift in control and responsibilities. People get autonomous and you then have resistance. I don't think it's due to the tool. But it could still be improved maybe on those aspects, by providing more controls or permissions and the like. And the other thing that is missing is the test automation or tracking. But I know it's coming.

**Disclosure: IT Central Station contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor. The reviewer's company has a business relationship with this vendor other than being a customer: Consultant.
Add a Comment