If you were talking to someone whose organization is considering erwin Data Intelligence (DI) for Data Governance, what would you say?
How would you rate it and why? Any other tips or advice?
erwin is very good for companies who have a very structured, data governance process. It puts every possible tool around a company's data. This is very good for companies who are mature with their data. However, if a company is just looking for a tool to showcase their data in a data catalog, then I would advise companies to be careful because erwin is sometimes really complex to master and configure. Once it is set up, you have to put your hands in the gears of the software to model how your data works. It is more of a company process modeler than a directory of all data available you need and can access. Industrial companies over 30 to 40 years in age are struggling to find what data they may have and it may prove to be difficult for them to use erwin directly. What we have done with the lineage is valuable, but manual. For the IT dictionary, automation is possible. However, we have not installed the plugin that allows us to do this. Right now, all the data that we have configured for the lineage has been inputted by hand. I would rate this feature as average. We have not tested the automation. I would rate this solution as seven (out of 10) since we have not experienced all the functionalities of the product yet.
If you have the ability to pull a steering committee together to talk about how your data asset metadata needs to be used in different processes or how you can connect it into mission-critical business processes so you slowly change the culture, because erwin DI is just part of the processes, that probably would be a smoother transition than what I am trying to do. I'm sitting in an office by myself trying to push it out. If I had a steering committee to help market or move it into different processes, this would be easier. Along the same lines as setting up an erwin Workgroup environment, you need to be thoughtful about how you are going to name things. You can set up catalogs and collection points for all your physical data, for instance. We had to think about if we did it by server, then every time we moved a server name, we'd have to change everything. You have to be a little careful and thoughtful about how you want to do the collections because you don't want the collection names to change every time you're changing something physically. What we did is I set up a more logical collection, so crossing all the servers. The following going into different catalogs: * The analytics reporting data sets * The business-purchased applications * External data sets * The custom applications. I'm collecting the physical metadata, and they can change that and update it. However, the structure of how I am keeping the data available for people searching for it is more logically-focused. You can update it. However, once people get used to looking in a library using the Dewey Decimal System, they don't understand if all of a sudden you reorganize by author name. So, you have to think a bit down the road as to what is going to be stable into the future. Because the more people start to get accustomed to it being organized a certain way, they're not going to understand if all of a sudden you pull the rug out from under them. I'm going to give the solution an eight (out of 10) because I'm really happy with what I've been able to do so far. The more that the community uses this tool, the more feedback they will get, and the better it will become.
Our first goals were data literacy and data as an asset. Those were our two big, ultimate goals three years ago. Data literacy turned out to be 10 times more important than a data warehouse. We could look at existing data sets and, just by educating people, it gave them an advantage almost immediately. The fact that the data governance was able to put a framework around data literacy helped us focus on the right answer, even if it wasn't the first one given. In other words, sometimes we'd have the same answer three or four times, and it would shift until we nailed it. But without governance, we would never have done that and we would have stayed the same. The secret to the success of this project was that we had a vision and we stuck to it. Governance was important to us, no matter how other people might have thought about it. In my very first data steward meeting I was introducing everybody to these brand-new terms they'd never seen, and someone in our analytical group totally derailed the meeting. So be aware that it's not going to be easy, but have a vision. And make sure that governance is important or don't bother. It's not something that a lot of people add value to at all. When you say, "Oh, I want governance so we can have a data dictionary and you can go look at it," they'll say, "I don't want to look at it, just give me a report." But the ability for those who need to do that is huge. Have a vision and stick to it and be willing to take a step back, sometimes, to go two forward. The neat thing is that we've pretty much done all of this with two to three people, for our entire organization. We do have three data teams that are using the Modeler for development — ETL SSIS stuff — but we have a pretty serious "wash, rinse, repeat" standard. If anything is in doubt, we just go back to the business rules and see what our rules are. What are our principles, and are we meeting them? As far as automating the changes through the environments, it has helped, but not a lot. It's not like it was a silver bullet. We need help there, because there's so much. There's the model, but once you promote that in different environments, sometimes you miss it because you only get three or four days to get out of QA to get it into stage. Obviously, you mitigate risks with automation. It does have an impact. As a company, we just haven't been able to take full advantage of it now, but that's our hope. We're only into it for about a year-and-a-half, even though we have run with the suite for almost three years. We're still immature. I wish I had everything at the push of one button, the "Easy" button. Some of it's over our heads. We could use some new training and we could use some additional support. erwin has been great with us, but it's also a matter of the appetite and the resources. The biggest issue is that I don't have a team of people doing what a team of people need to do to accomplish what we would like to. It's done by a small number of people on a consistent basis, and not full-time. The solution's generation of production code through automated code engineering would reduce the time it takes to go from initial concept to implementation, but we're a Microsoft shop and most of all that is done inside TFS or Visual Studio. That's how we manage all our codebase, including release management. That's all done separately and is automated. We're trying to create some interfaces between the two. We just haven't gotten there. In my three years using erwin, besides actually getting approval for the money to purchase the software, I don't think I've had a struggle with it. They've been great. When we first got on and we had some questions, they got me to the development team in England and set it all up with us without question, no extras. They just tried to make sure it worked. I would rate erwin DI for Data Governance at eight out of 10. I never give a 10 because I have yet to see perfection. It has some gaps, but I definitely think it's in the top third. As far as rates go, I don't have a lot to compare it with. It's easy now but it took going through a learning curve, but that's the case with any software. Does it need to mature a little? Possibly. But that would be it. With their roadmap, they're buying companies, and changing things, and doing things. I've been pleased.
I learned how to automate in the data area and how this is very different from any CI/CD development platforms that I was working on before. I learned that we need totally different things to automate properly in the data area. We need very accurate metadata. We need precise mappings reviewed by different data stakeholders. I would rate this product as an eight (out of 10). I can imagine some capabilities for this product that would make it even better.