Data governance became mainstream around 2005 and has now become institutionalized in many organizations. Since it first became an enterprise-level activity, it has evolved to address emerging trends and concerns. With today’s big-data-fast reality, we are now at a point where a new generation of this practice is emerging: agile (or bottom-up) data governance.
In its initial phase, data governance (DG) was typically centered on a Data Governance Council with representatives from around the business who collectively decided what data issues needed to be addressed and then set up working groups for execution. That did not always succeed, particularly when council members did not have the time nor experience needed in DG. Often, they could articulate the organizational pain caused by data, but nothing more. Similarly, working groups were staffed by people who worked with data, but had no knowledge of governance — and, so, could not come up with viable solutions.
Centralized Data Governance
After a few years, a new type of governance emerged: centralized data governance. This iteration gained momentum when executives realized data could provide a competitive advantage — and that their competitors were thinking the same way, too. Data-targeted regulations were also starting to heat up, which executives took note of and decided to invest in centralized DG units staffed by specialists with enterprise-wide governance responsibilities.
The centralized governance model had its successes and challenges. On the success side, the model addressed, at least to some extent, rules of the road on how data should be managed. Also, tools came into the marketplace to support centralized governance, helping it to scale. And Data Governance became a horizontal function, like Human Resources, Finance or Facilities Management, dedicated to ensuring well-managed enterprise data resources.
But new challenges emerged after the initial successes of centralized DG which became familiar and, often, taken for granted. The challenges fell into two categories; the first was heavy, top-down data governance processes that imposed burdens on the business. For example, labeling people as Data Stewards and Data Owners and declaring the individuals could be blamed if things went wrong with the data they were accountable for. There was no clear, detailed identification of their obligations, no effort to determine the required level of effort, and no real support for how to carry out the obligations. In the worst cases, it was not clear what benefit (if any) the organization realized from the bureaucracy.
Self-Service Analytics
The second challenge category centered on the emerging tools and capabilities that unlocked the value of enterprise data. Self-service analytics, in particular, rapidly gained adoption because of technology support. However, the individuals who took advantage of self-service analytics believed they weren’t getting support from DG and became frustrated when they could not find data, did not understand it, were unsure of its quality, and did not know its origin.
Agile Data Governance
Now we are in the third wave of governance, which is gaining momentum: agile data governance. This governance is oriented to provide support much closer to the point where data is used, with a particular focus on self-service analytics. Agile data governance is supported by tools that help deliver knowledge about the data to the data users. These tools, typically called data catalogs, differ from the tools supporting centralized governance in that they have machine learning and/or artificial intelligence built into them. Of course, these tools also have manual aspects — but rather than being pre-specified processes, they are more social in nature. This gives everyone in the organization the ability to add their knowledge to the enterprise data resource.
As agile data governance continues to develop, we will likely see adoption move more quickly across the enterprise. An emphasis on support, re-engineered governance processes and new technologies will make for a better fit. We will also see agile DG continue to evolve, providing new capabilities with tremendous promise.
Agile Data Governance Webinar
In September, First San Francisco Partners announced a strategic partnership with Alation, the data catalog company.
In two weeks, I join Alation’s co-founder and product head Aaron Kalb on the Agile-Driven Data Governance: a Bottom-up Approach webinar. We will cover agile data governance methodologies and discuss why scalable and sustainable governance is at the heart of digital transformation. I hope you consider attending this timely program on Tuesday, November 27 at 10 a.m. Pacific and 1 p.m. Eastern.
Article contributed by Malcolm Chisholm. He brings more than 25 years’ experience in data management, having worked in a variety of sectors including finance, insurance, manufacturing, government, defense and intelligence, pharmaceuticals and retail. Malcolm’s deep experience spans specializations in data governance, master/reference data management, metadata engineering, business rules management/execution, data architecture and design, and the organization of enterprise information management.
