Last November, our Chief Innovation Officer Malcolm Chisholm shared his thoughts on the evolution of data governance on our blog (Agile Data Governance: A Modern Approach for Self-Service Analytics). He also presented a webinar with one of our strategic partners in the areas of governance and metadata management, Alation.
With the blog post and webinar continuing to garner interest, we’re revisiting this topic with a focus on the critical role the data catalog plays on enabling a more modern approach to governance.
What is Agile Data Governance?
In recent years, a new phase of data governance has emerged: agile data governance or what some call Data Governance 3.0. It’s a bottom-up, data democratized approach fueled by technological maturity and increasing concerns and challenges about data quality, data privacy and data-related regulations.
It’s governance that unlocks the value of data for end users, empowering them to solve business problems, take advantage of self-service analytics and do their jobs better and more efficiently.
If anyone, anywhere, at any time in the enterprise must be supported in terms of their need to utilize or manage data, then information and guidance about the data must be immediately available to them.
Only an enterprise-class tool such as a data catalog enables a modern, self-service approach that is flexible enough to quickly and successfully satisfy the needs of individuals across the organization to use and manage data.
Shifting the Governance Paradigm
Agile data governance supports data usage and management across the organization but makes everyone responsible for contributing to the understanding of data. This “social contract” of empowerment with responsibility is seen in other aspects of agile data governance, too.
In the dynamic landscape of data privacy, the needs and requirements are robust and must be protected, yet the traditional Central Data Governance Office can’t become a bureaucratic bottleneck. How can these two needs be reconciled? One answer is to ensure the organization’s central reference point for all things data, the data catalog, makes potential users of personal information (PI) aware of what they can and can’t do with this data — and expect the users to act accordingly.
This type of an approach is more efficient than users submitting requests to the central governance area each time they want to do something with PI. Again, this is a reflection of a social contract balancing empowerment with responsibility by using the data catalog to publish guardrails that will guide business users.
Agile data requires a cultural shift in how the organization functions within the new paradigm of agile data governance. People say they want the support and empowerment but may not be prepared to live up to the responsibilities that come with it. Sometimes, they may even want the Data Governance Office to have a “command and control” function so they don’t have to be worried about being blamed for anything.
In agile data governance, it’s the function of the Data Governance Office to drive the cultural shift to the new social contract around data, with the data catalog as the means of implementation.
Data Acquisition with Agile Governance
Agile data governance also plays a role in data acquisition (i.e., discovering, qualifying and obtaining external data). With agile governance, it’s not just about the “ingest” (the process to load the data).
Consider this scenario of an organization concerned about retail sales, which are influenced by weather patterns:
- A business unit wants to acquire weather-related data.
- A data scientist kicks off the process and makes the initial request.
- Once acquired, a data analyst maps the external dataset to internal data elements.
- Governance provides guardrails on what is permissible usage of the ingested data.
- Data Operations set up the ingest job.
- Throughout the ingest process, the organization’s data catalog allows stakeholders to interact with one another.
With agile data governance’s principles in place in this scenario, each person and business area know the role they play. This makes the data ingest more efficient than a traditional top-down, “command and control” process single-threading through a central unit.
Agile Governance’s Dependence on the Data Catalog
Agile Data Governance is a natural response to the increased demand from individuals across the organization to be able to use and manage the data they work with. The functions of Centralized Data Governance are not replaced by agile data governance — they’re transformed to deliver a new kind of governance program that emphasizes support and empowerment, but also individual accountability.
The data catalog, as an enterprise-wide platform rather than a local solution to specific needs, plays a key role in agile data governance and is now an indispensable component of any governance program.
