Data mesh is a hot topic and for a good reason. It addresses the shortfalls of the “modern” data architecture that rely on the centralization of data stores. It increases data owners’ decision-making power and flexibility by emphasizing the concept of “data as a product.” And it enables ultimate flexibility through a distributed architecture focused on self-service. A solid data governance capability is critical for success.
Despite the shift from data warehouses to data lakes (and next-generation data lakes), we still need to solve the problem of managing data at scale. Although data lakes solved volume challenges, they didn’t significantly improve user experience. Further complicating matters, organizations are hesitant to restructure their data landscapes after investing in data lakes due to the current state of the economy.
In response to these shortcomings, data mesh emerged as an architectural concept. Data mesh, according to data thought leader Zhamak Dehghani, consists of four principles:
- Domain-oriented decentralized data ownership and architecture
- Data as a product
- Self-serve data infrastructure as a platform
- Federated computational governance
The principles emphasize serving data instead of ingestion, discovering and using data instead of extracting and loading, publishing events as streams versus flowing data around via centralized pipelines, and creating an ecosystem of data products instead of a centralized platform.
Dehghani explains, “A data mesh is an infrastructure that enables the idea of data as a product. This inverts the current mental model from a centralized data lake to an ecosystem of data products that play nicely together, a data mesh.” Instead of a landscape of fragmented silos of inaccessible data, the data mesh platform is an intentionally designed distributed data architecture governed and standardized by a central authority, enabled by a shared and harmonized self-service data infrastructure.
In existing analytical data architectures, finding, understanding, trusting and, ultimately, using quality data can be difficult and costly. Many organizations have invested in a central data lake and a data team to drive their business based on data. However, after a few early wins, they notice the central data team often becomes a bottleneck. This is where the “mesh” of domain-centric solutions, pipelines and teams seeks to break the centralized paradigm and accelerate innovation and analytic effectiveness in a self-service model.
However, as the number of locations and teams that provide data-as-a-product increases, the problem only intensifies with data mesh. Dehghani herself calls for “governed global standards and global access controls.” A business-oriented data governance capability can ensure that the technical standards needed to support a mesh are in place. The definition, context, and quality standards must support user understanding and trust. As our respected fellow data pioneer Barr Moses puts it, without this, the data mesh becomes a data mess.
The idea of data mesh is a good one. I encourage people to think about how their data governance program can support the evolution into a data mesh regarding the organization and infrastructure. I also firmly believe businesses should ensure that all their projects are data-centric, not just the analytic exercises. Fundamentally, data governance is a growth function, not only a control function. – Kelle O’Neal, FSFP CEO and Founder
A data governance discipline can help the transition to and adoption of a data mesh architecture.
Here are some things to consider:
Leverage your assets – Utilize data owners, stewards and custodians to manage the data product. They are probably domain-centric already but may need to expand their scope to include data creation, consumption and sharing standards.
Treat your users as customers – Your job as a data product owner may be more challenging if you have novice customers because you must upskill them and provide them with useful information. Implement a “know your customer” program to help you understand their needs.
Give power to your people – Ensure the data governance office is empowered to create the federated governance for the data mesh. Governance is already skilled at developing and implementing value-producing policies and standards, communicating them across the organization and driving adoption. They should do the same for the data mesh architecture as well.
Align data literacy and enablement – Rather than create a new organization, assess how the Data Literacy team could be the Data Mesh Enablement team. Governance should also drive data literacy to help people better understand what data is available, how to find or create it, and how to use and explain it.
Metadata matters! – Make sure you have a robust metadata platform that spans a glossary and a catalog, along with workflow and policy automation, so you can help data product owners build the best customer solutions.
The Bottom Line
An adept data governance capability enables domain data teams to think strategically and act locally through global standards and policies. Additionally, it offers local solutions to meet the evolving needs of the dynamic data consumer through flexibility, adaptability, and autonomy.
As part of our data mesh strategy, FSFP creates and supports a data product mentality that puts data users first. This mentality leverages metadata and standards for data literacy, enabling agile data governance.
It’s time to transform data value into measurable business value. We help data-driven companies navigate data change to scale the value of insights, with collaborative governance at the core. Learn more about FSFP here or contact us.
