Le maillage de données est-il adapté à mon organisation?
A description about the picture so the audience knows what is happening in the image.
If you operate anywhere near the tech or business sectors, you’re well aware of the value of data in today’s world. You’ve also likely heard of data mesh, the radical new approach to data architecture that could exponentially increase the value of your data.
As a business leader, you want to empower your teams with the information to generate insights that drive value and growth. These days, data mesh is widely positioned as the solution for democratizing data at scale, broadening access to a wealth of organizational information by putting ownership in the hands of the teams generating it.
But data mesh isn’t a plug-and-play solution, and in some cases, setting your sights on data mesh might actually be a distraction from your underlying business challenges. In other words, despite its promising potential, pursuing a data mesh architecture is not a guaranteed fast track to becoming a data-driven organization, and might not be the best approach for every organization.
The purpose of this article is to help you understand if and when data mesh is the right approach for your business, and offer insights for better data management, no matter what your current state is.
What exactly is data mesh?
Data mesh is a fairly recent data architecture paradigm that seeks to make data more available and accessible to business users by directly connecting data owners, data producers, and data consumers through four guiding principles:
- Domain-driven ownership/design
- Data as a product
- Self-service data infrastructure
- Federated computational governance
Unlike traditional monolithic data infrastructures that handle the consumption, storage, transformation, and output of data in one central data lake, a data mesh supports distributed, domain-specific data consumers and views data as a product, with each domain handling its own data pipelines. It does this by leveraging data experts within the business to own and define how various data domains are represented and accessed across the enterprise. This shifts the focus from a centralized, technically focused data team to business-focused data SMEs who take ownership of the data and its delivery.
Data mesh reference architecture: bringing together infrastructure and governance
So what does this approach look like in practice? In many ways, data mesh is a lot like Legos. It’s possible to make over 915 million different combinations from just six different Lego bricks. A data mesh can similarly be built in any way that works best for your organization: choose each component carefully and build the solution that most fits your needs. There is no one right or wrong way to do it, and you can swap out components as your data needs mature and shift.
Sounds great, right? And under the right circumstances it is! But it’s important to understand the limitations before deciding to embark on a data mesh journey so you can be sure the end product will achieve the desired results for your business.
Data mesh is not a magic fix
It’s not going to solve existing governance issues
If your organization is already struggling with governance and documentation around data resources, a data mesh is not going to inherently solve this problem.
Decentralized solutions often result in products of varying quality and reliability. Within a data mesh, business teams own their own data products, which also means they are responsible for their own quality and governance practices. The teams that own these data products may not be ideally incentivized to make their data easier to use for various use cases outside their teams (Swapnil Wani has a great article that discusses this issue at length).
While distributed ownership can expand reach and access to data, decentralization can also lead to data silos, conflicting sources of truth, and duplication of effort, worsening existing pain points coming from a data lake solution.
Business teams have to be ready to own their data products
Data mesh methodology can’t be pushed out from the IT organization. If your organization doesn’t already have a strong data culture in which business teams are equipped and eager to own their data products to improve agility and reduce friction, it could be a hard sell to get non-technical teams to take ownership of products they historically had little to no responsibility for.
Decentralized development is often more expensive
The traditional operating model of centralized data engineering requires fewer skilled technical resources as the business teams all share those resources. Decentralization can lead to each business team hiring and supporting their own technical teams, which requires more resources. On the one hand, this is one reason agility and speed-to-delivery is improved: there are more people delivering, perhaps with fewer competing demands on their time. On the other hand, business teams may not be able to fully staff certain roles, like testing, which can erode solution quality or cause erratic behavior. In the end, it’s a trade-off between having fewer people do the job for a longer duration, leading to longer time to market, or having a bigger team and faster time to market, leading to greater growth in a shorter time.
There are solutions that try to mitigate the technical demands on business users by providing a low-code platform, but simple, easy-to-use common platforms are rarely going to meet all your business requirements. Edge cases exist and will require technical teams to build toward and support.
The trade-off of flexibility vs. simplicity
Having a central IT organization designing the architectural solution that business teams will own can often result in a mismatch between the solution technical requirements and the technical abilities or needs of the team. If the goal is a truly a decentralized solution, business teams will go off in their own directions and push up against the limitations designed by centralized data architects.
A productive solution will fall somewhere on a continuum, with perfect agility on one end and perfect consistency and simplicity on the other. Each organization must arrive at a point on that spectrum that meets its needs, and that balance point will evolve over time, always involving compromise.
With data mesh, flexibility frequently comes at the cost of simplicity. Some in-between approach is certainly possible but must be arrived at iteratively, through trial and error, to meet the requirements abilities of the enterprise as a whole and each business unit individually.
Success depends on the candidate
The strongest candidate for a data mesh includes a compelling business case, strong buy-in and sufficient resources, and an organizational culture that supports it.
If you have an approach that’s working for you — say, your organization is not domain-oriented and has centralized IT with fungible resources that are implemented alongside various projects — then data mesh likely isn’t the right investment at this time.
If data mesh is so complex, why do I need it?
The desire for data mesh is connected to the more fundamental goal of becoming a data-driven organization. Depending on your current state, here are some tips for taking your next, or first, steps toward realizing a modern culture of data.
Foundational
If you’re just getting started, focus on three areas: governance, modern engineering, and data quality.
- Governance
As we stated earlier, if your organization already has governance issues, a data mesh is not going to fix this. Before embarking on a data mesh journey, ensure that you have a data governance framework in place, even if it’s for a section of a big organization. Work with business teams to improve data literacy best practices before decentralizing. Identify a business domain, list the data elements that are part of it and build a RACI matrix. Clear domain ownership and who’s responsible for what are critical for the self-service component of data mesh. Discuss with the concerned parties and ensure they are on board with the initiative; more than the technology, it’s the people who drive the process.
- Modern engineering
Data mesh is founded on modern engineering practices of automation, continuous delivery, and DevOps — all nonnegotiable aspects of the technology that will be required to support domain-oriented data-sharing. Ascertaining whether those engineering practices exist within your organization or if you have the capacity to support such an endeavor is essential.
- Data quality
Even with the best of people, processes, and technology, data mesh initiatives often fail as they cannot showcase the business value due to lack of data quality. Check the quality of the data coming in and fix it close to the source. The better the quality, the higher the value derived.
Enhanced
Once you have established a solid governance framework, modern engineering practices, and a strong data culture, here are three tips for setting up your data mesh implementation for success.
- Start small, but choose the right candidate
It’s important to start small and select the right candidate to do a proof of concept — something that demonstrates the value of the data mesh to the business community and sponsors to keep them engaged. After all, the key element is building the data culture in the organization. Find a project that will bring quantifiable business value with unambiguous goals. Make the goals and objectives very clear early on and discuss with data leaders within the organization to avoid conflicting projects and initiatives; this way, you can minimize pushback and garner maximum support.
- Work with what you’ve got
Most data mesh projects are brownfield projects, meaning you’re improving upon what’s already in place and is generally working, even if it’s not perfect. Proposing a fresh implementation is not only expensive but also will likely receive pushback from the team(s) that had implemented the previous solution (human nature!). You might require a new set of technologies, but also think on the upskilling element — it’s easier to reuse existing technology and leverage the knowledge your employees already have.
- Include your teams on the journey
As the saying goes, for many big organizations, the left hand doesn’t know what the right hand is doing. Multiple parallel initiatives often cloud a clear sense of the common goal. Doing roadshows and letting the business users speak for themselves will not only build up the momentum but also help you to identify use cases that you thought never existed.
Conclusion
At its core, data mesh is really a sociotechnical approach that relies on the synergies between people, processes, and technology to democratize data at scale. Aligning these three elements is an iterative process, often requiring a substantial investment of time and resources.
At Slalom, we partner with you to understand the business and human case for any approach, and help you prioritize the investments that will unlock value and allow you to achieve your purpose.
-1?fmt=webp-alpha)
.svg)
?fmt=webp-alpha)
