The first thing to know about data mesh is that people expect too much of a data mesh.
Data mesh is often misunderstood as a specific technology or solution – when instead, it’s more of a conceptual framework. To be exact, it’s a “sociotechnical approach,” as described by Zhamak Dehghani, the creator of the data mesh concept.
Because many mistake data mesh for a specific cure-all technology they can buy, organizations say they want to adopt data mesh, and then struggle to implement it on a practical level.
So, is data mesh already on its way out? Analysts have suggested that data mesh is at risk of failing, a dying trend in 2024. Gartner put data mesh in the innovation trigger phase of the Hype Cycle for Data Management but predicted it will be “obsolete before plateau.”
But we think the death of data mesh is greatly exaggerated. It hasn’t had a fair chance to live. Data mesh isn’t the solution; it’s a strategy. It’s a framework for making decisions. If you view it as such, it can be a helpful guide as you think about the people, process and technology changes that will set your business up for success.
In the age of AI, data mesh is a more useful framework than ever.
According to an MIT study, fewer than 10% of companies have generative AI (genAI) use cases in production. Almost 60% say that they have not made any changes to their data environment to support or enable genAI. These organizations need a framework that guides their data architecture strategy toward successful AI implementation. Cue data mesh!
The goal of data mesh is to empower business domain experts to manage their own data pipelines. And who is an untapped talent pool for AI use cases? Business domain experts.
Business users are a critical resource for acquiring data, use case identification, exploratory analysis, and evaluating outcomes. When the users with business logic are left out of AI initiatives, adoption stalls. Data mesh can help ensure business users have a seat at the table.
Line-of-business expertise is needed for each stage of the AI/ML journey. A truly business-friendly data framework will enable this participation.
If you’re thinking about using data mesh to build a data architecture that will support enterprise-wide AI adoption, here are some dos and don’ts to keep in mind:
DON’T: adopt a self-service data platform that isn’t actually self-service
A data mesh culture enables domain ownership. That means…enabling the domain owners! If you can’t do that, you’ve missed the point. Yet it’s a common pitfall.
Analytics leaders might get excited about data mesh, adopt an exciting new platform like Microsoft Fabric, and realize – whoops – that financial analysts used to working in Excel can’t realistically use Power Query. Or they’ll migrate to a SQL-based data warehouse, hand the keys over to the sales team, and the sales team doesn’t know what to do with a notebook.
To truly enable a data mesh culture, you need to make the actual analytics work accessible for the non-technical business experts who will be taking on the analytics process. It’s why a self-service platform is one of the core principles of data mesh.
Make sure your analytics platform really is self-service. Invest in technology solutions that are designed for the business to use, regardless of whether they have existing data skills.
Beware of platforms that claim to support self-service but aren’t realistically business-friendly and require more technical knowledge than you bargained for. Your progress will stall, and you’ll be among the 90% of companies who haven’t made it to their first GenAI use case.
DO: build a foundation of data literacy across roles
Data mesh requires organization-wide data literacy. When you shift from a centralized data team approach to a more decentralized model, you need to democratize the foundational data knowledge that was formerly reserved for a central data team.
Your line-of-business experts already have important data literacy to bring to the table – domain literacy and use case literacy. What does the data mean in the context of the business? What can you do with the data? What are the business outcomes you can get?
But business experts still likely need a foundational understanding of data concepts, such as:
- How to access data from the storage source your organization uses
- What it means to handle different data types
- How to recognize quality issues with your dataset
As you transition to a data mesh culture, make sure you launch a data literacy program that invests in the training and education that will set your domain experts up for success when it’s their turn to own the analytics and AI journey.
DON’T: abandon governance, as data quality becomes even more critical
You need to be able to trust the data behind your AI and ML models. That means building governance into your analytics journey at each stage, from data collection to model training.
Just because data mesh means decentralized data ownership, doesn’t mean governance and quality standards go out the window. If anything, data mesh helps give a framework for improving data quality as business users get more involved in the AI/ML process. One of the tenets of data mesh is federated governance, providing guidance for establishing governance across business teams.
Some suggestions for putting this into practice (and each of these should be a collaboration between business experts and IT/data experts):
- Create a cross-functional data governance committee
- Conduct regular data accuracy audits
- Establish clear data standards
- Draft data usage policies
The Alteryx AI Platform for Enterprise Analytics doesn’t just fit in with a data mesh approach – it facilitates it. The Alteryx AI Platform is a genuinely self-service platform that empowers business users to work with data. Domain owners will have the easy-to-use tooling and learning resources they need to realistically own the analytics journey.
At the same time, Alteryx supports the governance and infrastructure considerations that come with a self-service model. Because Alteryx provides an easy-to-use analytics interface for underlying infrastructure like Microsoft, Databricks and Snowflake, it respects the governance policies and role-based access controls that come with those technologies. Alteryx also has built-in auditability features, like annotated workflows that make it easy to document the steps of each data transformation.
Thinking about moving forward with data mesh at your organization? Talk to an Alteryx expert today about your goals for implementation or check out more resources below.
AWS + Snowflake + Alteryx: Technical Guide
Generative AI Use Cases: Driving Innovation Across Industries