This post continues our "Back to Basics" or "Data management is simple" series.
In a previous post, I described a commonly agreed classification of data and defined the idea behind Master Data, Reference Data and Golden Data.
This Master/Reference/Golden Data is usually stored and managed in a central database called the MDM Hub.
There are various patterns or styles identified in the industry for such an MDM hub.
Registry Hub Style
In this style of hub, master data is authored and remains in the source systems. The hub stores an index of this source data, keeping track of the cross-references between matching source data. This hub style also typically stores the attributes used for matching purposes.
Master data distribution is performed downstream by federating the source data. The transformation and consolidation process for building the golden data is performed on demand.
This approach is completely non-intrusive and maintains data exclusively in the source applications. This style allows read-only access to the golden data via federated queries. Federating (assembling) golden data on the fly may raise performance issues and technology challenges, especially when large datasets or complex transformation/consolidation strategies come into play.
Consolidation Hub Style
In this pattern, master data is copied from source applications, then matched and consolidated in the hub; The golden data physically created in the hub becomes available for distribution to downstream applications or for direct consumption by business users and data stewards.
This style is as non-intrusive as the registry hub style. Since golden data is stored in the hub, the performance concerns with complex transformation/consolidation strategies disappear, and stewardship processes can be executed and orchestrated in the hub.
Stewardship is a set of processes and policies that run on top of the hub. They include data entry (for fixing data into the hub), duplicates handling (for false matches or specific matching cases) and rejects management.
Co-Existence Hub Style
This pattern is similar to the Consolidation hub. It adds an integration loop back to the source applications for Golden Data.
This style is non-intrusive from the end-user perspective, but brings major organizational and technical challenges to approve and implement the integration loop.
Performing a loop not at the application data level, but at the application UI level alleviates these challenges. Such a UI-level loop could be implemented as portlets included in the end-user application to show the "similar customers in other systems" or the "golden customer record" corresponding to the source application record being viewed or edited.
In this pattern, master data is entirely migrated and cleansed in the MDM hub. The hub becomes the single provider for master data, and master data is exclusively authored in the hub through data entry workflows. All applications refer to the hub for their master data.
This pattern guarantees the highest level of quality for the master data, but is highly intrusive on existing business and technical processes, as it forces to rethink these processes around the MDM hub.
This style is applicable when you want to create golden and master data ex nihilo, when this data does not exist in a formal place at the operational level. For example, you may want to replace all the "Cost Center Excel Spreadsheets", or the "Employee MS Access Database" stored in users' drives and mailboxes.
One radical use of this pattern is the replacement of the operational master data authoring point by an MDM Hub authoring point. Although it is possible to do so, moving data authoring to the MDM hub is a major risk to existing operational processes, as illustrated in the following scenario:
A Sales Rep. is on the phone with a new customer. He switches from his CRM/Order application to the MDM application to start creating a new customer, then waits for the new customer to be validated by data stewards before appearing in his CRM application which then enables him to place the order.
No need to say that such a process can be time consuming while the customer holds the line! This process may indeed guarantee a superior quality of the customer record, but would turn this simple order process into your worst nightmare.
One Pattern to Rule them All?
None of the styles described above is a magic pattern (or golden ring) to rule all the master data.
Depending on the domain, the source applications, or the master data that you plan to have in your MDM hub, you may want to store all the data or only the references, or something in between. You may want to enable consolidation only, with some data entry capabilities, or data entry only for certain data.
Add to this the fact that your MDM hub will evolve over time to include new domains or data, with different requirements that will mandate a switch to a different pattern.
For example, in a mature multi-domain hub:
- Customer data would be managed in a Co-Existence style, aggregating data from the CRM, Website, etc., with a loop back to the CRM system.
- Product data would be managed in a Consolidation style (from the PLM and ERP applications), with some additional attributes and validation processes handled in a Transactional style.
- Critical data would be handled in a Registry style for regulatory compliance.
- Financial data would be handled in a Transactional style to replace Excel spreadsheets and support authoring and validation workflows.
The choice of a style is not a technical decision. It must be driven by the business requirements. In addition, this choice is not written into stone. It evolves over time. Hence, the hub must support all these styles out of the box. We can define this multiple-style hub as a Convergence Hub.
Semarchy Convergence 1.3 was designed specifically not only to support all these hub patterns, but also to provide the comprehensive framework allowing mixing all the styles simultaneously - Registry, Consolidation, Co-existence and Centralized/Transaction - in a single MDM hub. This simple but fundamental differentiator allows true Evolutionary MDM implementations that evolve at the pace of the business requirement's maturity.