The Intelligent Data Management Blog

Distributing and Consuming Golden Data

I have described in a previous post how the Convergence Hub pattern implemented in Semarchy Convergence for MDM allowed you to simply certify golden records from the source operational systems.

One question that came up was: How do you make this golden data available to the end-users?

Why is Golden Data Availability Critical?

Before answering, it is important to understand that this question is a critical one.

Golden Data is extremely valuable, but unlike very valuable things, it should not be stored in a vault for the future, or remain available for a small group of power users.

The true value of golden data only appears when it is broadly consumed and used.

For example:

  • Business Intelligence relying on accurate and trusted golden data help decision-makers drive the business with greater confidence.
  • Processes become more efficient when they rely on operational applications that share the same master data.
  • Having a clear and single vision of enterprise data assets (customers, products, etc.) allows discovering, planning and executing new business opportunities.

As I explained in the Successful Adoption Cycle post, having Golden Data authored in a master data entry system is not right for creating golden data. It becomes very wrong when it comes to consuming, as master data fluidity is impeded by the master data entry application itself.

Consumption or Distribution?

To facilitate adoption and avoid existing processes disruption and changes, Consumption for business users should take place in existing applications or systems. For example:

  • Decision Makers should use their existing dashboards and use the golden data stored in the datawarehouse or datamarts.
  • Business Users access golden data through their operational applications and modify it there directly.

These two cases imply a mechanism in the MDM Platform for Golden Data Distribution to the consuming applications.

Handling the Distribution

Now, the hundred dollar question: How can you do this?

Well, very simply! The convergence hub pattern implementation in Semarchy Convergence for MDM not only includes the data structures and certification process, but also the endpoints to make this data available for consumers. These endpoints are generated from the metadata and support methods such as SQL, JMS, Web Services, etc, depending on the consuming application or user.

The following example shows the response for Customer golden data. This web service - automatically generated from the Customer entity metadata - includes not only the golden data, but also tagging and version information generated by the platform.

Golden Data Webservice Example

The exposed endpoints not only include the golden data as shown in the example, but also the entire history and lineage of this golden data. Using similar endpoints, a consumer can access previous versions of the golden data, as well as the entire process that led to this golden data.

Convergence and Distribution

Distribution is the counterpart of the convergence process that takes place in the convergence hub.

Convergence assembles golden data from the sources; distribution sends (back) golden data to existing applications where it is consumed by end-users.

Having both Convergence and Distribution guarantees that the MDM platform remains totally non-intrusive, even when master data becomes highly pervasive across the enterprise.