Semarchy xDM 5.1: On-board new and old data with xDM Discovery

We recently announced the availability of the new xDM Dashboards component in our Intelligent Data Hub™ platform. Since xDM 5.0, data architects, data champions, and business users can build amazing charts and data visualizations providing insights and quantitative metrics that blend together data sets from the hub and from any other external sources such as the Data Warehouse or the Data Lake. This component is one of the cornerstones of the platform to support the data governance initiative in building 360° views of critical data assets.

The Semarchy xDM 5.1 release extends the platform again and introduces xDM Discovery to help profile and assess the quality of any data set before connecting it with the Intelligent Data Hub. This new component complements the xDM platform and completes our Intelligent Data Hub™ vision.

The Intelligent Data Hub™

The xDM Intelligent Data Hub Platform drives an end-to-end data hub project by offering the following capabilities, in a single integrated platform:

  • Discover data: Connect to data sources, profile their content and learn about existing critical data assets.
  • Integrate any upstream and downstream applications in batch and real-time, and leverage any datasource or service to enhance the enterprise data.
  • Govern the data hub by defining and enforcing business terms, processes, rules, policies, etc.
  • Manage data using apps designed for data champions and business users, with built-in data quality, match and merge, workflows, generated from the governance definitions and decisions.
  • Measure the efficiency of the operations and the relevance of the governance choices using dashboards, KPIs and metrics based on data from the data hub or from external data sources.

These capabilities are required to support a lean and agile approach to delivering the highest business value in fast iterations.

xDM Discovery

Semarchy xDM Discovery covers the first step of the data management initiative as well as future iterations that will involve onboarding new and old data sources into the data hub. This new component enables data architects and business users to gather the profiling metrics required to prepare their data management initiative iterations.

xDM Discovery connects to data sources, profiles the existing data and exposes these profiles in the xDM repository. Profiles include high-level table metrics and detailed column metrics, such as record counts, values frequency, values completeness, uniqueness, recognition of data patterns, and dynamic usage or remediation recommendations.

 

xDM Discovery - Profiling Process

 

Business users and data champions analyze the profiles using the built-in xDM Discovery dashboards and drill down to the source data to identify root causes and/or potential remediation processes. Advanced profiling is also possible with a fully customizable dashboarding experience using xDM Dashboards.

 

xDM Discovery - Profiling Dashboard

 

Using the metrics, data architects and data champions have a clear assessment of the quality of their data sources. They can discuss and infer the structure and data rules to apply to the data hub entities, build automatic remediation rules, create key performance indicators based on their findings, and follow the recommendations made by xDM Discovery to enhance their data governance initiative. xDM Discovery can therefore be used in any phase of a project, including the design phase, the data preparation phase, the data integration phase, and in the on-going data cleansing and maintenance of the Intelligent Data Hub™.

xDM Discovery dashboards and metrics can be customized and embedded into any xDM data management application. Business users become major stakeholders in the data governance initiative by continuously monitoring and transparently acting on the data quality following meaningful recommendations.

What’s Next?

This release also brings great new exciting features to enhance the user experience in data management and dashboard applications such as mass-actions, SemQL Lookups, matcher enhancements, drilldown, and more.

You can read more about these other new features in the next section of this blog post.

Try out xDM Discovery and all of the other great features available in Semarchy xDM release 5.1, install it on premises or in the Cloud.

 

Download xDM