The Intelligent Data Management Blog

The Big, the Bad and the Ugly Data

In the Good, the Bad and the Ugly, the italian epic spaghetti western from film director Sergio Leone, a mythic scene takes place in a desert graveyard. In this scene, Blondie (Clint Eastwood) who came there to dig $20,000 of stolen confederate money tells his partner Tuco (Eli Wallach), the following:

"You see, in this world there's two kinds of people, my friend: Those with loaded guns and those who dig. You dig."

This scene tells everything about tooling. With the right tool you'll have more gold with less effort!

Today, companies rush to new fields of untapped data gold, namely the big data sources. These include very large internal data stores and social data sources. Corporations turn to data voodoo doctors and scientists to process big piles of data rubble, and arm them with big data tools. Yet, the obvious choices (the shovel), R, Map/Reduce, Pig, Hive, etc, are not enough to mine the data-yard.

Master Data guarantees analysis accuracy and speed for Small and Big Data projects. Big Data, associated with bad and ugly Master Data is extremely likely to simply turn into Big Rubbish. As a project foundation, Master Data should always be taken into account before or during Big Data initiatives.

Depending on the master data management tool they will choose when entering the big data-yard, data scientists will end up in Tuco's or Blondie's situation:

"Tuco digs up the loot from the grave only to find himself once again staring down the barrel of Blondie's gun, who now holds a noose in his hand. After placing Tuco into the noose, fastening it to a nearby tree and making Tuco stand on the unstable wooden cross of one of the graves, Blondie takes half the coins and rides away while Tuco cries for help. In a dramatic twist, Blondie turns around to shoot the rope above Tuco's head, as he used to do in their times of partnership, freeing him one last time before riding off as Tuco screams in rage."

In this world there's two kinds of people, my friend: Those with Semarchy and those who dig. Where do you stand?