Follow the Data

In his recent blog post Multiple Data Touch Points, David Loshin wrote about how common it is for many organizations to not document how processes acquire, read, or modify data.  As a result, when an error occurs, it manifests itself in a downstream application, and it takes a long time to figure out where the error occurred and how it was related to the negative impacts.

Data is often seen as just a by-product of business and technical processes, but a common root cause of poor data quality is this lack of awareness of the end-to-end process of how the organization is using its data to support its business activities.

For example, imagine we have discovered an error in a report.  Do we know the business and technical processes the data passed through before appearing in the report?  Do we know the chain of custody for the report data?  In other words, do we know the business analyst who prepared it, the data steward who verified its data quality, the technical architect who designed its database, and the data entry clerk who created the data?  And if we can’t answers these questions, do we even know where to start looking?

When an organization doesn’t understand its multiple data touch points, it’s blindsided by events caused by the negative business impacts of poor data quality, e.g., a customer service nightmare, a regulatory compliance failure, or a financial reporting scandal.

“Follow the money” is an expression often used during the investigation of criminal activities or political corruption.  I remember the phrase from the 1976 Academy Award winning movie All the President’s Men, which was based on the non-fiction book of the same name written by Carl Bernstein and Bob Woodward, two of the journalists who investigated the Watergate scandal.

“Follow the data” is an expression sometimes used during the investigation of incidents of poor data quality.  However, it’s often limited to reactive data cleansing projects where the only remediation will be finding and fixing the critical data problems, but without taking corrective action to resolve the root cause—and in some cases, without even identifying the root cause.

A more proactive approach is establishing a formal process to follow the data from its inception and document every step of its journey throughout the organization, including the processes and people that the data encountered.  This makes it much easier to retrace data’s steps, recover more quickly when something goes awry, and prevent similar problems from recurring in the future.

Deep Throat told Woodward and Bernstein to: “Follow the money.”

Deep Thought told me 42 times to tell you to: “Follow the data.”