Scary Calendar Effects

OCDQ Radio is a vendor-neutral podcast about data quality and its related disciplines, produced and hosted by Jim Harris.

During this episode, recorded on the first of three occurrences of Friday the 13th in 2012, I discuss scary calendar effects.

In other words, I discuss how schedules, deadlines, and other date-related aspects can negatively affect enterprise initiatives such as data quality, master data management, and data governance.

Please Beware: This episode concludes with the OCDQ Radio Theater production of Data Quality and Friday the 13th.

Popular OCDQ Radio Episodes

Clicking on the link will take you to the episode’s blog post:

• Demystifying Data Science — Guest Melinda Thielbar, a Ph.D. Statistician, discusses what a data scientist does and provides a straightforward explanation of key concepts such as signal-to-noise ratio, uncertainty, and correlation.

• Data Quality and Big Data — Guest Tom Redman (aka the “Data Doc”) discusses Data Quality and Big Data, including if data quality matters less in larger data sets, and if statistical outliers represent business insights or data quality issues.

• Gaining a Competitive Advantage with Data — Guest William McKnight discusses some of the practical, hands-on guidance provided by his book Information Management: Strategies for Gaining a Competitive Advantage with Data.

• Doing Data Governance — Guest John Ladley discusses his book How to Design, Deploy and Sustain Data Governance and how to understand the difference and relationship between data governance and enterprise information management.

• Demystifying Master Data Management — Guest John Owens explains the three types of data (Transaction, Domain, Master), the four master data entities (Party, Product, Location, Asset), and the Party-Role Relationship, which is where we find many of the terms commonly used to describe the Party master data entity (e.g., Customer, Supplier, Employee).

• Measuring Data Quality for Ongoing Improvement — Guest Laura Sebastian-Coleman discusses bringing together a better understanding of what is represented in data with the expectations for use in order to improve the overall quality of data.

• The Blue Box of Information Quality — Guest Daragh O Brien on why Information Quality is bigger on the inside, using stories as an analytical tool and change management technique, and why we must never forget that “people are cool.”

• Data Governance Star Wars — Special Guests Rob Karel and Gwen Thomas joined this extended, and Star Wars themed, discussion about how to balance bureaucracy and business agility during the execution of data governance programs.

• Good-Enough Data for Fast-Enough Decisions — Guest Julie Hunt discusses Data Quality and Business Intelligence, including the speed versus quality debate of near-real-time decision making, and the future of predictive analytics.

• The Johari Window of Data Quality — Guest Martin Doyle discusses helping people better understand their data and assess its business impacts, not just the negative impacts of bad data quality, but also the positive impacts of good data quality.

• The Art of Data Matching — Guest Henrik Liliendahl Sørensen discusses data matching concepts and practices, including different match techniques, candidate selection, presentation of match results, and business applications of data matching.

• Studying Data Quality — Guest Gordon Hamilton discusses the key concepts from recommended data quality books, including those which he has implemented in his career as a data quality practitioner.

Two Weeks Before Christmas

OCDQ Radio is a vendor-neutral podcast about data quality and its related disciplines, produced and hosted by Jim Harris.

Season’s Greetings fellow data management enthusiasts and welcome to a special holiday-themed episode of OCDQ Radio.

With the Christmas, Hanukkah, Kwanzaa, and Festivus seasons now upon us, I revisited my ‘Twas Two Weeks Before Christmas blog post from 2009, which is based on the poem A Visit from St. Nicholas.  During this brief podcast, I perform a recital.

The entire OCDQ Blog family wishes you and yours all the best during this holiday season and the coming new year.

Popular OCDQ Radio Episodes

Clicking on the link will take you to the episode’s blog post:

• Demystifying Data Science — Guest Melinda Thielbar, a Ph.D. Statistician, discusses what a data scientist does and provides a straightforward explanation of key concepts such as signal-to-noise ratio, uncertainty, and correlation.

• Data Quality and Big Data — Guest Tom Redman (aka the “Data Doc”) discusses Data Quality and Big Data, including if data quality matters less in larger data sets, and if statistical outliers represent business insights or data quality issues.

• Gaining a Competitive Advantage with Data — Guest William McKnight discusses some of the practical, hands-on guidance provided by his book Information Management: Strategies for Gaining a Competitive Advantage with Data.

• Doing Data Governance — Guest John Ladley discusses his book How to Design, Deploy and Sustain Data Governance and how to understand the difference and relationship between data governance and enterprise information management.

• Demystifying Master Data Management — Guest John Owens explains the three types of data (Transaction, Domain, Master), the four master data entities (Party, Product, Location, Asset), and the Party-Role Relationship, which is where we find many of the terms commonly used to describe the Party master data entity (e.g., Customer, Supplier, Employee).

• Measuring Data Quality for Ongoing Improvement — Guest Laura Sebastian-Coleman discusses bringing together a better understanding of what is represented in data with the expectations for use in order to improve the overall quality of data.

• The Blue Box of Information Quality — Guest Daragh O Brien on why Information Quality is bigger on the inside, using stories as an analytical tool and change management technique, and why we must never forget that “people are cool.”

• Data Governance Star Wars — Special Guests Rob Karel and Gwen Thomas joined this extended, and Star Wars themed, discussion about how to balance bureaucracy and business agility during the execution of data governance programs.

• Good-Enough Data for Fast-Enough Decisions — Guest Julie Hunt discusses Data Quality and Business Intelligence, including the speed versus quality debate of near-real-time decision making, and the future of predictive analytics.

• The Johari Window of Data Quality — Guest Martin Doyle discusses helping people better understand their data and assess its business impacts, not just the negative impacts of bad data quality, but also the positive impacts of good data quality.

• The Art of Data Matching — Guest Henrik Liliendahl Sørensen discusses data matching concepts and practices, including different match techniques, candidate selection, presentation of match results, and business applications of data matching.

• Studying Data Quality — Guest Gordon Hamilton discusses the key concepts from recommended data quality books, including those which he has implemented in his career as a data quality practitioner.

Data Quality and #FollowFriday the 13th

As Alice Hardy arrived at her desk at Crystal Lake Insurance, it seemed like a normal Friday morning.  Her thoughts about her weekend camping trip were interrupted by an eerie sound emanating from one of the adjacent cubicles:

Da da da, ta ta ta.  Da da da, ta ta ta.

“What’s that sound?” Alice wondered out loud.

“Sorry, am I typing too loud again?” responded Tommy Jarvis from another adjacent cubicle.  “Can you come take a look at something for me?”

“Sure, I’ll be right over,” Alice replied as she quickly circumnavigated their cluster of cubicles, puzzled and unsettled to find the other desks unoccupied with their computers turned off, wondering, to herself this time, where did that eerie sound come from?  Where are the other data counselors today?

“What’s up?” she casually asked upon entering Tommy’s cubicle, trying, as always, to conceal her discomfort about being alone in the office with the one colleague that always gave her the creeps.  Visiting his cubicle required a constant vigilance in order to avoid making prolonged eye contact, not only with Tommy Jarvis, but also with the horrifying hockey mask hanging above his computer screen like some possessed demon spawn from a horror movie.

“I’m analyzing the Date of Death in the life insurance database,” Tommy explained.  “And I’m receiving really strange results.  First of all, there are no NULLs, which indicates all of our policyholders are dead, right?  And if that wasn’t weird enough, there are only 12 unique values: January 13, 1978, February 13, 1981, March 13, 1987, April 13, 1990, May 13, 2011, June 13, 1997, July 13, 2001, August 13, 1971, September 13, 2002, October 13, 2006, November 13, 2009, and December 13, 1985.”

“That is strange,” said Alice.  “All of our policyholders can’t be dead.  And why is Date of Death always the 13th of the month?”

“It’s not just always the 13th of the month,” Tommy responded, almost cheerily.  “It’s always a Friday the 13th.”

“Well,” Alice slowly, and nervously, replied.  “I have a life insurance policy with Crystal Lake Insurance.  Pull up my policy.”

After a few, quick, loud pounding keystrokes, Tommy ominously read aloud the results now displaying on his computer screen, just below the hockey mask that Alice could swear was staring at her.  “Date of Death: May 13, 2011 . . . Wait, isn’t that today?”

Da da da, ta ta ta.  Da da da, ta ta ta.

“Did you hear that?” asked Alice.  “Hear what?” responded Tommy with a devilish grin.

“Never mind,” replied Alice quickly while trying to focus her attention on only the computer screen.  “Are you sure you pulled up the right policy?  I don’t recognize the name of the Primary Beneficiary . . . Who the hell is Jason Voorhees?”

“How the hell could you not know who Jason Voorhees is?” asked Tommy, with anger sharply crackling throughout his words.  “Jason Voorhees is now rightfully the sole beneficiary of every life insurance policy ever issued by Crystal Lake Insurance.”

Da da da, ta ta ta.  Da da da, ta ta ta.

“What?  That’s impossible!” Alice screamed.  “This has to be some kind of sick data quality joke.”

“It’s a data quality masterpiece!” Tommy retorted with rage.  “I just finished implementing my data machete, er I mean, my data matching solution.  From now on, Crystal Lake Insurance will never experience another data quality issue.”

“There’s just one last thing that I need to take care of.”

Da da da, ta ta ta.  Da da da, ta ta ta.

“And what’s that?” Alice asked, smiling nervously while quickly backing away into the hallway—and preparing to run for her life.

Da da da, ta ta ta.  Da da da, ta ta ta.

“Real-world alignment,” replied Tommy.  Rising to his feet, he put on the hockey mask, and pulled an actual machete out of the bottom drawer of his desk.  “Your Date of Death is entered as May 13, 2011.  Therefore, I must ensure real-world alignment.”

Da da da, ta ta ta.  Da da da, ta ta ta.  Da da da, ta ta ta.  Da da da, ta ta ta.  Data Quality.

The End.

(Note — You can also listen to the OCDQ Radio Theater production of this DQ-Tale in the Scary Calendar Effects episode.)

#FollowFriday Recommendations

#FollowFriday is when Twitter users recommend other users you should follow, so here are some great tweeps who provide tweets mostly about Data Quality, Data Governance, Master Data Management, Business Intelligence, and Big Data Analytics:

(Please Note: This is by no means a comprehensive list, is listed in no particular order whatsoever, and no offense is intended to any of my tweeps not listed below.  I hope that everyone has a great #FollowFriday and an even greater weekend.)

Henrik Liliendahl Sørensen – @hlsdk Dylan Jones – @DataQualityPro Prashanta Chandramohan – @MDMGeek John Owens – @JohnIMM Phil Simon – @PhilSimon Rich Murnane – @murnane Alan David Duncan – @Alan_D_Duncan Axel Troike – @AxelTroike Chris Hale – @HaleChris Nicola Askham – @Nicola_Askham Karen Way – @KarenAWay1 Melinda Thielbar – @mthielbar Carla Gentry – @data_nerd Jill Wanless – @sheezaredhead Julie Hunt – @juliebhunt Jacqueline Roberts – @JackieMRoberts Lyndsay Wise – @wiseanalytics Loraine Lawson – @LoraineLawson Corinna Martinez – @Futureratti Jen Stirrup – @JenStirrup Loretta Mahon Smith – @silverdata April Reeve – @Datagrrl Tamara Dull – @tamaradull Tracy Austin – @tracykowal Ina Felsheim – @InaSAP Marinka Voorhout – @marinkavoorhout Meta Brown – @metabrown312 Kelle O’Neal – @1stSanFrancisco Robin Stehlik – @RobinPStehlik Natalie Eschen – @NatalieEschen Cindy Balon Harder – @CindyBHarder Kristin McMahon – @Kristin_McMahon Carrie Byrum – @CByrum Marie Haggberg – @HaggbergConsult Terri Rylander – @BIMarcom Nicole Carriere – @carrni Beth Breidenbach – @bbreidenbach Giedre Aleknonyte – @googlea Sarah Burnett – @SarahBurnett Teresa Cottam – @Teresacottam Kelly Lautt – @KellyLautt Margaret van Engers – @EngersWijs Karen Lopez – @datachick Rob Drysdale – @projmgr William Sharp – @dqchronicle Augusto Albeghi – @Stray__Cat Ted Louie – @TedLouie Gordon Hamilton – @DQStudent Juan Carlos Ore – @juancore Richard Eudes, PhD – @RichardEudes Julian Schwarzenbach – @jschwa1 Ken O’Connor – @KenOConnorData Mark Horseman – @MarkHorseman Graham Rhind – @GrahamRhind Rich Northwood – @TheDataGeek Jarrett Goldfedder – @JGoldfed Dalton Cervo – @dcervo Phil Wright – @faropress Garry Ure – @dq_midnightblue Jaime Fitzgerald – @jaimefitzgerald Simon Daniels – @mktginsightguy Ajay Ohri – @decisionstats Marcus Borba – @marcusborba Alexej Freund – @alexej_freund Vish Agashe – @VishAgashe Jean-Michel Franco – @jmichel_franco Jorge García – @jgptec Rohin Bhargava – @RohinBhargava Krish Krishnan – @datagenius Sunil Soares – @SunilSoares1 Mady Korada – @MostlyKnown Craig Mullins – @craigmullins Chris Sorensen – @wjdataguy Ted Cuzzillo – @datadoodle Mark Lorion – @mark_lorion Steve Dine – @steve_dine Doug Newdick – @dougnewdick Richard Jarvis – @ManBagBlog David Pratt – @DataMgmtWonk Gary Allemann – @Gary_Allemann David Ho – @David_Ho_DQ Jay Zaidi – @JayZaidi Ralph Winters – @RDub2 Rob Paller – @RobPaller Stephen Putman – @SJPutman JJ Burnam – @AmbivalentGeek Garnie Bolling – @GarnieBolling Benjamin Wolfe – @BenjaminWolfe Nick Giuliano – @Nick_Giuliano Scott Delaney – @scottyd99999 FX Nicolas – @FxNicolas Kimmo Kontra – @kimmokontra Leif Tietje – @Leif74 Jeff Cutler – @JeffCutler Amanda Woolf – @AmandWoolf Ron Miller – @ron_miller

IAIDQ – @IAIDQ Daragh O Brien – @daraghobrien Thomas Redman – @thedatadoc1 David Loshin – @DavidLoshin Neil Raden – @NeilRaden James Taylor – @jamet123 John Ladley – @jladley Art Petty – @ArtPetty Paul Gillin – @pgillin Shawn Rogers – @shawnrog Wayne Eckerson – @weckerson Peter Thomas – @PeterJThomas Timo Elliott – @TimoElliott Dan Power – @dan_power William McKnight – @williammcknight Kirk Borne – @KirkDBorne Michael Cavaretta – @mjcavaretta Peter Aiken – @paiken Gwen Thomas – @gwenthomasdgi Claudia Imhoff – @Claudia_Imhoff Tom Davenport – @tdav Gartner Research – @Gartner_inc Ted Friedman – @ted_friedman Andy Bitterer – @bitterer Merv Adrian – @merv Doug Laney – @Doug_Laney Forrester Research – @forrester Michele Goetz – @FORR_Mgoetz IBM Midsize Business – @MidmarketIBM IBM Big Data – @IBMbigdata Crysta Anderson – @CrystaAnderson David Pittman – @TheSocialPitt James Kobielus – @jameskobielus SAS DataFlux – @SASDataFlux Jill Dyché – @JillDyche Data Roundtable – @DataRoundtable SAS Software – @SASsoftware SAS Analytics – @SASanalytics Mark Troester – @mtroester All Analytics – @AllAnalytics Service Objects – @ServiceObjects Wendy Breakstone – @socialwen Actian Corporation – @ActianCorp Actian DataCloud – @DataIntegrate Paige Roberts – @RobertsPaige Bill Hewitt – @BillHewittCEO Kalido – @Kalido Mike Wheeler – @MWheeler_DDG Informatica – @InformaticaCorp Robert Karel – @rbkarel Clarke Patterson – @ClarkePatterson Jakki Geiger – @JakkiGeiger Ravi Shankar – @Ravi_Shankar_ Talend – @Talend Steve Sarsfield – @SteveSarsfield Trillium Software – @TrilliumSW RedPoint Global – @RedPointGlobal George Corugedo – @GeorgeCorugedo Carol Wolicki – @cwolicki TIBCO Spotfire – @TibcoSpotfire Brett Stupakevich – @Brett2point0 Melissa Data – @MelissaData Ira Warren Whiteside – @irawhiteside Dun & Bradstreet – @DnBUS Shelly Lucas – @pisarose helpIT Systems – @helpIT Joshua Buckler – @bucklerjosh Datamartist – @Datamartist Experian QAS – @Experian_QAS Ataccama – @Ataccama DataMentors – @DataMentors DQ Global – @DQ_Global Pitney Bowes Software – @PitneyBowes Utopia, Inc. – @UtopiaInc Enterworks – @enterworks Fergus Cloughley – @Master_OBASHI Match2Lists – @Match2Lists DMG Federal – @dmgfederal Data Science at UC Berkeley – @BerkeleyData MIKE2.0 – @OpenMethodology The Data Manager’s Public Library – @TheDMPL Dataversity – @Dataversity ECCMA – @ECCMA SmartData Collective – @SmartDataCo Information Management – @InfoMgmt Tony Carrini – @TonyCarrini Julie Langenkamp – @JulieLangenkamp Jim Ericson – @JimEricson Eric Kavanagh – @Eric_Kavanagh Justin Kern – @IMJustinKern Whitney Eden – @WhitneyAEden Enterprise CIO Forum – @ECIOForum John Dodge – @Thedodgeretort Paul Calento – @pcalento Pearl Zhu – @pearl_zhu Eric D. Brown – @ericdbrown

Spartan Data Quality

My recent Twitter conservation with Dylan Jones, Henrik Liliendahl Sørensen, and Daragh O Brien was sparked by the blog post Case study with Data blogs, from 300 to 1000, which included a list of the top 500 data blogs ranked by influence.

Data Quality Pro was ranked #57, Liliendahl on Data Quality was ranked #87, The DOBlog was a glaring omission, and I was proud OCDQ Blog was ranked #33 – at least until, being the data quality geeks we are, we noticed that it was also ranked #165.

In other words, there was an ironic data quality issue—a data quality blog was listed twice (i.e., a duplicate record in the list)!

Hilarity ensued, including some epic photo shopping by Daragh, leading, quite inevitably, to the writing of this Data Quality Tale, which is obviously loosely based on the epic movie 300—and perhaps also the epically terrible comedy Meet the Spartans.  Enjoy!

Spartan Data Quality

In 1989, an alliance of Data Geeks, lead by the Spartans, an unrivaled group of data quality warriors, battled against an invading data deluge in the mountain data center of Thermopylae, caused by the complexities of the Greco-Persian Corporate Merger.

Although they were vastly outnumbered, the Data Geeks overcame epic data quality challenges in one of the most famous enterprise data management initiatives in history—The Data Integration of Thermopylae.

This is their story.

Leonidas, leader of the Spartans, espoused an enterprise data management approach known as Spartan Data Quality, defined by its ethos of collaboration amongst business, data, and technology experts, collectively and affectionately known as Data Geeks.

Therefore, Leonidas was chosen as the Thermopylae Project Lead.  However, Xerxes, the new Greco-Persian CIO, believed that the data integration project was pointless, Spartan Data Quality was a fool’s errand, and the technology-only Persian approach, known as Magic Beans, should be implemented instead.  Xerxes saw the Thermopylae project as an unnecessary sacrifice.

“There will be no glory in your sacrifice,” explained Xerxes.  “I will erase even the memory of Sparta from the database log files!  Every bit and byte of Data Geek tablespace shall be purged.  Every data quality historian and every data blogger shall have their Ethernet cables pulled out, and their network connections cut from the Greco-Persian mainframe.  Why, uttering the very name of Sparta, or Leonidas, will be punishable by employee termination!  The corporate world will never know you existed at all!”

“The corporate world will know,” replied Leonidas, “that Data Geeks stood against a data deluge, that few stood against many, and before this battle was over, a CIO blinded by technology saw what it truly takes to manage data as a corporate asset.”

Addressing his small army of 300 Data Geeks, Leonidas declared: “Gather round!  No retreat, no surrender.  That is Spartan law.  And by Spartan law we will stand and fight.  And together, united by our collaboration, our communication, our transparency, and our trust in each other, we shall overcome this challenge.”

“A new Information Age has begun.  An age of data-driven business decisions, an age of data-empowered consumers, an age of a world connected by a web of linked data.  And all will know, that 300 Data Geeks gave their last breath to defend it!”

“But there will be so many data defects, they will blot out the sun!” exclaimed Xerxes.

“Then we will fight poor data quality in the shade,” Leonidas replied, with a sly smile.

“This is madness!” Xerxes nervously responded as the new servers came on-line in the data center of Thermopylae.

“Madness?  No,” Leonidas calmly said as the first wave of the data deluge descended upon them.  “THIS . . . IS . . . DATA !!!”

Related Posts

Pirates of the Computer: The Curse of the Poor Data Quality

Video: Oh, the Data You’ll Show!

The Quest for the Golden Copy (Part 1)

The Quest for the Golden Copy (Part 2)

The Quest for the Golden Copy (Part 3)

The Quest for the Golden Copy (Part 4)

‘Twas Two Weeks Before Christmas

My Own Private Data

The Tell-Tale Data

Data Quality is People!

Pirates of the Computer: The Curse of the Poor Data Quality

This recent tweet (expanded using TwitLonger) by Ted Friedman of Gartner Research conspired with the swashbuckling movie Pirates of the Caribbean: The Curse of the Black Pearl, leading, really quite inevitably, to the writing of this Data Quality Tale.

Pirates of the Computer: The Curse of the Poor Data Quality

Jack Sparrow was once the Captain of Information Technology (IT) at the world famous Es el Pueblo Estúpido Corporation. 

However, when Jack revealed his plans for recommending to executive management the production implementation of the new Dystopian Automated Transactional Analysis (DATA) system and its seamlessly integrated Magic Beans software, his First Mate Barbossa mutinied by stealing the plans and successfully pitching the idea to the CIO—thereby getting Captain Sparrow fired.

As the new officially appointed Captain of IT, Barbossa implemented DATA and Magic Beans, which migrated and consolidated all of the organization’s information assets, clairvoyantly detected and corrected existing data quality problems, and once fully implemented into production, was preventing any future data quality problems from happening.

As soon as a source was absorbed into DATA, Magic Beans automatically freed up disk space by deleting all traces of the source, including all backups—somehow even the off-site archives.

DATA was then the only system of record, truly becoming the organization’s Single Version of the Truth.

DATA and Magic Beans seemed almost too good to be true.

And that’s because they were.

A few weeks after the last of the organization’s information assets had been fully integrated into DATA, it was discovered that Magic Beans was apparently infected with a nasty computer virus known as The Curse of the Poor Data Quality.

Mysterious “computer glitches” began causing bizarre data quality issues.  At first, the glitches seemed rather innocuous, such as resetting all user names to “TED FRIEDMAN” and all passwords to “GARTNER RESEARCH.”

But that’s hardly worth mentioning, especially when compared with what happened next.

All of the business-critical information stored in DATA—and all new information added—suddenly became completely inaccurate and totally useless as the basis for making any business decisions.

DATA and Magic Beans were cursed!  It was believed that the only way The Curse of the Poor Data Quality could be lifted was by re-installing the organization’s original systems and software.

William “Backup Bill” Turner, Jack’s only supporter, believing the organization deserved to remain cursed for betraying Jack, sent a USB drive to his young son, Will, which contained the only surviving backup copy of the original systems and software.

Many years later, Will Turner, still wearing his father’s old USB drive around his neck, but not knowing its alleged value, is told by Jack Sparrow that Captain Barbossa killed Will’s father and kidnapped Will’s ex-girlfriend, Elizabeth Swann.

Jack and Will infiltrate the DATA center disguised as PIRATEs (Professional Information Retrieval and Technology Experts). 

Jack tells Will that he needs the USB drive to determine where Elizabeth is being held.  Will gives Jack the USB drive and he uses it to begin restoring the original systems and software.  Moments later, Barbossa and Elizabeth walk into the DATA center.

“Elizabeth!  Don’t worry, I’m here to save you!” Will proudly declares.

“Will?” Elizabeth responds, confused.  “What are you talking about?  You’re here to save me from what?  My new job?”

Embarrassed, and turning toward Jack, Will shouts, “You told me Barbossa killed my father and kidnapped Elizabeth!”

“I’m terribly sorry, but I lied,” replies Jack.  “I’m a PIRATE, that’s what we do.”

“Killed your father?” Barbossa interjects.  “No, not literally.  Years ago, I killed a UNIX process he was running in production, and he threw a temper tantrum then quit.  I just hired Elizabeth last week in order to help us overcome our DATA problems.”

“You are Jack Sparrow?” asks Elizabeth.  “You are, without doubt, the worst PIRATE I’ve ever heard of.”

“But you have heard of me,” replies Jack, proudly smiling.

“Security!” yells Barbossa.  “Please escort Mr. Sparrow out of the building—immediately!”

“That’s Captain Sparrow,” Jack retorts.  “And it’s too late, Barbossa!  I just restored the original systems and software.  Ha ha!  DATA and Magic Beans are no more!  Without doubt, this will earn my rightful reinstatement as the Captain of IT!”

“Oh no it won’t,” Barbossa responds slowly, while staring at his monitor in disbelief.  “DATA and Magic Beans are gone alright, but The Curse of the Poor Data Quality remains!”

“The what?” asks Elizabeth.

“The Curse of the Poor Data Quality,” Barbossa angrily replies.  “All of our information assets are still completely inaccurate and totally useless as the basis for making any business decisions.  Therefore, we are still cursed with unresolved data quality issues!”

“What did you expect to happen?” remarks Will.  “Technology is never the solution to any problem.  Technology is the problem.  And unabated advancements in technology will eventually lead to computers becoming self-aware and taking over the world.”

Laughing, Barbossa asks, “You do realize that only happens in really bad movies, right?”

“No, curses only happen in really bad movies,” replies Will.  “Sentient computers taking over the world is really going to happen.  After all, it was very clearly explained in that excellent documentary series produced by the governor of California.”

“Oh, shut up Will!” shouts Elizabeth.  “I don’t won’t to hear another one of your anti-technology rants!  That’s why I broke up with you in the first place.  Although technology didn’t cause the data quality problems, Luddite Will is right about one thing, technology is not the solution.”

“What in blazes are you talking about?” Jack and Barbossa retort in unison.

“Seriously, I actually have to explain this?” replies Elizabeth.  “After all, the name of this corporation is Es el Pueblo Estúpido!”

Jack, Barbossa, and Will just stare at Elizabeth with puzzled looks on their faces.

“It’s Spanish for,” explains Elizabeth, “It’s the People, Stupid!”

“Well, we don’t speak Spanish,” Barbossa and Jack reply.  “The only languages we speak are Machine Language, FORTRAN, LISP, COBOL, PL/I, BASIC, Pascal, C, C++, C#, Java, JavaScript, Perl, SQL, HTML, XML, PHP, Python, SPARQL . . .”

“Enough!” Elizabeth finally screams. 

“The point that I am trying to make is that although people, business processes, and yes, of course, technology, are all important for successful data quality management, by far the most important of all is . . . Do I really have to say it one more time?”

“It’s the People, Stupid!”

“This corporation should really be renamed to Todos los hombres son idiotas!” Elizabeth concludes, while shaking her head and looking at the clock.  “We can discuss all of this in more detail next week after I return from my Labor Day Weekend vacation.”

“You’re going away for Labor Day Weekend?” asks Will cheerily.  “Perhaps you would be so kind as to invite me to join you?”

“It’s a good thing you’re cute,” replies Elizabeth.  “Yes, you’re invited to join me, but you’ll have to carry my purse—all weekend.”

“Can we pretend,” Will says, grimacing as he reluctantly accepts her purse, “that I am carrying your laptop computer bag?”

“Oh sure, why not?” replies Elizabeth sarcastically with a sly smile.  “And while we’re at it, let’s all just continue pretending that the key to ongoing data quality improvement isn’t focusing more on people, their work processes, and their behaviors . . .”

Related Posts

Data Quality is People!

The Tell-Tale Data

There are no Magic Beans for Data Quality

Do you believe in Magic (Quadrants)?

Data Quality is not a Magic Trick

The Tooth Fairy of Data Quality

Which came first, the Data Quality Tool or the Business Need?

Predictably Poor Data Quality

The Scarlet DQ

The Poor Data Quality Jar

Video: Oh, the Data You’ll Show!

In May, I wrote a Dr. Seuss style blog post called Oh, the Data You’ll Show! inspired by the great book Oh, the Places You'll Go!

In the following video, I have recorded my narration of the presentation format of my original blog post.  Enjoy!

Oh, the Data You’ll Show!

If you are having trouble viewing this video, then you can watch it on Vimeo by clicking on this link: Oh, the Data You’ll Show!

And you can download the presentation (PDF file) used in the video by clicking on this link: Oh, the Data You’ll Show! (Slides)

And you can listen to and/or download the podcast (MP3 file) by clicking on this link: Oh, the Data You’ll Show! (Podcast)

Oh, the Data You’ll Show!

Congratulations!
Today is your day.
You’re off to make your data presentations!
You’re off and away!

You have brains in your head.
You have pretty charts and graphs in your slides.
With data transparency, you’ll show you have nothing to hide.
You’re on your own, and you know what you know.

But your Data Quality may decide, where it is you may go.

You looked up and down columns, then across every row, with patience and care.
About some data you said, “I think that we need better quality here.”
With your head full of brains, and data under your supervision, 
You’re too smart to advise a not-so-good business decision.

Oh! the Data You’ll Show!

You’ll be on your way up!
You’ll be providing great business insights!
You’ll join the high fliers who soar to high heights.

You won’t lag behind, because you have parallel processing speed.
You’ll analyze the whole database, and you’ll soon take the lead.
Wherever you fly, you’ll be the best of the best.
Wherever you go, your data analysis will help you top all the rest.

Except when you don’t.
Because, sometimes, you won’t.

I’m sorry to say so but, sadly, it’s true.
Poor Data Quality and Bad Business Decisions can happen—yes, even to you.

You will discover data sources without any meta-mark. 
Nothing is labeled, leaving all business context in the dark. 
Data that could cause quite a chagrin!  Do you dare to stay out?  Do you dare to go in?
How much could you lose?  How much could you win?

And if you go in, should you JOIN LEFT or JOIN RIGHT—or JOIN LEFT-and-three-quarters?
With this data, you will feel like you are SQL querying blind.
Simple it’s not, I’m afraid you will find,
For a fine mind-maker-upper to make up their mind.

You can get so confused that you’ll start racing down long winding rows at a break-necking pace,
Grinding on for gigabytes across a weirdish and wild tablespace, headed, I fear, toward a most useless place.

The Analysis Paralysis Place—for people just analyzing.

Analyzing and analyzing, with no end in sight,
Analyzing and analyzing, with no way to know what’s wrong or what’s right.
Analyzing and analyzing, until three in the morning, and until the Nth degree,
Analyzing and analyzing, refusing to seek help from any business or data SME.

No!  That’s not for you!

Somehow you’ll escape all that Analysis Paralysis,
And hopefully without any of that costly psychoanalysis.
You’ll discover a way out of that place, so dismal and so dark,
Because when it comes to clear thinking, you’re a bright little spark.

Oh! the Data You’ll Show!

There is fun to be done!  And work too, that’s for sure.  But even work feels like a game you have already won.
The magical things that you can do with data, will make you the winning-est winner of all.
Among your co-workers and friends, everyone and all, you will truly stand the tallest of the tall.
You’ll be famous as famous can be, with the whole World Wide Web watching you win on YouTube and Google TV.

Except when they don’t.
Because, sometimes, they won’t.

I’m afraid that sometimes you’ll play lonely games too.
Games you can’t win because you’ll play against you.

All Alone!  Whether you like it or not,
Alone will be something you’ll be quite a lot.

And when you’re alone, there’s a very good chance you will meet, 
Data that scares you and convinces you it’s time to retreat.
There are some operational source systems that regularly do spawn,
Data that can scare you so very much, you won’t want to go on.

But on you will go though the data quality be most foul.
On you will go though the hidden data defects do prowl.
On you will go though it might take quite awhile, and leave quite a scar, 
You’ll overcome your data’s problems, whatever they are.

Oh! the Data You’ll Show!

Proceed with great care and with great tact, always remembering that,
Data Quality is a Great Balancing Act.
Just never forget to be dexterous and deft,
And never mix up your RIGHT JOIN with your LEFT.

Kid, you’ll move data mountains!
Today is your day!
Your data analysis is waiting.
So you had better get underway!

And will you succeed?
Yes!  You will, indeed!
(99.999 percent guaranteed.)

* * *

As you probably already do know,
Since it really does quite easily show,
This blog post was inspired by Oh, the Places You'll Go!

‘Twas Two Weeks Before Christmas

‘Twas two weeks before Christmas, and all about the data warehouse,
Every employee was stirring, busy clicking their mouse;
The stockings were hung on our cubicle walls with care,
In hopes that year-end bonus checks soon would be there.

The data were nestled all snug in their test beds,
While visions of sugar-plums danced in DBA's heads; 
Working together, the Business and IT, for collaboration is best,
All had just settled in, for a winter night's long, pre-production test.

When out in the parking lot there arose such a clatter,
We all sprang from our desk chairs to see what was the matter;
Away to the window we flew like a flash,
Tore open the shutters and threw up the sash.

The moon on the crest of the new-fallen snow,
Gave the luster of mid-day to objects below;
When, what to our wondering eyes should appear?

The Big Boss Man dressed up as Santa,
Carrying eight tiny candles, to Light the Menorah.

We descended the stairs to the lobby, so lively and quick,
We wanted to know in mere moments, if this was some trick;
The Big Boss Man greeted us, as into the lobby we all did file,
He whistled, and shouted, then gave us a big grinning smile.

He was dressed all in faux fur, from his head to his toes,
And his clothes were well-tailored with buttons and bows;
A bundle of bonus checks he had flung on his back,
We were as giddy as young children as he opened the sack.

His eyes—how they twinkled, his dimples how merry!
His cheeks were like roses, his nose like a cherry!
His droll little mouth was drawn up like a bow,
And the beard of his chin was as white as the snow.

The stump of a pipe he held tight in his teeth,
And the smoke it encircled his head like a wreath;
He had a broad face and a little round belly,
That shook when he laughed, like a bowlful of jelly.

He was chubby and plump, a right jolly old elf,
And we laughed when we saw him, in spite of ourselves;
A wink of his eye and a twist of his head,
Soon gave us to know, we had nothing to dread.

And these were the words that carefully he said:

“Whether you celebrate Christmas or Hanukkah, Kwanzaa or Festivus,
Whether for you, these are Holy Days or holidays, or simply a rest for us,
My words are the same, and they are just as bright:

Peace, Love, and Happiness to All,
And to all—A Good Night.”

To you and yours, from the entire OCDQ Blog family.

The Tell-Tale Data

It is a dark and stormy night in the data center.  The constant humming of hard drives is mimicking the sound of a hard rain falling in torrents, except at occasional intervals, when it is checked by a violent gust of conditioned air sweeping through the seemingly endless aisles of empty cubicles, rattling along desktops, fiercely agitating the flickering glow from flat panel monitors that are struggling against the darkness.

Tonight, amid this foreboding gloom with only my thoughts for company, I race to complete the production implementation of the Dystopian Automated Transactional Analysis (DATA) system.  Nervous, very, very dreadfully nervous I have been, and am, but why will you say that I am mad?  Observe how calmly I can tell you the whole story.

Eighteen months ago, I was ordered by executive management to implement the DATA system.  The vendor's salesperson was an oddly charming fellow named Machiavelli, who had the eye of a vulture — a pale blue eye, with a film over it.  Whenever this eye fell upon me, my blood ran cold. 

Machiavelli assured us all that DATA's seamlessly integrated Magic Beans software would migrate and consolidate all of our organization's information, clairvoyantly detecting and correcting our existing data quality problems, and once DATA was implemented into production, Magic Beans would prevent all future data quality problems from happening.

As soon as a source was absorbed into DATA, Magic Beans automatically did us the favor of freeing up disk space by deleting all traces of the source, somehow even including our off-site archives.  DATA would then become our only system of record, truly our Single Version of the Truth.

It is impossible to say when doubt first entered my brain, but once conceived, it haunted me day and night.  Whenever I thought about it, my blood ran cold — as cold as when that vulture eye was gazing upon me — very gradually, I made up my mind to simply load DATA and rid myself of my doubt forever.

Now this is the point where you will fancy me quite mad.  But madmen know nothing.  You should have seen how wisely I proceeded — with what caution — with what foresight — with what Zen-like tranquility, I went to work! 

I was never happier than I was these past eighteen months while I simply followed the vendor's instructions step by step and loaded DATA!  Would a madman have been so wise as this?  I think not.

Tomorrow morning, DATA goes live.  I can imagine how wonderful that will be.  I will be sitting at my desk, grinning wildly, deliriously happy with a job well done.  DATA will be loaded, data quality will trouble me no more.

It is now four o'clock in the morning, but still it is as dark as midnight.  But as bright as the coming dawn, I can now see three strange men as they gather around my desk. 

Apparently, a shriek had been heard from the business analysts and subject matter experts as soon as they started using DATA.  Suspicions had been aroused, complaints had been lodged, and they (now identifying themselves as auditors) had been called in by a regulatory agency to investigate.

I smile — for what have I to fear?  I welcome these fine gentlemen.  I give them a guided tour of DATA using its remarkably intuitive user interface.  I urge them audit — audit well.  They seemed satisfied.  My manner has convinced them.  I am singularly at ease.  They sit, and while I answer cheerily, they chat away about trivial things.  But before long, I feel myself growing pale and wish them gone.

My head aches and I hear a ringing in my ears, but still they sit and chat.  The ringing becomes more distinct.  I talk more freely, to get rid of the feeling, but it continues and gains volume — until I find that this noise is not within my ears.

No doubt I now grow very pale — but I talk more fluently, and with a heightened voice.  Yet the sound increases — and what can I do?  It is a low, dull, quick sound.  I gasp for breath — and yet the auditors hear it not. 

I talk more quickly — more vehemently — but the noise steadily increases.  I arise, and argue about trifles, in a high key and with violent gesticulations — but the noise steadily increases.  Why will they not be gone?  I pace the floor back and forth, with heavy strides, as if excited to fury by the unrelenting observations of the auditors — but the noise steadily increases.

What could I do?  I raved — I ranted — I raged!  I swung my chair and smashed my computer with it — but the noise rises over all of my attempts to silence it.  It grows louder — louder — louder!  And still the auditors chat pleasantly, and smile.  Is it really possible they can not hear it?  Is it really possible they did not notice me smashing my computer?

They hear! — they suspect! — they know! — they are making a mockery of my horror! — this I thought, and this I think.  But anything is better than this agony!  Anything is more tolerable than this derision!  I can not bear their hypocritical smiles any longer!  I feel that I must scream or die! — and now — again! — the noise!  Louder!  Louder!!  LOUDER!!!

“DATA!” I finally shriek.  “DATA has no quality!  NO DATA QUALITY!!!  What have I done?  What — Have — I — Done?!?”

With a sudden jolt, I awaken at my desk, with my old friend Edgar shaking me by the shoulders. 

“Hey, wake up!  Executive management wants us in the conference room in five minutes.  Apparently, there is a vendor here today pitching a new system called DATA using software called Magic Beans...” 

“...and the salesperson has this weird eye...”

Data Quality is People!

New York City, 2022 - Technical Architect Robert Thorn and Business Analyst Sol Roth have been called in by Business Director Harry Harrison and IT Director Tab Fielding to investigate an unsolved series of anomalies that have been plaguing the company's Dystopian Automated Transactional Analysis (DATA) system.

Harry Harrison (Business Director):

“Thank you for coming on such short notice.  We hope that you will be able to help us with our DATA problems.”

Robert Thorn (Technical Architect): 

“You're welcome.”

Sol Roth (Business Analyst): 

“I am sure that we can help.  Can you provide us with an overview of the situation?”

Harry Harrison (Business Director): 

“We hired quality expert William Simonson from Soylent Consulting to fix our DATA problems.”

Robert Thorn (Technical Architect): 

“Soylent Consulting?  Never heard of them.”

Sol Roth (Business Analyst): 

“They are the professional services division of Green, Incorporated.”

Harry Harrison (Business Director): 

“Yes, that's right - the High-Energy, Environmentally-Minded Corporation”

Tab Fielding (IT Director): 

“More like the High-Rate, Weak-Minded Corporation, if you ask me.”

Harry Harrison (Business Director): 

“Well anyway, Mr. Simonson first met with me to receive the business requirements.”

Sol Roth (Business Analyst): 

“Receive?  You mean he was handed the completed business requirements document?”

Harry Harrison (Business Director): 

“Yes, of course.”

Sol Roth (Business Analyst): 

“So...he didn't meet directly with anyone on the business team?”

Harry Harrison (Business Director): 

“No, I write the business requirements document so that meeting with the business team is unnecessary.”

Sol Roth (Business Analyst): 

“O...K...then what happened?”

Tab Fielding (IT Director): 

“Mr. Simonson met with me to receive the functional specifications.”

Robert Thorn (Technical Architect): 

“Receive?  So...did you write the functional specifications?”

Tab Fielding (IT Director): 

“Yes, I write the functional specifications after reading Mr. Harrison's business requirements document.”

Robert Thorn (Technical Architect): 

“Reading?  So...you didn't even meet directly with Mr. Harrison?”

Tab Fielding (IT Director): 

“No, before today's meeting, I haven't even seen him or anyone from the business team in months.”

Robert Thorn (Technical Architect): 

“O...K...then what happened?”

Tab Fielding (IT Director): 

“Mr. Simonson spent a few months coding, implemented our solution, then told us he was ‘going home.’”

Harry Harrison (Business Director): 

“And now our DATA is worse than ever!”

Sol Roth (Business Analyst): 

“And both of you are wondering how it came to this?”

Harry Harrison (Business Director) and Tab Fielding (IT Director): 

“Yes!”

Robert Thorn (Technical Architect): 

“I'll tell you how.  Because you both forgot the most important aspect of DATA.”

Tab Fielding (IT Director): 

“What are you talking about?”

Robert Thorn (Technical Architect): 

“It's people.  Data's Quality is made by People.  You've gotta tell them.  You've gotta tell them!”

Harry Harrison (Business Director): 

“I promise, Thorn.  I promise.  We will tell executive management.”

Robert Thorn (Technical Architect): 

“You tell everybody.  Listen to me, both of you!  You've gotta tell everybody that Data Quality is People!”