Who profits from poor data quality?

In the course of reading I found something that I believe puts a very different perspective on data management. We talk about how real time fast retrieval based analytics reporting can be used to grow the business. The truth it that it is not always so. Modern business systems have high integrity built in and can present  data to you on demand, so you can ask questions that perhaps can be of high impact . That is the real value

But when a system process is broken or poorly designed, aside from fixing the cause, it seems bad data can provide business opportunities for the unscrupulous?

The question posed in the research material I read  asked, imageWho profits from poor data quality? 

Well apparently, the retail industry does—sometimes.

Poor data quality (and poor information quality in the case of intentionally confusing fine print) definitely has a role to play with things such as mail-in rebates—and it’s a supporting role that can definitely lead to increased profits.

In a post by Jim Harris  dated 06/16/2010 he describes how this works. When I read it I saw how poor data engineered to be supplied this way allows some dubious business practices  to scam the consumer.  His post is also re-published in the Smart Data Collective,

It is an excellent read with congratulations to the author. I have also reposted here ~

A few months ago, during an e-mail correspondence with one of my blog readers from Brazil (I’ll let him decide if he wishes to remain anonymous or identify himself in the comments section), I was asked the following intriguing question:

“Who profits from poor data quality?”

The specific choice of verb (i.e., “profits”) may have been a linguistic issue, by which I mean that since I don’t know Portuguese, our correspondence had to be conducted in English.

Please don’t misunderstand me—his writing was perfectly understandable.

As I discussed in my blog post Can Social Media become a Universal Translator?, my native language is English, and like many people from the United States, it is the only language I am fluent in.  My friends from Great Britain would most likely point that I am only fluent in the American “version” of the English language, but that’s a topic for another day—and another blog post.

When anyone communicates in another language—and especially in writing—not every word may be exactly right.

For example: Muito obrigado por sua pergunta!

Hopefully (and with help from Google Translate), I just wrote “thank you for your question” in Portuguese.

My point is that I believe he was asking why poor data quality continues to persist as an extremely prevalent issue, especially when its detrimental effects on effective business decisions has become painfully obvious given the recent global financial crisis.

However, being mentally stuck on my literal interpretation of the word “profit” has delayed my blog post response—until now.

Promoting Poor Data Quality

In economics, the term “flight to quality” describes the aftermath of a financial crisis (e.g., a stock market crash) when people become highly risk-averse and move their money into safer, more reliable investments.  A similar “flight to data quality” often occurs in the aftermath of an event when poor data quality negatively impacted decision-critical enterprise information.

The recent recession provides many examples of the financial aspect of this negative impact.  Therefore, even companies that may not have viewed poor data quality as a major risk—and a huge cost greatly decreasing their profits—are doing so now.

However, the retail industry has always been known for its paper thin profit margins, which are due, in large part, to often being forced into the highly competitive game of pricing.  Although dropping the price is the easiest way to sell just about any product, it is also virtually impossible to sustain this rather effective, but short-term, tactic as a viable long-term business strategy.

Therefore, a common approach used to compete on price without risking too much on profit is to promote sales using a rebate, which I believe is a business strategy intentionally promoting poor data quality for the purposes of increasing profits.

You break it, you slip it—either way—you buy it, we profit

The most common form of a rebate is a mail-in rebate.  The basic premise is simple.  Instead of reducing the in-store price of a product, it is sold at full price, but a rebate form is provided that the customer can fill out and mail to the product’s manufacturer, which will then mail a rebate check to the customer—usually within a few business weeks after approving the rebate form.

For example, you could purchase a new mobile phone for $250 with a $125 mail-in rebate, which would make the “sale price” only $125—which is what the store will advertise as the actual sale price with “after a $125 mail-in rebate” written in small print.

Two key statistics significantly impact the profitability of these type of rebate programs, breakage and slippage.

Breakage is the percentage of customers who, for reasons I will get to in a moment, fail to take advantage of the rebate, and therefore end up paying full price for the product.  Returning to my example, the mobile phone that would have cost $125 if you received the $125 mail-in rebate, instead becomes exactly what you paid for it—$250 (plus applicable taxes, of course).

Slippage is the percentage of customers who either don’t mail in the rebate form at all, or don’t cash their received rebate check.  The former is the most common “slip,” while the latter is usually caused by failing to cash the rebate check before it expires, which is typically 30 to 90 days after it is processed (i.e., expiration dated)—and regardless of when it is actually received.

Breakage, and the most common form of slippage, are generally the result of making the rebate process intentionally complex.

Rebate forms often require you to provide a significant amount of information, both about yourself and the product, as well as attach several “proofs of purchase” such as a copy of the receipt and the barcode cut out of the product’s package.

Data entry errors are perhaps the most commonly cited root cause of poor data quality.

Rebates seem designed to guarantee data entry errors (by encouraging the customer to fill out the rebate form incorrectly).

In this particular situation, the manufacturer is hyper-vigilant about data quality and for an excellent reason—poor data quality will either delay or void the customer’s rebate.

Additionally, the fine print of the rebate form can include other “terms and conditions” voiding the rebate—even if the form is filled out perfectly.  A common example is the limitation of “only one rebate per postal address.”  This sounds reasonable, right?

Well, one major electronics manufacturer used this disclaimer to disqualify all customers who lived in multiple unit dwellings, such as an apartment building, where another customer “at the same postal address” had already applied for a rebate.


Statistics vary by product and region, but estimates show that breakage and slippage combine on average to result in 40% of retail customers paying full price when making a purchasing decision based on a promotional price requiring a mail-in rebate.

So who profits from poor data quality?  Apparently, the retail industry does—sometimes.

Poor data quality (and poor information quality in the case of intentionally confusing fine print) definitely has a role to play with mail-in rebates—and it’s a supporting role that can definitely lead to increased profits.

Of course, the long-term risks and costs associated with alienating the marketplace with gimmicky promotions take their toll.

In fact, the major electronics manufacturer mentioned above was actually substantially fined in the United States and forced to pay hundreds of thousands of dollars worth of denied mail-in rebates to customers.

Therefore, poor data quality, much like crime, doesn’t pay—at least not for very long.

I am not trying to demonize the retail industry.

Excluding criminal acts of intentional fraud, such as identity theft and money laundering, this was the best example I could think of that allowed me to respond to a reader’s request—without using the far more complex example of the mortgage crisis.

10 thoughts on “Who profits from poor data quality?

  1. Just experienced a similar case, where a booking company lost over 30K USD from poor data quality when it failed to analyze the number of overbookings for that day.

  2. Gordan,
    Reading your eloquent reply caused me to reflect back on the old “tax shelter” days of years gone by. Boy, did unwitting investors take their lumps when they realized that all those wonderful losses morphed into evil tax consequences.

    My website is hosted by a terrific organization called Hubspot. It was founded by some guys from MIT, and is fueled by data and super analytic tools. The Hubspot motto is “In God we trust, but show me the data.”

    Speak to you soon.

  3. I was thinking of data integrity of retail world from another angle. Where cashier made mistakes during double entry on the payment terminal and on the POS workstation. This creates problem for the accounts department and month end closing is hell with poorly consolidated reports.

    Good article that provokes thinking

  4. Larry

    I love your metaphor as I now I imagine people en-mass going into Macys with whiskers brislings looking for some cheese. I can also imagine you as a champion of fair play as steam is coming from your collar and making your ears red as you watch them getting wacked.

    In the good old days when I was studying commercial tax law at university our professor, who was very wise ways of the tax had a favorite expression. His wry delivery said it all when he was presented with some logical question on the validity of paying money with a plan get it back later under some scheme offered by a vendor. This often found him answering with “Did you ever see the pea and thimble trick being performed?”

    One of the worst such rebate schemes I can think of is a tax scheme on a mass scale that was promoted in investment property. In my youth in the now long forgotten growth and prosperity period that we lived in, people on good incomes often sought to reduce their tax bill by borrowing money on such investments. This had the effect to make the investment negative in the initial years but they got the offset in so call tax relief while inflation discounted capital gains over time on naturally rising property values caught it up.

    It seemed a great scheme then and was followed by so many who took advice from property salesmen and tax planners who made the immediate buck. But in fact all the investor was doing was subsidizing the tenant who got market rates based on yield that by then took the tax credit into account and caused the investment loss that gave rise to the tax rebate. The pea and thimble trick on this one was the best ever as banks, the government and property developers all made a killing for decades.

    In the world of understanding data it is also said figures can lie but lies can’t figure!! I learned that saying from another wise man, my father, who sometimes talked about Politian’s in the same context.

    And if you don’t believe that he would add, you better believe there is no such thing as a free lunch except of course if you are a public servant working for the government.

    Larry, I can send you some burn cream if it helps but do I hope the ear pain passes soon. In the interests of animal welfare too I do hope your comments do cause some mice to take note and be spared to indulge is some nice healthy cheese instead.


  5. Gordan,
    What a terrific, thought provoking article. When I see a mail in rebate form, it makes my ears turn red. These forms are like the cheese on a mouse trap. They lure you in with a promise of significant savings on an overpriced product, and then cause a lot of pain. 40% failure to collect rate…amazing.

    I’m trying to think about other industries that profit from poor data quality. My first thought is Government. Can you imagine what might happen if we understood the tax laws of our country?

    Can you come up with any other examples?

    Looking forward to continuing our conversation.
    Great post!

Leave a Reply