Data quality is a business problem, not an IT problem

Bad data is rarely created by databases. It is created by processes, incentives and forms that make the wrong thing easy.

7rayfi StudioProduct, data & engineering
Published
Reading time3 min
A computer screen full of analytics data
Photo: 1981 Digital on Unsplash

When a report shows numbers nobody believes, the first reaction is usually to blame the system: the database, the export, the integration. Sometimes that is fair. Far more often, the data is an accurate record of an inaccurate process.

Where bad data really comes from

Duplicate customers appear because searching for an existing one takes longer than creating a new one. Missing fields appear because the form allows them to be skipped and nobody downstream complains quickly enough. Inconsistent product names appear because three people type them freely, each with their own habits.

In every case, the data reflects what the process makes easy. Cleaning it after the fact helps for a while, but the problem returns with next week's entries.

Fix it where it is created

The most effective data-quality work we do is not in the data warehouse. It is in the product and the process:

  • Constrain choices that should be constrained: a list instead of free text, a lookup instead of retyping.
  • Make the right action the fast one: finding an existing customer should take one search, not five clicks.
  • Validate at entry, with messages that explain the rule in business language.
  • Assign ownership: every important field has someone who cares if it is wrong.

Data quality improves the day someone who creates the data also benefits from it being right.

Measure it like any other KPI

"Our data is bad" is not actionable. "12% of orders have no delivery date" is. We like to define a few simple quality indicators — completeness, duplicates, freshness, consistency — for the data that feeds important decisions, and to track them over time. Making them visible is often enough to start improving them.

What a first data-quality sprint looks like

When a client asks us to "clean up the data", we rarely start with a cleaning script. We start with a short investigation, usually a week or two long.

  1. Pick the decision. Which report or process suffers most from unreliable data? Margin by product? Customer follow-up? Stock levels?
  2. Trace the fields. For the handful of fields that feed that decision, we find out where each one is created, by whom, in which screen, and what happens when it is wrong.
  3. Measure the gap. Simple queries tell us how often each field is missing, duplicated, stale or inconsistent. Those become the baseline.
  4. Fix the source. We change the form, the default, the lookup or the ownership — the cause, not the symptom.
  5. Clean once, then watch. Only after the source is fixed do we clean the history, and we keep the quality indicators on a small dashboard so drift is noticed early.

The result is modest in scope but durable: one decision backed by data people trust, and a method the team can repeat on the next one.

Start small and close to value

Trying to fix all data everywhere is a multi-year programme that rarely finishes. It is better to start with the data behind one decision that matters — margin by product, late deliveries, patient follow-ups — and make that trustworthy end to end. Once people see a report they can believe, the appetite for the next one follows naturally.

  • Data quality
  • Data-driven decisions
  • Processes

Working on something similar?

We help companies turn problems like this one into working products. Tell us what you’re dealing with.

Start a conversation