Data Quality: What It Is, How to Measure It, and Where It Costs Least to Fix
In short
Data quality is the degree to which a dataset meets the requirements of the use it will be put to. It is evaluated through dimensions such as accuracy, completeness, consistency, validity, uniqueness, and timeliness, and the international reference standard is ISO/IEC 25012.
Measuring it is the familiar part. The part almost nobody discusses is where the control sits: at the origin, in the pipeline, or at consumption. That decision determines which errors you can catch and how much they cost to fix.
The industry overwhelmingly measures at the consumption layer, which is the most expensive and the latest. That bias has a historical reason, and it is already being corrected.
What Is Data Quality?
Data quality is the degree to which a dataset meets the requirements of the operational or analytical use it will be put to. It is not an absolute property of the data but a relationship between the data and its purpose: the same record can be good enough for a monthly report and inadequate for feeding a pricing model.
That conditionality is why quality is never solved once. A record that is correct today can stop being fit for use tomorrow because the use changed, not because the data did.
The Dimensions of Data Quality
Dimensions are the categories a dataset is evaluated against. Several taxonomies exist; the one most often cited as a formal reference is ISO/IEC 25012, which breaks quality into roughly fifteen characteristics. In day-to-day operations, these six account for nearly every real problem:
Dimension | What it measures | Example of failure |
Accuracy | Whether the value corresponds to reality | The recorded weight of a shipment does not match the physical one |
Completeness | Whether all required fields are present | A supplier with no bank account on file |
Consistency | Whether the value is the same across systems | The same customer with two different credit limits |
Validity | Whether it conforms to the defined format and rules | A tax identifier with the wrong structure |
Uniqueness | Whether the entity appears only once | The same material under two part numbers |
Timeliness | Whether it reflects the current state | A code catalog that was not refreshed after an official change |
A seventh dimension, fitness for purpose, is not measured like the others because it depends on whoever consumes the data. It is what explains how a dashboard can be entirely green while the finance team still does not trust it.
What Happens When Quality Fails
The cost of poor quality is hard to calculate because it spreads across systems, teams, and time. IBM frames it this way in its January 2026 analysis: poor quality rarely shows up at the point where it fails. It surfaces downstream, as lost revenue, inefficiency, and compliance exposure. That lag between cause and symptom is what makes it dangerous.
The same analysis cites the IBM Institute for Business Value 2025 CDO Study, where 43% of COOs identify data quality issues as their single most significant data priority.
Three ways it shows up, across different domains:
An outdated catalog stops the operation. Tax and logistics systems validate against official catalogs that change over time. A code that no longer exists triggers rejections at the moment a document is issued, not in next month's report. The cost is paid on the road, not on the dashboard.
A duplicate ties up capital. The same component created under two naming conventions means two plants each see half the real inventory and both place emergency orders. Consolidated inventory says coverage is fine.
An ingestion error poisons a model. The most widely cited public case is Unity Technologies, which in 2022 reported roughly USD 110 million in lost revenue after bad ingested data corrupted the datasets training its advertising models. It was not a model problem. It was an input problem.
Where a Record Gets Fixed: The Three Control Layers
This is the decision most articles on the subject never put on the table. Quality can be controlled in three different places, and each one sees things the others cannot.
Origin layer. The control lives at the moment the record is created or captured: format rules, conditional mandatory fields, validations against external sources, approvals before the record exists at all. It catches the error while it still costs nothing.
Pipeline layer. The control lives in ingestion and transformation: schema tests, volume counts, anomaly detection, quarantine for partial loads. It catches what the origin cannot see, because it only becomes visible when sources are crossed.
Consumption layer. The control lives at the end: the report that does not reconcile, the model that drifts, the user who raises a hand. It catches almost everything, always late.
Layer | What it catches well | What it cannot see | Relative cost to fix |
Origin | Format, business rules, invalid entities, missing fields | Errors that only appear when systems are crossed | Low |
Pipeline | Schema changes, partial loads, cross-source duplicates, volume anomalies | Data that is formally valid but false | Medium |
Consumption | Inconsistencies the business recognizes as impossible | Everything above, until it has already happened | High |
The cost progression is not intuition. The 1-10-100 rule, used for years in incident management, holds that fixing a problem at the point of entry costs roughly one unit, that once propagated through the system it costs around ten, and that if it reaches the end user or a business decision it can cost a hundred. The exact figures vary by organization. The shape of the curve does not.
No layer replaces the others. A serious program has controls in all three. The problem is that most organizations only have controls in the last one.
Why the Industry Measures in the Wrong Layer
The bias has a concrete origin: data quality tooling grew out of the analytics side, gets installed where analytical data lives, and therefore measures where the data has already arrived.
This is stated openly. In its article on the subject, Databricks says that any data entering an analytics platform is unlikely to meet quality requirements, and that quality is achieved by cleaning and transforming data over time. Its entire improvement section is organized around quality during the ETL process.
That is an honest description of how things work today. It is also a description of a problem accepted as a law of nature. If we take for granted that data arrives broken, the only question left is how much cleanup we can afford.
What is interesting is that the industry itself is already moving. In the analysis cited above, IBM argues that traditional approaches, meaning reviewing quality exclusively inside the data warehouse, no longer scale, and that organizations need to shift left on data integrity: pushing detection, prevention, and remediation closer to the moment data is created rather than waiting for problems to surface downstream. Among the practices it recommends is validating at the point of entry rather than after consumption.
The dates are worth noting. IBM's definitional article that currently dominates searches on this topic was published in 2022 and mentions none of this. The shift-left argument is from January 2026. The distance between those two texts is exactly the distance between how data quality is explained and how it is being solved.
What Controlling at the Origin Actually Means
Moving control upstream does not mean adding a form with required fields. It means three concrete things.
That the business rule executes before the record exists, not after. A validation that flags the record and lets it through produces a marker somebody will review someday. A validation that stops the process produces the absence of the problem.
That verification against external sources happens inside the flow, not in a later review. Checking a restriction list after the record is already in the ERP and has already generated transactions is auditing, not quality.
That the process producing the data can be changed. Rules change: official catalogs get updated, regulations shift, internal policies are adjusted. If changing one validation requires a multi-week development cycle, the origin control degrades on its own, without anyone touching it.
When the data in question describes an entity that several systems share, such as a supplier or a material, origin control is implemented by concentrating authoring in a single place: the record is created once, under rules, and distributed from there. For everything else, origin control lives in the ingestion layer and in the contracts between whoever produces the data and whoever consumes it.
How to Measure Data Quality: Metrics and Thresholds
Measuring quality is not calculating a global percentage of good data. It is instrumenting specific dimensions with thresholds defined by whoever consumes the data. The standard repertoire (error rate by dimension, validation coverage, mean time to detection and resolution, the quality matrix, and KPIs by industry) is developed in detail in our guide to data quality metrics.
There are two metrics that rarely appear in those frameworks and that reveal the structural health of the program:
Distance from the origin at detection. Which layer caught each problem. If practically everything is caught at consumption, the program has no preventive control, whatever the dashboards say.
Time from a rule change to production. Whether your controls can keep pace with reality. A control that takes weeks to update has stopped being a control.
Neither appears in the classic frameworks because both assume the measurement point has already been decided. Once it becomes a decision, it has to be measured.
Quality as Evidence, Not Just as a Report
There is a function of data quality that traditional frameworks do not cover: serving as proof. In processes where several teams argue over the same numbers, the problem is not always that the data is wrong. Sometimes it is that nobody can demonstrate what state it was in.
Three mechanisms solve that:
Initial-state snapshot. Capturing the exact state of the data at the instant a process is declared open. It eliminates disputes over transactions that arrived late, because there is a record of what existed and when.
Dry runs. Running the full process before the real one, so anomalies surface with time to react instead of being discovered in production.
An auditable log by stage. Recording times and results per table or per batch, so that when something is slow or fails you can point at exactly where, instead of debating impressions.
None of this appears in any taxonomy of dimensions, and in practice it is what turns an hour of complaints into a five-minute review.
Data Quality and AI-Ready Data
A model inherits every defect in its input data and applies it at scale. An agent querying a catalog with duplicates propagates them into every answer. AI does not filter poor quality. It amplifies it and makes it harder to trace.
That is why moving controls upstream stopped being an architectural preference. When systems consume data continuously rather than episodically, checking quality in an overnight processing window stops making sense: by the time the report catches the problem, the agent has already acted on it dozens of times.
Three Cases: Where the Control Was Placed
Unifying 21 countries in 120 days. A global food company ran its logistics and commercial model through three separate providers, with schemas that changed without notice and outside its control. Arkon Data ran exhaustive validation of historical loads and current data at the ingestion layer, completely avoiding duplicates and integrity loss across the 200 migrated tables. The result: 21 countries operating under one business logic, roughly 4,200 data entities governed by a team of 9, and a transition with zero downtime. The control was placed at the entrance, with two dedicated QA specialists covering consistency across about 200 tables per country.
Financial close with transactional evidence. The same client consolidated millions of transactions in a process that ran up to 6 hours, saturated the ERP, and ended in complaint meetings. An initial-snapshot mechanism was implemented to capture the exact state of the data the second the client declares the close, plus internal dry runs ahead of the real close and an auditable log of extraction times per table. Times dropped 72%, from up to 6 hours to 1 hour 40 minutes while supporting twice the accounting tables, with zero impact on the source system. Hour-long complaint meetings became 10-minute retrospectives.
Reconciliation with external validation. Matching millions of invoices against accounting records was classified as impossible: the ERP's native report would not run, and Power BI hit a structural limit of 150,000 rows against millions of records. With the logic migrated and exhaustive historical test cycles, automated reconciliation raised the match rate from 96% to 99%, so that today only 1% requires manual validation, across 42 data entities spanning multiple countries. The external auditor, KPMG, adopted the resulting report for the audit of the most critical accounts over its own queries.
Frequently Asked Questions About Data Quality
What is the difference between data quality and data integrity?
Quality is the broad category: accuracy, completeness, validity, consistency, and the other dimensions. Integrity is a subset focused on data not being corrupted or altered across its lifecycle, and it is usually approached from a security and change-control perspective.
What is data profiling?
It is the technical process of examining a dataset in detail, comparing it against its metadata, and computing statistics to surface incorrect patterns, null values, and anomalies. Profiling measures; quality is what gets measured.
Can you reach zero errors?
No, and chasing it is expensive and unproductive. The goal is to define tolerance thresholds by dimension, based on what the data consumer needs, and measure deviation against those thresholds.
What is shift left in data quality?
It means moving detection, prevention, and correction toward the moment data is created, instead of waiting to catch problems in the warehouse or in the report.
Is data quality the same as data governance?
No. Governance defines who decides and who is accountable for the data. Quality is one of the properties governance is responsible for guaranteeing, with its own metrics and thresholds.
Where do you start a data quality program?
By identifying which layer is catching most of your problems today. If nearly all of them are caught at consumption, the first job is not buying a tool, it is moving controls upstream. The full step-by-step path is in our guide on crafting an effective data quality plan.
From Diagnosis to Execution
The question worth answering before investing in tooling is simple, and almost nobody has the answer: of the quality problems your organization caught last quarter, how many were caught at the origin, how many in the pipeline, and how many once they were already in a report or in a customer's hands?
Arkon Data's data quality solution works on two fronts at once. It connects business processes to the data they generate, so rules apply at the moment of capture and the error stops being produced. And it corrects the data that is already damaged, because the records that entered badly before any control existed are still there and someone has to clean them.
That second part is what shift-left thinking usually leaves out. Moving control upstream stops the problem, but it does not repair what accumulated. A complete program does both, and in that order: close the tap first, then clean up.
The work does not start with a tool. It starts with process engineers mapping where each critical data element is actually created, which rules apply to it today, including the ones that live only in the head of whoever applies them, and how damaged the historical record is. Two tracks come out of that: the controls that prevent the next error, and the remediation of what already entered.
Let's set up a short conversation, no obligation, to review which layer you are correcting in today and how large the backlog is that needs repairing.