What is data lineage?
Data lineage is the record of where data originated, how it moved between systems and how it was transformed before reaching its current form. It can show the path from a source application through integrations, calculations, data warehouses, reports, machine-learning pipelines and downstream products. Lineage helps an organisation understand not only where a value is stored, but how it came to exist.
Lineage may be documented at different levels. Technical lineage traces tables, fields, files and transformation jobs, while business lineage connects those technical elements to concepts, owners, purposes and decisions. Both views are valuable because a technically correct flow may still lack clear business meaning or permitted use.
Why is data lineage important?
Organisations often rely on reports or models without knowing which source created the information or which transformations altered it. When an error appears, teams may spend days tracing dependencies manually. Lineage makes it easier to investigate quality issues, explain results and understand which products will be affected by a system change.
It also supports privacy and AI governance. Teams can identify where personal data travels, which systems must respond to correction or deletion, whether training data has an appropriate source and which downstream outputs may need to be updated after a problem is found.
How is data lineage captured?
Lineage can be collected automatically from databases, integration platforms, transformation tools and data catalogues. Metadata may reveal which field was copied, joined, filtered or calculated. Architecture diagrams and owner interviews can add context where automated tools cannot see manual exports, spreadsheets or external vendor processes.
The record should include meaningful transformations and dependencies rather than every minor technical event. Critical data elements, regulated information and high-impact AI systems normally deserve the most detailed coverage.
How should data lineage be governed?
Each important source and transformation should have an owner, description and review process. Changes to data structures or calculations should be tested for downstream impact. Lineage should connect to classification, quality rules, retention, access and approved purposes so that it becomes part of operational governance rather than a static diagram.
Where complete lineage is not available, organisations should record limitations and prioritise the flows that create the greatest business or privacy risk.
Frequently asked questions
Is data lineage the same as data mapping?
No. Lineage focuses on origin, movement and transformation, while data mapping often focuses on processing purposes, systems, recipients and legal or privacy context.
Does lineage include manual spreadsheets?
It should when they materially transform or distribute important data. Manual steps are often difficult to detect and may require owner interviews.
How does lineage help with AI?
It documents the provenance and transformations of training and operational data, making errors, permissions and model limitations easier to assess.
Can lineage be fully automated?
Technical lineage can be automated substantially, but business meaning, offline processes and external uses usually require human input.
Who is responsible for lineage accuracy?
Data and system owners should validate their flows, while central data-governance or architecture teams maintain standards and tooling.