Data Governance for AI

AI data governance is the part of an AI program that decides which data a model or assistant may use, who approved it, how good it is, where it came from, who is allowed to see what it returns, and how long prompts, outputs and derived data are kept. It is the prerequisite nobody wants to fund because it looks like plumbing, and it is why most stalled AI pilots stalled: the demo ran on a clean extract and production ran on the real data. This guide shows where AI projects break on data, gives an AI-ready checklist, and explains how to govern retrieval-augmented generation (RAG) and training data differently, for a 20-person company and a 2,000-person one.

Where AI projects break on data

Nobody owns the data the model needs

An AI use case usually draws on several domains at once: customer records, support tickets, contracts, product data. If none of them has an accountable owner, there is no one to approve the use, answer quality questions or sign off on access. The project either waits or proceeds without approval, and the second option is how incidents happen.

Definitions differ between the pilot and production

The pilot used a dataset where “active customer” meant one thing; the production system uses three systems where it means three things. A model trained or prompted on one definition produces confident answers against another. Governance fixes this with a glossary and a system of record per term, before the model sees any data.

Quality defects become model behavior

Duplicate customers, stale prices, missing fields and inconsistent codes do not average out; they are learned or retrieved and repeated with fluency. Quality rules on the critical data elements, measured before the data enters a training set or an index, are the simplest fix available, and nearly always skipped under deadline pressure.

Access rights do not carry into retrieval

A RAG assistant indexes a document store and answers questions from it. If the index ignores the permissions on the source documents, anyone who can ask the assistant can read the HR folder. This is the most common governance failure in generative AI deployments, and it is a data governance problem: classification and access rules have to be enforced at retrieval time, not just at the source.

Purpose and consent were never recorded

Data collected for one purpose (billing, support, clinical care) may not be usable for another (training a model, profiling) under privacy law, contract terms or your own privacy notice. If the purpose for which data was collected is not recorded anywhere, nobody can say whether the AI use is permitted. This is general information, not legal advice; check the current text of the privacy laws and contracts that apply to you.

No lineage, so nothing can be explained or reproduced

When a model gives a wrong answer, the first question is which data produced it. Without data lineage from source systems through transformations to the training set or index, that question has no answer, and neither does an auditor’s question about what the model was trained on.

Prompts, outputs and embeddings are retained by accident

AI systems create new data: prompts that contain customer details, outputs that may be wrong, embeddings derived from confidential documents, logs that hold all three. If the retention schedule does not cover them, they accumulate indefinitely in places the classification scheme never considered, including third-party vendors’ systems.

The AI-ready checklist for AI data governance

Run this data governance and AI checklist for each AI use case before it leaves the pilot stage. Every item should be a clear yes, with a name next to it. A 20-person company can complete it in a day for one assistant; a 2,000-person company should make it the standard intake form for AI projects.

• Ownership: every dataset the use case touches has a named owner who has approved this use in writing.
• Definitions: the business terms the model relies on are defined in the glossary, with a system of record for each.
• Classification: every dataset and document collection in scope carries a classification label; restricted and sensitive data are either excluded, de-identified, or explicitly approved with compensating controls.
• Quality: quality rules exist for the critical data elements, have been measured on the actual data (not a sample extract), and the results are acceptable to the owner.
• Lineage: the path from source systems to the training set or index is documented, including transformations and filters, and can be reproduced.
• Access: the AI system enforces the same access rules as the source systems, at retrieval or query time, and this has been tested with users of different roles.
• Purpose and consent: the purpose for which the data was collected permits this use, or consent and notices have been updated; third-party data is covered by license or contract terms that allow it.
• Vendor terms: contracts with AI vendors state whether prompts and outputs are retained or used for training, where data is processed, and how deletion is handled; regulated data has the required agreements in place.
• Retention: prompts, outputs, logs, embeddings and fine-tuned models are in the retention schedule with a classification and a deletion path.
• Monitoring: someone reviews data quality, access logs and output samples on a schedule after launch, and there is a route for users to report wrong or inappropriate outputs.
• Record: the use case, its datasets, approvals and controls are entered in the AI inventory that the AI governance program maintains.

Governing RAG and training data

Retrieval-augmented generation and model training use data differently, so the governance controls differ too. RAG retrieves documents at question time; training or fine-tuning bakes data into a model. The table shows what changes.

Question	RAG (retrieval at query time)	Training and fine-tuning
What data is in scope	Document collections and structured sources that are indexed; changes continuously	A fixed training set, assembled and versioned once per model version
Freshness	Index must be refreshed as sources change; stale chunks produce stale answers	Frozen at training time; drift is managed by retraining
Access control	Enforced per query: the index stores the source permissions and filters results by the asking user’s rights	Enforced before training: anything in the set is effectively visible to every user of the model, so restricted data is excluded or de-identified
Sensitive data	Excluded from the index or filtered by classification label and user role	Removed or de-identified before training; cannot be deleted from a trained model later
Lineage	Each chunk carries source, owner, classification and date as metadata, and answers cite them	A training data register (dataset card) lists sources, filters, versions and approvals
Deletion	Remove the source document and its chunks and embeddings from the index	Requires retraining without the data; plan for it before training on anything deletable
Evaluation	A governed question set with expected answers and sources, rerun on each index refresh	A held-out evaluation set governed like any other dataset, with an owner and a version

Practical rules for RAG

Index only collections with a named owner and a classification label. Store the source document’s permissions with each chunk and filter at query time; test it by asking the same question as a manager and as a new hire. Attach owner, classification, source and last-modified date as chunk metadata, and have the assistant cite sources so users can check. Schedule index refreshes against the source systems’ change rate, and propagate deletions. Keep the question-and-answer evaluation set under version control, with the data steward responsible for it.

Practical rules for training data

Treat every training set as a dataset with an owner, a card describing its sources, filters, versions and approvals, and a classification. Exclude or de-identify restricted data before assembling the set, because a trained model cannot forget selectively. Check license and contract terms for any third-party data. Keep the set and the evaluation set versioned so a model can be reproduced and its data explained to an auditor. Record the model version and its training set in the AI inventory.

At two sizes

A 20-person company connecting an assistant to its shared drive needs an owner for the drive, a sweep that moves HR and financial folders out of the indexed scope, permission-aware retrieval, and a one-page record of what was indexed and who approved it. A 2,000-person company training a forecasting model on five years of sales data needs a training data register, de-identification of customer identifiers, lineage from the warehouse to the training set, a governed evaluation set, and sign-off from the customer and finance data owners before the first training run. The controls scale; the questions do not change.

These data controls are the foundation that AI governance frameworks assume. The NIST AI Risk Management Framework, published at nist.gov, and the ISO/IEC 42001 management system standard both treat data quality, provenance and access as inputs that the AI program must be able to show. Our guide to AI governance vs data governance maps where one discipline ends and the other begins, and what AI governance is covers the model-side controls.

Where PS3G fits

PS3G runs data governance consulting and implementation as the first phase of AI work, so the checklist above is completed before a pilot becomes a product. Our AI development team builds RAG assistants, agents and custom models with permission-aware retrieval, lineage and evaluation sets designed in from the start, and our AI governance practice supplies the inventory, risk assessment and policies the data controls feed into. We work with organizations of every size, on Microsoft and open-source stacks, and we apply the same controls to the AI features in our own regulated software products.

Related reading

• What Is AI Governance? A Practical Guide for Any Size of Organization
• AI Governance vs Data Governance: Differences and Overlaps
• Data Lineage: What It Is, Why It Matters and How to Capture It
• Data governance consulting and implementation services
• AI governance services

Frequently asked questions

What is AI data governance?

AI data governance is the application of data governance to the data that AI systems consume and create: deciding which datasets may be used for training or retrieval and who approved it, classifying and quality-checking that data, recording its lineage, enforcing access rights at query time, honoring the purpose and consent under which data was collected, and managing retention of prompts, outputs, embeddings and models. It is the data half of AI governance.

Why do AI projects fail because of data governance?

Because the pilot ran on a clean extract and production runs on real data with no owner, inconsistent definitions, quality defects, permissions that do not carry into retrieval, and consent nobody recorded. Each gap either blocks approval or causes an incident after launch. Fixing them before the pilot ends is slower for a month and faster for every month after, which is why it is the prerequisite.

How is governing RAG data different from governing training data?

RAG retrieves documents at question time, so governance focuses on keeping the index fresh, storing source permissions with each chunk and filtering by the user’s rights, and propagating deletions. Training bakes data into a model, so governance focuses on assembling a versioned, approved, de-identified training set with a register of sources, because restricted data cannot be removed from a trained model later.

Can a small company do AI data governance?

Yes, and it is short. For an assistant connected to company documents, a small company needs an owner for the document store, a sweep that removes HR, financial and other restricted folders from the indexed scope, permission-aware retrieval that has been tested with two different user roles, vendor terms that cover retention of prompts and outputs, and a one-page record of what was indexed and who approved it.

Does data governance for AI require a special tool?

No. The controls are the same ones a data governance program already runs: ownership, glossary, classification, quality rules, lineage and access reviews, extended to indexes, training sets, prompts and outputs. Catalogs such as Microsoft Purview, Collibra, Alation, Atlan or OpenMetadata can record the AI datasets alongside everything else, and the AI inventory can start as a shared workbook. The gap is usually process, not software.