Problem
Most data-governance tooling makes a claim no one verifies: that it finds the violations. But how do you know an audit caught everything, when you don't know what was there to begin with? This project flips the usual approach — it manufactures a dataset with a known number of planted violations, so the audit can be graded against an answer key instead of taken on faith.
Data
50,000 synthetic customer records generated with Faker, deliberately seeded with 4,447 known violations — retention-period breaches, unmasked PII, and consent gaps — spread across the dataset, giving an exact ground truth to measure detection against.
Method
A layered audit: rule-based detectors for retention and consent violations, pattern scanners for unmasked PII (emails, card numbers, national IDs), and dbt models that apply and verify masking. Great Expectations enforces data-quality contracts, and a DuckDB query layer answers regulator-style requests — including a Subject Access Request lookup — on demand.
Finding
The audit caught 100% of the 4,447 seeded violations — including 847 unmasked PII patterns and roughly 2,400 retention breaches — with zero known misses against the answer key.
The Subject Access Request lookup query took 3.2 milliseconds in the project benchmark; this measures one query, not the complete request-handling process.
Recommendation
Validate the audit rules against the target system before scheduling checks and alerts. Retain seeded violations as regression fixtures after schema changes, and supplement them with tests for unseen formats and false positives.
Limitations and next steps
Detection is measured against seeded violations in synthetic records. It does not establish coverage of unknown patterns or certify regulatory compliance. Next steps are testing unseen formats and false positives, checking larger workloads, and reviewing rules against the requirements of a specific data system.