A data scientist copies a few thousand production records into a notebook to debug a model. The experiment ends in March. The notebook is still there in November — synced to a laptop, duplicated into a shared workspace, holding customer data that no inventory has ever listed.
AI/ML programs do not merely consume sensitive data. They create new forms of it. A training corpus may hold customer records, internal decisions, private communications, or proprietary process knowledge. The work around that corpus then creates derivatives: embeddings, prompts, evaluation sets, labels, notebooks, traces, and tuning decisions.
The derivatives can be worth as much as the source. An embedding index reveals semantic relationships. A prompt log exposes business logic. A notebook may contain credentials, sample records, or a shortcut that quietly became production practice. An evaluation set encodes exactly what the organization fears its system will mishandle.
This is the new sensitive data estate, and it rarely looks like a database. It looks like a bucket, a vector index, a cache, a transcript, a dashboard export, a pile of operational logs kept because they were useful during development.
The estate grows through normal work, which is what makes it hard to see. Teams experiment, copy samples, run evaluations, and preserve traces for debugging. None of that is careless. The risk arrives when the security model still watches only the original source systems while the pipeline has been quietly building new stores of concentrated context. A model project can create a sensitive data estate faster than governance can name it.
The defensive principle is simple: treat AI/ML artifacts according to what they can reveal, not according to their file extension, product category, or team boundary. The first control is knowing what now exists.
What to check now: build an inventory of AI/ML data stores and their derivatives — training data, embeddings, prompt and response logs, review queues, notebooks, model artifacts, and the exports made along the way.
What to check now: map access by task, not by team. Evaluating model quality rarely requires raw sensitive records. A support workflow may need a summary, not the prompt history. A development environment may need synthetic examples, not production traces.
What to check now: review retention defaults, then go looking for silent multiplication. Logs and intermediate artifacts get kept because they are useful, and usefulness needs an expiration date. The official system is often governed while the working copies — notebook downloads, copied evaluation sets, cached retrieval results, ad hoc exports — carry the real exposure.
We will keep returning to this estate, because it is where many AI security failures become practical. The problem is not only how the model behaves. It is the shape, movement, and memory of the data around it.