What Actually Happens to Your Data Inside an AI Tool

Every AI data-privacy incident this year asks a version of the same four questions. Answer them once, for every tool your team already has connected, and you stop re-deriving the checklist from scratch each time a new one surfaces.

Three incidents, one shape

A personal AI assistant kept full copies of synced email after users revoked its Google access — disconnecting stopped it from reading new mail, but did nothing to what it had already copied. Atlassian started training its AI on Jira and Confluence content this year, opted in by default below Enterprise tier — most teams never noticed the plan-tier line that decided whether they had a choice. A free PII filter released this spring catches names and account numbers at 96% accuracy — and misses deal values, unreleased plans, and client names entirely, because that was never the category it screens for.

Three vendors, three different failures, and the same shape underneath: each team had assumed an answer to a question nobody had actually put to the vendor. Retention, training use, and screening scope are three separate settings. A tool can get one right and the other two wrong, and the marketing page rarely says which.

The audit, once

Run this against every tool that touches anything you wouldn't want repeated, and keep the answers somewhere your team can find them again the next time a vendor changes its policy:

1. Retention. What's kept, and for how long? Zero retention and a 30-day window are both legitimate answers. "We don't really specify" is not.
2. Training. Does this plan tier train on your inputs by default, and is opting out available below Enterprise — or only above it?
3. Disconnection. If you revoke access, does that delete what the tool already copied, or only stop it from reading anything new?
4. Screening scope. If a PII filter sits in front of the tool, does anything still need a separate pass for commercially sensitive content — deal terms, roadmaps, client names — that a PII filter was never built to catch?

Four questions, one pass, before anything goes in that you'd regret being retained, trained on, or half-screened.

Worked example: the finance controller choosing a forecasting tool

Renata is finance controller at a 90-person manufacturing company evaluating an AI forecasting assistant that would connect to the company's accounting system and pull three years of transaction history. The vendor's site says "bank-level security" and stops there. Before she signs anything, she asks support the four questions directly: how long is transaction data kept, does the Growth-tier plan she's on train on customer data by default, and if the company cancels next year, does that history get deleted or just made inaccessible to her. The answer to the third question — "inaccessible, not deleted, per our data processing addendum" — is what she takes to legal, not the marketing page.

Second example: the marketing director with client campaign data

Dev runs marketing at a 20-person creative agency and wants to use an AI tool to draft campaign copy from client briefs that include unreleased product names and launch dates. He already runs briefs through a PII filter out of habit, the same one covered in April's release. It catches the client contact's name and email every time. It has never once flagged the unreleased product name sitting two lines below — because a launch date isn't personally identifiable information, and the filter was never built to know it's confidential. Dev's actual fix isn't a better filter. It's treating "screened for PII" and "cleared for release" as two separate checks, with the second one done by a person, every time.

The takeaway

The vendor's marketing page will tell you what it wants you to conclude. These four questions tell you what's actually true, and they only need answering once per tool.