Vetting an AI tool before you approve it

the stance: match the tool to a data class

Start from the right framing, because it changes every check that follows. You are not vetting the tool against an absolute standard. You are deciding which of your data classes the tool is fit to handle, and under what conditions, so the output is a mapping rather than a single grade.

A useful set of data classes, from least to most sensitive: public or low-sensitivity (marketing copy, public-facing content, general questions with no company specifics; most tools are fine here and vetting can be light); internal, non-sensitive (internal notes, meeting summaries, drafts that contain company context but nothing regulated or confidential); confidential (source code, non-public financials, roadmaps, strategy, anything under an NDA); and personal data (customer or employee records, anything under GDPR). The same tool can be approved for the first class, restricted to the second, and forbidden for the last. That is a normal outcome, not a failure of the vetting. Each check below is really asking the same thing from a different angle: how far up the data classes can this tool be trusted, and what conditions move the line.

what to check

Eight checks. The first six are about what happens to the data once it reaches the vendor; the last two are about what the tool can reach on your side and how the vendor behaves.

Data processing agreement. Under GDPR, if the tool processes personal data on your behalf, you need a DPA in place; no DPA means you cannot lawfully put personal data through the tool. A good answer is a DPA that can be signed without enterprise sales theatre and names the processing the tool actually does. A DPA that exists but excludes the AI feature is not a real DPA for your use.

Training on your data. If the vendor trains its models on your inputs, your data leaves your control in a way you cannot reverse, so a prompt containing a client name or a code snippet becomes part of a model other people query. A good answer is a no-training tier, or a clear opt-out on by default for business accounts, with terms stating plainly that inputs and outputs are not used for training. Consumer free tiers usually fail this.

Data retention. How long the vendor keeps your inputs and outputs determines your exposure window if they are breached and your ability to honour a deletion request. A good answer is a stated retention period, a way to delete data on request, and ideally a zero-retention or short-retention option for the API or business tier.

Sub-processors. The vendor rarely does everything itself: it runs on cloud infrastructure, uses other AI providers, and routes data through tools you have not vetted, each part of your data path. A good answer is a maintained, public sub-processor list with notice of changes. A vendor that cannot tell you its sub-processors does not know its own data flows.

Hosting and data residency. Where the data is processed decides which laws apply and whether a transfer outside the EU is happening; for some of your data, EU hosting is a requirement, not a preference. A good answer is clear documentation of where processing happens, an EU hosting or residency option where you need it, and a lawful transfer mechanism for anything that leaves the EU.

Security certifications. A SOC 2 Type II or ISO 27001 certificate is evidence that an independent party checked the vendor's controls; it is not a guarantee, but its absence in a vendor handling sensitive data is a real gap. A good answer is a current report available on request with a scope that actually covers the product you are vetting.

Access model. A standalone tool you paste text into is one decision; a tool that asks to connect to your Google Workspace or Microsoft 365 and read mail, files, or calendars is a much bigger one, because the OAuth grant persists and the scope is often broader than the feature needs. A good answer is the narrowest scope that does the job, scopes visible and explained, and broad read access justified by the function rather than convenient for the vendor.

Vendor security posture and incident history. A young vendor moving fast may have thin practices and a short track record, and past incidents tell you how it behaves under pressure. A good answer is a published security page, a way to report vulnerabilities, no pattern of unreported or badly handled breaches, and a vendor mature enough to answer security questions without friction.

A note on the access check, because it is the one most often skipped. An AI tool that only ever sees what a person pastes into it has a bounded blast radius. An AI tool with an OAuth grant into your mail or drive has a standing one: it can reach data nobody chose to share with it, and it keeps that reach long after the person who approved it has moved on. Treat a connection request as a category above a paste-in tool, and look at the scopes the way you would look at any other standing access. Two of these checks overlap with general vendor due diligence, so if you already run a vendor security questionnaire for software purchases, the AI vetting is the same instinct narrowed to the questions that matter when a model is in the path.

the rubric

The checks tell you what to look at. The rubric turns what you found into a decision, so two people vetting the same tool reach roughly the same place. Score each of the eight checks on a three-point scale: 2 met (the good answer is in place, clear and documented), 1 partial (something is there but conditional, such as a DPA that excludes the AI feature, EU hosting on the enterprise tier only, or a no-training option off by default), and 0 missing or unknown (no answer, no documentation, or a clear no).

The score feeds a decision per data class rather than a single grade. Approve for a data class when every check that matters for that class is a 2 and the rest are 1 or above; for low-sensitivity data the checks that matter are few, the access model and vendor posture carry most of the weight, and a tool can be approved for public content even with a thin DPA story. Restrict when the tool scores well overall but one or two checks are partial in a way that rules out the higher data classes; a tool with a strong product, EU hosting, and a no-training tier but a DPA only on the annual enterprise plan gets restricted to non-personal data until the plan and DPA are in place, and restriction is the most common and most useful verdict. Forbid for the use being asked about when a check that is load-bearing for the requested data class is a 0 with no path to fixing it, such as a tool that trains on inputs with no opt-out asked to handle source code; the same tool may be a fine approve for drafting public copy. Two checks act as gates rather than points: if a tool will touch personal data and there is no DPA, it cannot be approved for personal data regardless of how it scores elsewhere, and if it requests a broad OAuth grant into mail or drives and cannot justify the scope, that caps the data classes it can be approved for until the scope is narrowed. A high score on the other checks does not buy back a failed gate.

running it without a bottleneck

A vetting method that takes a week per tool will be skipped, and skipped vetting is worse than light vetting because it hides the risk instead of pricing it. The fix is to match the depth of the check to the risk of the request, with two paths. The fast path, for low-risk tools, covers a standalone tool with no connection to your systems used for low-sensitivity or internal non-sensitive data: confirm three things, that it does not connect into your mail or drive, that it has a no-training option or the use is so low-sensitivity that training does not create exposure, and that the approval itself restricts it to the data class it is fit for. That is a fifteen-minute check and it covers the large majority of requests. The deeper check, for tools that touch sensitive data or request broad access, runs the full rubric: any tool that will see confidential or personal data, or that asks for an OAuth grant, gets all eight checks scored, the gates applied, and the verdict recorded with its conditions. The routing rule is one question asked first: does the tool connect to our systems, or will it see confidential or personal data? No to both is the fast path; yes to either is the deeper check. Two practices keep the whole thing from rotting: record every verdict with its date, conditions, and data class, so the approved list is also an audit trail, and re-check on a cadence, quarterly for tools that touch sensitive data and annual for the rest, because vendors change their terms.