Before accepting an AI vendor claim

Can the AI vendor prove it?

Turn AI vendor claims into common demonstrations, evidence requests, acceptance conditions, monitoring duties, and exit questions.

Time

About 5 minutes

You leave with

AI vendor evidence agenda

Privacy

No account. No submission.

Use operational facts, not confidential records.

Do not enter confidential procurement, student, personnel, resident, vendor, security, or incident information. Answers are saved only in this browser and are not sent to JS Technology Solutions.

Question 1: Is the AI role separated from the human decision and authority that must remain accountable?

Is the AI role separated from the human decision and authority that must remain accountable?

Name what the system may recommend or produce, what it may not decide, and who reviews, overrides, and records the result.

Question 2: Will every vendor complete the same realistic scenario with the same constraints?

Will every vendor complete the same realistic scenario with the same constraints?

A scripted demonstration is more useful when evaluators can compare outputs, failure handling, evidence, and user effort.

Question 3: Are claimed accuracy, quality, speed, or savings tied to a defined population, method, baseline, and limitation?

Are claimed accuracy, quality, speed, or savings tied to a defined population, method, baseline, and limitation?

Ask what was measured, on which data, by whom, when, and how the result transfers to the public organization.

Question 4: Are collection, use, training, retention, disclosure, location, deletion, export, and subcontractor terms testable?

Are collection, use, training, retention, disclosure, location, deletion, export, and subcontractor terms testable?

Include prompts, inputs, outputs, metadata, logs, feedback, derived data, and model-improvement use where applicable.

Question 5: Has the team defined who could be harmed, excluded, delayed, surveilled, or denied recourse when the system fails?

Has the team defined who could be harmed, excluded, delayed, surveilled, or denied recourse when the system fails?

The evidence plan should reflect the context, affected groups, accessibility, privacy, security, and impact of a wrong result.

Question 6: Are production logging, review cadence, thresholds, incidents, drift, complaints, and corrective actions defined?

Are production logging, review cadence, thresholds, incidents, drift, complaints, and corrective actions defined?

Pre-award evidence does not replace monitoring after data, users, models, vendors, or operating conditions change.

Question 7: Must the vendor disclose and test material model, data, feature, subcontractor, or policy changes?

Must the vendor disclose and test material model, data, feature, subcontractor, or policy changes?

Define notice, revalidation, approval, rollback, and remedy before a material service change reaches production.

Question 8: Can the organization export its records, preserve required evidence, transition service, and verify deletion?

Can the organization export its records, preserve required evidence, transition service, and verify deletion?

A useful exit plan covers format, timing, assistance, continuity, cost, retention, deletion, and locally required records.

Source record

Primary sources behind this pressure test

Last verified August 26, 2026

These sources provide useful disciplines and risk frames. They do not make every cited rule applicable to every state, local, or K-12 organization.