FDA AI Guardrail Dossier (Draft)

Purpose

Capture controls for any adaptive or AI-assisted output before FDA-enabled release.

1. Intended AI Functions (Gated)

• Daily AI wellness summary text
• AI-assisted routine drafting
• Optional data normalization/validation assistance for scanner workflows

Current App Store profile: disabled by default for adaptive claims-sensitive surfaces.

2. Model and Runtime Constraints

• Cloud model is invoked only through controlled backend endpoints.
• No on-device autonomous model retraining.
• No self-modifying clinical logic.
• Prompt templates are versioned and reviewed.

3. Guardrail Controls

1. Feature-gate control
2. aiInsights, advisoryCopy, adaptiveProtocol, labInterpretation are required gates.
3. Input constraints
4. Structured prompt schema with bounded fields only.
5. Output constraints
6. Policy filters remove or block diagnosis/treatment language.
7. Disclosure control
8. Explicit user consent gate before cloud processing.
9. Fallback control
10. If AI unavailable, app continues with non-adaptive organizer behavior.

4. Safety and Abuse Cases

• Hallucinated supplement or dose instruction
• Overconfident risk statements without evidence
• Inference from sparse data presented as certainty
• Contradictory guidance relative to known interaction data

Mitigation: strict output schema + moderation filter + disable path via flag and kill switch.

5. Validation Plan (Pre-Clearance)

• Prompt/response test suite using representative scenarios
• Red-team tests for prohibited medical claims
• Regression tests for guardrail bypass attempts
• Human review sampling of production-like outputs in staging

6. Monitoring and Change Control

• Version pinning for prompts and model configuration
• Incident logging and rollback procedure for AI output anomalies
• Re-validation required on model/provider or prompt policy changes

7. Open Items

• Finalize hazard analysis references and risk classification.
• Define formal acceptance criteria with regulatory reviewer sign-off.
• Integrate dossier with Design History File structure.