VA production cutover runbook¶
Last updated: 2026-07-15
Scope: Patient Health (FHIR) + Veteran Service History & Eligibility only.
Goal: When VA Lighthouse production access is granted, going live is a config change + smoke test, not an engineering sprint.
Architecture truth¶
| Layer | Calls VA Lighthouse? | Cutover impact |
|---|---|---|
| last.vet-ios | Yes — OAuth, FHIR, Service History | All VA cutover work is here |
| last.vet-api | No — stores veteran-synced snapshots in RDS | No VA endpoint changes; may need snapshot invalidation |
| last.vet-provider | No — reads consent-gated API snapshots | No VA changes |
VA OAuth tokens live in iOS Keychain (TokenStore). API va_patient_id is the opaque identifier from the iOS OAuth patient claim.
Configuration.environment (LastVetAPIEnvironment in Xcode) controls LastVet API (api-staging.last.vet vs api.last.vet) only. It does not control VA Lighthouse today.
1. Variable flip table (sandbox → production)¶
Patient Health (FHIR)¶
| Setting | Sandbox (today) | Production (target) | Where set | Env-driven? |
|---|---|---|---|---|
| OAuth authorize | https://sandbox-api.va.gov/oauth2/health/v1/authorization |
https://api.va.gov/oauth2/health/v1/authorization |
VAAPIType.patientHealth via VAEnvironment |
Yes — LASTVET_VA_ENVIRONMENT in Info.plist |
| OAuth token | https://sandbox-api.va.gov/oauth2/health/v1/token |
https://api.va.gov/oauth2/health/v1/token |
same | Yes |
| FHIR API base | https://sandbox-api.va.gov/services/fhir/v0/r4/ |
https://api.va.gov/services/fhir/v0/r4/ |
same | Yes |
| OAuth client ID | 0oa1aei8cwdcib1U22p8 |
VA-issued production app ID | VAOAuthCredentials in VAAuthConfig.swift |
Partial — flip IDs when VA issues prod apps |
| Redirect URI | https://last.vet/oauth/callback |
Same (register on prod app) | VAAuthConfig.swift:75 |
No |
| Scopes | patient/*.read, launch/patient, offline_access, openid, profile |
Same (verify against prod app registration) | VAAuthConfig.swift:46-58 |
No |
SMART aud |
FHIR apiBaseURL |
Production FHIR base | AuthManager.swift:192-194 |
Derived from base URL |
| PKCE | S256 authorization code | Same | AuthManager.swift |
N/A |
| Keychain account | oauth.health |
Same | TokenStore.swift |
N/A |
Veteran Service History & Eligibility¶
| Setting | Sandbox (today) | Production (target) | Where set | Env-driven? |
|---|---|---|---|---|
| OAuth authorize | https://sandbox-api.va.gov/oauth2/veteran-verification/v1/authorization |
https://api.va.gov/oauth2/veteran-verification/v1/authorization |
VAAPIType.serviceHistory |
No |
| OAuth token | https://sandbox-api.va.gov/oauth2/veteran-verification/v1/token |
https://api.va.gov/oauth2/veteran-verification/v1/token |
same | No |
| API base | https://sandbox-api.va.gov/services/veteran_verification/v2/ |
https://api.va.gov/services/veteran_verification/v2/ |
VAAPIType.serviceHistory |
Yes |
| OAuth client ID | 0oa1ap5991l14cLjw2p8 |
VA-issued production app ID | VAOAuthCredentials |
Partial |
| Redirect URI | https://last.vet/oauth/callback |
Same | VAAuthConfig.swift:75 |
No |
| Scopes | service_history.read, disability_rating.read, veteran_status.read, offline_access, openid, profile |
Same (verify) | VAAuthConfig.swift:61-67 |
No |
| Keychain account | oauth.service-history |
Same | TokenStore.swift |
N/A |
ICN / patient identity¶
| Setting | Behavior | Where |
|---|---|---|
Token patient claim |
Opaque patient ID from VA; used as FHIR patient reference | TokenStore, AuthManager.activeHealthPatientId() |
| ICN identifier search | http://hl7.org/fhir/sid/us-icn + VA MVI naming system |
Endpoints.swift:20-26 |
| Identity guard | Health patient must match Service History patient when both present |
AuthManager.hasServiceIdentityMismatch() |
Not in scope (unchanged at VA cutover)¶
VA_OPEN_DATA_API_KEY— Facilities/Forms open data (separate apikey,Secrets.xcconfig)- last1.id OAuth — independent of VA Lighthouse
- LastVet API URLs — already AWS (
api.last.vet)
Finding: single VA environment switch (fixed 2026-07-15)¶
VAEnvironment + LASTVET_VA_ENVIRONMENT in Info.plist drive all OAuth hosts and API bases via VAAPIType. Remaining cutover trap: production OAuth client IDs still mirror sandbox until VA issues prod apps — flip URLs and client IDs together on cutover day.
Required fix before cutover: VAEnvironment enum tied to build config (see P0 sprint #4).
2. Pre-cutover checklist¶
VA / legal¶
- [ ] VA Lighthouse production approval for both APIs (track in
docs/compliance/VA_PRODUCTION_ACCESS_TRACKER.md) - [ ] Production OAuth apps registered; client IDs in Bitwarden (not committed)
- [ ] Redirect URI
https://last.vet/oauth/callbackon both prod apps - [ ] Hogan Lovells Part 2 copy locked (blocks real veteran pilot — D10)
Engineering (P0 sprint — must complete before cutover)¶
See Appendix A — all buildable against sandbox today.
- [x] FHIR pagination (
Bundle.link[relation=next]) — 2026-07-15 - [x] 404 / empty-bundle semantics fixed (no silent truncation) — 2026-07-15
- [x] FHIR decode tolerance (nullable
id, dates, codes; undecodable entries surfaced) — 2026-07-15 - [x]
VAEnvironmentenum + removeServiceHistoryClient.swiftduplicate base — 2026-07-15 - [x] Service History 429 retry (match FHIR) — 2026-07-15
- [x]
AuthManagerpatient-claim guard made defensive — 2026-07-15 - [x] P0.7 provider-side sync honesty (Phase A + B built 2026-07-15 — see §P0.7)
Infra¶
- [x] Railway LastVet rollback net removed (done 2026-07-15; dump at
backups/railway-lastvet-prod-2026-07-15.dump, gitignored) - [x]
SECURE_LINK_E2E_LOG_OTPremoved from prod (2026-07-16) — SM + Terraform task def:7+ logging code deleted; inbox-only harness; seeE2E_RUN_LOG_2026-07-16_VA_PARALLEL_TRACK.md - [ ] GitHub Actions repo secrets for SendGrid→AWS SM workflow (see §9 — org secrets confirmed absent)
AWS prod RDS — wipe synthetic test accounts¶
Before first real veteran: production must not carry A1/E2E fixtures beside real accounts.
As of 2026-07-15 after A1, AWS RDS holds synthetic veterans (incl. va_patient_id=2000163) and provider rows created by harness seeds. All current prod rows are synthetic.
Procedure:
# 1. Identify (read-only) — ECS one-off or local with DATABASE_URL from Bitwarden
psql "$DATABASE_URL" -c "
SELECT id, va_patient_id, call_sign, email, created_at FROM veterans ORDER BY created_at;
SELECT id, email, created_at FROM providers ORDER BY created_at;
"
# 2. Dry run
node scripts/reset-demo-veterans.js
# 3. Apply — resets veteran-scoped PHI rows; does NOT delete provider accounts
node scripts/reset-demo-veterans.js --apply
# 4. Delete remaining synthetic providers manually if any (E2E seed may create grantees)
# Review FK order: consent_grants, secure_links, documents, then providers/veterans
Operator rule: After wipe, prod RDS should have zero veterans and zero providers until the first real onboarding. Re-run A1 only against staging, never prod, once real pilots begin.
iOS build¶
- [ ]
VAEnvironment→ production in Release TestFlight build - [ ]
LastVetAPIEnvironment=production(already Release default) - [ ] Prod VA client IDs injected via build setting or
Secrets.xcconfig— not hardcoded sandbox IDs in Release
3. Cutover procedure (ordered)¶
| Step | Action | Owner |
|---|---|---|
| 1 | Complete P0 sprint; merge to main |
Engineering |
| 2 | Receive VA production client IDs; store in Bitwarden | Ryan |
| 3 | Wipe synthetic AWS prod RDS accounts (§2) | Ryan/ops |
| 4 | Set VAEnvironment.production in iOS Release config |
Engineering |
| 5 | Archive TestFlight build; internal smoke (§4) | Ryan |
| 6 | Submit App Store build when pilot-ready | Ryan |
| 7 | Monitor first real veteran OAuth + sync | On-call |
last.vet-api deploy: No VA config change. Optional: deploy latest API image so snapshot schema matches iOS build.
DNS: api.last.vet already points to AWS ALB. No DNS step for VA cutover.
Prod DB migration pre-flight¶
Problem: APP_ENV=production gates when migrate.js runs, not which migrations apply. A one-off task applies every pending file in the image. Approving migration 050 silently applied unreviewed 049 on 2026-07-16.
Required operator flow (prod):
- List pending —
run-ecs-db-setup.sh prodalways runsnode scripts/migrate.js --list-pendingfirst (read-only; changes nothing). - Confirm intent — compare the printed set to what you intend to apply. If any filename is unexpected, stop.
- Apply with explicit expect — rerun with the exact pending set:
This passes
migrate.js --expect <set>; apply fails if pending ≠ expected.
Local equivalents:
node scripts/migrate.js --list-pending
node scripts/migrate.js --expect 050_fhir_sync_manifest.sql
node scripts/migrate.js --expect 050_fhir_sync_manifest.sql # list + verify + apply
Staging: ./infra/scripts/run-ecs-db-setup.sh staging lists pending, then applies all pending (no --expect required).
Deploy order when new API code reads new columns or depends on migration output (e.g. 050, 051):
- Push image only —
./infra/scripts/deploy-api-image.sh <env> --push-only(migration files must be in the container). - Migrate one-off —
./infra/scripts/run-ecs-db-setup.sh <env> 'NNN_foo.sql,...'(prod requires explicit--expectset). - Wait for completion —
run-ecs-db-setup.shblocks onaws ecs wait tasks-stopped; do not roll out until exit 0. - Roll out —
./infra/scripts/deploy-api-image.sh <env> --rollout-only.
Do not push + force rollout in one step when migrations are pending. 2026-07-21 051 prod deploy raced migrate vs rollout; verify passed but that was luck.
Code-only deploys (no pending migrations): ./infra/scripts/deploy-api-image.sh <env> (push + rollout). See Phase B ship 2026-07-16.
4. Immediate post-flip smoke test¶
Run on a physical device, Release build, api.last.vet, within 30 minutes of cutover.
| # | Step | Pass criteria |
|---|---|---|
| 1 | Fresh install or sign out all VA sessions | No stale Keychain tokens |
| 2 | Connect Patient Health OAuth | Authorize completes; no invalid_client |
| 3 | Connect Service History OAuth | Both tokens present; identity guard does not discard session |
| 4 | Record → Sync My Data | FHIR resources load; no "partial sync" banner without explanation |
| 5 | Profile → demographics | Birth date/gender from FHIR or explicit "unavailable" — not fake-complete |
| 6 | Facilities (open data) | SD wedge still resolves (separate apikey) |
| 7 | ROI → sign → secure link → provider OTP | Full Gate 0 loop on real VA identity |
| 8 | API session exchange | POST /api/auth/veteran/session → 200 with Bearer JWT |
| 9 | Revoke secure link | Post-revoke blocked |
Scripted API leg (optional): scripts/smoke-va-demo.sh — update to check api.va.gov reachability when cut over.
5. Rollback¶
iOS / VA layer (fast — minutes)¶
- Revert
VAEnvironmentto sandbox in Release (or ship hotfix TestFlight). - Veterans must re-authenticate (Keychain tokens are environment-specific in practice).
- Do not mix sandbox tokens against prod FHIR or vice versa.
Persisted snapshot problem (the hard part)¶
iOS sync pushes health record items and verification snapshots to last.vet-api RDS. Sandbox-shaped data may include:
- Stub
FHIRPatientrows (nil demographics) promoted as if complete - Truncated FHIR bundles (pre-pagination fix) stored as full records
- Service History path-alias responses normalized to sandbox envelope shapes
va_patient_idfrom sandbox MPI that does not equal productionpatientclaim for the same human
Rollback actions:
| Asset | Action |
|---|---|
| iOS Keychain VA tokens | Clear both oauth.health and oauth.service-history on sign-out |
iOS UserDefaults service snapshots |
Cleared by AuthManager.removeAllServiceSnapshots() on mismatch — verify prod mismatch does not brick session |
API health_record_items |
Veteran-initiated re-sync after VA revert, or operator reset-demo-veterans.js --apply --id <uuid> for pilot veterans |
API documents / consent_grants |
Consent-scoped; revoke + re-sign if ROI was generated from bad snapshot |
| Provider-visible artifacts | Treat as potentially wrong if synced during prod FHIR experiment; revoke secure links |
Operator rule: If production FHIR behaves unexpectedly mid-pilot, stop new shares, revoke active secure links, force veteran re-sync from corrected iOS build, and document in incident log. Do not assume RDS snapshots are valid after a VA layer rollback.
AWS / LastVet API¶
No rollback needed for VA cutover — API does not call VA. Roll back iOS only unless schema migration was coupled (it should not be).
Railway¶
Removed 2026-07-15. Rollback net no longer exists. AWS is sole prod.
6. Ranked breakage list (real MPI data)¶
| Rank | Risk | Symptom | Location | Mitigation (P0) |
|---|---|---|---|---|
| P0 | FHIR pagination absent | Veteran sees truncated record; shares incomplete data with provider | FHIRBundleLoading.swift |
✅ Fixed 2026-07-15: link[relation=next] pagination + incomplete banner |
| P0 | 404 → empty bundle | "No conditions" when endpoint failed or wrong patient ref | FHIRClient + FHIRBundleLoading |
✅ Fixed 2026-07-15: .empty vs .unavailable vs .partial |
| P0 | resilientBundle swallows errors |
Silent partial health record | FHIRClient.swift |
✅ Removed; per-resource resourceStatuses on VAHealthRecord |
| P0 | Non-optional FHIR id |
Decode drops entire resource bundle entries | FHIRBundleLoading.swift |
✅ Fixed 2026-07-15: tolerant per-entry decode + droppedEntryCount telemetry |
| P0 | VAAuthConfig all sandbox |
App hits sandbox from Release build after cutover | VAAuthConfig.swift + VAEnvironment |
✅ Fixed 2026-07-15: LASTVET_VA_ENVIRONMENT → VAAPIType (Release VA stays sandbox until prod OAuth apps issued) |
| P0 | Duplicate Service History base URL | Client bypasses enum; prod URL missed on cutover | ServiceHistoryClient.swift |
✅ Fixed 2026-07-15: uses VAAPIType.serviceHistory.apiBaseURL only |
| P1 | hasServiceIdentityMismatch discards service session |
Service History connected but verification blank | AuthManager.swift + VAPatientIdentity |
✅ Fixed 2026-07-15: normalize ICN; defensive guard; manual reconnect |
| P1 | Service History no 429 retry | Sync fails under prod rate limits | ServiceHistoryClient.executeRequest |
✅ Fixed 2026-07-15: VAHTTPRetry (30s × 2, matches FHIR) |
| P1 | Path alias fallbacks | Wrong prod path silently 404 → veteranNotFound | ServiceHistoryClient firstOf paths |
Confirm prod paths from OpenAPI; remove wrong aliases |
| P2 | Token patient optional in sandbox |
Prod always returns patient — untested merge path | AuthManager.canUseServiceHistorySession L648-650 |
Integration test with both claims |
| P2 | Date parsing | Sort/display wrong for VistA date formats | VADemographicsFormatting, FHIR date fields |
Fuzzy ISO + sentinel "unknown date" |
| P2 | Deprecated FHIR codes | Empty display strings in UI | CodeableConcept rendering | Fall back to text field |
| P3 | 2000163 in E2E scripts only |
No prod leak (scripts only) | seed-roi-demo.sh, etc. |
Keep harness on staging |
7. Test / synthetic paths that must not leak¶
| Item | Leak risk | Mitigation |
|---|---|---|
va_patient_id=2000163 |
Harness only | Never seed prod after pilot start |
FHIRPatient stub with nil demographics |
Degraded UX looks "connected but empty" | Show explicit incomplete state |
ensure-demo-veteran.js |
API-only bootstrap | Staging/dev DATABASE_URL only |
Configuration.showsVADiagnostics |
Sandbox builds only | Release uses environment == .sandbox gate |
8. Rate limits & pagination¶
| API | 429 handling | Pagination |
|---|---|---|
| FHIR | 30s sleep, 2 retries (FHIRClient.swift) |
✅ FHIRBundleLoading.loadPaginated follows link[relation=next] |
| Service History | ✅ 30s sleep, 2 retries (ServiceHistoryClient.executeRequest) |
N/A (single GET resources) |
| last.vet-api | Per-route 429 | N/A |
Prod rate limits may differ from sandbox. Document actual limits from VA when prod access is granted.
9. GitHub Actions secrets (SendGrid → AWS SM)¶
| Scope | Result (2026-07-15) |
|---|---|
Repo Last-1-Enterprises/last.vet-api |
total_count: 0 — no repo secrets |
Org Last-1-Enterprises |
Confirmed absent (Ryan verified 2026-07-15) |
sync-provider-email-templates.yml reads repo-level secrets only (SENDGRID_API_KEY, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY). Org secrets are not in play.
If you want CI dispatch: add those three as repository secrets on last.vet-api. Until then, run node scripts/sync-sendgrid-provider-templates.mjs locally with env from Bitwarden (same outcome, no GitHub dependency).
Appendix A — P0 pre-cutover engineering sprint (PLAN ONLY)¶
Buildable today against VA sandbox. No VA production dependency. Separate PR(s) from this runbook.
A1. FHIR pagination¶
Problem: Single bundle request per resource. Prod veterans with long VistA history return link[relation=next]; we store page 1 only.
Plan:
- Extend
FHIRBundle(FHIRBundle.swift) withlink: [FHIRBundleLink]?(relation,url). - Add
FHIRClient.fetchBundlePages(path:)— loop until nonextlink; mergeentryarrays. - Optional safety cap (e.g. 50 pages) with explicit
truncated: trueonVAHealthRecordmetadata. - Unit test: multi-page fixture JSON in
LastVetTests/FHIRClientPaginationTests.swift(2–3 pages, verify all entries merged). - UI: if truncated, show banner on Record tab — "Some records may not be shown yet."
Files: FHIRBundle.swift, FHIRClient.swift, HealthRecordViewModel.swift, new test fixture.
A2. 404 vs empty vs error¶
Problem: requestBundle maps 404 → empty bundle; indistinguishable from veteran with no data.
Plan:
- Introduce
FHIRBundleResult<T>enum:.empty,.populated(FHIRBundle),.unavailable(FHIRError). - 404 on known resource type search →
.emptyonly whenOperationOutcomeindicates no results; otherwise.unavailable. - Remove silent empty return at
requestBundleWithPatientFallbacksL108 — return.emptyonly after all patient query variants return valid empty bundles. resilientBundle→ per-resource status in sync manifest;VAHealthRecordcarriesresourceStatuses: [String: SyncStatus].- Block provider share / ROI prefill if any critical resource is
.failedor.unavailable(conditions, medications at minimum).
A3. FHIR decode tolerance¶
Problem: let id: String on all resource structs — one bad entry fails JSONDecoder for whole bundle.
Plan:
- Two-phase decode:
FHIRBundleEntryRawwith[String: Any]orJSONValue, then per-entrydecodeIfPresentinto typed resource. - Surrogate id:
"\(resourceType)-\(index)-\(hash(prefix))"whenidabsent. - Dates:
VADemographicsFormattingacceptsYYYY,YYYY-MM, full ISO; nil → store nil, don't throw. - Codes: prefer
coding[].display, thentext, thencodesystem|code string. - Fixture tests:
Fixtures/fhir-messy-bundle.jsonwith null ids, missing fields, deprecated codes.
A4. VAEnvironment enum¶
Problem: N hardcoded sandbox values; ServiceHistoryClient.swift:6 duplicate.
Plan:
- Add
enum VAEnvironment: String { case sandbox, production }inConfiguration.swiftor newVAEnvironment.swift. - Xcode build setting
LASTVET_VA_ENVIRONMENT→ Info.plist (mirrorLastVetAPIEnvironmentpattern). - Refactor
VAAPITypeto read hosts + client IDs fromVAEnvironment.current. - Debug/sandbox → sandbox VA; Release → production VA (when approved; until then Release can stay sandbox with explicit comment).
- Delete
ServiceHistoryClient.swift:6string; initbaseURLfromVAAPIType.serviceHistory.apiBaseURL. - Single smoke: log
VAEnvironment.currentat OAuth start in debug.
A5. Service History 429 retry¶
Plan:
- Extract shared
VAHTTPRetryPolicy(429: 30s × 2, network exponential like FHIR). - Apply in
ServiceHistoryClient.executeRequestbefore decode. - Test: mock
URLProtocolreturning 429 then 200.
A6. AuthManager patient claim mismatch — defensive mode¶
Current behavior (AuthManager.swift:621-633):
- If health
patientand servicepatientboth non-empty and string-differ →hasServiceIdentityMismatch()true → clears service token + all service snapshots (refreshServiceVerificationSnapshotL560-573).
Prod risk: MPI may return ICN in one token and a different compatible identifier in another; formatting differences (2000163 vs 2000163V123456).
Plan:
- Add
normalizeVAPatientIdentifier(_:)— strip whitespace, uppercase, optional ICN suffix normalization per VA docs. - Compare normalized forms before mismatch.
- On mismatch: do not auto-clear in production — set
identityGuardState = .mismatch(health, service); show Profile banner with "Reconnect Service History". - Log structured event (no PII in release logs).
- Fallback: if one token lacks
patientbut FHIRPatient.idresolves, use FHIR id as canonical for comparison (read-after-auth). - Unit tests: mismatch, suffix variant, one-sided missing patient, both missing (allow sync per L648-650).
Status: ✅ Implemented 2026-07-15 (VAPatientIdentity, vaIdentityGuardState, VAPatientIdentityTests).
P0.7 — Provider-side sync honesty (BUILT 2026-07-15)¶
Question: When a veteran with .partial or .unavailable FHIR resources generates an ROI and shares it, does the provider see that the clinical record may be incomplete?
Built surfaces¶
| Surface | Carries sync status? | Evidence |
|---|---|---|
| iOS ROI sign | Yes (informed consent) | ROIViewModel.requestSignAuthorization() + FHIRSyncPlanner.roiInformedConsentRequired |
| iOS → API sync | Yes | fhirResourceFetchStatuses on HealthRecordSyncRequest; FHIRSyncPlanner sync gate |
| API manifest | Yes | veterans.fhir_sync_manifest; buildFhirSyncPlan in fhirSyncManifest.ts |
| Provider record view | Yes | RecordView.tsx alerts from fhirSyncManifestAlerts.ts |
| Shared secure-link artifact | No | Phase C (parked) |
| Signed ROI PDF | No | Phase C (parked) |
Phase A — iOS informed consent (built)¶
Before ROIViewModel.signAuthorization(), when Condition or MedicationRequest is .partial or .unavailable:
Your VA record is incomplete. The provider will be told. Share anyway?
Veteran decides. Does not block ROI sign.
Files: ROIViewModel.swift, ROIFlowView.swift, FHIRSyncPlanner.swift
Phase B — API manifest + sync gate (built)¶
Rule: Never replace a more-complete stored state with a less-complete one.
lastFetch |
Sync behavior | stored after sync |
|---|---|---|
.complete |
Always sync | complete |
.empty |
Always sync (deactivate all) | empty |
.partial |
If stored unknown/empty: store items; else skip | partial or stale |
.unavailable |
Never sync | stale (or unchanged unknown/empty) |
Provider alerts (distinct copy):
stored=partial→ "This record is incomplete. Some VA data could not be loaded."stored=stale→ "Data shown is from the last complete sync. The most recent VA fetch was incomplete."- Also fires on
lastFetch = partial | unavailablewhen stored is complete/empty.
Tests: tests/fhir-sync-manifest.test.ts (three two-sync scenarios); tests/health-record-routes.test.ts; iOS FHIRSyncPlannerTests.swift.
Key files: migrations/050_fhir_sync_manifest.sql, src/lib/fhirSyncManifest.ts, src/repositories/health-record.repo.ts, HealthRecordViewModel.syncRemoteHealthRecord, RecordView.tsx.
Phase C — parked¶
Point-in-time manifest on consent grants / secure-link sidecar. Hogan Lovells, not engineering. Form 10-5345 is federal authorization, not a clinical record.
Appendix B — E2E harness inbox OTP¶
Status (2026-07-16): Inbox path proven on prod. SECURE_LINK_E2E_LOG_OTP torn down — prod no longer logs OTPs.
| Item | Value |
|---|---|
| Mailbox | e2e-harness@last1.enterprises (dedicated; not ryan@last1.enterprises) |
| Delivery | Real SendGrid → Gmail IMAP (imap.gmail.com) |
| Secrets | E2E_OTP_IMAP_* in Bitwarden only |
| Script | scripts/lib/fetch-e2e-otp-inbox.mjs + fetch-e2e-otp-inbox.sh |
| Harness | e2e-roi-gate0-full.sh — inbox only (CloudWatch fallback removed) |
| Default recipient | DEMO_RECIPIENT_EMAIL=e2e-harness@last1.enterprises |
| Teardown | ✅ Done 2026-07-16 — removed from Terraform + SM lastvet-prod/app + API code |
Run:
# Load Bitwarden E2E_OTP_IMAP_* + VETERAN_SESSION_EXCHANGE_SECRET, then:
API_BASE=https://api.last.vet/api ./scripts/e2e-roi-gate0-full.sh
Log: docs/runbooks/gate-0.5/E2E_RUN_LOG_2026-07-16_VA_PARALLEL_TRACK.md
Related docs¶
docs/compliance/VA_PRODUCTION_ACCESS_TRACKER.md— D12 tracker (Ryan fill-in)docs/api/VA_ARCHITECTURE_AND_REVIEW_GUIDE.md— OAuth comparison tabledocs/api/VA_SANDBOX_SMOKE_TEST.md— pre-demo checklistlastvet-gates/.cursor/rules/00-invariants.mdc— AWS-only prod, Railway teardown before VA prod accessdocs/runbooks/gate-0.5/E2E_RUN_LOG_2026-07-15.md— A1 earned on AWS