End-to-end smoke runbook¶

Step-by-step verification of the full link → check-in → revoke chain once the relevant PRs land. Pairs with docs/ROADMAP.md §3, which has the higher-level checklist; this file has the actual click paths, curl commands, and SQL verifications.

You can run this against:

Local dev — last1.id + RadioCheck on supabase start, LastVet on Railway preview / local Postgres
Production — full prod stack

Use a throwaway test account in both modes. The revoke flow permanently destroys session state; you don't want to do this to a real veteran or provider.

The verification half of every revoke check uses the helper script scripts/smoke-verify-revoke.mjs rather than hand-refreshing SQL.

Prerequisites¶

These must be true before any of the chains below will work. Tick each as you confirm.

Deploys¶

Reference: docs/ROADMAP.md §1.

[ ] last1.id is on main with at least PRs #46 (admin force-unlink), #52 (Connected Apps enrich) deployed
[ ] last1-admin is on main with PR #2 deployed (if testing admin force-unlink, §3 below)
[ ] last.vet-api has #12 → #13 → #14 all deployed; migration 023 ran; LAST1_TOKEN_ENCRYPTION_KEY set on Railway
[ ] last.vet-provider has PR #4 deployed
[ ] last.vet-ios has PRs #2 and #3 in TestFlight
[ ] RadioCheck has PRs #6 (connected screen) and #7 (revoke webhook) deployed

Webhook configuration¶

For each partner you're testing the revoke chain on, both sides must be configured:

[ ] last1.id app_clients row exists for the partner with:
webhook_url pointing at the partner's inbound endpoint
webhook_secret_b64 provisioned (32 random bytes, base64)
webhook_enabled = true
[ ] Partner has the same webhook_secret_b64 set in its environment:
LastVet: LAST1_WEBHOOK_SECRET on Railway
RadioCheck: supabase secrets set LAST1_INBOUND_WEBHOOK_SECRET=<b64>

Throwaway test account¶

You need one veteran identity that exists on last1.id. Easiest recipe:

# Sign in via the regular flow at https://auth.last1.id (or http://localhost:5003
# in dev) using an @last1-test.local email. The first sign-in auto-
# creates the identity row.

Take note of the resulting identity.id UUID. You'll paste it into the verify script in §3.

To find your own identity id after sign-in:

-- against last1.id Supabase
select id from identities where supabase_user_id = (
  select id from auth.users where email = 'YOUR-TEST-EMAIL@last1-test.local'
);

Environment variables for the verify script¶

Set once in your shell session:

# last1.id (always required)
export LAST1_SUPABASE_URL="https://<last1id-project>.supabase.co"
export LAST1_SUPABASE_SERVICE_ROLE_KEY="..."   # from Supabase dashboard

# RadioCheck (required when testing the RadioCheck chain)
export RADIOCHECK_SUPABASE_URL="https://<radiocheck-project>.supabase.co"
export RADIOCHECK_SUPABASE_SERVICE_ROLE_KEY="..."

The service-role key bypasses RLS. Never commit it. Set it in your shell, not in .env checked into git.

1. Chain: Link veteran → check-in → revoke (RadioCheck)¶

This is the canonical end-to-end. It exercises every PR from §1 of the roadmap that involves RadioCheck.

1.1 Link¶

[ ] In the RadioCheck iOS app (TestFlight build with PR #6), tap "Sign in with Last1 ID"
[ ] Complete the OAuth roundtrip in the SFAuthenticationSession sheet:
Redirect to last1.id
Sign in with your test account
Authorize the requested scopes
Redirect back to radiocheck://
[ ] Verify the post-link "Connected to Last1" screen appears with your masked identity id

Verification SQL (against last1.id Supabase):

select identity_id, app_id, linked_at, revoked_at
from identity_app_links
where identity_id = '<YOUR-IDENTITY-ID>'
  and app_id = 'radiocheck';
-- expected: 1 row, revoked_at IS NULL

select g.scopes, g.granted_at, g.revoked_at
from consent_grants g
join app_clients a on a.id = g.app_client_id
where g.identity_id = '<YOUR-IDENTITY-ID>'
  and a.app_id = 'radiocheck';
-- expected: 1 row with the scopes the iOS authorize page showed

select count(*) as live_sessions
from oauth_refresh_tokens t
join app_clients a on a.id = t.app_client_id
where t.identity_id = '<YOUR-IDENTITY-ID>'
  and a.app_id = 'radiocheck'
  and t.revoked_at is null
  and t.expires_at > now();
-- expected: 1 (or however many devices you've used)

1.2 Check-in¶

[ ] Trigger a check-in on RadioCheck (any standard radio check-in flow — voice / SMS / app)
[ ] Wait up to 5 minutes for the proof outbox drainer to flush

Verification SQL (against last1.id Supabase):

select id, source, event_type, created_at
from trust_events
where identity_id = '<YOUR-IDENTITY-ID>'
  and app_id = 'radiocheck'
order by created_at desc
limit 3;
-- expected: at least 1 row dated within the last 5 minutes, event_type
-- something like 'radiocheck.checkin.completed'

If the row doesn't appear after 5 minutes, check the RadioCheck proof_outbox table for stuck rows:

-- against RadioCheck Supabase
select id, status, attempts, last_error, payload->>'event_type' as event_type, created_at
from proof_outbox
where status != 'delivered'
order by created_at desc
limit 10;

1.3 Revoke (from end-user dashboard)¶

[ ] On last1.id, visit /dashboard/apps
[ ] Verify the RadioCheck row shows:
The permissions you granted (with descriptions, not raw scope ids — PR #52)
"1 active session" badge
"Last used N ago" with a real timestamp (not "—")
[ ] Click Revoke access
[ ] In the confirm modal, verify the copy spells out the count of permissions and active sessions that will be cut
[ ] Click Revoke access to confirm

Verification — run the helper script:

node scripts/smoke-verify-revoke.mjs \
  --identity-id '<YOUR-IDENTITY-ID>' \
  --app-id radiocheck

The script polls every 2s for up to 60s and prints a live table:

✅  last1.identity_app_links.revoked_at IS NOT NULL
     revoked_at=2026-05-22T23:50:11.123Z
✅  last1.consent_grants.revoked_at IS NOT NULL
     revoked_at=2026-05-22T23:50:11.124Z
✅  last1.oauth_refresh_tokens — all revoked OR expired
     0 live refresh tokens remain
✅  last1.webhook_deliveries — app_link.revoked queued
     id=... status=delivered created=2026-05-22T23:50:11.224Z
✅  radiocheck.users.last1_identity_id IS NULL
     no RadioCheck user bound to this identity
✅  radiocheck.last1_webhook_deliveries — identity_unlinked received
     delivery_id=... received=2026-05-22T23:50:11.405Z

6/6 passed

If any check stays ⏳ after 60s, common causes (also printed by the script):

Webhook drainer not running on last1.id side (pm2 status / Railway logs)
Partner endpoint returning 5xx (check webhook_deliveries.last_error)
webhook_enabled = false on the app_clients row
Secret mismatch between last1.id and the partner

1.4 Re-link is possible¶

After a successful revoke, the user should be able to re-link without manual cleanup.

[ ] In the RadioCheck app, tap "Sign in with Last1 ID" again
[ ] Complete OAuth as before
[ ] Verify it succeeds (does NOT 409 with identity_already_linked or veteran_already_linked)
[ ] Verify a NEW identity_app_links row exists (or the existing row's revoked_at was nulled — implementation detail, both are valid)

2. Chain: Disconnect from LastVet provider portal¶

This exercises last.vet-provider #4 + last.vet-api #14 + last1.id #52.

2.1 Link¶

[ ] Sign in to the LastVet provider portal
[ ] Visit /dashboard/profile → Linked Accounts card
[ ] Click Link Last1 ID → OAuth roundtrip
[ ] Verify the card flips to "Linked" with a masked identity id

Verification SQL (against LastVet Postgres on Railway):

select id, last1_identity_id, last1_tokens_updated_at
from providers
where id = '<YOUR-PROVIDER-ID>';
-- expected: last1_identity_id is the same UUID you saw at sign-in,
--           last1_tokens_updated_at within the last 60s

2.2 Disconnect¶

[ ] On the provider portal, click Disconnect in the Linked Accounts card
[ ] In the confirm modal, verify the copy mentions upstream revoke + local clear + re-link required
[ ] Click Disconnect

Verification SQL (LastVet Postgres):

select last1_identity_id,
       last1_access_token_ciphertext is null as access_cleared,
       last1_refresh_token_ciphertext is null as refresh_cleared,
       last1_tokens_updated_at
from providers
where id = '<YOUR-PROVIDER-ID>';
-- expected: last1_identity_id = NULL,
--           access_cleared = true,
--           refresh_cleared = true

Verification SQL (LastVet Postgres, audit trail):

select event_type, metadata->>'revoke_result' as revoke_result, created_at
from provider_auth_audit_events
where provider_id = '<YOUR-PROVIDER-ID>'
  and event_type = 'unlink_last1_id'
order by created_at desc
limit 3;
-- expected: most recent row, revoke_result either 'revoked' or
--           a categorical reason (network_error, no_stored_refresh_token, etc.)

Verification — the upstream revoke at last1.id (best-effort):

node scripts/smoke-verify-revoke.mjs \
  --identity-id '<YOUR-IDENTITY-ID>' \
  --app-id lastvet

Note: when the revoke_result above is no_stored_refresh_token or network_error, the LastVet-side clear still happens (best-effort contract), but the last1.id-side oauth_refresh_tokens may not all be revoked. That's expected — the next user re-link replaces them anyway.

3. Chain: Admin force-unlink¶

This exercises last1.id #46 + last1-admin #2.

3.1 Link¶

Pre-state: the test identity must be linked to at least one partner. Use §1.1 (RadioCheck) or §2.1 (LastVet) to establish a link.

3.2 Force unlink¶

[ ] Sign in to last1-admin with operator credentials
[ ] Navigate to Identities → search for the test identity
[ ] Click the Apps button on the row
[ ] In the Connected Apps dialog, click Force unlink on the partner row
[ ] In the confirm dialog, type a reason (min 5 chars, e.g. "smoke test")
[ ] Click Force unlink to confirm

Verification — same script as §1.3:

node scripts/smoke-verify-revoke.mjs \
  --identity-id '<TEST-IDENTITY-ID>' \
  --app-id <radiocheck|lastvet>

Plus an admin-audit-trail check:

-- against last1.id Supabase
select action, metadata->>'reason' as reason, metadata->>'source' as source, created_at
from admin_audit_logs
where action = 'identity.app_link.force_revoke'
  and metadata->>'identity_id' = '<TEST-IDENTITY-ID>'
order by created_at desc
limit 1;
-- expected: 1 row, source='admin_initiated', reason="smoke test"

3.3 Half-state recovery (optional but recommended)¶

A subtle scenario: the link row was already revoked but a stray refresh token still has access. The force-unlink endpoint should still kill that token even though identity_app_links.revoked_at was already set.

To test:

[ ] Manually update identity_app_links.revoked_at = now() for a test pair, but leave an oauth_refresh_tokens row live
[ ] Trigger force unlink via the admin UI
[ ] Verify oauth_refresh_tokens.revoked_at is now set
[ ] Verify a new admin_audit_logs row exists (the action is not idempotent at the audit-trail level — every click records the intent)
[ ] Verify a partner webhook was queued even though the link state didn't visibly change (the half-state IS a state change worth notifying about)

4. Smoke test for the "Already revoked" idempotency path¶

The end-user /dashboard/apps revoke path is idempotent. Hitting it twice should not fail.

[ ] Link partner via §1.1 or §2.1
[ ] Revoke via §1.3 — confirm green
[ ] Browser dev-tools → re-fire the same POST /api/identity/app-links/:app_id/revoke request
[ ] Expected response: { unlinked: false, alreadyUnlinked: true } with status 200, no new webhook queued
[ ] Confirm by re-running smoke-verify-revoke.mjs and checking that webhook_deliveries still has exactly 1 app_link.revoked row for this identity, not 2

5. Common debug commands¶

last1.id webhook drainer¶

-- queue depth
select status, count(*) from webhook_deliveries group by status;

-- most recent failures
select id, app_client_id, event_type, status, attempts, last_error,
       last_response_status, next_attempt_at
from webhook_deliveries
where status in ('pending', 'failed', 'dead_lettered')
order by created_at desc
limit 10;

RadioCheck inbound webhook idempotency¶

-- against RadioCheck Supabase
select delivery_id, event_id, event_type, action, identity_id, received_at, last_error
from last1_webhook_deliveries
order by received_at desc
limit 10;
-- delivery_id == X-Last1-Delivery header. Re-deliveries collide on
-- the PK and short-circuit via INSERT ... ON CONFLICT DO NOTHING.

LastVet inbound webhook idempotency¶

-- against LastVet Postgres
select event_id, event_type, processed_at, error
from last1_webhook_audit
order by processed_at desc
limit 10;

Re-fire a stuck delivery¶

If webhook_deliveries has a row stuck in failed or dead_lettered, the drainer will not retry on its own (terminal state). To force an immediate retry:

update webhook_deliveries
set status = 'pending', next_attempt_at = now(), attempts = 0
where id = '<DELIVERY-ID>';

6. Cross-app trust activity (Identity alignment)¶

Verifies the unified activity story: partner events land in last1.id, the dashboard trust summary reflects them, and optional public dimension sharing works. Run after deploying last1.id identity-alignment PRs, Last1 trust-event outbox, and RadioCheck check-in-proof-events updates.

Prerequisites¶

[ ] last1.id migration 20260602180000_public_trust_dimensions.sql applied
[ ] Last1 migration 20260615_last1_trust_event_outbox.sql applied
[ ] Last1 LAST1_APP_SECRET matches last1.id app_clients row for last1
[ ] Last1 cron or manual drain: POST /api/internal/cron/drain-last1-trust-events (with internal auth)
[ ] RadioCheck check-in-proof-events Edge Function redeployed with radio_check_responder_added

6.1 RadioCheck check-in → Activity + Care Connections + dimensions¶

Follow §1.1 to link, then §1.2 for check-in verification. Additionally:

[ ] Open last1.id Dashboard — Trust summary shows responsiveness and/or participation > 0
[ ] Open Care Connections — RadioCheck card shows event count ≥ 1; timeline lists check-in with human-readable label
[ ] Open Activity — filter Partner activity + RadioCheck; check-in event appears with category trust

6.2 RadioCheck responder added¶

[ ] Add a responder to a check-in in RadioCheck (any flow that emits radio_check_responder_added)
[ ] Wait up to 5 min for proof drainer
[ ] Verify event in Activity: kind radio_check_responder_added, metadata includes responder_role only (no free text)

-- against last1.id Supabase
select event_type, metadata
from trust_events
where identity_id = '<YOUR-IDENTITY-ID>'
  and app_id = 'radiocheck'
  and event_type = 'radio_check_responder_added'
order by created_at desc
limit 1;
-- expected: metadata has responder_role; must NOT contain response_message

6.3 Last1.app operational events¶

Requires a Last1 user whose beneficiary profile has last1_identity_id set (linked via Last1 ID OAuth).

[ ] Enroll beneficiary in a program (org admin) OR record an outcome / confirm validation
[ ] Drain outbox:

curl -X POST "$LAST1_API_BASE/api/internal/cron/drain-last1-trust-events" \
  -H "Authorization: Bearer $LAST1_INTERNAL_CRON_SECRET"

[ ] Verify trust event in last1.id:

select event_type, app_id, metadata, created_at
from trust_events
where identity_id = '<YOUR-IDENTITY-ID>'
  and app_id = 'last1'
order by created_at desc
limit 5;
-- expected: last1.program.enrolled, last1.outcome.recorded, last1.validation.passed/failed, or last1.org.member_joined

[ ] Dashboard trust summary updates after snapshot refresh (sign out/in or wait for lazy recompute on fetch)

[ ] Settings → Public profile → Trust dimensions — enable Participation (or another dimension with a non-zero value)
[ ] Visit /id/<username> in a private window — dimension bar visible for opted-in key only
[ ] Disable all dimension toggles — public profile hides trust summary bars (credentials unchanged)

6.5 Activity filters¶

[ ] Activity → All categories → Partner activity — only trust-category rows
[ ] Activity → RadioCheck — only RadioCheck-sourced events
[ ] Verifications only — credential and verification events only (no check-ins)

7. Cleanup¶

After running these smokes, clean up your test data so the next run starts fresh:

-- against last1.id Supabase
delete from oauth_refresh_tokens where identity_id = '<TEST-IDENTITY-ID>';
delete from consent_grants where identity_id = '<TEST-IDENTITY-ID>';
delete from identity_app_links where identity_id = '<TEST-IDENTITY-ID>';
delete from webhook_deliveries
  where payload->'data'->>'identity_id' = '<TEST-IDENTITY-ID>';
delete from trust_events where identity_id = '<TEST-IDENTITY-ID>';
delete from identities where id = '<TEST-IDENTITY-ID>';
delete from auth.users where email = 'YOUR-TEST-EMAIL@last1-test.local';

-- against RadioCheck Supabase
delete from last1_webhook_deliveries where identity_id = '<TEST-IDENTITY-ID>';
update users set last1_identity_id = null where last1_identity_id = '<TEST-IDENTITY-ID>';

-- against LastVet Postgres
update providers
set last1_identity_id = null,
    last1_access_token_ciphertext = null,
    last1_refresh_token_ciphertext = null,
    last1_token_scope = null,
    last1_access_token_expires_at = null
where last1_identity_id = '<TEST-IDENTITY-ID>';

End-to-end smoke runbook¶

Prerequisites¶

Deploys¶

Webhook configuration¶

Throwaway test account¶

Environment variables for the verify script¶

1. Chain: Link veteran → check-in → revoke (RadioCheck)¶

1.1 Link¶

1.2 Check-in¶

1.3 Revoke (from end-user dashboard)¶

1.4 Re-link is possible¶

2. Chain: Disconnect from LastVet provider portal¶

2.1 Link¶

2.2 Disconnect¶

3. Chain: Admin force-unlink¶

3.1 Link¶

3.2 Force unlink¶

3.3 Half-state recovery (optional but recommended)¶

4. Smoke test for the "Already revoked" idempotency path¶

5. Common debug commands¶

last1.id webhook drainer¶

RadioCheck inbound webhook idempotency¶

LastVet inbound webhook idempotency¶

Re-fire a stuck delivery¶

6. Cross-app trust activity (Identity alignment)¶

Prerequisites¶

6.1 RadioCheck check-in → Activity + Care Connections + dimensions¶

6.2 RadioCheck responder added¶

6.3 Last1.app operational events¶

6.4 Public dimension sharing¶

6.5 Activity filters¶

7. Cleanup¶