last1.id Sovereign Ops Runbook (R940)

Status: Active production runbook post–Tier 2 closeout (2026-08-26)
Primary URL: https://auth.last1.id
Plan archive: LAST1_ID_MIGRATION_PLAN.md (Tier 2 CLOSED)

---

Infrastructure map

Component	VM / host	LAN IP	Tailscale	Notes
auth-last1id (VM 114)	GoTrue + Kong + last1.id app + MinIO + Caddy	192.168.1.96	100.85.114.46	App at /opt/last1.id/; GoTrue stack at /opt/auth-last1id/; 8 GB RAM (bumped 2026-08-26)
sovereign-db-last1id (VM 113)	Postgres 17 + postgres_exporter	192.168.1.95	—	DB last1id_prod; roles last1id_owner, last1id_app
Ingress	Cloudflare Tunnel	—	—	auth.last1.id → auth-last1id
Marketing	Cloudflare Pages	—	—	last1.id (not auth)

---

Bitwarden Secrets (BWS)

Field	Value
Project	Last1.id
Project ID	cffb320a-5f61-4e1d-85df-b480012151ec
Prefix	LAST1_ID_R940_PROD_* (33 keys as of Tier 2 closeout)
Access token	/etc/bitwarden/access-token on operator host

Key map (by purpose)

Purpose	BWS keys
Postgres (app role)	LAST1_ID_R940_PROD_DATABASE_URL
Postgres (owner/migrations)	LAST1_ID_R940_PROD_DATABASE_OWNER_URL
GoTrue session JWT	LAST1_ID_R940_PROD_GOTRUE_JWT_SECRET, GOTRUE_ADMIN_SECRET
PostgREST / anon + service JWTs	LAST1_ID_R940_PROD_VITE_SUPABASE_ANON_KEY, SOVEREIGN_SERVICE_ROLE_KEY
Public API base (app env)	LAST1_ID_R940_PROD_SUPABASE_TLS_URL
Partner OIDC PEM	LAST1_ID_R940_PROD_OAUTH_JWT_{PRIVATE_KEY,PUBLIC_KEY,KID}
Google OAuth	LAST1_ID_R940_PROD_GOOGLE_OAUTH_{CLIENT_ID,CLIENT_SECRET}
Partner app secrets	LAST1_ID_R940_PROD_{LASTVET,LAST1,ONSET,RADIOCHECK}_APP_SECRET
Credential encryption	LAST1_ID_R940_PROD_CREDENTIAL_SIGNING_KEY
Admin / bridge HMAC	LAST1_ID_R940_PROD_{ADMIN_GATEWAY,LAST1ID_ADMIN_GATEWAY,BRIDGE_AUTHORIZE_INTENT}_SECRET
MinIO profile photos	LAST1_ID_R940_PROD_MINIO_{ACCESS_KEY,SECRET_KEY,ENDPOINT}
Cutover URLs	LAST1_ID_R940_PROD_{SITE_URL,PUBLIC_BASE_URL,LAST1_ID_ISSUER_URL}
Turnstile / Twilio	LAST1_ID_R940_PROD_TURNSTILE_*, TWILIO_*, VITE_TURNSTILE_SITE_KEY
Metrics	LAST1_ID_R940_POSTGRES_EXPORTER_PASSWORD

Hygiene note: Week 3 copy-railway-secrets-week3.sh could create duplicate BWS entries if re-run. Always use bws secret list + key name uniqueness before writes. See migration plan § duplicate BWS follow-up.

---

Redeploy path (last1.id app)

From operator host with BWS access:

cd /path/to/lastvet-gates/scripts/last1-id

# 1. Render env from BWS (never commit output)
./render-last1id-env-week3.sh /tmp/last1id-week3.env

# 2. Copy to auth VM and rebuild container (see deploy-last1id-app-week3.sh)
./deploy-last1id-app-week3.sh   # or manual: scp env + docker build/run on 192.168.1.96

On auth-last1id directly:

# Env lives at /opt/last1.id/.env (mode 600)
# Container: last1id-app on 127.0.0.1:5000 → Caddy → tunnel
docker ps | grep last1id

GoTrue/Kong stack: /opt/auth-last1id/docker-compose.yml — restart only when GoTrue config or image changes.

Kong route-only updates (L3+):

cd /path/to/lastvet-gates/scripts/last1-id
./apply-kong-config.sh   # copies kong/kong.yml, backs up prior, restarts kong

---

Health checks

curl -sf https://auth.last1.id/auth/v1/health | jq .
curl -sf https://auth.last1.id/.well-known/openid-configuration | jq '{issuer,jwks_uri}'
curl -sf https://auth.last1.id/.well-known/jwks.json | jq '.keys[].kid'
# Login UI ingress (L3+ — should be HTML 200, not Kong 404 JSON)
curl -sf -o /dev/null -w "login %{http_code}\n" https://auth.last1.id/login
curl -sf https://auth.last1.id/api/config | jq '{supabaseUrl, turnstileSiteKey: .turnstileSiteKey != ""}'

Synthetic OIDC (on auth VM):

/opt/last1.id/week6b-oidc-e2e.sh

Prometheus: PostgresLast1idDatabaseDown, GoTrueDown, KongDown, Last1idAuth5xxRateHigh (see LAST1_ID_WEEK6B1_REPORT.md).

---

Backup cadence

Asset	Location	Cadence	Script / method
Postgres last1id_prod	sovereign-db-last1id	Daily recommended (operator)	pg_dump -Fc auth+public+storage schemas; store encrypted off-repo
MinIO profile-photos	auth-last1id	With DB or weekly	mc mirror or bucket sync
BWS secrets	Bitwarden SM	On change	Export via BWS CLI; never commit values
Pre-teardown archive	lastvet-gates/backups/last1id-railway-teardown-2026-08-26/	One-time (W7-2)	Railway env + hosted Supabase dump before delete

Gate B reference dump: supabase-last1id.dump in archive (397 KB, PG 17 format). Live sovereign backups should use pg_dump against 192.168.1.95:5432/last1id_prod with owner credentials from BWS.

Restore drill: Annually or before major GoTrue upgrades. Requires PG 17 pg_restore.

---

Teardown archive (Tier 2 complete)

All legacy Railway + hosted Supabase artifacts:

lastvet-gates/backups/last1id-railway-teardown-2026-08-26/

Deletion records: supabase-deletion-record.json, railway-deletion-record.json, gcp-oauth-deletion-record.json, bws-deletion-record.json.

---

Deferred follow-ups (not in this runbook)

Item	Status
iOS issuer still https://last1.id	Rebuild when Ryan ready
my.last.vet Last1 sign-in	New scope; separate muster item
Provider portal staging issuer cleanup	Small separate task
Duplicate BWS key guard on copy-railway-secrets-week3.sh	Engineering hygiene

---

Related docs

Doc	Purpose
LAST1_ID_MIGRATION_PLAN.md	Full Tier 2 history + closeout
LAST1_INFRASTRUCTURE_CONSOLIDATION_PLAN.md	Multi-tier roadmap
LAST1_ID_WEEK6B_REPORT.md	Cutover details
last.vet-api/docs/api/LAST1_ID_INTEGRATION.md	LastVet partner integration