LastVet dev & staging environments¶
LastVet uses a paired dev stack: the staging API on AWS ECS plus the dev provider portal on Cloudflare Pages. Production API is a separate AWS stack at api.last.vet.
Railway staging (last.vet-api-staging) stays online as a rollback net during the migration window. Client apps should call api-staging.last.vet, not the .railway.app hostname.
There is no separate staging marketing site or iOS backend today. See iOS below.
Quick reference¶
| Component | Dev / staging | Production |
|---|---|---|
| API | https://api-staging.last.vet |
https://api.last.vet |
| API base (clients) | https://api-staging.last.vet/api |
https://api.last.vet/api |
| Postgres | AWS RDS lastvet-staging-postgres |
AWS RDS lastvet-prod-postgres |
| App secrets | Secrets Manager lastvet-staging/app |
Secrets Manager lastvet-prod/app |
| Provider portal | https://dev.provider.last.vet |
https://provider.last.vet |
| Marketing web | https://last.vet (shared) |
Same |
| iOS LastVet API | Staging when Configuration.environment == .sandbox |
Prod when .production |
| Onset → LastVet OAuth | onset-ios-sandbox client; set LAST1_ID_OAUTH_REDIRECT_URI_ONSET to staging callback URL |
onset-ios client on prod API |
| VA Lighthouse | Sandbox (all non-App-Store builds) | Sandbox until VA production access |
See docs/deployment/ENVIRONMENT_TOPOLOGY.md for the permanent staging lane, VA credential walls, fixture suite, and deploy sequence.
DNS (BAA): api-staging.last.vet and api.last.vet are grey-cloud (DNS only) in Cloudflare. TLS terminates at the AWS ALB with ACM certs. Do not re-enable Cloudflare proxy on API hostnames.
Ops console paths: AWS_OPS_CONSOLE.md
Staging API (AWS)¶
Health check¶
Grey-cloud verification:
dig +short api-staging.last.vet A # ALB IPs, not 104.21.x / 172.67.x
openssl s_client -connect api-staging.last.vet:443 -servername api-staging.last.vet </dev/null 2>/dev/null \
| openssl x509 -noout -subject # CN=api-staging.last.vet
Deploy and secrets¶
From last.vet-api repo root:
source ./infra/scripts/load-aws-env.sh
./infra/scripts/sync-railway-secrets.sh staging # Railway → Secrets Manager (bootstrap / parity)
Code-only deploy (no pending migrations):
Schema / migration deploy (required order; avoids rollout racing migration one-off):
./infra/scripts/deploy-api-image.sh staging --push-only
./infra/scripts/run-ecs-db-setup.sh staging # blocks until migration task exits 0
./infra/scripts/deploy-api-image.sh staging --rollout-only
2026-07-21 prod 051 deploy rolled out before migrate finished; RLS verify still passed but treat that as luck, not procedure.
After editing Secrets Manager lastvet-staging/app, force a new ECS deployment on lastvet-staging-cluster / lastvet-staging-api.
Key environment variables (staging)¶
Stored in Secrets Manager lastvet-staging/app (synced from Railway last.vet-api-staging):
| Variable | Expected value |
|---|---|
NODE_ENV |
staging |
DATABASE_URL |
RDS master URL (lastvet-staging/rds/master) |
DATABASE_APP_URL |
RDS lastvet_app URL (lastvet-staging/rds/lastvet_app) |
PROVIDER_PORTAL_URL |
https://dev.provider.last.vet |
LAST1_ID_OAUTH_REDIRECT_URI_PROVIDER |
https://dev.provider.last.vet/auth/callback |
LAST1_ID_OAUTH_CLIENT_ID |
Staging UUID app client on last1.id |
LAST1_ID_OAUTH_EXPECTED_AUDIENCE |
lastvet-staging |
GOOGLE_OAUTH_CLIENT_ID |
LastVet Google OAuth client ID |
GOOGLE_OAUTH_CLIENT_SECRET |
Server-only. Never commit. |
GOOGLE_OAUTH_REDIRECT_URI_PROVIDER |
https://dev.provider.last.vet/auth/google/callback |
RUN_MIGRATIONS_ON_BOOT |
true |
LAST1_PROOF_DRAINER_ENABLED |
false (typical for staging) |
CORS_ORIGIN may be a comma-separated allowlist. The API always also allows https://provider.last.vet, https://dev.provider.last.vet, and PROVIDER_PORTAL_URL so the secure-link landing can call /api/secure-access/* from the browser.
Staging auto-runs migrations on deploy. Production never auto-migrates. Apply migrations manually after verifying on staging.
Provider registration admin email notifications are on by default when PROVIDER_PORTAL_URL contains dev.provider.
Local API development¶
Local .env may still point at the Railway staging Postgres proxy for convenience (see repo .env / .env.example). That keeps local API work off production data. To hit the AWS staging API from a local server, set BASE_URL / client URLs to https://api-staging.last.vet.
Dev provider portal (Cloudflare Pages)¶
Required build-time variables¶
The provider app reads NEXT_PUBLIC_* at build time. If unset, it defaults to production API, a common source of mixed-environment bugs.
For dev.provider.last.vet, set these in Cloudflare Pages → last-vet-provider → Settings → Environment variables (Preview or a dedicated dev branch / production slot for that hostname):
| Variable | Dev / staging value |
|---|---|
NEXT_PUBLIC_API_URL |
https://api-staging.last.vet/api |
NEXT_PUBLIC_LAST1_ID_OAUTH_CLIENT_ID |
Staging UUID (same as LAST1_ID_OAUTH_CLIENT_ID in staging secrets) |
NEXT_PUBLIC_LAST1_ID_OAUTH_REDIRECT_URI |
https://dev.provider.last.vet/auth/callback |
NEXT_PUBLIC_LAST1_ID_ISSUER |
https://last1.id |
NEXT_PUBLIC_GOOGLE_OAUTH_CLIENT_ID |
LastVet Google OAuth client ID |
NEXT_PUBLIC_GOOGLE_OAUTH_REDIRECT_URI |
https://dev.provider.last.vet/auth/google/callback |
Copy from last.vet-provider/.env.staging.example.
See GOOGLE_SIGNIN.md for Google Cloud Console setup and OAuth consent screen branding (logo asset).
Local provider dev (against staging API)¶
Open http://localhost:3000. A yellow staging API banner appears when not pointed at prod.
Manual deploy to dev (Cloudflare)¶
cd last.vet-provider
cp .env.staging.example .env.staging.local # fill in if needed
npm run cf:deploy:dev
Deploys to Cloudflare Pages project last-vet-provider-staging (custom domain dev.provider.last.vet).
Verify wiring¶
- Open
https://dev.provider.last.vet. Confirm the Staging API banner is visible. - Browser devtools → Network → API calls should go to
api-staging.last.vet, notapi.last.vet. - Last1 sign-in should use redirect
https://dev.provider.last.vet/auth/callbackand staging OAuth client id.
iOS app¶
API target¶
LastVet/App/Configuration.swift:
AppEnvironment.sandbox(default for TestFlight/internal builds) → staging API
https://api-staging.last.vet/apiAppEnvironment.production(App Store release builds) → prod API
https://api.last.vet/api
LastVetAPIClient uses Configuration.apiBaseURL. Do not hardcode URLs elsewhere.
Last1 ID OAuth (Profile → Link Last1 ID)¶
Sandbox builds must authorize against the same last1.id app client the staging API uses for token exchange:
| Build | Configuration.last1OAuthClientID |
Staging LAST1_ID_OAUTH_CLIENT_ID |
|---|---|---|
| Sandbox / TestFlight | f3713b66-71da-4392-8758-d5f96b19f64f |
same UUID |
| App Store / production | lastvet |
lastvet |
Redirect URI stays https://last.vet/auth/last1/callback in both environments (registered on the staging UUID client in last1.id).
If the iOS app uses lastvet while the staging API uses the UUID client, authorize completes but /auth/last1/veteran/link fails with token_exchange_failed.
Onset → LastVet OAuth redirect (migration 035)¶
Staging/Debug Onset builds use onset://lastvet/callback instead of the Universal Link so direct Xcode installs can finish Connect without Associated Domains. Apply migration 035_oauth_onset_custom_scheme_redirect.sql (or set RUN_MIGRATIONS_ON_BOOT=true on deploy).
Before App Store release¶
Set Configuration.environment to .production in the release branch/build, archive, and ship. VA OAuth remains sandbox until VA production credentials are approved.
Verify on device¶
Profile or debug surfaces can log the active API base. After install, pull-to-refresh on Record should hit api-staging.last.vet when running a sandbox-configured build.
Smoke tests¶
Health + bootstrap work on staging:
API_BASE=https://api-staging.last.vet/api ./scripts/seed-roi-demo.sh
API_BASE=https://api-staging.last.vet/api ./scripts/e2e-roi-gate0.sh
Staging fixture suite (staging RDS; all security-boundary proof — do not use prod):
source ./infra/scripts/load-aws-env.sh staging
./infra/scripts/run-ecs-seed-staging-fixtures.sh staging
./infra/scripts/run-ecs-verify-portal-documents.sh staging
Seeds Veterans A/B/C and two providers. Veteran A carries merged dual-grant, signed ROI, DD-214 (service_records), and both Part 2 paths (revoked signed_roi row + active sheet row for HL issue #2 promotion verify). UUIDs in scripts/staging-fixtures.js.
Legacy alias: run-ecs-seed-staging-dual-grant.sh → same fixture suite.
document_share allowlist (Gate 1) — staging status¶
Verified on staging (2026-07-23): live POST /api/veteran/me/secure-links rejects ROI artifacts with 400 document_share_requires_service_records and accepts DD-214 (service_records) with 201. Run:
source ./infra/scripts/load-aws-env.sh staging
./infra/scripts/run-ecs-verify-document-share-guards.sh staging
Prerequisites once per environment: merge-app-secret-keys.sh (populates VETERAN_JWT_SECRET + SECURE_LINK_HMAC_SECRET in SM), terraform apply (wires keys into task definition), pin service to the new task def revision, then deploy-api-image.sh staging.
Do not record allowlist enforcement as verified until that live POST path passes. Mirror/DB-only checks do not count.
If Docker is unavailable, apply migrations only via the normal deploy sequence (--push-only → run-ecs-db-setup.sh → --rollout-only). Do not treat ECS SQL injection (below) as a substitute for deploy + live verify.
ECS SQL injection for migrations (workaround only)¶
When the running image does not contain new migration files, injecting SQL via a one-off ECS task can unblock schema work temporarily. This is not the normal pattern. It decouples schema from code — exactly what the deploy runbook exists to prevent.
| Acceptable when | Required follow-up |
|---|---|
| Docker unavailable and you need schema ahead of image (emergency / time-boxed) | Record decoupled state explicitly (schema ahead of code) |
| Never as the default deploy path | ./infra/scripts/deploy-api-image.sh staging so running tasks include migration files and allowlist code |
| Never mark security guards verified | Re-run run-ecs-verify-document-share-guards.sh against the live API after rollout |
Normal order remains: deploy-api-image.sh --push-only → run-ecs-db-setup.sh → wait exit 0 → --rollout-only → verify.
Legacy Railway rollback hostname (do not use for new client wiring):
The consent step in smoke-consent.sh may fail on both staging and prod due to veteran id header vs internal UUID mismatch. Use provider portal E2E for consent validation.
Production deploy checklist (after staging verification)¶
- Merge to
mainonlast.vet-api: export LASTVET_ALLOW_PROD_DEPLOY=1source ./infra/scripts/load-aws-env.sh prod- Pending migrations:
deploy-api-image.sh prod --push-only→run-ecs-db-setup.sh prod '...'→ wait →--rollout-only - Code only:
./infra/scripts/deploy-api-image.sh prod - Merge to
mainonlast.vet-provider→ Cloudflare Pages deploysprovider.last.vet(prod env vars must keepNEXT_PUBLIC_API_URL=https://api.last.vet/api). - iOS: App Store release only after VA approval (P3) — prod OAuth client IDs in
Secrets.xcconfig,LASTVET_VA_ENVIRONMENT=production. - Do not run staging seed scripts on prod. Use staging fixtures for all security-boundary verification.
Related docs¶
- ENVIRONMENT_TOPOLOGY.md
- AWS_OPS_CONSOLE.md
- VA_SANDBOX_SMOKE_TEST.md
- LAST1_ID_INTEGRATION.md
last.vet-provider/.env.staging.example