Products Isolation Architecture — Network + Identity Clarification¶

Status: Audit report for Ryan review. No execution.
Date: 2026-08-29
Trigger: Ryan architectural correction before T3-4 continues (ARCH-1)
Scope: Ingress model, auth.last1.id tunnel migration plan, auth unification vs RealOutcomes Tier 3, full portfolio impact

Ryan principles (restated)¶

#	Principle
1	`auth.last1.id` moves to a dedicated `last1id-edge` tunnel — separate from `lastvet-edge`.
2	Products do not get dedicated Cloudflare Tunnels — RealOutcomes, Onset, last1.app, RadioCheck, admin.
3	Only LastVet needs its own tunnel — product-specific requirement (clinical / PHI ingress posture).
4	Products consume `auth.last1.id` via NPM SDK (`@last1id/sdk`, login-button, OIDC) like any external integrator — not product-local GoTrue.

These principles conflict with in-flight Tier 3 RealOutcomes work (product-local GoTrue on VM 117, staging hostnames on lastvet-edge tunnel). T3-4 is correctly STOPped until Ryan confirms target architecture.

Executive summary¶

Topic	As-built today	Ryan target (inferred)	Gap
Public ingress to R940 LAN	Hybrid: LastVet API on Cox static IP + Caddy; everything else mostly `lastvet-edge` Cloudflare Tunnel	Platform identity on `last1id-edge`; LastVet on `lastvet-edge`; other products via shared reverse proxy, not per-product tunnels	T3-4 planned RealOutcomes + Supabase staging routes on `lastvet-edge` — violates principle 2
`auth.last1.id`	Live on R940; DNS → `lastvet-edge` tunnel → `http://192.168.1.96:8000`	Dedicated `last1id-edge` tunnel	Migration required; live users authenticating today
Product auth	Mixed: sovereign IdP for partners; most products still on hosted/product Supabase Auth	Single identity layer at `auth.last1.id`	RealOutcomes T3-1 explicitly deferred unification; VM 117 runs product GoTrue
RealOutcomes Tier 3	VMs 116/117 provisioned; soak on LAN; T3-4 blocked on tunnel + GoTrue	Architecture decision required before any T3-4 resume	See §3

Recommendation (Cursor, for Ryan to confirm or reject):

Freeze T3-4 until ingress + identity decisions are locked (already STOPped).
Do not add RealOutcomes (or admin) hostnames to lastvet-edge cloudflared.
Plan last1id-edge migration as a discrete, live-traffic workstream (§2).
Choose one auth path for RealOutcomes (§3): unify now vs temporary product GoTrue vs pause sovereignty until unified.
Define shared product ingress (§1): Cox + Caddy multi-hostname on a platform edge, not N tunnels.

1. How does external traffic reach VMs on `192.168.1.0/24`?¶

1.1 As-built ingress map (audit, 2026-08-29)¶

The portfolio uses three overlapping patterns, not one:

Internet
   │
   ├─── (A) Cox static IP 98.186.224.178 ──► Orbi NAT :443/:80 ──► lastvet-edge .89 (Caddy TLS)
   │         Grey-cloud DNS: api.last.vet, api-staging.last.vet
   │         Status: LIVE (2026-08-12 cutover). PHI API path; no Cloudflare proxy.
   │
   ├─── (B) Cloudflare Tunnel (cloudflared on lastvet-edge .89, tunnel lastvet-r940)
   │         Proxied CNAME → *.cfargotunnel.com
   │         Current routes (from infra preview + scripts):
   │           auth.last1.id          → 192.168.1.96:8000   ← LIVE identity
   │           n8n.last1.enterprises  → 192.168.1.94:5678   ← LIVE automation
   │           admin.last1.enterprises→ 192.168.1.97:5000   ← staged; prod still Railway
   │           api.last.vet / api-staging (also listed — overlap/rollback with Caddy)
   │         Planned (T3-4, NOT applied): realoutcomes-app*, realoutcomes-db-r940*
   │
   ├─── (C) Cloudflare Pages / Railway (vendor compute)
   │         realoutcomes.app, admin prod, most product apex domains
   │         Traffic never hits R940 until DNS cutover
   │
   └─── (D) Tailscale mesh — admin/operator only, NOT public ingress
             East-west Postgres over 100.x blocked by ACL; BWS uses LAN IPs

Physical path to LAN: Cox Business → Netgear Orbi RBR860 (interim NAT) → JGS524Ev2 switch → Proxmox VMs. SonicWall 3000 is inventory, not documented in-service. Netgate 5200 is the planned firewall (multi-IP NAT, second static IP .186 for Angel).

Sources: Muster r940-ingress-migration-cox-static-ips-caddy-reti, HIPAA_ARCHITECTURE.md §1.5, infra/lastvet-edge/cloudflared-config-t3-4-staging.preview.yml, RealOutcomes/admin tunnel scripts.

1.2 Answer to Q1 — which pattern applies going forward?¶

Option	Used today?	Fits Ryan principles?	Notes
Public IP + port forward (Orbi → edge VM Caddy)	Yes for LastVet API	Yes for non-tunnel products if hostnames share one reverse proxy	Documented target in Muster: grey-cloud DNS → static IP → Caddy SNI routing. Not per-product tunnels.
Cloudflare Tunnel per hostname on `lastvet-edge`	Yes for auth, n8n, staged admin/RO	No for products (principle 2). No for auth (principle 1 — must move to `last1id-edge`).	Current T3-4 plan assumes this — needs replacement.
Cloudflare orange-cloud proxy terminating at origin	Marketing Pages only	Partial	Acceptable for non-PHI marketing; not for LastVet API (grey-cloud invariant).
Dedicated bastion / VPN for users	Tailscale for ops	No for public products	Stays admin-only.
Something in ENVIRONMENT_TOPOLOGY.md	File deleted/missing from repo at audit time	—	Canonical detail lived in `docs/deployment/R940_ENVIRONMENT_TOPOLOGY.md` (referenced by HIPAA doc, not present in workspace). Recreate or restore as follow-up.

1.3 Target product ingress (Cursor proposal — Ryan to confirm)¶

If products must not get dedicated Cloudflare Tunnels:

Proposed model: shared platform ingress via Cox + Caddy (no new tunnel credentials per product)

realoutcomes.app ──grey-cloud──► 98.186.224.178:443 ──► Caddy on platform edge
onset.app        ──grey-cloud──► (same IP, different hostname)
admin.last1...   ──grey-cloud──► (same IP, different hostname)
                                      │
                                      ├──► 192.168.1.99:5000  (RealOutcomes app VM)
                                      ├──► 192.168.1.97:5000  (admin VM)
                                      └──► … per product LAN origin

Component	Role
Cox static IP + NAT	Single (or dual) public entry; port 443/80 → edge VM
Caddy (or equivalent)	TLS + hostname routing to LAN origins
Cloudflare DNS	Grey-cloud A/AAAA to static IP for product API/app hostnames
`last1id-edge` tunnel	Only `auth.last1.id` (and maybe `/.well-known/*`) — platform identity, not a "product"
`lastvet-edge` tunnel	Only LastVet clinical surfaces if Ryan requires tunnel-specific controls beyond Caddy; otherwise LastVet may also be Caddy-only on `.89`

Implications:

T3-4 staging URLs like realoutcomes-app.last1.enterprises should not be added to lastvet-edge cloudflared. Staging should use either:
Grey-cloud staging hostnames on the same Caddy edge, or
Tailscale/LAN-only testing until cutover, or
Temporary Ryan-approved exception (explicitly documented as debt).
No realoutcomes-db-r940 public hostname if auth is unified at auth.last1.id (browser never talks to product Kong for login). If product GoTrue stays, Supabase Kong must be reachable publicly — that either violates "no product tunnel" or requires shared Caddy route to Kong (still not a separate tunnel).
n8n (n8n.last1.enterprises on lastvet-edge today) is unclassified under new principles — platform automation vs product. Ryan should label it: move to platform Caddy, keep on lastvet-edge, or separate ops tunnel.

2. `auth.last1.id` on `lastvet-edge` → dedicated `last1id-edge`¶

2.1 Current state¶

Item	Value
Public URL	`https://auth.last1.id` — LIVE (Tier 2 closed 2026-08-26)
Compute	`auth-last1id` VMID 114; Kong/app at LAN `192.168.1.96:8000` (tunnel target)
Database	`sovereign-db-last1id` VMID 113; BWS `DATABASE_URL` uses LAN `192.168.1.95` per muster
Ingress today	Cloudflare proxied DNS → `lastvet-edge` tunnel `lastvet-r940` → `.96:8000`
Stack	Custom GoTrue + Kong + Express monolith (not official Supabase compose)
BWS	`LAST1_ID_R940_*` (33 live keys post-teardown)

2.2 Live traffic / blast radius¶

Who authenticates against auth.last1.id today (production):

Consumer	Integration	Live?
Human users	Login, signup, dashboard, OAuth consent	Yes
Provider portal	OIDC issuer `auth.last1.id` (`provider.last.vet`, W6B-3)	Yes
Partner OIDC	Registered app clients, redirect URIs on auth host	Yes (Week 6b E2E PASS)
last1-admin	HMAC proxy to `LAST1_ID_API_BASE=https://auth.last1.id` (server-side; admin user login still shared Supabase until Tier 6)	Partial — proxy yes; admin login split
LastVet API	`@last1id/sdk` webhooks, profile, proof drainer (targets migrating to sovereign URLs)	Partial
last1.id iOS	Issuer still `https://last1.id` in places	Deferred rebuild
my.last.vet Last1 sign-in	Not shipped	No
RealOutcomes	Optional cert registry URL only; not IdP	No
Onset / last1.app / RadioCheck	Hosted Supabase Auth on product projects	No (product-local today)

Any tunnel migration is a live identity outage risk if cutover is not atomic (DNS TTL, dual-run, session cookies, OAuth redirect URIs unchanged).

2.3 Migration plan (plan only — do not execute)¶

Goal: auth.last1.id terminates on last1id-edge cloudflared (new tunnel credential + VM or systemd unit), removed from lastvet-edge ingress.

Phase	Owner	Actions	Verification
P0 — Design	Ryan + Cursor	Confirm `last1id-edge` placement: new VM (e.g. `.88`) vs dedicated cloudflared on existing edge VM that is not `lastvet-edge`. Name tunnel `last1id-r940`.	Written decision in this doc
P1 — Provision	Cursor (when authorized)	Create Cloudflare tunnel `last1id-edge`; install `cloudflared` on chosen host; UFW allow tunnel host → `192.168.1.96:8000` only	`cloudflared tunnel ingress validate`
P2 — Parallel ingress	Cursor	Add route `auth.last1.id` → `http://192.168.1.96:8000` on new tunnel config; do not remove `lastvet-edge` route yet	Staging hostname test: `auth-staging.last1.id` or low-TTL parallel CNAME (Ryan choice)
P3 — OAuth / redirect audit	Ryan	GCP OAuth client authorized redirect URIs unchanged (`https://auth.last1.id/...`); partner client redirect matrix export	Spreadsheet of all registered redirect URIs
P4 — DNS cutover	Ryan	Cloudflare DNS: `auth.last1.id` CNAME → new tunnel UUID `.cfargotunnel.com` (proxied)	External `curl -sI https://auth.last1.id/health`; login E2E
P5 — Decommission	Cursor	Remove `auth.last1.id` from `lastvet-edge` `/etc/cloudflared/config.yml`; restart; confirm no 404 on old tunnel	Monitor `Last1idAuth5xxRateHigh`, provider login
P6 — Docs / BWS	Cursor	Update runbooks, HIPAA asset table ("which hostnames on which tunnel"), BWS if tunnel secrets stored	Muster item

Rollback: Revert DNS CNAME to lastvet-edge tunnel target; keep old route in config until P6 complete.

Coordination checklist (every product consuming auth):

[ ] Provider portal OIDC discovery + callback URLs
[ ] Partner app clients (Onset, MATR, etc.)
[ ] last1-admin proxy (unchanged URL if hostname stable — preferred)
[ ] LastVet API JWKS / issuer validation env vars
[ ] iOS / mobile issuer strings (when rebuilt)
[ ] Developer docs / @last1id/sdk default issuer URLs
[ ] UptimeRobot / Prometheus probes hitting auth.last1.id

Stable hostname invariant: Migration should not change the public URL https://auth.last1.id — only the tunnel backend behind DNS. That minimizes product code changes.

3. Auth unification vs RealOutcomes Tier 3¶

3.1 The contradiction¶

Decision	When	Statement
T3-1 (Ryan 2026-08-28)	RealOutcomes planning	Defer `auth.last1.id` unification; keep product-local GoTrue; lift `auth.users` dump
ARCH-1 (Ryan 2026-08-29)	Portfolio principles	Products consume `auth.last1.id` via NPM SDK; product-local GoTrue should not exist

Both cannot be true at steady state. Tier 3 execution assumed the first; architecture principles assert the second.

3.2 RealOutcomes auth as-built on R940¶

Layer	State
VM 117	Official Supabase Docker includes GoTrue at Kong `:8000/auth/v1`
Users	3 Google-only accounts lifted in Gate B (`auth.users`; `auth.identities` empty — restore gap)
App (VM 116)	`supabase-js` login UI: password, OTP, Google OAuth → `/auth/callback`
BWS	`SUPABASE_URL=http://192.168.1.98:8000` (LAN); no `@last1id/sdk` in product
T3-4 plan	Public staging Supabase hostname + GoTrue redirect allow-list + Google OAuth on product GoTrue

3.3 Three paths (Ryan must pick one)¶

Option A — Unify now (aligns with ARCH-1 principles)¶

Work	Effort (from T3-1)	Notes
Register RealOutcomes OIDC client on `auth.last1.id`	~0.5 d	Redirect: `https://realoutcomes.app/auth/callback` (and staging hostname)
Replace login UI with `@last1id/sdk` / OIDC redirect	~1 d	Remove product GoTrue dependency
Server: validate JWT via `auth.last1.id` JWKS	~0.5–1 d	Replace `supabaseAdmin.auth.getUser`
Migrate 3 users to sovereign identities	~0.5 d	Google accounts may map 1:1; FK remap in `public.users`
Decommission GoTrue on VM 117	~0.5 d	Keep Postgres/PostgREST/Edge Functions; auth schema optional/archive
GCP OAuth redirect matrix	~0.5 d	On last1.id client, not product Supabase

Pros: One identity layer; no public Supabase Kong for auth; T3-4 browser tests use real portfolio pattern; closes architectural debt early.
Cons: Adds 2–4 days to Tier 3 critical path; couples RealOutcomes cutover to auth.last1.id uptime; last1id-edge migration should ideally land first to avoid auth ingress churn mid-tier.

T3-4 impact: Staging needs app hostname only (shared Caddy ingress). Auth tests exercise auth.last1.id, not VM 117 GoTrue. Delete configure-gotrue-staging-redirects.sh / Supabase staging tunnel from plan.

Option B — Defer unification (aligns with T3-1; violates ARCH-1 steady-state)¶

Work	Notes
Keep VM 117 GoTrue	Lifted users, Google OAuth on product stack
Product ingress without dedicated tunnel	Still need shared Caddy route to app and possibly Kong for browser `supabase-js`
Document explicit architecture debt	"Temporary product GoTrue until identity workstream" with muster item + expiry

Pros: Fastest continuation of current T3-2/T3-3 investment; minimal auth code change.
Cons: Two identity layers on R940; duplicate Google OAuth clients; T3-4 Supabase public hostname still required; contradicts Ryan 2026-08-29 principles unless principles are "target state" not "Tier 3 gate."

Option C — Pause RealOutcomes sovereignty until identity + ingress locked (audit-first)¶

Work	Notes
Keep VMs 116/117 for rehearsal / LAN soak only	No public cutover
Production stays Railway + hosted Supabase	Zero user impact
Sequence: `last1id-edge` migration → ingress model doc → RealOutcomes Option A → Tier 3 resume

Pros: Cleanest alignment with new principles; avoids throwaway GoTrue/public-Kong work.
Cons: Delays SBIR "first full product on R940" narrative; sunk provisioning cost until resumed.

3.4 Cursor recommendation¶

Priority	Action
1	Ryan picks A, B, or C explicitly — do not resume T3-4 without this
2	If A or C: treat `last1id-edge` migration as prerequisite
3	If B: Ryan acknowledges principle 4 is future state, not Tier 3 enforcement — update ARCH-1 or T3-1 so docs agree
4	Regardless: replace per-product tunnel staging plan with shared Caddy ingress design (§1.3)

Account migration note (Option A): All 3 RealOutcomes users are Google-only on product GoTrue today. Unification means linking those emails to auth.last1.id identities (may already exist for ryan@last1.org / ryan@last1.enterprises). Requires identity dedup script + FK update on public.users, not password export.

4. Full architectural impact¶

4.1 Work invalidated or at risk (if principles enforced)¶

In-flight artifact	Impact
`configure-tunnel-staging-app.sh`	Adds product route to `lastvet-edge` — non-compliant
`configure-tunnel-staging-db.sh`	Public product Kong tunnel — non-compliant; unnecessary if Option A
`configure-gotrue-staging-redirects.sh`	Product GoTrue — non-compliant if Option A
T3-4 test plan § auth (A-2/A-3)	Assumes product Supabase URL — rewrite for Option A
`REALOUTCOMES_MIGRATION_PLAN.md` diagram (`lastvet-edge tunnel` → app)	Update ingress + auth
VM 117 GoTrue enablement / Google OAuth on product stack	Wasted if Option A
UFW rules allowing `lastvet-edge` → VM 116/117	May shift to platform edge IP
BWS `SUPABASE_URL` / anon keys for auth	Becomes PostgREST-only if GoTrue removed; auth keys from `auth.last1.id` client

4.2 Portfolio-wide identity matrix (target vs today)¶

Product	Auth today	Target (principle 4)	Tier
last1.id	`auth.last1.id` sovereign	Same	Done (Tier 2)
LastVet provider	OIDC → `auth.last1.id`	Same	W6B live
LastVet veteran	VA OAuth + (future Last1)	Last1 via SDK when shipped	Deferred
last1-admin	Shared Supabase + proxies	Admin login → `auth.last1.id`? (Tier 6)	Paused Tier 3
RealOutcomes	Product Supabase Auth	`@last1id/sdk`	Decision point
Onset	Product Supabase	SDK	Tier 4
last1.app + RadioCheck	Shared Supabase `tcthzakppndkykatoefx`	SDK	Tier 5
Angel	Separate (Sign in with Apple, etc.)	TBD (`angel.report` → auth.last1.id in Tier 6 notes)	Later
n8n	Basic auth / SSO?	Platform ops — not user identity	Tier 1 done

4.3 Ingress matrix (target vs today)¶

Hostname class	Today	Target (principles 1–3)
`auth.last1.id`	`lastvet-edge` tunnel	`last1id-edge` tunnel
`api.last.vet`	Cox + Caddy grey-cloud	LastVet edge (Caddy and/or lastvet tunnel)
`n8n.last1.enterprises`	`lastvet-edge` tunnel	Ryan: classify (platform Caddy vs stay)
Product app apex	Railway or staged tunnel	Shared Caddy, no product tunnel
Marketing apex	Cloudflare Pages	Unchanged

4.4 Compliance / narrative¶

Topic	Impact
Sovereignty story	Stronger with single IdP + on-metal products; weaker if product GoTrue persists
HIPAA / PHI	LastVet keeps grey-cloud API invariant; product apps without PHI may use shared Caddy
GitGuardian D-live	RealOutcomes JWT rotation tied to which auth stack issues tokens at cutover
SBIR / pitch	"One identity layer, many products" requires Option A narrative
BAA / Cloudflare	Fewer proxied tunnels for products reduces Cloudflare-in-path debates; Cox+Caddy matches LastVet precedent

4.5 Sequencing dependency graph¶

flowchart TD
  ARCH[Ryan confirms ARCH-1 principles]
  IDEDGE[last1id-edge tunnel migration]
  INGRESS[Shared product ingress design\nCox + Caddy hostnames]
  AUTHCHOICE[RealOutcomes auth: A / B / C]
  T34[T3-4 integration + browser E2E]
  T35[T3-5 DNS cutover realoutcomes.app]

  ARCH --> IDEDGE
  ARCH --> INGRESS
  ARCH --> AUTHCHOICE
  IDEDGE --> AUTHCHOICE
  INGRESS --> T34
  AUTHCHOICE --> T34
  T34 --> T35

4.6 Open questions for Ryan¶

last1id-edge host: New VM vs cloudflared on dedicated lightweight VM vs co-locate with auth-last1id VM 114?
LastVet "own tunnel": Does this mean keep cloudflared on lastvet-edge for LastVet-only hostnames while also using Cox+Caddy for API — or Caddy-only and retire LastVet tunnel entries entirely?
Shared product ingress: Single Caddy on lastvet-edge .89 for all non-LastVet products, or separate platform-edge VM (still one tunnel or zero tunnels)?
n8n: Platform exception on lastvet-edge, or move to platform Caddy?
Principle 4 timing: Tier 3 gate (Option A/C) or target state allowing Option B temporary GoTrue?
Restore R940_ENVIRONMENT_TOPOLOGY.md as canonical ingress doc?

5. T3-4 status (explicit STOP)¶

Item	Status
T3-4 tunnel dry-run on `lastvet-edge`	Done technically; do not apply under ARCH-1
Ryan R-1/R-2 CNAMEs for RealOutcomes staging	Hold until ingress model confirmed — CNAMEs to `lastvet-edge` tunnel conflict with principle 2
GoTrue allow-list R-3 / GCP R-4	Hold — depends on Option A vs B
Browser E2E test plan	Valid for directory/soak; auth section stale pending §3 decision

6. References¶

Doc	Relevance
TIER_3_REALOUTCOMES_T3-1_REPORT.md	Auth deferral decision
REALOUTCOMES_MIGRATION_PLAN.md	`lastvet-edge` tunnel assumption
TIER_3_REALOUTCOMES_T3-4_REPORT.md	STOP state
HIPAA_ARCHITECTURE.md	Cox / Caddy / cloudflared hybrid
Muster `r940-ingress-migration-cox-static-ips-caddy-reti`	LastVet direct-to-origin
Muster `migrate-last1-id-railway-supabase-to-r940`	Tier 2 closed; auth live
`infra/lastvet-edge/cloudflared-config-t3-4-staging.preview.yml`	Current tunnel routes

7. Decision log (blank — Ryan fills)¶

ID	Decision	Choice
D-1	Product ingress pattern	☐ Shared Caddy ☐ Other: ___
D-2	`last1id-edge` placement	☐ New VM ☐ Other: ___
D-3	RealOutcomes auth	☐ A unify now ☐ B defer GoTrue ☐ C pause tier
D-4	Principle 4 enforcement	☐ Tier 3 gate ☐ Future state only
D-5	Resume T3-4	☐ Yes after D-1…D-4 ☐ No

END — audit only. No provisioning, DNS, tunnel, or auth changes executed.