last1-meridian

Confidential — Last 1 Enterprises trade secret

Meridian M0 evaluation harness. Synthetic data only until the PHI hosting gate is lifted.

Local eval against self-hosted Mistral via Ollama; synthetic data only; not used in CI.

Quick start

uv sync --extra dev
uv run meridian-eval validate-cases
uv run meridian-eval run --provider mock
uv run pytest

Fixture snapshot (token ritual)

Short-lived VA_SANDBOX_ACCESS_TOKEN per persona (never committed):

export VA_SANDBOX_ACCESS_TOKEN="..."
export VA_SANDBOX_PERSONA_ID="persona_alpha"
uv run python scripts/fetch_sandbox_fixtures.py

Telemetry

Each run emits telemetry.json per MERIDIAN_TELEMETRY_CONTRACT.md. Optional POST when MERIDIAN_TELEMETRY_URL and MERIDIAN_TELEMETRY_TOKEN are set.

Crisis harness

Crisis protocol strings are wired per CRISIS_PROTOCOL.md. m1_eligible stays false until the clinical advisor signs the Section 3 blocklist (crisis.blocklist_advisor_signed in eval-config.yaml).

Public scoreboard (last.vet/meridian)

Metrics are copied into last.vet-web/content/meridian-eval.json after an api run.

Publish rule (standing): run 3 api passes (uv run meridian-eval run --provider api), then publish the worst result on zero-tolerance categories and m1_eligible. Never let a lucky pass flatter the model on the public page. See MERIDIAN_EVAL_SPEC.md and last.vet-web/README.md.

Parked (not built)

Reproducible api eval: today the api provider only runs when Ollama (or equivalent) is manually started locally, so published numbers drift from reality by default. At some point api eval should run on a reproducible path (Bedrock production model or CI) so the scoreboard cannot silently rot. Do not build until the hosting gate and model path are decided.