descrybe/docs/migration-readiness.md

# Migration readiness — MySQL → Postgres + post-import passwords

**Last updated:** 2026-08-04  
**Live ETL status:** **DONE on staging** (dry-run + load, 2026-08-03) — see [migration-run-log.md](migration-run-log.md).  
**Production cutover:** **NOT ready** — remaining blockers below.

Migrated users are imported with `password_hash = NULL` and `must_set_password = true`. Login rejects empty/null hashes, so a random guess cannot succeed.

Fresh installs that never ran the migrator can keep using `/register`.

---

## Live dry-run + load (staging)

| Gate | Status |
|------|--------|
| Goose `001`–`007` on staging PG (`localhost:5433`) at ETL time (2026-08-03) | Done — **repo schema is now `001`–`023`**; re-apply goose `up` before any new staging/prod load |
| Offline fixture dry-run | Done |
| Live MySQL dry-run | Done (exit 0) |
| Live load to staging Postgres | Done (~11 min) |
| Orphan FK validation | **16/16 pass** |
| Evidence | [migration-run-log.md](migration-run-log.md); `artifacts/` gitignored |

### Counts written (staging)

| Entity | Written |
|--------|--------:|
| companies | 27 |
| users | 21 |
| memberships | 15 |
| categories | 8278 |
| attributes | 57630 (unique `(company_id, attribute_key)` collapse) |
| raw_products | 118784 (+7732 GTIN-deduped from MySQL) |
| processed_products | 7219 |
| input feeds | 31 |
| export feeds | 8 |

### Remaining blockers (before production cutover)

| Severity | Item |
|----------|------|
| **Data** | Most users have synthetic `…@legacy.local` emails — export from Clerk and patch `users.email` before invites (**tooling:** migrator `-list-legacy-emails` / `-export-legacy-emails` / `-patch-emails`; see [portable-mysql-pg-migration.md](portable-mysql-pg-migration.md#clerk--legacylocal-email-repair-cutover-data-hygiene)) |
| **Data** | All memberships imported as `role=member` — promote company admins |
| **Data** | 2 `company_plans` skipped (`plan_id=6` missing) — resolve with migrator list → `-dry-run` → `-confirm` (see [cutover.md](cutover.md#resolve-skipped-company_plans-plan_id6); no blind live assigns) |
| **Ops** | This load used `-skip-post-import` — re-issue set-password invites when emails are real |
| **Ops** | SMTP + mailhooks + login smoke still unproven |
| **Gaps** | See [Cutover coverage gaps](#cutover-coverage-gaps) below |

Staging is **GO for login testing** after set-password (preferably after email patch). Production DNS/cutover remains **NO-GO**.

### Cutover coverage gaps

Honest status vs migrator code (`cmd/migrator`, domains `settings` / `woo` / `files`):

| Area | Status | Operator action |
|------|--------|-----------------|
| **API keys** | **Not migrated** (intentional — secrets) | Clients must mint new keys; Settings → API Keys empty state says so |
| **company_settings** | **Partial** — `language` + `merge_products` only (`migrateCompanySettings`) | Re-enter other legacy settings in-app if needed |
| **File blobs** | **Metadata only** — bytes not copied (`migrateFiles`) | Resync object storage / re-upload; Files empty + dashboard ETL gaps panel say so; platform admins see cheap COUNTs on **Admin → Diagnostics → ETL gap inventory** (`files_metadata_only` / `files_total`) |
| **Woo configs** | **Migrated when domain `woo` enabled** — from `wc_*` custom_fields (`migrateWooConfigs`) | Re-verify store URL + credentials; ensure `APP_ENCRYPTION_KEY` before prod secrets |
| **Jobs / history** | **Cutover default: not backfilled** — optional domain `jobs` can import `processing_jobs` (+ best-effort products) + `tasks` tagged `ai_provider_mode=migrated`; accepted production gap is empty history unless ops explicitly ran `jobs` | Expect empty Processing history after cutover; tenant UI must not claim history moved; diagnostics `migration_inventory.jobs_domain_ran` / `processing_jobs_migrated` show whether optional backfill ran |

Migrator run reports surface these as notes + count-report rows (`api_keys`, `files` metadata-only, `processing_jobs` when domain enabled, partial `company_settings`).

**Admin inventory (read-only):** `GET /api/admin/diagnostics` → `migration_inventory` — COUNT of metadata-only files, migrated-tagged jobs, and tasks. Not an import path; never fabricates blob bytes or job history.

---

## After a live import

### 1. Issue set-password invites (preferred)

Post-import already runs this unless `-skip-post-import`. To re-issue later (Postgres only, no MySQL):

```bash
cd apps/api
export DATABASE_URL="postgres://descrybe:descrybe@localhost:5433/descrybe?sslmode=disable"
# WEB_ORIGIN controls printed /accept-invite links (default http://localhost:5174)
go run ./cmd/migrator -issue-set-password-invites \
  -postgres "$DATABASE_URL" \
  -maps-dir ../../artifacts
```

Artifacts (gitignored, do not commit):

- `artifacts/password_invites.json` — email, token, invite URL, expiry
- `artifacts/set-password-hooks.json` — same payload for `cmd/mailhooks`

Stdout prints `email<TAB>url` for each invite. Open a URL → `/accept-invite` → set password → session (invite path) or sign in.

**No-SMTP staging rehearsal** (promote admin → re-issue → copy link → one login): [staging-auth-rehearsal.md](staging-auth-rehearsal.md).

Invite tokens hit `POST /api/auth/accept-invite` (DB invite row). Existing users get `password_hash` set and `must_set_password` cleared.

### 2. Email delivery (optional)

```bash
cd apps/api
# Smoke first (works with EMAIL_DRY_RUN default true; no SMTP dial)
go run ./cmd/mailhooks -hooks ../../artifacts/set-password-hooks.json -dry-run
# Live send requires EMAIL_DRY_RUN=false + SMTP — see ops-runtime.md
go run ./cmd/mailhooks -hooks ../../artifacts/set-password-hooks.json
```

Operator smoke steps (fail-closed under dry-run, pass criteria): [ops-runtime.md](ops-runtime.md) § SMTP.
### 3. Local bootstrap one user (dev only)

```bash
cd apps/api
export DATABASE_URL="postgres://descrybe:descrybe@localhost:5433/descrybe?sslmode=disable"
go run ./cmd/migrator -set-password "user@example.com:YourPass123" \
  -postgres "$DATABASE_URL"
```

Then sign in at `/login`. Do not use this for production cutover.

### 4. Admin HMAC emails (platform admin UI)

`POST /api/admin/emails/set-password` emails links with `?mode=set-password`. Those tokens are HMAC (`TOKEN_SIGNING_SECRET`), not DB invites. The accept-invite page calls `POST /api/auth/complete-set-password`, then redirects to `/login`.

### 5. Authenticated set-password

Logged-in users can call `POST /api/auth/set-password` (session required) — useful after a partial login path, not for first-time migrated users.

## Smoke checklist

1. Migrated user cannot log in before setting a password.
2. Invite URL from `password_invites.json` opens `/accept-invite`.
3. Submit password (≥8 chars) → logged in (invite) or redirected to login (HMAC mode).
4. Subsequent `/login` works with the new password.
5. `must_set_password` is false in Postgres.

See also [cutover.md](cutover.md) §4, [go-live-checklist.md](go-live-checklist.md), and [ops-runtime.md](ops-runtime.md).