greeneclipse
8580c996c3
Drop one-shot tmp/axe scripts and agent i18n scratch so the Gitea tree is deployable.
134 lines
6.9 KiB
Markdown
134 lines
6.9 KiB
Markdown
# Migration readiness — MySQL → Postgres + post-import passwords
|
||
|
||
**Last updated:** 2026-08-04
|
||
**Live ETL status:** **DONE on staging** (dry-run + load, 2026-08-03) — see [migration-run-log.md](migration-run-log.md).
|
||
**Production cutover:** **NOT ready** — remaining blockers below.
|
||
|
||
Migrated users are imported with `password_hash = NULL` and `must_set_password = true`. Login rejects empty/null hashes, so a random guess cannot succeed.
|
||
|
||
Fresh installs that never ran the migrator can keep using `/register`.
|
||
|
||
---
|
||
|
||
## Live dry-run + load (staging)
|
||
|
||
| Gate | Status |
|
||
|------|--------|
|
||
| Goose `001`–`007` on staging PG (`localhost:5433`) at ETL time (2026-08-03) | Done — **repo schema is now `001`–`023`**; re-apply goose `up` before any new staging/prod load |
|
||
| Offline fixture dry-run | Done |
|
||
| Live MySQL dry-run | Done (exit 0) |
|
||
| Live load to staging Postgres | Done (~11 min) |
|
||
| Orphan FK validation | **16/16 pass** |
|
||
| Evidence | [migration-run-log.md](migration-run-log.md); `artifacts/` gitignored |
|
||
|
||
### Counts written (staging)
|
||
|
||
| Entity | Written |
|
||
|--------|--------:|
|
||
| companies | 27 |
|
||
| users | 21 |
|
||
| memberships | 15 |
|
||
| categories | 8278 |
|
||
| attributes | 57630 (unique `(company_id, attribute_key)` collapse) |
|
||
| raw_products | 118784 (+7732 GTIN-deduped from MySQL) |
|
||
| processed_products | 7219 |
|
||
| input feeds | 31 |
|
||
| export feeds | 8 |
|
||
|
||
### Remaining blockers (before production cutover)
|
||
|
||
| Severity | Item |
|
||
|----------|------|
|
||
| **Data** | Most users have synthetic `…@legacy.local` emails — export from Clerk and patch `users.email` before invites (**tooling:** migrator `-list-legacy-emails` / `-export-legacy-emails` / `-patch-emails`; see [portable-mysql-pg-migration.md](portable-mysql-pg-migration.md#clerk--legacylocal-email-repair-cutover-data-hygiene)) |
|
||
| **Data** | All memberships imported as `role=member` — promote company admins |
|
||
| **Data** | 2 `company_plans` skipped (`plan_id=6` missing) — resolve with migrator list → `-dry-run` → `-confirm` (see [cutover.md](cutover.md#resolve-skipped-company_plans-plan_id6); no blind live assigns) |
|
||
| **Ops** | This load used `-skip-post-import` — re-issue set-password invites when emails are real |
|
||
| **Ops** | SMTP + mailhooks + login smoke still unproven |
|
||
| **Gaps** | See [Cutover coverage gaps](#cutover-coverage-gaps) below |
|
||
|
||
Staging is **GO for login testing** after set-password (preferably after email patch). Production DNS/cutover remains **NO-GO**.
|
||
|
||
### Cutover coverage gaps
|
||
|
||
Honest status vs migrator code (`cmd/migrator`, domains `settings` / `woo` / `files`):
|
||
|
||
| Area | Status | Operator action |
|
||
|------|--------|-----------------|
|
||
| **API keys** | **Not migrated** (intentional — secrets) | Clients must mint new keys; Settings → API Keys empty state says so |
|
||
| **company_settings** | **Partial** — `language` + `merge_products` only (`migrateCompanySettings`) | Re-enter other legacy settings in-app if needed |
|
||
| **File blobs** | **Metadata only** — bytes not copied (`migrateFiles`) | Resync object storage / re-upload; Files empty + dashboard ETL gaps panel say so; platform admins see cheap COUNTs on **Admin → Diagnostics → ETL gap inventory** (`files_metadata_only` / `files_total`) |
|
||
| **Woo configs** | **Migrated when domain `woo` enabled** — from `wc_*` custom_fields (`migrateWooConfigs`) | Re-verify store URL + credentials; ensure `APP_ENCRYPTION_KEY` before prod secrets |
|
||
| **Jobs / history** | **Cutover default: not backfilled** — optional domain `jobs` can import `processing_jobs` (+ best-effort products) + `tasks` tagged `ai_provider_mode=migrated`; accepted production gap is empty history unless ops explicitly ran `jobs` | Expect empty Processing history after cutover; tenant UI must not claim history moved; diagnostics `migration_inventory.jobs_domain_ran` / `processing_jobs_migrated` show whether optional backfill ran |
|
||
|
||
Migrator run reports surface these as notes + count-report rows (`api_keys`, `files` metadata-only, `processing_jobs` when domain enabled, partial `company_settings`).
|
||
|
||
**Admin inventory (read-only):** `GET /api/admin/diagnostics` → `migration_inventory` — COUNT of metadata-only files, migrated-tagged jobs, and tasks. Not an import path; never fabricates blob bytes or job history.
|
||
|
||
---
|
||
|
||
## After a live import
|
||
|
||
### 1. Issue set-password invites (preferred)
|
||
|
||
Post-import already runs this unless `-skip-post-import`. To re-issue later (Postgres only, no MySQL):
|
||
|
||
```bash
|
||
cd apps/api
|
||
export DATABASE_URL="postgres://descrybe:descrybe@localhost:5433/descrybe?sslmode=disable"
|
||
# WEB_ORIGIN controls printed /accept-invite links (default http://localhost:5174)
|
||
go run ./cmd/migrator -issue-set-password-invites \
|
||
-postgres "$DATABASE_URL" \
|
||
-maps-dir ../../artifacts
|
||
```
|
||
|
||
Artifacts (gitignored, do not commit):
|
||
|
||
- `artifacts/password_invites.json` — email, token, invite URL, expiry
|
||
- `artifacts/set-password-hooks.json` — same payload for `cmd/mailhooks`
|
||
|
||
Stdout prints `email<TAB>url` for each invite. Open a URL → `/accept-invite` → set password → session (invite path) or sign in.
|
||
|
||
**No-SMTP staging rehearsal** (promote admin → re-issue → copy link → one login): [staging-auth-rehearsal.md](staging-auth-rehearsal.md).
|
||
|
||
Invite tokens hit `POST /api/auth/accept-invite` (DB invite row). Existing users get `password_hash` set and `must_set_password` cleared.
|
||
|
||
### 2. Email delivery (optional)
|
||
|
||
```bash
|
||
cd apps/api
|
||
# Smoke first (works with EMAIL_DRY_RUN default true; no SMTP dial)
|
||
go run ./cmd/mailhooks -hooks ../../artifacts/set-password-hooks.json -dry-run
|
||
# Live send requires EMAIL_DRY_RUN=false + SMTP — see ops-runtime.md
|
||
go run ./cmd/mailhooks -hooks ../../artifacts/set-password-hooks.json
|
||
```
|
||
|
||
Operator smoke steps (fail-closed under dry-run, pass criteria): [ops-runtime.md](ops-runtime.md) § SMTP.
|
||
### 3. Local bootstrap one user (dev only)
|
||
|
||
```bash
|
||
cd apps/api
|
||
export DATABASE_URL="postgres://descrybe:descrybe@localhost:5433/descrybe?sslmode=disable"
|
||
go run ./cmd/migrator -set-password "user@example.com:YourPass123" \
|
||
-postgres "$DATABASE_URL"
|
||
```
|
||
|
||
Then sign in at `/login`. Do not use this for production cutover.
|
||
|
||
### 4. Admin HMAC emails (platform admin UI)
|
||
|
||
`POST /api/admin/emails/set-password` emails links with `?mode=set-password`. Those tokens are HMAC (`TOKEN_SIGNING_SECRET`), not DB invites. The accept-invite page calls `POST /api/auth/complete-set-password`, then redirects to `/login`.
|
||
|
||
### 5. Authenticated set-password
|
||
|
||
Logged-in users can call `POST /api/auth/set-password` (session required) — useful after a partial login path, not for first-time migrated users.
|
||
|
||
## Smoke checklist
|
||
|
||
1. Migrated user cannot log in before setting a password.
|
||
2. Invite URL from `password_invites.json` opens `/accept-invite`.
|
||
3. Submit password (≥8 chars) → logged in (invite) or redirected to login (HMAC mode).
|
||
4. Subsequent `/login` works with the new password.
|
||
5. `must_set_password` is false in Postgres.
|
||
|
||
See also [cutover.md](cutover.md) §4, [go-live-checklist.md](go-live-checklist.md), and [ops-runtime.md](ops-runtime.md).
|