descrybe/docs/migration-run-log.md

# Migration run log — legacy MySQL → v2 Postgres

**Date:** 2026-08-03 (local)  
**Status:** **SUCCESS** (dry-run + live load, exit 0)  
**Elapsed (live):** ~11 minutes

Secrets are masked below. Do not commit `artifacts/` maps or hook tokens.

## Environment

| Item | Value |
|---|---|
| Legacy MySQL source | `f:/laragon/www/_MY/descrybe/.env.local` → `DATABASE_URL` |
| MySQL (masked) | `root:****@tcp(localhost:3306)/descrybe_v2` |
| MySQL also present | `descrybe_v1`, `descrybe_21_05_2025` (not used this run) |
| Postgres | Docker Compose service `postgres` (`descrybe-v2-postgres`) |
| Postgres URL (masked) | `postgres://descrybe:****@localhost:5433/descrybe?sslmode=disable` |
| Goose | migrations **001–007** applied (see below) |
| Migrator | `apps/api/cmd/migrator` — `go build ./cmd/migrator` **PASS** |

### Goose notes

- `docker compose up -d` — container healthy on host port **5433**.
- `scripts/migrate.ps1` failed on this machine: `goose@latest` needs Go ≥ 1.25.7 (runner had 1.25.5); `sqlc` not on PATH.
- Worked: `go run github.com/pressly/goose/v3/cmd/goose@v3.24.3 -dir sql/schema postgres "$DATABASE_URL" up`
- Fixed **UTF-8 BOM** on `apps/api/sql/schema/006_feed_sync.sql` (goose parse error `\ufeff-- +goose Up`).

## Migrator fixes applied this session

1. **`profiles.role` missing** — memberships default to `member` (Clerk held org roles).
2. **`profiles.email` / no `users` table** — synthetic `…@legacy.local` emails; enrich from `admin_users` when present.
3. **`companies.language` missing** — load from `company_settings` / default `en`.
4. **MySQL DSN** — accept `mysql://…` URLs; normalize to `user:pass@tcp(host:port)/db?parseTime=true`; clearer ping/open errors with Laragon hints (passwords masked in logs).
5. **`raw_products.feed_ids`** — optional on MySQL (column absent); PG still gets `NULL` / jsonb.
6. **GTIN unique index** — dedupe `(company_id, gtin)` before insert; report `raw_products_gtin_deduped`.
7. **Dry-run `planMap`** — populate with legacy plan IDs so `company_plans` linkage is counted.

## Dry-run

```text
go run ./cmd/migrator -mysql "root:****@tcp(localhost:3306)/descrybe_v2?parseTime=true" \
  -postgres "postgres://descrybe:****@localhost:5433/descrybe?sslmode=disable" \
  -dry-run -maps-dir ../../artifacts -id-map ../../artifacts/id-map.json
```

| Entity | Count |
|---|---:|
| companies | 27 |
| users | 21 |
| memberships | 15 |
| admin_users (flagged) | 1 |
| plans | 5 |
| company_plans | 3 (+2 skipped: orphan `plan_id=6`) |
| credit_balances | 5 (+1 skipped: orphan company `…-0`) |
| categories | 8278 |
| attributes | 95292 |
| category_attributes | 74377 |
| custom_variables | 36 |
| input_feeds (xml) | 31 |
| feed_mappings | 24 |
| raw_products | 126516 |
| processed_products | 7219 |
| export_feeds | 8 |
| files | 1 |
| Validation | `ok=true` (orphan checks skipped in dry-run) |

## Live load

Same command without `-dry-run` (used `-skip-post-import` this run). Artifacts under `artifacts/` (gitignored).

### Migration report (attempts / written)

| Entity | Report | Notes |
|---|---:|---|
| companies | 27 | PG total 29 (includes 2 pre-existing seed rows) |
| users | 21 | PG total 23 (seed + migrate); **no password hashes imported** |
| memberships | 15 | 6 legacy profiles have `company_id` NULL → skipped |
| admin_users | 1 | → `users.is_platform_admin` |
| plans | 5 | |
| company_plans | 3 | 2 skipped (`plan_id=6` missing in `plans`) |
| credit_balances | 5 | 1 skipped (orphan company id) |
| categories | 8278 | PG 8279 (+1 seed) |
| attributes | 95292 | PG **57630** = MySQL `COUNT(DISTINCT company_id, attribute_key)` (unique constraint collapse) |
| category_attributes | 74377 | |
| custom_variables | 36 | PG 37 (+1 seed) |
| input_feeds | 31 | PG 33 (+2 seed) |
| feed_mappings | 24 | |
| raw_products | 118784 | + **7732** GTIN-deduped (= 126516 MySQL rows) |
| processed_products | 7219 | |
| export_feeds | 8 | PG 9 (+1 seed) |
| files | 1 | metadata only; blobs not copied |

### Validation (live)

- `mode: live ok=true`
- Orphan FK checks: **16 passed / 0 failed** (companies, memberships, feeds, products, mappings, platform admin, etc.)

## Blockers / follow-ups

| Severity | Item | Action needed |
|---|---|---|
| **Data** | Most users have synthetic emails (`user_…@legacy.local`) — Clerk emails are not in MySQL | Export emails from Clerk (or another source) and patch `users.email` before set-password invites |
| **Data** | All memberships imported as `role=member` (`profiles.role` absent) | Promote company admins post-cutover |
| **Data** | `company_plans` rows with `plan_id=6` skipped | Create missing plan or remap those companies |
| **Data** | 1 orphan credit balance (`company_id` suffix `-0`) | Ignore or repair source row |
| **Ops** | Attribute / GTIN collapses are intentional under v2 unique constraints | Accept PG counts; ID map still covers remapped FKs |
| **Ops** | Goose pin / BOM / sqlc PATH | Prefer pinned goose ≤ Go toolchain; keep schema files BOM-free; install sqlc if regenerating queries |
| **Ops** | Set-password hooks | Re-run without `-skip-post-import` (or `-issue-set-password-invites`) when ready to email invites |

## CLI verification

```text
cd apps/api
go build ./cmd/migrator          # PASS
go test ./cmd/migrator -count=1  # PASS
```

Offline smoke (no MySQL):

```text
go run ./cmd/migrator -dry-run -fixture ./cmd/migrator/testdata/fixture.json -maps-dir ../../artifacts
```

## How to re-run

1. `docker compose up -d` in `descrybe-v2`
2. Apply goose (`goose@v3.24.3` or newer matching local Go)
3. Set DSN from legacy `.env.local` (convert `mysql://` → go-sql-driver form, or pass URL — migrator converts)
4. Dry-run, then live; reuse `artifacts/id-map.json` for idempotent remaps