descrybe/docs/migration-run-log.md

Migration run log — legacy MySQL → v2 Postgres

Date: 2026-08-03 (local)
Status: SUCCESS (dry-run + live load, exit 0)
Elapsed (live): ~11 minutes

Secrets are masked below. Do not commit artifacts/ maps or hook tokens.

Environment

Item	Value
Legacy MySQL source	f:/laragon/www/_MY/descrybe/.env.local → DATABASE_URL
MySQL (masked)	root:****@tcp(localhost:3306)/descrybe_v2
MySQL also present	descrybe_v1, descrybe_21_05_2025 (not used this run)
Postgres	Docker Compose service postgres (descrybe-v2-postgres)
Postgres URL (masked)	postgres://descrybe:****@localhost:5433/descrybe?sslmode=disable
Goose	migrations 001–007 applied (see below)
Migrator	apps/api/cmd/migrator — go build ./cmd/migrator PASS

Goose notes

• docker compose up -d — container healthy on host port 5433.
• scripts/migrate.ps1 failed on this machine: goose@latest needs Go ≥ 1.25.7 (runner had 1.25.5); sqlc not on PATH.
• Worked: go run github.com/pressly/goose/v3/cmd/goose@v3.24.3 -dir sql/schema postgres "$DATABASE_URL" up
• Fixed UTF-8 BOM on apps/api/sql/schema/006_feed_sync.sql (goose parse error \ufeff-- +goose Up).

Migrator fixes applied this session

1. profiles.role missing — memberships default to member (Clerk held org roles).
2. profiles.email / no users table — synthetic …@legacy.local emails; enrich from admin_users when present.
3. companies.language missing — load from company_settings / default en.
4. MySQL DSN — accept mysql://… URLs; normalize to user:pass@tcp(host:port)/db?parseTime=true; clearer ping/open errors with Laragon hints (passwords masked in logs).
5. raw_products.feed_ids — optional on MySQL (column absent); PG still gets NULL / jsonb.
6. GTIN unique index — dedupe (company_id, gtin) before insert; report raw_products_gtin_deduped.
7. Dry-run planMap — populate with legacy plan IDs so company_plans linkage is counted.

Dry-run

go run ./cmd/migrator -mysql "root:****@tcp(localhost:3306)/descrybe_v2?parseTime=true" \
  -postgres "postgres://descrybe:****@localhost:5433/descrybe?sslmode=disable" \
  -dry-run -maps-dir ../../artifacts -id-map ../../artifacts/id-map.json

Entity	Count
companies	27
users	21
memberships	15
admin_users (flagged)	1
plans	5
company_plans	3 (+2 skipped: orphan plan_id=6)
credit_balances	5 (+1 skipped: orphan company …-0)
categories	8278
attributes	95292
category_attributes	74377
custom_variables	36
input_feeds (xml)	31
feed_mappings	24
raw_products	126516
processed_products	7219
export_feeds	8
files	1
Validation	ok=true (orphan checks skipped in dry-run)

Live load

Same command without -dry-run (used -skip-post-import this run). Artifacts under artifacts/ (gitignored).

Migration report (attempts / written)

Entity	Report	Notes
companies	27	PG total 29 (includes 2 pre-existing seed rows)
users	21	PG total 23 (seed + migrate); no password hashes imported
memberships	15	6 legacy profiles have company_id NULL → skipped
admin_users	1	→ users.is_platform_admin
plans	5
company_plans	3	2 skipped (plan_id=6 missing in plans)
credit_balances	5	1 skipped (orphan company id)
categories	8278	PG 8279 (+1 seed)
attributes	95292	PG 57630 = MySQL COUNT(DISTINCT company_id, attribute_key) (unique constraint collapse)
category_attributes	74377
custom_variables	36	PG 37 (+1 seed)
input_feeds	31	PG 33 (+2 seed)
feed_mappings	24
raw_products	118784	+ 7732 GTIN-deduped (= 126516 MySQL rows)
processed_products	7219
export_feeds	8	PG 9 (+1 seed)
files	1	metadata only; blobs not copied

Validation (live)

• mode: live ok=true
• Orphan FK checks: 16 passed / 0 failed (companies, memberships, feeds, products, mappings, platform admin, etc.)

Blockers / follow-ups

Severity	Item	Action needed
Data	Most users have synthetic emails (user_…@legacy.local) — Clerk emails are not in MySQL	Export emails from Clerk (or another source) and patch users.email before set-password invites
Data	All memberships imported as role=member (profiles.role absent)	Promote company admins post-cutover
Data	company_plans rows with plan_id=6 skipped	Create missing plan or remap those companies
Data	1 orphan credit balance (company_id suffix -0)	Ignore or repair source row
Ops	Attribute / GTIN collapses are intentional under v2 unique constraints	Accept PG counts; ID map still covers remapped FKs
Ops	Goose pin / BOM / sqlc PATH	Prefer pinned goose ≤ Go toolchain; keep schema files BOM-free; install sqlc if regenerating queries
Ops	Set-password hooks	Re-run without -skip-post-import (or -issue-set-password-invites) when ready to email invites

CLI verification

cd apps/api
go build ./cmd/migrator          # PASS
go test ./cmd/migrator -count=1  # PASS

Offline smoke (no MySQL):

go run ./cmd/migrator -dry-run -fixture ./cmd/migrator/testdata/fixture.json -maps-dir ../../artifacts

How to re-run

1. docker compose up -d in descrybe-v2
2. Apply goose (goose@v3.24.3 or newer matching local Go)
3. Set DSN from legacy .env.local (convert mysql:// → go-sql-driver form, or pass URL — migrator converts)
4. Dry-run, then live; reuse artifacts/id-map.json for idempotent remaps