descrybe/README.md

# Descrybe v2

Go API + SvelteKit frontend + PostgreSQL rewrite of Descrybe (no Clerk).

## Stack

- **API:** Go, chi, pgx, sqlc, goose, River jobs
- **Web:** SvelteKit 2, Svelte 5, Tailwind
- **DB:** PostgreSQL 16
- **Migrator:** MySQL → Postgres ETL with Clerk ID remapping

## Local setup (any OS)

Docker Compose runs **Postgres only** (host **:5433**). API, web, and worker run on the host via `npm run setup` + `npm run dev` (fast reload; no Laragon/XAMPP required). OS-specific Docker notes: [docs/docker.md](docs/docker.md). Ports match `docker-compose.yml` + `scripts/dev-ports.mjs`.

**After the stack is up:** operator path (login → feed → process → export) → [docs/getting-started.md](docs/getting-started.md).

### Two-step path (recommended)

```bash
npm run setup        # .env (if missing) + docker Postgres :5433 + goose migrate
npm install && npm run dev   # API :28471 + web :28472 + worker (needed for /readyz + jobs)
```

Same on Windows PowerShell, macOS, and Linux (Node scripts; Docker Desktop or Engine + Compose v2). Equivalents: `make setup` then `make` targets / `npm run …`.

Optional: `npm run seed` (demo login) · `npm run health` (`/healthz` + `/readyz`).

### Prerequisites

| Tool | Version / notes |
|------|-----------------|
| **Docker** | Docker Engine + Compose v2 (`docker compose`) — Postgres only on **:5433** |
| **Node.js** | **≥ 20** (`package.json` `engines.node`) |
| **Go** | **1.25.0** (`apps/api/go.mod`) |
| **sqlc** (optional) | On `PATH` for migrate regen; scripts fall back to `go run …/sqlc` if missing |
| **Make** (optional) | `make setup` / `make up` / `make migrate` — macOS/Linux/WSL; Windows can use `npm run …` |

### Local ports (canonical)

| Service | Host |
|---------|------|
| Postgres | `localhost:5433` → container `5432` (`postgres:16-alpine`) |
| API | `http://localhost:28471` (`HTTP_ADDR=:28471`) |
| Web | `http://localhost:28472` (Vite `strictPort`; proxies `/api`, `/healthz`, `/readyz`) |

Do **not** use older docs that mention `:8080` / `:5174` for day-to-day `npm run dev`.

### Environment (one file)

Use a **single root `.env`** (copy from [`.env.example`](.env.example)). Do **not** create `apps/api/.env`. `npm run setup` creates `.env` from the example and fills empty `TOKEN_SIGNING_SECRET` / `APP_ENCRYPTION_KEY` with local random hex (never commit `.env`).

| Variable | Local default / note |
|----------|----------------------|
| `DATABASE_URL` | `postgres://descrybe:descrybe@localhost:5433/descrybe?sslmode=disable` (matches `docker-compose.yml`) |
| `HTTP_ADDR` | `:28471` |
| `WEB_ORIGIN` | `http://localhost:28472` |
| `PUBLIC_API_URL` | `http://localhost:28471` |
| `APP_ENV` | `development` |
| `SESSION_SECURE` | `false` locally; `true` in production |
| `TOKEN_SIGNING_SECRET` | `openssl rand -hex 32` (or let `npm run setup` generate) |
| `APP_ENCRYPTION_KEY` | Prefer this for at-rest secrets (`CREDENTIALS_ENCRYPTION_KEY` is legacy). Same generate rule |

Leave OpenAI, Stripe, EPREL, marketing mail, Woo, and Shopify **out of** `.env` for day-to-day setup. Configure them in the app after login:

| Integration | Where |
|-------------|--------|
| AI (OpenAI-compatible / BYOK) | `/integrations/ai` (tenant) and/or `/admin/settings` (platform) |
| Stripe, EPREL, feed private-URL allowlist | `/admin/settings` (`GET/PUT /api/admin/settings` → `values.*`; env optional fallback) |
| Marketing email (Resend / SMTP) | `/integrations/email` |
| Stores (Woo / Shopify / feeds) | `/stores` |

Platform set-password / invite SMTP may still use process `SMTP_*` when you enable live invite mail (see [docs/ops-runtime.md](docs/ops-runtime.md)). Session/encryption fail-closed rules are in `.env.example` and [docs/production-checklist.md](docs/production-checklist.md). Production does **not** require Stripe secrets at boot.

The API/worker load the **monorepo-root** `.env` automatically (`apps/api/internal/config` `loadDotEnv`).

### Manual steps (if you skip `npm run setup`)

`docker-compose.yml` defines **only** Postgres (`descrybe-v2-postgres`, user/db/password `descrybe`, volume `descrybe_v2_pg`, `pg_isready` healthcheck). OS-specific Docker Desktop / Engine notes: [docs/docker.md](docs/docker.md).

```bash
cp .env.example .env          # PowerShell: Copy-Item .env.example .env
docker compose up -d          # or: npm run db:up / make up
npm run migrate               # goose up + sqlc (loads root .env)
npm install && npm run dev
```

Migrate wrappers: `bash scripts/migrate.sh` · `.\scripts\migrate.ps1` · `make migrate` (all call `scripts/migrate.mjs`).

### Worker and `/readyz`

`npm run dev` starts **API + web + worker**. Processing jobs and `GET /readyz` need that worker heartbeat (stale after **60s**).

| Script | What runs |
|--------|-----------|
| `npm run dev` / `npm run dev:app` | api + web + worker |
| `npm run dev:backend` / `make backend` | api + worker (no web; enough for `/readyz`) |
| `npm run dev:api` | api only → `/readyz` 503 until a worker is started |
| `npm run dev:worker` | worker only |
| `make worker` / `.\scripts\run-api.ps1 worker` | worker only |

### Verify

```bash
npm run health
# or:
curl -sS http://127.0.0.1:28471/healthz
curl -sS http://127.0.0.1:28471/readyz
```

Open the app: [http://localhost:28472](http://localhost:28472). Demo seed: `npm run seed`.

### Split processes (optional)

```bash
npm run dev:web      # web only
npm run dev:api      # API only (readyz 503 without worker)
npm run dev:worker   # worker only
npm run dev:backend  # api + worker
npm run dev:ps1      # PowerShell: api + web + worker
```

### OS notes

- **Windows / macOS / Linux Docker details** (Desktop vs Engine, WSL2, ports): [docs/docker.md](docs/docker.md).
- **Windows:** Docker Desktop running; use `npm run setup` / `npm run migrate` / `npm run seed` (Node) — no Git Bash required. Optional: `.\scripts\migrate.ps1`, `.\scripts\dev.ps1`.
- **macOS / Linux:** same npm commands; `make setup` / `make migrate` work if Make is installed.
- **WSL2:** Docker Desktop WSL integration or Linux engine; keep `DATABASE_URL` on `localhost:5433` from the same environment as the API.

### Troubleshooting: `/readyz` returns 503

`GET /healthz` = process liveness. `GET /readyz` = Postgres ping **and** a fresh worker heartbeat (`worker_id=processing`, stale after **60s**). On worker failure the JSON keeps a short `error` and adds operator `reason` (how to start the host worker). See also [docs/ops-runtime.md](docs/ops-runtime.md) and [docs/production-checklist.md](docs/production-checklist.md).

| Symptom | Meaning | Fix |
|---------|---------|-----|
| `/healthz` 200, `/readyz` 503, `checks.worker` = `missing`, error `worker heartbeat missing` | API up, **worker never started** (or migrations before `039_worker_heartbeats`) | Use `npm run dev` (includes worker), or `npm run dev:worker` / `make worker`; ensure goose is up through **039**. Read JSON `reason`. |
| `/readyz` 503, `checks.worker` = `stale`, error `worker heartbeat stale` | Worker was up but heartbeat older than 60s | Restart the worker; confirm it stays running. Read JSON `reason`. |
| `/readyz` 503, `checks.database` = `fail` / `unavailable` | DB down or bad `DATABASE_URL` | `docker compose ps`, fix URL (host **5433**), re-run migrate |
| `/readyz` still 503 after starting worker | Heartbeat table missing or wrong DB | Re-run migrate; confirm worker and API share the same `DATABASE_URL` |

**Expected:** API-only local (`cmd/api` without `cmd/worker`, or `npm run dev:api`) → `/readyz` **503**. That is not a broken API binary — start the worker (`npm run dev`, `npm run dev:backend`, or `npm run dev:worker`) before treating readiness as green. Jobs on `/processing` also need the worker. Compose does **not** start the worker (Postgres only).

### Routes

| Path | Purpose |
|------|---------|
| `/` | Public marketing homepage (sell copy, FAQ, pricing teaser; no app shell) |
| `/pricing` | Public marketing pricing (Free→Enterprise; no client deals) |
| `/features` | Optional features deep-dive (not primary nav) |
| `/privacy`, `/terms` | Legal pages |
| `/login`, `/register` | Auth (login success → `/dashboard`) |
| `/dashboard` | App home (sidebar shell) |
| `/plans` | In-app plans (authenticated) |
| `/standard-fields`, `/feeds`, `/feeds/{id}/mapping` | Catalog ingest + mapping |
| `/products`, `/processing` | Catalog + background process jobs |
| `/export-feeds` | Outbound CSV/XML templates + generate |
| `/stores`, `/stores/shopify` | Store connectors hub (Woo, Shopify sibling, feed URL, CSV) |
| `/campaigns`, `/seo`, `/brand`, `/marketing/calendar` | Marketing suite |
| `/reviews` | Alias to WooCommerce Reviews tab |
| `/admin/*` | Platform admin |

### Demo login (migrated staging data)

Alias `demo@descrybe.test` is also seeded. Platform admin + **admin of Platform Demo only** (not A1). Default session company: **Platform Demo**. Act for A1 via Admin → Users → Switch to user (`a1-primary@descrybe.local`). Details: [docs/demo-user.md](docs/demo-user.md), [docs/safe-test-fixtures.md](docs/safe-test-fixtures.md).

```bash
npm run seed
# or: node scripts/seed-local.mjs
# or: pwsh -File .\scripts\seed-local.ps1
```

## Migrations

Prefer the cross-platform script (loads root `.env`):

```bash
npm run migrate
# equivalents: node scripts/migrate.mjs  |  make migrate  |  .\scripts\migrate.ps1
```

Manual:

```bash
cd apps/api
go run github.com/pressly/goose/v3/cmd/goose@v3.24.3 -dir sql/schema postgres "$DATABASE_URL" up
sqlc generate
```

## Migrator

Live MySQL → staging Postgres load **succeeded** (2026-08-03); production cutover is still **NO-GO** until emails/roles/SMTP login are proven. Evidence: [docs/migration-run-log.md](docs/migration-run-log.md).

```bash
cd apps/api
go run ./cmd/migrator -mysql "$MIGRATE_MYSQL_DSN" -postgres "$DATABASE_URL" -dry-run
```

## Docs
- **[Local setup (README)](README.md#local-setup-any-os)** — any-OS Docker/Node/Go/migrate; `/readyz` 503 troubleshooting
- **[docs/getting-started.md](docs/getting-started.md)** — operator checklist: login → feed → process → export
- **[docs/store-connectors.md](docs/store-connectors.md)** — Woo / Shopify / feed URL / CSV / export REST
- **[docs/marketing-suite-user-guide.md](docs/marketing-suite-user-guide.md)** — seasons, Black Friday in 5 clicks, tutorial start
- **[docs/free-tier.md](docs/free-tier.md)** — Free plan 0 AI credits + marketing gates
- **[docs/green-chat-ai.md](docs/green-chat-ai.md)** — Green Chat / OPENAI_BASE_URL for processing + campaign AI
- **[docs/mock-llm.md](docs/mock-llm.md)** — Tiny OpenAI-compatible stub (`cmd/mock-llm`) for CI/dev processing without production keys
- [Email sending](docs/email-sending.md) — Resend/SMTP, encryption, dry-run

- **[docs/process-and-sell-summary.md](docs/process-and-sell-summary.md)** — process & sell E2E test path (standard fields, sync/process, EPREL, export, public API)
- **[docs/demo-user.md](docs/demo-user.md)** — demo login + company data counts for staging testing
- **[docs/woocommerce-demo.md](docs/woocommerce-demo.md)** — Woo store setup, WOO_* env, seed-woo-demo, audience/campaigns
- **[docs/go-live-checklist.md](docs/go-live-checklist.md)** — go/no-go cutover checklist (design, stubs, migrate, set-password)
- [docs/migration-run-log.md](docs/migration-run-log.md) — live dry-run + staging load report
- [docs/status-and-gaps.md](docs/status-and-gaps.md) — what works, what's stubbed, what's missing, priority order
- [docs/design-gaps.md](docs/design-gaps.md) — UI/UX parity vs legacy
- [docs/migration-readiness.md](docs/migration-readiness.md) — ETL status + post-import set-password
- [docs/features.md](docs/features.md) — phase checklist
- [docs/schema-map.md](docs/schema-map.md) — MySQL → Postgres ID maps
- [docs/cutover.md](docs/cutover.md) — production cutover runbook
- [docs/ops-runtime.md](docs/ops-runtime.md) — SMTP, sessions, Woo encryption