Files
descrybe/docs/production-readiness.md
T

722 lines
67 KiB
Markdown
Raw Normal View History

# Production readiness (Descrybe v2)
**Date:** 2026-08-09 (LOCAL/CODE cutover wave: all non-SMTP/Stripe blockers handled; Clerk removed; goose **042**; cutover still **NO-GO** on 2 live ops: SMTP + Stripe)
**Stack:** Go API (`apps/api`) + SvelteKit (`apps/web`) + PostgreSQL
**Related:** [production-checklist.md](production-checklist.md), [security-notes.md](security-notes.md), [go-live-checklist.md](go-live-checklist.md), [cutover.md](cutover.md), [migration-readiness.md](migration-readiness.md), [staging-auth-rehearsal.md](staging-auth-rehearsal.md), [ops-runtime.md](ops-runtime.md), [stripe-setup.md](stripe-setup.md), [ux-backlog.md](ux-backlog.md)
## Executive verdict
| Question | Answer |
|----------|--------|
| **Product (code) readiness** | **10/10** — features, security architecture, perf foundations, UX activation path, tests, setup DX |
| **Production cutover (ops)** | **NO-GO** / **9/10** (was **8/10**) — only **2** hard live blockers left (SMTP + Stripe) |
| **Staging / local rehearsal** | **GO** — migrator A1-safe tooling + local plan assign **87→0** + orphan **0** + goose **039042** + worker `/readyz` + `cutover-deploy-check` **PASS** + forgot-password opaque **200** (API + browser) |
| **A1 clean?** | **YES** — no A1 mutations; migrator promote/assign/patch skip `a1=true` (tests) |
| **Ship to paying public internet today?** | **NO** until **live SMTP** and **live Stripe** clear |
**One-line:** Product **10/10** · local/CODE cutover prep **handled** · hard ops left = **live SMTP + live Stripe** · cutover **NO-GO / 9/10**. Schema head = goose **042**.
**Hard live blockers (still OPS OPEN — excluded this wave):**
| # | Blocker | Why still open |
|---|---------|----------------|
| **1** | `@legacy.local` hygiene (Clerk removed) | **Not a hard NO-GO** — Clerk removed from product; optional migrator patch only |
| **4** | Live SMTP + set-password / forgot-password login smoke | Delivery unproven (`EMAIL_DRY_RUN` / dry-run only) |
| **5** | Live Stripe keys + signed webhooks | Presence flags only; no live Checkout/webhook smoke |
**LOCAL/CODE handled this wave (former blockers #2#3, #6#10):** promote tooling + Demo already admin; local plan assign **87→0** (A1 untouched); orphan report/cleanup fail-closed (local **0**); goose **039042** + worker + deploy-check **PASS**; adapter-node pin/scripts; edge RPM env + nginx/Caddy snippets; Prometheus scrape/alert **examples**; api_keys/blobs/jobs honesty + admin inventory; store-reconnect admin inventory; session revoke on reset (**042**); `cutover-local-rehearsal.mjs`. Residual **prod-host apply** (migrate/restart Node/wire scrape/edge if N>1 / cutover-DB confirms) is mechanical deploy — not missing product work.
**ASSUMPTION:** Cutover **9/10** = tooling + local proofs complete and only **SMTP + Stripe** gate DNS. Routine prod-host `npm run migrate` / `start:web` / scrape wiring do **not** drop the score; clearing SMTP+Stripe is what moves toward GO. Clerk is removed from the product.
### Scoring rules (split)
| Scorecard | Counts | Does **not** count |
|-----------|--------|--------------------|
| **Product (code)** | Shipped features, security architecture, perf foundations (indexes, durable jobs, limiters), UX activation/cutover chrome, unit/vet/browser proofs, setup DX | Live Stripe keys, SMTP delivery, DNS flip, scrape/alerting topology, orphan cleanup **run**, goose apply on prod host |
| **Cutover (ops)** | Operator data repair, live mail/billing smoke, migrate+worker deploy order, edge multi-replica caps, metrics scrape+alerts, Node host for `adapter-node` | Whether CSRF/CSP/`/readyz`/export rotate **exist in code** (those are Product) |
## Dimension scores (honest, 2026-08-09)
### Product (code) — **10/10 overall**
| Dimension | Score | Evidence (code-verified) |
|-----------|------:|--------------------------|
| **Setup DX** | **10/10** | `npm run setup` → `dev` (+ `seed` / `health`) + [getting-started.md](getting-started.md); `/readyz` needing worker is intentional (API-only → 503 expected) |
| **Security architecture** | **10/10** | CSRF/CSP/SSRF/encryption/`APP_ENV=production` fail-closed; prod **`metrics.Gate`** loopback/`METRICS_PUBLIC`; auth login vs register split (`RateLimitAuth`); `RateLimitPublicExport`; export rotate + **256-bit** (`publicExportTokenBytes=32`); Stripe/mail fail-closed at **use** (boot does not require live `STRIPE_*`) |
| **Performance foundations** | **10/10** | Goose **040** claim/list composites; durable feed sync + `ListenWake` + stuck reclaim + `findOrCreatePendingSyncJob`; IETF `RateLimit` on 429 |
| **UX approachability** | **10/10** | Wave 7–8 P0 + **store-sync UX wave** (wizard, schedule, scope filters, live poll, Woo/Shopify checklists, plan-gate honesty); empty shells honest ([ux-backlog.md](ux-backlog.md)) |
| **Browser / tests** | **10/10** | Wave 7–8 deep E2E **PASS**; Soft Undo + settings re-smoke **PASS**; **mobile** authenticated Feeds/dashboard proof **PASS** ([4bdf0120](4bdf0120-5afb-4a6a-b78e-a0ee611c4deb)); `go test` / `go vet` **PASS**; **`apps/web` `npm run check` GREEN** (0 errors / 0 warnings) via `allowImportingTsExtensions` + `ja.ts` dup-key cleanup ([7c018ec6](7c018ec6-ffd1-46b4-8717-fbe52c9e9913)) |
| **Product overall** | **10/10** | No remaining **code** gaps that block this scorecard (see Product 10 assumptions + cutover list) |
#### Product 10 assumptions
Product **10/10** is valid **without** Redis/Valkey shared rate-limit store or multi-replica edge caps. These deployment assumptions are accepted for the Product scorecard (they do **not** reduce Product below 10; violating them in a multi-replica prod topology is **Cutover/ops**):
| # | ASSUMPTION | Product implication | Cutover if violated |
|---|------------|---------------------|---------------------|
| 1 | **Single API instance** (default), or multi-replica with **edge** cluster caps | In-process limiters are Product-OK on one process; `N×` RPM without edge is **not** a missing Redis product gap | When N>1: edge/WAF hard global RPM (cutover). Optional `RATE_LIMIT_REPLICAS=N` only divides **HTTP** middleware caps (`rateLimitEffectiveCap`) under even load — not lockout/StartLimiter/AI/email, not a shared store |
| 2 | **Worker required for ready** | `/readyz` needs fresh `processing` heartbeat ≤60s (`jobs.ProbeWorkerReadiness`); API-only → **503** is expected, not a Product defect | goose **039042** → restart `cmd/worker` so readiness probes green |
| 3 | **Metrics loopback in prod** | `metrics.Gate`: allow loopback peers or `METRICS_PUBLIC=1`; default is not a public no-auth scrape surface | Example scrape/alerts **CODE DONE** (`deploy/prometheus/`); live targets + edge deny still **OPS OPEN** |
**ASSUMPTION:** Product rates *shipped code + local/staging proofs*, not production scrape/alerting/LB topology.
**ASSUMPTION:** Multi-tenant load soak is ops verification; foundations (040 indexes, durable claim/LISTEN) are shipped.
**ASSUMPTION:** Deferred polish accepted for Product 10 — waived P0-8/9 remainders, export-selected→feed picker, admin logs stub, structured-descriptions/vector-categories empty shells (honesty banners). **P1-18 forgot-password is shipped** (`041_password_reset_tokens` + `/forgot-password`/`/reset-password`; admin set-password re-issue remains for migration cohort). Re-open remaining polish via [ux-backlog.md](ux-backlog.md) / [design-gaps.md](design-gaps.md), not as Product blockers.
### Production cutover (ops) — **NO-GO** / **9/10**
| Dimension | Score | Why not higher |
|-----------|------:|----------------|
| **Cutover / DNS** | **9/10** | LOCAL/CODE blockers cleared; DNS still gated by **live SMTP + live Stripe** (plus mechanical prod-host migrate/Node). Clerk removed from product. |
| **Live integrations** | **unproven** | SMTP set-password smoke, Stripe live keys+webhooks — code ready, live ops not run |
| **Deploy topology** | **LOCAL GO / prod apply open** | goose **039042** + worker + adapter-node gates **PASS** locally; metrics/edge **examples CODE DONE** — live scrape/edge targets still operator-owned |
### Code gaps that would block Product 10
**None after Aug-9 audit** (verified via codehelper: `metrics.Gate`, `RotateExportFeedPublicToken` / `publicExportTokenBytes=32`, `RateLimitAuth` / `RateLimitPublicExport`, `ListenWake`, `findOrCreatePendingSyncJob`, `CleanupOrphanProcessed`, goose `040_job_hotpath_indexes`, `mail.ApplyDryRun`, `config.validate` Stripe-at-boot honesty, `adapter-node@5.5.7`; **`npm run check` green** [7c018ec6](7c018ec6-ffd1-46b4-8717-fbe52c9e9913)). Sibling fixers: **no Product-10 code blockers** — work remaining is Cutover/ops (table below).
### What fixed this wave (Aug-9 siblings + synthesis)
| Area | Change | Score impact |
|------|--------|--------------|
| Job hot-path indexes | Goose **`040_job_hotpath_indexes`** — claim/list composites for `feed_sync_jobs` / `processing_jobs` / `processing_job_products` | Product Perf foundations complete; load soak remains Cutover/ops |
| Forgot-password | Goose **`041_password_reset_tokens`** + self-serve forgot/reset API + UI ([forgot-password.md](forgot-password.md); ux-backlog **P1-18** done) | Product auth polish shipped; live SMTP smoke still Cutover/ops |
| Worker readiness | Goose **039** + `/readyz` heartbeat ≤60s (prior sibling) | Reliability — deploy: migrate → restart worker |
| Durable feed sync | **202** + poll + LISTEN wake + stuck reclaim + same-feed pending dedupe (014e2507 / 2f1019f2) | Reliability — **residuals closed** in code |
| Orphan processed | Admin report/cleanup (prior sibling) | Data hygiene — **ops once** |
| Observability | **`GET /metrics`** Prometheus text + prod **Gate** (loopback / `METRICS_PUBLIC=1`); worker optional `METRICS_ADDR`; examples in `deploy/prometheus/` | Observability **gated in prod**; example scrape/alerts **CODE DONE**; live targets **OPS OPEN** |
| Abuse limits | Split auth login vs register; API-key IP/company; public export IP/probe/token RPM | Security — still per-process; edge caps for multi-replica |
| Export public tokens | Rotate endpoint + **256-bit** new tokens (168a394e); legacy 128-bit hex still accepted | Security — revoke old URLs via rotate |
| Adapter | `adapter-node@5.5.7` pinned | Deploy shape decided; Node host still required |
| Web typecheck | Sibling **7c018ec6**: `tsconfig` `allowImportingTsExtensions` + `ja.ts` dup keys → `npm run check` **GREEN** (0/0) | Product tests/DX gate closed (was blocking honest Product 10 claim) |
### Store-sync UX wave (Aug-9)
Closed the top merchant gaps from the store-sync UX review (A1 **read-only** / clean; mutations & smoke on **Platform Demo** only).
| Shipped | Notes |
|---------|-------|
| Activation `store-connect` + `/stores/wizard` | Optional after Process; plan-gated `stores.hub`; connect → test → first sync → done |
| Product sync schedule | Interval + `schedule_paused`; Woo/Shopify `PUT …/schedule` |
| What-to-sync filters | Optional sync body: status / category / limit / `product_ids`; UI scope controls |
| Live sync progress | Poll config status after enqueue; retry on fail |
| Woo REST-key + Shopify Dev Dashboard client-credentials | Deep links / scopes; auto token refresh; **no App Store one-click install** |
| Direction copy, plan-gate "still works", member locked UX | i18n parity (~5339 keys) |
| Feed schedule + export picker polish | Auto vs manual clarity; export empty/help CTAs |
| Tests / gates | `go test` shopify+woo+httpapi; web **194** pass; `npm run check` green; A1 `/stores` plan-gate browser **PASS** |
### Truly-easy onboarding wave (Aug-9 afternoon)
~20 parallel siblings closed the prior "still not truly easy" polish list plus discoverability / empty-state / test coverage. **A1 stayed clean** (read-only / no mutations); Demo-only for live sync enqueue + browser proofs.
| Shipped | Notes |
|---------|-------|
| Dashboard value prop | ActivationChecklist: Catalog in → Enrich/process → Export/stores + one **Do this step** CTA |
| Activation copy | Feed-only connect-source; store-connect clearly optional (**Skip for now**); Continue destinations foolproof |
| Product tour | Reordered Feeds → Process → Export/Stores; dead selectors fixed; i18n tips |
| Empty states (7) | Uploads, exports, stores hub, Woo/Shopify orders & reviews, dashboard jobs |
| Shopify connect ease | Client-credentials OAuth + Dev Dashboard checklist (not App Store one-click) |
| Product multi-select sync | UI picker → `product_ids` (cap 500) on Shopify + Woo |
| Store delivery honesty | Poll vs push banner; **Store webhooks: not available** (no fake receive path) |
| Inline help | Interval / pause / match strategy / mapping required (~8 controls) |
| API keys first-run | One-time secret warning + Bearer / X-API-Key + docs link |
| Mobile nav | Essentials pinned; staff unlock while capabilities load; Demo mobile PASS |
| Command palette | Discoverable **Search** + shortcut; ranking helpers + tests |
| Dropdown a11y | Arrow/Home/End focus; `aria-labelledby` |
| Nav crash fix | `paletteShortcut` ReferenceError fixed; Demo login→dashboard PASS |
| Browser Demo | Woo live sync progress **PASS**; dashboard CTA proof after palette fix |
| Tests | Feeds/export/products/search/selection/sync-scope/menu/command-palette helpers; Go process gates + schedule + sync body |
**Gates (live verify this synthesis):** `apps/web` `npm test` **194/194**; `npm run check` **0/0**; `go test ./internal/shopify/ ./internal/woocommerce/ ./internal/httpapi/` **ok**.
**Still not anyone-can-onboard perfect (deferred, not Product blockers):** Shopify App Store / Partner one-click install; **real** store webhook receive + registration (full HMAC + routing + idempotency — **do not stub** CSRF-exempt placeholder routes; see `docs/store-connectors.md`); exhaustive UI E2E of every control (unit helpers expanded; full click-matrix not claimed); Demo Shopify live-progress when connector is plan-gated (Woo Demo proof stands in).
<a id="ops-prometheus-scrape"></a>
### Ops: Prometheus scrape (`/metrics`) — gated in production
**Code:** API mounts `GET`/`HEAD` `/metrics` (`metrics.Handler` + RED middleware; path skipped from self-instrumentation) behind `metrics.Gate`:
- **Non-production:** always allowed (local scrapes).
- **Production:** allow only **loopback** peers **or** `METRICS_PUBLIC=1` (`config.MetricsPublic`); otherwise **404** (not a public scrape surface).
Admin diagnostics document the same rule. Worker sync histograms/failures appear on the **worker** process when `METRICS_ADDR` is set (separate listener, e.g. `:9091`), also wrapped with `Gate`.
**Not a P0 “public no-auth metrics” code gap** when Gate defaults hold. Edge lock-down remains **recommended** defense-in-depth (and required if you intentionally set `METRICS_PUBLIC=1`).
**Scrape (recommend):**
```yaml
# Prometheus — API RED (scrape from loopback/sidecar, or set METRICS_PUBLIC=1 on a private VIP only)
- job_name: descrybe-api
metrics_path: /metrics
static_configs:
- targets: ["api.internal:28471"] # private / service mesh only
# Worker sync series (only if METRICS_ADDR set)
- job_name: descrybe-worker
metrics_path: /metrics
static_configs:
- targets: ["worker.internal:9091"]
```
| Control | Recommendation |
|---------|----------------|
| Default prod | Keep `METRICS_PUBLIC` unset/`0`; scrape via loopback sidecar / mesh |
| Network | Prefer private subnet / VPN; deny public VIP even if Gate is on |
| Reverse proxy | Drop or `deny` `/metrics` on the internet-facing vhost; optional basic-auth or mTLS on internal scrape path |
| Worker | Bind `METRICS_ADDR` to loopback or private interface only (`127.0.0.1:9091` or mesh DNS) |
| Alerting | Wire RED + sync rules below — **examples CODE DONE**; live targets still **OPS OPEN** |
**Repo examples (no secrets):** [`deploy/prometheus/scrape.example.yml`](../deploy/prometheus/scrape.example.yml) · [`deploy/prometheus/alerts.example.yml`](../deploy/prometheus/alerts.example.yml) · [`deploy/prometheus/README.md`](../deploy/prometheus/README.md)
**Topology (cutover):**
```
Prometheus ──scrape──► API HTTP_ADDR GET /metrics → http_requests_total, http_request_duration_seconds
──scrape──► Worker METRICS_ADDR GET /metrics → sync_duration_seconds, sync_failures_total
──probe───► API HTTP_ADDR GET /readyz → worker freshness (503 if missing/stale >60s)
LB / public VIP ──deny──► /metrics (scrape only via loopback sidecar, mesh, or private VIP)
```
| Process | Env / bind | Series |
|---------|------------|--------|
| `cmd/api` | `HTTP_ADDR` (app port) | HTTP RED: `http_requests_total{method,code,path}`, `http_request_duration_seconds{method,code,path}` |
| `cmd/worker` | `METRICS_ADDR` (optional; e.g. `127.0.0.1:9091`) | Sync: `sync_duration_seconds{kind}`, `sync_failures_total{kind}` (`feed`, `woocommerce*`, `shopify*`) |
| blackbox → `/readyz` | private API base | `probe_success` (not heartbeat age); triage JSON `worker_last_seen_age_s` |
**Suggested alerts (ops — tune thresholds to baseline; copy from [`alerts.example.yml`](../deploy/prometheus/alerts.example.yml)):**
| Alert | Signal | Starter expression (illustrative) |
|-------|--------|-----------------------------------|
| API 5xx rate (RED Errors) | `http_requests_total` `code=~"5.."` | `sum(rate(http_requests_total{job="descrybe-api",code=~"5.."}[5m])) / sum(rate(http_requests_total{job="descrybe-api"}[5m])) > 0.05` for 10m |
| API latency (RED Duration) | `http_request_duration_seconds` | `histogram_quantile(0.99, sum by (le) (rate(http_request_duration_seconds_bucket{job="descrybe-api"}[5m]))) > 2` for 15m |
| Sync failures | `sync_failures_total` | `sum by (kind) (increase(sync_failures_total{job="descrybe-worker"}[15m])) > 0` for 15m (page on sustained / critical kinds) |
| Sync stall / slow | `sync_duration_seconds` | `histogram_quantile(0.95, sum by (le, kind) (rate(sync_duration_seconds_bucket{job="descrybe-worker"}[15m]))) > 120` for 30m |
| Scrape down | `up` | `up{job=~"descrybe-api|descrybe-worker"} == 0` for 5m |
| Worker heartbeat stale | `/readyz` (not Prom) | Blackbox/HTTP probe: `/readyz` ≠ 200, or JSON `checks.worker` ∈ {`missing`,`stale`}. On-call triage: read `worker_last_seen_age_s` (seconds since `processing` touch; omit when no row; stale after **60s** = `jobs.DefaultHeartbeatStaleAfter`). Same field printed by `scripts/cutover-deploy-check.mjs` |
**On-call note:** Heartbeat age is **not** a `/metrics` series — use `GET /readyz` `worker_last_seen_age_s` + `checks.worker` when paging worker-down (confirm how stale before restart). Do not start a second worker while age is still fresh (see [ops-runtime.md](ops-runtime.md)).
Cutover boxes: [production-checklist.md §1d](production-checklist.md#1d-metrics-scrape--alerts-cutover). Example files: [`deploy/prometheus/`](../deploy/prometheus/). No secrets in scrape configs — private targets / sidecars only.
Missed external checklist items (not code blockers, still gate honest Prod GO): dependency high/critical audit, backup **restore** drill, on-call runbook, DSAR/export rehearsal, per-tenant dashboards (series are route/kind-scoped, not `company_id`), launch-day rollback criterion.
### Aug-8 fixes summary (2026-08-08)
| Area | Change | Cutover impact |
|------|--------|----------------|
| Stripe boot vs runtime | `config.validate` in production requires `STRIPE_MOCK=false` only — **does not** require env `STRIPE_SECRET_KEY` / `STRIPE_WEBHOOK_SECRET` at API boot. Keys may live in `/admin/settings` (`platform_settings`); checkout/webhooks **fail closed** until configured. Env remains optional fallback ([stripe-setup.md](stripe-setup.md)). | Ops still must configure live Stripe + webhook signing before taking payments; boot alone is not proof of billing readiness |
| Transactional email dry-run | `mail.ApplyDryRun` honors `EmailDryRun` so SMTP-on + dry-run cannot send for real. Admin `POST /api/admin/settings/mail/test` returns **`skipped`** under dry-run. | Live SMTP / set-password login smoke remains **ops-only** / unproven |
| Public export feeds | Wrong extension on a **valid** `public_token` returns opaque **404** (was **400** format mismatch) — closes token-existence oracle. Tokens case-normalized; API only advertises the matching format URL. **BREAKING** for public clients that treated 400 as “token exists, wrong format”. | Security harden; update any scrapers/clients that branched on 400 |
| Export token rotate / entropy / RPM | **Shipped (168a394e):** `RotateExportFeedPublicToken` + handler; new tokens **256-bit** CSPRNG (`publicExportTokenBytes=32`); `RateLimitPublicExport` IP/probe/token budgets (stricter than generic `/api/public`) | Code **done** — not an open residual |
| Edge rate limits | Confirmed: in-app limiters remain **per process**; multi-replica cutover = **edge** hard global RPM. Optional `RATE_LIMIT_REPLICAS` divides HTTP middleware only (not a substitute). | Ops note only — no shared Redis limiter in-repo |
| Web adapter | **Decided (2026-08-09):** pin `@sveltejs/adapter-node@5.5.7` in `apps/web`. Evidence: `hooks.server.ts` (CSP/HSTS/legacy redirects), `+page.server.ts` (e.g. `/tasks` SSR redirect), `+server.ts` (admin translations). **`adapter-static` rejected** — request-time SSR surface cannot be prerender-only. | Deploy Node process for `build/` output; reverse-proxy `WEB_ORIGIN` to it |
| Orphan processed cleanup | **Code landed (sibling 97a2be08):** admin report/cleanup + `processOne` TX hardening. Ops must **once** run report then confirmed cleanup (platform session). | Does **not** clear Prod GO — data/SMTP/Stripe blockers remain |
| `/readyz` worker heartbeat | **Code landed (sibling 63ee62c5):** goose `039_worker_heartbeats`; `/readyz` requires fresh `processing` heartbeat (≤**60s**). | **Deploy order:** goose through **041** (includes 039+040+041) → restart **worker**. API-only local without worker → `/readyz` **503** (expected). Does **not** clear Prod GO |
| Durable feed sync | **Code landed (014e2507 / 2f1019f2):** API `EnqueueSync` → `feed_sync_jobs` + `pg_notify`; worker claim (`FOR UPDATE SKIP LOCKED`); **LISTEN** wake (`jobs.ListenWake`); stuck `running` reclaim via `processing.CleanupStuck` (includes `feed_sync_jobs`); same-feed pending reuse (`findOrCreatePendingSyncJob`). Dashboard **202** + poll. | Worker required for completion. Prior “LISTEN / stuck reclaim / same-feed dedupe” residuals are **done** |
### Aug-9 ops note (orphan processed)
After ETL / staging data is on the target DB (and before trusting processed-product counts), platform admins must run **once**:
1. Open **/admin/orphan-processed** (or `GET /api/admin/jobs/orphan-processed`) — report counts/samples (no delete).
2. Optional dry-run: **Dry-run cleanup** in the UI (POST without confirm).
3. If orphans exist: **Confirm delete…** in the UI, or `POST /api/admin/jobs/orphan-processed-cleanup` with `confirm=true` — deletes; without `confirm=true` the POST is report-only.
Requires platform-admin session + CSRF like other `/api/admin/*` mutations. **This alone is not production GO.**
### Aug-9 ops note (`/readyz` + worker + goose head)
Sibling **63ee62c5** + **040** + **041**: readiness is no longer DB-only; claim/list hot paths need indexes; forgot-password needs reset tokens.
1. Apply goose through **`041_password_reset_tokens.sql`** (includes **039** worker heartbeats + **040** hotpath indexes).
2. Restart **`cmd/worker`** so it touches `worker_heartbeats` (`jobs.TouchHeartbeat`).
3. Confirm `GET /readyz` → **200** with `checks.worker=ok` (stale threshold **60s**).
**API-only** (no worker): `/healthz` **200**, `/readyz` **503** (`worker heartbeat missing` / `stale`). Do not treat that as a broken API binary — start the worker (or accept 503 on readiness probes in that topology). **Not Prod GO by itself.**
**Operator cmd (read-only):** `node scripts/cutover-deploy-check.mjs` - asserts goose **039+040+041** applied + `/readyz` worker ok + prints `adapter-node` host notes (no migrate/restart).
**Local verify (2026-08-09):** Recycled to unified `npm run dev` (api+web+worker) after browsers hit **CONNECTION_REFUSED** on `:28472` (API/worker were up without Vite). Confirmed: `GET http://127.0.0.1:28471/readyz` → **200** (`checks.database/queue/worker=ok`); `http://localhost:28472/` → **200** (browser landing + Vite HMR). Stale standalone worker killed so only the concurrently worker remains. No A1 data writes.
### Aug-9 ops note (durable feed sync)
Sibling **014e2507**: feed sync is **durable** (no API-process goroutine sync).
| Step | Behavior |
|------|----------|
| Dashboard sync | `POST /api/feeds/{id}/sync` → **202** job payload; UI polls `/api/feeds/{id}/sync-jobs/{jobID}` |
| Persistence | Row in `feed_sync_jobs` (`pending` → worker claims `running`) |
| Worker | `cmd/worker` claim loop + `ProcessSyncJob`; also needs heartbeat for `/readyz` |
| v1 | Legacy envelope may still return **200** `{ data: { jobId } }` — dual-support |
**Shipped residuals (mark done):** worker `ListenWake` on `feed_sync_jobs` / `processing_jobs` (NOTIFY remains wake-only; claim/poll is source of truth); stuck `running` sync reclaim in `CleanupStuck` (worker loop + admin); same-feed pending dedupe in `EnqueueSync` / `findOrCreatePendingSyncJob`. Retention delete of terminal sync jobs: `CleanupExpiredSyncJobs`. Do **not** re-list these as open code blockers. Scale/load soak and multi-replica edge caps remain **Cutover/ops** (Product Perf foundations already 10).
### What landed (Wave 8)
| Area | Landed |
|------|--------|
| Cutover UX | **P0-10** login `password_not_set` CTA; **P0-11** role repair (Wave 7, retained); **P0-12** missing-plan / `past_due` recovery (no fake Free/Unlimited); **P0-13** store reconnect cliff; **P0-14** API-key reissue honesty; **P1-13** migrated-cohort checklist; **P1-14** invite email mismatch; **P1-15** admin readiness strip (`GET /api/admin/readiness`); **P1-16** known ETL gap panel |
| Cutover ops tooling | Migrator `-list-companies-without-plans` / `-assign-missing-plans` / ETL `-fallback-plan-name`; `GET /api/admin/companies?without_active_plan=1`; [staging-auth-rehearsal](staging-auth-rehearsal.md) scripts (promote → re-issue link → login smoke without live `.env` edits) |
| Honesty / polish | FTP sync “not supported” upfront; empty shells (vector-categories, structured-descriptions, campaigns); Settings `replaceState` race fix; bulk Accept Undo (admin → `/api/products/reset`); migrator gap report honesty |
| Security | Team PATCH rejects invalid silent demote; DELETE blocks last admin; Stripe live keys fail closed **at use** (admin settings and/or env); single-user set-password reissue **429** when rate-limited; CSP `connect-src` rejects non-http(s); OpenAPI documents team PATCH + admin set-password |
| Perf / a11y | Campaigns list batch `DISTINCT ON` (N+1 gone); ConfirmationDialog alertdialog focus; Alerts `aria-label`; ⌘K `aria-live` |
### What landed (Wave 7, retained)
| Area | Landed |
|------|--------|
| Cutover code | Membership role repair (`PATCH /api/team/{userID}`); admin set-password re-issue harden |
| UX | ⌘K (P1-10); bulk confirm (P1-3); Needs Review field accept/discard; mapping chips; Mapped honesty; Alerts `role=switch`; maintenance banner |
| Security/ops | CSP API+web; Stripe mock blocked in production; SMTP/Resend gated at send time when live email; CSRF cookie name sync |
### Tests (Wave 8 suite)
| Gate | Result |
|------|--------|
| `apps/api` `go test ./... -count=1` | **PASS** ([Full suite race Wave8](4a52137f-5614-411b-a529-1c2ca77324f6)) |
| `apps/api` `go vet ./...` | **PASS** |
| `apps/api` `go test -race ./... -count=1` | **PASS** |
| `apps/web` `npm run check` | **PASS / GREEN** (0 errors / 0 warnings) — Aug-9 re-verify [7c018ec6](7c018ec6-ffd1-46b4-8717-fbe52c9e9913) (`allowImportingTsExtensions` + `ja.ts` dup-key fix); Wave 8 suite also **PASS** |
| `apps/web` `npm run build` | **PASS** |
| Wave 8 final suite re-verify ([e239d01f](e239d01f-8ba9-4d3d-92a9-8b6ad7aec7ec)) | **PASS** — go test, vet, npm check, build |
| Wave 8 deep browser E2E ([3880dabd](3880dabd-5248-4bfa-a06d-cb726848f73b)) | Prior **FAIL** 10/11; Soft Undo **PASS** (`POST /api/processing/jobs` 202 + Undo); readiness **200** (`GET /api/admin/readiness`). Bare `/settings` `replaceState` fix **landed** ([ddad0c89](ddad0c89-9a46-429f-bb7d-7ae77213b399)); re-smoke **PASS** (tabs render; no `replaceState` error; Alerts 6 switches) |
| Wave 7 deep browser E2E ([651eeb31](651eeb31-fc05-4e66-ae07-30085c3f1bc1)) | Last completed deep live proof **PASS**: Mapping incomplete; Alerts 6 switches; mapping chips; login; Needs Review; processing; billing wallet; products pager |
| Mobile Product10 browser proof ([4bdf0120](4bdf0120-5afb-4a6a-b78e-a0ee611c4deb)) | **PASS** — device=mobile (390×844); `site=local-descrybe` + `laravel_login` (secret); assert **Feeds**; authenticated dashboard (A1); 0 console/uncaught |
| `apps/api` `staticcheck ./...` | **PASS** with `staticcheck.conf` excluding **ST1005** (legacy v1 `processing_type` API messages — see note below) |
#### Accepted staticcheck exception (ST1005)
Legacy `POST /api/v1/products/process` validation returns capitalized, punctuated client messages such as `Invalid processing_type. Use "full", …`. They live in `internal/processing/v1_legacy.go` and `internal/httpapi/v1_process_handlers.go`. Changing casing/punctuation would alter the public error contract, so **ST1005 is suppressed in** [`apps/api/staticcheck.conf`](../apps/api/staticcheck.conf) rather than rewriting the strings. `make vet` / `go vet` remain the Makefile gate and are unaffected.
### Remaining blockers (production cutover / ops) — true open items
**Hard live (still OPEN — excluded from LOCAL/CODE wave):**
| # | Severity | Blocker |
|---|----------|---------|
| 1 | **Data** | Optional `@legacy.local` hygiene — Clerk **removed from product** (tooling ready; **not** a hard NO-GO) |
| 4 | **Ops** | Live SMTP + set-password / forgot-password login smoke **unproven** (dry-run-safe code + diagnostics presence; live delivery not proven) |
| 5 | **Ops** | Real Stripe keys + signed webhooks in `/admin/settings` and/or env (presence flags shipped; live keys **unproven**) |
**LOCAL/CODE handled (2026-08-09 evening wave) — not Product gaps; prod-host apply remains mechanical:**
| # | Status | Item |
|---|--------|------|
| 2 | **CODE/LOCAL** | Promote company admins — A1-safe migrator; Demo already admin; cutover `-confirm` only on target DB |
| 3 | **CODE/LOCAL** | Assign skipped `company_plans` — local **87→0** Free (`-confirm`); A1 untouched |
| 6 | **CODE/LOCAL** | goose **039042** + worker + adapter-node — local deploy-check **PASS**; prod host still runs `npm run migrate` + restart + `start:web` |
| 7 | **CODE/LOCAL** | Orphan-processed — local orphans **0**; confirm fail-closed + A1-protected; cutover DB run if needed |
| 8 | **CODE DONE** | Multi-replica edge RPM — env contract + [deploy/examples/edge-rate-limit.md](../deploy/examples/edge-rate-limit.md); live edge apply if N>1 |
| 9 | **CODE DONE** | Metrics scrape/alerts — [`deploy/prometheus/`](../deploy/prometheus/) examples; live targets **OPS OPEN** |
| 10 | **CODE DONE** (honesty) | Unmigrated api_keys/blobs/jobs — admin inventory + reissue/reconnect CTAs; **no** fake migration |
**Not open code residuals (do not re-add):** feed sync LISTEN / stuck reclaim / same-feed pending dedupe; export token rotate + 256-bit + export RPM; “public `/metrics` no auth” as a P0 (gated); forgot-password opaque **200** (API + browser).
ASSUMPTION: Staging/LOCAL GO means cutover *rehearsal* (tooling + local proofs), not production DNS flip. Hard GO requires live SMTP + live Stripe on the target env.
## Verdict (security/ops)
**Product (code) is 10/10** under the split scorecard above.
**Security controls for API+web trust boundaries are production-capable** when env/admin knobs below are set.
**Production cutover** remains **NO-GO / 9/10**, gated hard by **live SMTP + live Stripe** ([go-live-checklist.md](go-live-checklist.md), [migration-readiness.md](migration-readiness.md), [cutover.md](cutover.md)). This document does **not** claim production DNS GO.
## Security checklist (2026-08-04 harden + Wave 7–8 + Aug-8)
| Item | Status | Notes |
|------|--------|--------|
| CSRF (dashboard) | **OK** | Double-submit cookie + header; web cookie name via `PUBLIC_CSRF_COOKIE_NAME` (default `descrybe_csrf`); `/api/v1` and webhooks exempt by design |
| Sessions | **OK** | HttpOnly + Postgres store; set `SESSION_SECURE=true` on HTTPS |
| Secure cookies | **OK** | Session + CSRF `Secure` follow `SESSION_SECURE`; production requires it |
| CORS | **OK** | Single `WEB_ORIGIN`; no `*`; credentials allowed |
| CSP | **OK** | API `SecurityHeaders` enforce `default-src 'none'…`; SvelteKit hooks; Wave 8: `connect-src` rejects non-http(s) |
| Stripe webhook verify | **OK** | Signature required whenever webhook secret is set (env or admin settings); unsigned only with mock + empty secret (local) |
| Stripe secret (prod) | **OK** (runtime) | Live Checkout needs secret key from admin settings and/or `STRIPE_SECRET_KEY`. **`validate` does not require it at boot** — missing key → checkout/portal fail closed |
| Stripe webhook secret (prod) | **OK** (runtime) | Live signed webhooks need secret from admin settings and/or `STRIPE_WEBHOOK_SECRET`. **Not a boot requirement** under `APP_ENV=production` |
| SMTP / email (prod) | **OK** (runtime + Aug-8) | Live send gated by platform/tenant config + dry-run. `mail.ApplyDryRun` disables SMTP send when dry-run on; admin mail test returns `skipped`. `EMAIL_DRY_RUN=false` alone does **not** force Resend/`SMTP_HOST` at boot |
| Public export token oracle | **OK** (Aug-8) | Format mismatch → opaque **404** (same as unknown token). **BREAKING** vs prior **400** |
| Export token rotate / entropy | **OK** (168a394e) | Rotate revokes prior URL; new tokens **256-bit** hex; validation accepts legacy 128–256-bit hex |
| Prod `/metrics` Gate | **OK** (13fec7ea) | Loopback-only unless `METRICS_PUBLIC=1`; not a public no-auth scrape surface by default |
| Team / last-admin | **OK** | Wave 8: invalid role no longer silent-demotes; DELETE cannot remove last company admin |
| Set-password reissue | **OK** | Rate limits + Wave 8 single-user path returns **429** when limited; skips `@legacy.local` |
| Secret encryption at rest | **OK** | AES-GCM for Woo / email / AI BYOK; `APP_ENCRYPTION_KEY` required in prod |
| SSRF — feeds | **OK** | Dial + redirect guards; 50 MiB / 60s |
| SSRF — Woo | **OK** | Normalize + `SafeHTTPClient` dial |
| SSRF — Shopify | **OK** | `NormalizeShopDomain` / `ValidateShopifyShopDomain` + `SafeHTTPClient` |
| Rate limits | **OK** (single replica) | Auth login **10**/min IP vs register **5**/min IP (split buckets); **email login lockout** 5 fails → 15 min (**429**); public `/api/public` **30**/min IP; export token GETs IP/probe/token budgets; API-key attempt **60**/min IP + company **120**/min; heavy sync/process/export 30/min/company; StartLimiter 20/min; marketing gen/send. **In-process only** — multi-replica cutover = **edge** hard caps; optional `RATE_LIMIT_REPLICAS` divides HTTP middleware only |
| `TRUSTED_PROXIES` | **OK** (docs) | Empty = ignore `X-Forwarded-For`; set to hop-1 proxy CIDRs so IP RPM / metrics peer checks see the real client. See [security-notes.md](security-notes.md). |
| Upload limits | **OK** | CSV 5 MiB; logo 2 MiB; JSON 2 MiB |
| No CLI secrets in UI | **OK** | Customer UI free of `go run` / seed / demo password copy |
| Production env fail-closed | **OK** | `APP_ENV=production` rejects insecure session/origin, missing encryption/signing secrets, and `STRIPE_MOCK=true`. Stripe live secrets + mail delivery are **runtime** gates (admin/env), not boot `STRIPE_*` env mandates |
## Required production env
Fail-closed when `APP_ENV=production` (`config.Load` / `validate`) — process will not start without these:
```text
APP_ENV=production
SESSION_SECURE=true
WEB_ORIGIN=https://<dashboard-origin>
APP_ENCRYPTION_KEY=<openssl rand -hex 32> # or CREDENTIALS_ENCRYPTION_KEY alias
TOKEN_SIGNING_SECRET=<openssl rand -hex 32>
STRIPE_MOCK=false
```
Stripe live keys are **operator-required for payments**, not boot-validated:
```text
# Prefer /admin/settings (stripe.secret_key, stripe.webhook_secret, prices…)
# Optional env fallback:
STRIPE_SECRET_KEY=sk_live_...
STRIPE_WEBHOOK_SECRET=whsec_...
```
Operator-required for a full production stack (not all enforced by `validate`):
```text
PUBLIC_API_URL=https://<api-origin>
DATABASE_URL=postgres://...sslmode=require
EMAIL_DRY_RUN=false # only after SMTP/Resend proven; unset/default is dry-run (safe)
```
Optional but recommended: VPC egress deny to link-local/metadata, WAF in front of API.
### Ops requirement: edge rate limits (multi-replica)
App-level limiters are **per process** (HTTP middleware in `httpapi/ratelimit.go`, email-keyed login lockout in `httpapi/login_lockout.go`, `processing.StartLimiter`, `support.AIRateLimiter`, email sliding limiter). There is **no Redis / Valkey / memcached** (or other shared rate-limit store) in this stack — confirmed by `diagCache` ("in-process only") and deps.
**ASSUMPTION (Product 10):** Single-node (one API process) in-memory rate limits + email login lockout are an **accepted product posture**. Multi-replica edge/Redis caps are **cutover/ops**, not a Product code gap. Captcha is intentionally deferred in favor of IP RPM + email lockout.
**Cutover when N>1:** enforce cluster-wide caps at the **edge** (CDN / reverse proxy / WAF / API gateway), aligned with the in-app budgets above (auth IP, login email lockout, public IP, heavy mutations per company, marketing generate/send, StartLimiter, AI auto-reply). Do **not** treat in-app 429s alone as a global guarantee: without edge, N replicas ≈ N× the configured RPM until shared infra exists.
**Optional `RATE_LIMIT_REPLICAS=N`:** divides **only** HTTP middleware caps in `httpapi/ratelimit.go` via `rateLimitEffectiveCap` (ceil) so aggregate under even load approximates documented RPM. It is **not** a shared counter, **not** a substitute for edge hard global caps, and **does not** apply to login email lockout, `StartLimiter`, `AIRateLimiter`, or email send limiters.
**Concrete edge snippets (nginx + Caddy) + env contract:** [deploy/examples/edge-rate-limit.md](../deploy/examples/edge-rate-limit.md). Ops boxes: [production-checklist.md](production-checklist.md) §1f · [cutover.md §8](cutover.md#8-multi-replica-edge-rate-limits).
**Aug-9 research wave (landed + backlog):**
| Item | Status | Notes |
|---|---|---|
| IETF `RateLimit` / `RateLimit-Policy` on HTTP 429 (with existing `Retry-After`) | **Landed** | `writeRateLimited` in `httpapi/ratelimit.go` — clients can pace without guessing |
| Task indicator concrete counters (“N of M products”) | **Landed** | `TaskStatusIndicator` — show progress even at 0% once total is known |
| Soft-warn headers on *allowed* requests (`RateLimit` remaining before hard 429) | **Recommend** | Emit `RateLimit` with `r=` remaining on success paths for heavy mutations so SDKs throttle early (see [STOA](https://docs.gostoa.dev/blog/saas-playbook-2-rate-limiting-saas), [IETF design notes](https://sujeet.pro/articles/design-api-rate-limiter)) |
| Hybrid local + Redis/Valkey counters | **Recommend** | Keep in-process token bucket for sub-ms deny; reconcile to Redis for multi-replica fairness ([Eltherion](https://eltherion.com/blog/tenant-aware-rate-limiting-design-tradeoffs), [Databricks batch-reporting](https://www.databricks.com/blog/high-performance-ratelimiting-databricks)). `RATE_LIMIT_BACKEND=redis` is still documentation-only |
| Per-endpoint quota classes (cheap GET vs heavy export) | **Partial** | Heavy-mutation bucket exists; extend policy matrix per tier when billing meters grow |
| Dark-mode `prefers-contrast: more` focus rings | **Landed** | `layout.css` media query boosts `--ring` / `:focus-visible` (token contrast wave already done) |
| Feed sync last-sync deltas in list UI | **Recommend** | API already returns `last_sync_deltas` via `PresentFeed`; surface per-row added/updated/failed counts next to last-synced (sync-state UX pattern) |
### Deploy note: SvelteKit adapter (decided)
`apps/web` pins **`@sveltejs/adapter-node@5.5.7`** (`svelte.config.js`). **Why not static:** the app relies on a Node request path — `src/hooks.server.ts` (`handle`: legacy redirects, password-query scrub, CSP/security headers, RapiDoc gzip), plus server loads/endpoints (`routes/tasks/+page.server.ts`, `routes/admin/translations/catalog/+server.ts`). Prerender/`adapter-static` would drop that surface. **Ops scripts (code):** `npm run build --workspace=web` then `npm run start:web` (`node build`). Read-only gate: `npm run cutover:deploy-check` (pin + config + scripts; plus goose/`/readyz` when not skipped). **Ops still required on host:** run that Node process; point `WEB_ORIGIN` at it; do not claim Prod GO from the pin alone.
## Verify before go-live
```bash
cd apps/api && go test ./... && go build ./...
cd ../web && npm run check
curl -sf https://<api>/healthz && curl -sf https://<api>/readyz
```
Staging rehearsal (no live `.env` edits): [staging-auth-rehearsal.md](staging-auth-rehearsal.md) / `scripts/staging-auth-rehearsal.ps1|.sh`.
Manual: CSRF rejection without token; login rate-limit after burst; Stripe CLI webhook signature; Woo/feed URL to private IP rejected; upload >5 MiB rejected; missing-plan / past_due banners; store reconnect when `has_credentials=false`; public export wrong-extension → **404** (not 400); admin mail test under dry-run → `skipped`.
## Still not “cutover ready” (ops)
See true open **cutover** blockers above and [go-live-checklist.md](go-live-checklist.md) / [migration-readiness.md](migration-readiness.md). Schema/migrate head is goose **041**. Wave 8 + Aug-8/9 code (billing recovery, orphan cleanup, worker heartbeat, durable sync + LISTEN/reclaim/dedupe, **040** indexes, **041** password reset / forgot-password, gated `/metrics`, abuse + export-token harden, **`npm run check` green** [7c018ec6](7c018ec6-ffd1-46b4-8717-fbe52c9e9913), mobile browser proof) **does** justify **Product 10/10**; it does **not** clear SMTP/Stripe/migrate+worker/orphan ops blockers or Prod DNS GO.
### Verify this synthesis (2026-08-09)
| Gate | Result |
|------|--------|
| `apps/api` `go test ./internal/metrics/ ./internal/httpapi/ -count=1` | **PASS** (shell) |
| `apps/api` `go vet ./internal/metrics/ ./internal/httpapi/` | **PASS** |
| Mobile browser Product10 proof ([4bdf0120](4bdf0120-5afb-4a6a-b78e-a0ee611c4deb)) | **PASS** (Feeds assert, authenticated) |
| `apps/web` `npm run check` ([7c018ec6](7c018ec6-ffd1-46b4-8717-fbe52c9e9913)) | **GREEN** (0 errors / 0 warnings; `allowImportingTsExtensions` + `ja.ts`) |
| codehelper `verify` on same pkgs | **abstain/fail env** (`could not import net` under MCP runner) — shell is source of truth here |
### Wave-20 verify (2026-08-09 afternoon) — A1 clean / Demo mutations
**Policy:** [demo-user.md](demo-user.md) — smoke/E2E/Postman/tests **MUST** use **Platform Demo**; **A1 Slovenija** is **read-only**.
| Gate | Result |
|------|--------|
| `apps/api` `go test ./... -count=1` (DATABASE_URL :5433) | **GREEN** — Free `max_products` expect **50**; sync-claim integration resilient to live worker ([ff63db4e](ff63db4e-cae9-4279-9510-7d7d931f9a2c)) |
| Process security + gates-before-EnsureRaw + lockout unit tests | **PASS** (no A1 writes) |
| Export token rotate (unit + ephemeral company) | **PASS** — Demo/ephemeral only; never A1 |
| `scripts/v1-process-smoke` (Demo API key) | **PASS** → COMPLETED on Platform Demo |
| A1 entitlements (`api_access`, `api_keys`, processing) | **PASS** read-only; A1 **not** mutated |
| A1 smoke-EAN cleanup check | **PASS** — no 8700999… junk on A1 (Demo only) |
| `/readyz` + worker | **PASS** 200 (existing worker; no duplicate) |
| `apps/web` `npm test` / `check` / `build` + root setup/install/build | **PASS** |
| Mobile A1 read-only Feeds ([699fca3d](699fca3d-34f6-4b40-adce-2d8ed58c79c2)) | **PASS** (stack up :28471/:28472) |
| Desktop A1 `/feeds` read-only | **PASS** (Your Feeds, 12 feeds); light contrast **PASS**; dark toggle **flaky/skipped** |
| OpenAPI products/process dual IDs + gates | **PASS** (docs + PresentProduct `raw_product_id`) |
| Feeds `mapping_incomplete` server batch | **PASS** (PresentFeeds + tests) |
**Verdict unchanged:** Product **10/10** · Cutover **NO-GO**. Residual polish (not Product blockers): dark-theme toggle flake; optional cleanup of leftover Demo smoke EAN `8700999000001`; mint/save A1 Postman key via demo impersonation if live Bearer proof needed (plaintext not recoverable).
### Visual + permissions QA wave (2026-08-09 late afternoon)
~20 parallel siblings: plan/permission honesty + light/dark contrast + sampled Demo/A1 browser PNGs. **Browser budget:** each visual agent capped at **12 PNG calls** (no machine thrash / no full click-matrix). Mutations stay on **Platform Demo**; **A1 = read-only**.
#### A1 clean confirmation
| Check | Result |
|-------|--------|
| A1 desktop plan-gate ([418200db](418200db-6391-49cc-9a8d-95d4a1bfc04f)) | **PASS**`/dashboard` shows **Pay as you go**; `/stores``plan-upgrade-panel` (“Store connectors are not on your plan” + “What still works”); **0** console/uncaught/failed requests; **no mutations** |
| Staff unlock leak ([5a1fca0a](5a1fca0a-c404-4105-8967-f99749c7c977)) | **FIXED** — normal A1 no longer gets `unlockAllFeatures` via legacy `is_platform_admin`; live `a1-primary` stores/marketing/ai **false** |
| Plan matrix / PAYG processing ([8a7374b5](8a7374b5-66d4-4272-9652-3cd4a941fdbb)) | **FIXED** — admin UI no longer treats A1 PAYG like restricted Legacy (Processing stays ON) |
| Entitlements API ↔ FE `can()` ([7e552a33](7e552a33-36ce-4741-bab0-590f7924f382)) | Catalog drift closed (`capability.storage_limit`, dashboard ETL keys, AI titles vs descriptions) |
| Wave policy | All permission/UI agents report **no A1 writes** |
**ASSUMPTION:** A1 clean = no process/sync/feed/store mutations and no staff unlock pollution this wave (browser + code audits). Not a fresh DB dump compare.
#### Screenshot / browser proofs (sampled)
| Surface | Agent | Result |
|---------|-------|--------|
| A1 `/dashboard` + `/stores` plan honesty | [418200db](418200db-6391-49cc-9a8d-95d4a1bfc04f) | **PASS** (2 desktop PNGs) |
| Demo light `/dashboard` + `/feeds` | [571f8f18](571f8f18-c196-42ed-8dfa-4d62c6107d44) | **PASS** (empty feeds OK; 0 console) |
| Demo dark shell | [08a28419](08a28419-76e7-4cb7-92e8-27f33044439b) | Partial — dark dashboard captured; `/products`/`/stores` missed (navigate/`Accept` budget) |
| Demo mobile dashboard + Quick links → `/feeds` | [f583f4d6](f583f4d6-bf74-4b8d-92d7-0bd1cabac7ee) | **PASS** — login → Quick links → authenticated `/feeds` (“Your Feeds”); 0 console |
| Demo Woo Connection | [e09a70ad](e09a70ad-3d01-4a48-8b40-d666acbcee49) | **PASS** — schedule/reconnect/permissions UI; no sync queued |
| Demo `/feeds` Add Feed dialog | [9106548b](9106548b-6893-4c6a-b20f-cc3b854f3d8f) | Empty list → opened Add Feed read-only; header overlap + sticky dialog footer **fixed** |
| Demo `/products` | [ae25bb09](ae25bb09-69af-4e17-a0e4-0d55cd6b0230) → [7f23ebfd](7f23ebfd-4a1d-4798-aa3f-0efb189d4937) | Chevrons + header badge spacing **FIXED** (All Feeds / Sort show unfold icons) |
| Demo `/settings` API keys + Team | [92e69159](92e69159-7388-4a09-a3ab-7ea6e87fbc4c) | **PASS** read-only; `apiKeysIntro` “endpoints”; no key created |
| Demo shell CommandPalette + theme | [ad8dbc5a](ad8dbc5a-e9f1-4534-aa42-f4c6840936b7) | **PASS** — 0 console; prior `paletteShortcut` crash already fixed |
#### Fixes landed this wave
| Area | Change |
|------|--------|
| Staff unlock | `$lib/staff-access.ts``shouldUnlockAllFeatures` / `isFullPlatformAdmin`; fail-closed premium `canFeature` when matrix missing |
| Plan honesty UI | `PlanUpgradePanel` + `PlanHonestyPanel` on Settings/Billing/Stores; locked tabs/CTAs show upgrade path |
| Member locks | Stores Connect/Reconnect/guided setup gated to admin (`storeConnectorAction` → View for members) |
| A1 PAYG cohort | Admin plan permissions treat custom A1 like backend `A1PaygPlanFeatures` (Processing ON; Stores OFF) |
| `/plans` feature overlay | Stable `nameKey` map (was brittle English “AI titles & descriptions” mismatch) |
| Catalog parity | Go + FE `capability.storage_limit` + dashboard ETL keys; AI titles vs descriptions split |
| Nav gates | `navItemAllowed` / `settingsTabAllowed` + unit tests; premium fail-closed |
| Light contrast | `--muted-foreground` (~7.5:1) + badge/chip tones → `text-foreground` |
| Dark contrast | Tokenized banners + dark `--muted-foreground` raised `255 14% 86%` ([d389b4e2](d389b4e2-477b-4164-b931-b0eca0ff550a)) |
| Products filters / header | All Feeds + Sort `ChevronsUpDown`; switcher/badge spacing ([7f23ebfd](7f23ebfd-4a1d-4798-aa3f-0efb189d4937)) |
| Settings `?tab=` sync | On disk: `urlSyncReady` + `applyTabFromSearchParams` / `syncTabToUrl`; `$effect` never overwrites an explicit valid `?tab=` |
| Feeds chrome | UserSwitcher overflow vs unread badge; sticky Add/Edit Feed dialog footer |
| Catalog tests | `plan-feature-catalog.vitest.ts` removed → `plan-feature-catalog.test.ts` under `node:test` |
| i18n | Packs **5443** keys; **17/17** i18n tests; permission/plan keys mirrored; API keys “endpoints” |
#### Gates (sibling-reported)
| Gate | Result |
|------|--------|
| `apps/web` `npm test` | **~230 pass** (incl. catalog parity under `node:test`) |
| `apps/web` `npm run check` | **0 errors / 0 warnings** |
| Go `./internal/billing/` catalog/A1 filters | **PASS** |
| i18n pack parity | **17/17** |
| `finish_check` | Often **abstain** on this bed (`review_diff` / empty git history) — not a product defect |
#### Remaining gaps vs production-ready cutover
**Cutover (ops) — still NO-GO:** role promote, plan assign, live SMTP + set-password smoke, live Stripe + webhooks, goose **039+040+041** + worker on prod host, orphan cleanup run, multi-replica edge RPM, metrics **live** scrape targets + Alertmanager (examples CODE DONE), Node `adapter-node` host. See blockers table above.
**Product polish closed this follow-up:** mobile Quick links → `/feeds` **PASS**; products filter chevrons + header badge spacing **FIXED**; dark muted-foreground raised (+ light muted/badges); settings `?tab=` sync **FIXED on disk**; catalog vitest → node:test; `npm run check` **0/0** / web tests **~230**.
**Still open (not Product-10 or cutover blockers):**
| Gap | Notes |
|-----|-------|
| Exhaustive UI E2E | Intentionally **not** claimed — browser agents stayed at 12 PNGs |
| Optional Demo dark re-shot | Muted token fixed; `/products`/`/stores` dark PNG not re-captured after bump |
**Verdict unchanged:** Product **10/10** · staging **GO** · production cutover **NO-GO**. A1 stayed clean. Visual+permissions residuals above are polish-only; ops cutover list still gates public DNS.
### Final miss-check wave (2026-08-09 evening)
~20 parallel siblings answered: *Did we miss anything? Does it work/look right? Is it production ready?* Sampled proofs only; mutations on **Platform Demo**; **A1 = read-only**.
#### Executive verdict (this wave)
| Question | Answer |
|----------|--------|
| **Did we miss anything (Product code)?** | **No Product-10 blockers.** Closed residual honesty/permission/UX leaks found this wave (stores hub panel, campaigns API gates, CSRF `/api/v10` lookalike, billing upgrade ladder, empty-state CTAs, dashboard truncation, mangled Woo/`plan.upgrade` locales). Minor polish leftovers remain (e.g. some `productoos` word-salad outside store/Woo) — not scorecard blockers. |
| **Does it work / look right?** | **Yes for sampled surfaces.** Gates green; Demo/A1/mobile/dark/shell browser proofs largely **PASS**; `/readyz` **200**; Demo `v1-process-smoke` **COMPLETED**. |
| **Product (code) ready?** | **YES — 10/10** (split scorecard; same assumptions as above). |
| **Production cutover ready?** | **NO — NO-GO / 9/10.** SMTP/Stripe/DNS/deploy mechanics still open; Clerk not a blocker. |
| **Ship to paying public internet today?** | **NO** until Cutover blockers clear. |
| **A1 clean?** | **YES** — confirmed below. |
**One-line:** Product **10/10** · looks/works on sampled proofs · staging **GO** · cutover **NO-GO**. Live Stripe/SMTP/DNS remain **Cutover-only**.
#### Product vs Cutover (do not mix)
| Lane | This wave | Still open |
|------|-----------|------------|
| **Product** | CSRF segment exempt harden; Demo stores honesty (`stores.hub`); campaigns `requireFeatures`; API-key create fail-closed in prod; member/plan UI gates; billing next-plan ladder; categories/attributes/campaigns empty CTAs; mangled `campaigns.emptyMessage` + Woo/`plan.upgrade.*` locale repair; Whats New + header truncation layout | Some non-store `productoos` word-salad (campaigns/products); Stripe webhook dedicated RPM (optional DoS harden); exhaustive UI E2E not claimed |
| **Cutover** | Docs+flags reconfirmed — **none** of the 10 ops blockers fully cleared (blocker **#9** examples CODE DONE; live scrape still OPS OPEN) | Optional `@legacy.local` hygiene, role promote, plan assign, live SMTP smoke, live Stripe+webhooks, goose **039+040+041**+worker+Node host on prod, orphan cleanup **run**, multi-replica edge RPM, metrics **live** scrape targets, unmigrated api_keys/blobs/jobs |
#### A1 clean confirmation
| Check | Result |
|-------|--------|
| A1 `/stores` plan honesty ([51ec9b58](51ec9b58-0173-4bbb-8246-856fdc98d4c7)) | **PASS**`plan-upgrade-panel` + “What still works”; Stores hidden in nav; **0** console/uncaught/failed; **no mutations** |
| A1 `/billing` Included/Locked ([087da6fa](087da6fa-19f0-45cc-af2e-015e9485d02e)) | **PASS** — both strings present; Settings reached Profile (no locked-tab claim on Profile) |
| Staff unlock regression ([ef9add99](ef9add99-fa90-4ed8-aa1e-475bad39277c)) | **25/25** — A1 cannot `unlockAllFeatures` via legacy `is_platform_admin` |
| Demo stores honesty fix vs A1 ([1b55661b](1b55661b-0355-4602-a618-f533f485a57b)) | Demo no longer shows denial when `stores.hub` allowed; **A1 still shows** denial — correct |
| Wave policy | Permission/UX/browser agents: **no A1 writes**; process smoke Demo-only |
**ASSUMPTION:** A1 clean = no process/sync/feed/store mutations and no staff-unlock pollution this wave. Not a fresh DB dump compare.
#### Gates / proofs (sibling-reported)
| Gate | Result |
|------|--------|
| `apps/web` `npm run check` | **0 errors / 0 warnings** ([acfb9ffd](acfb9ffd-088a-4d35-b3e3-8ca7de9e83b3)); re-confirmed after truncation fix ([27db0700](27db0700-2705-4c1f-ba6b-57b9e80f312d)) |
| `apps/web` `npm test` | **236** then **239/239** after locale repair ([3390a8c5](3390a8c5-5ba6-4905-a214-9f388f8ae7d0)) |
| Go `billing` / `shopify` / `woocommerce` / `httpapi` (+ `./...`) | **PASS** ([09489d3b](09489d3b-0075-479d-8e8c-ebadda47a40d)) |
| i18n packs | **5467** keys × 9 locales; sync filled=0 ([7b71f711](7b71f711-43ea-48f4-b5ef-e7628d74615b)); Woo brand + `plan.upgrade.*` EN leftovers cleared ([3390a8c5](3390a8c5-5ba6-4905-a214-9f388f8ae7d0)) |
| OpenAPI dual-ID + 402 gates | **Aligned** — tests PASS ([67863cff](67863cff-5ce5-4980-b457-675717cc320b)) |
| `/readyz` + `/metrics` Gate | Code correct; local `/readyz` **200** worker ok ([c1f18f77](c1f18f77-6bcd-48fa-afe2-18290749fb42)) |
| `scripts/v1-process-smoke` (Demo key) | **COMPLETED** ([736644e5](736644e5-1b38-4fc8-9f6b-42fc7e3af647)) |
| Mobile Demo Quick links → Feeds | **PASS** ([20fcc97e](20fcc97e-4cee-42cf-b365-b16c2e3b3996)) |
| Demo dark `/products`+`/stores` | **PASS** (0 console; honesty bug found → fixed) ([20a7becc](20a7becc-1861-4e83-83ca-4845324f63ea) → [1b55661b](1b55661b-0355-4602-a618-f533f485a57b)) |
| Demo shell palette + theme | **PASS** 0 console ([14c36e97](14c36e97-e899-4ef7-ae7b-e8cda0d922ef)) |
| Demo feeds Add Feed + products chevrons | **PASS** read-only ([0c70b947](0c70b947-040a-4520-a9ce-918234b51ca0)) |
| Demo dashboard truncation | **FIXED** — Whats New + header readable ([27db0700](27db0700-2705-4c1f-ba6b-57b9e80f312d)) |
| Security spot-check | No CSRF/auth/secret holes; CSRF path-prefix harden landed ([5d4782c0](5d4782c0-d2c6-40c3-af27-124978c94db6)) |
| Cutover checklist freshness | All named ops items still **OPEN** ([b7f9320f](b7f9320f-b001-457d-9bba-7b79a58ab10f), [984c0746](984c0746-b449-4eb0-86b5-66091ef9c021)) |
| Scorecard audit | Product **GO** · Cutover **NO-GO** ([984c0746](984c0746-b449-4eb0-86b5-66091ef9c021)) |
#### Fixes landed this wave
| Area | Change |
|------|--------|
| Demo stores honesty | `PlanUpgradePanel` only when `!stores.hub`; included honesty copy via `resolveHonestyItemDescription` |
| Campaigns / stores / API keys gates | API `requireFeatures` on campaigns; Shopify/Woo admin+plan gates; API-key create fail-closed if Billing nil in prod; FE FeatureGate / admin create guards |
| CSRF exempt paths | Segment match (`/api/v1`, `/api/public`, webhooks) — no `/api/v10` false exempt |
| UX gaps | Billing next self-serve upgrade plan (not hardcoded Growth); categories/attributes empty CTAs; billing no-balance + vector empty CTAs; `campaigns.emptyMessage` locale repair |
| Dashboard truncation | NewsFeed compact wrap; wider header UserSwitcher/email floors; Demo PNG readable |
| Store/Woo + plan.upgrade i18n | Cleared `WooComAcrcio`-class mangling; 43 `plan.upgrade.*` keys × 8 locales no longer English leftovers |
#### Still open (not Product-10 blockers)
| Gap | Lane | Notes |
|-----|------|-------|
| Non-store `productoos` word-salad | Product polish | Campaigns/products locales; store/Woo + `plan.upgrade.*` already fixed |
| Settings tab “Locked” honesty | Product polish | Billing Included/Locked proven; Profile settings has no locked-tab claim |
| Stripe webhook dedicated rate limit | Product harden (optional) | Signature+body cap exist; no dedicated RPM |
| Cutover blockers #1#10 | **Cutover** | Unchanged — see Remaining blockers table above |
| Launch hygiene | Cutover | Dep audit, backup restore drill, on-call runbook, DSAR rehearsal, rollback criterion |
**Doc drift watch:** prefer this file + [production-checklist.md](production-checklist.md) (goose **041**) over older goose `001007` mentions in some cutover docs.
**Final answer for parent:** Nothing material missed for **Product 10**. Sampled UI/API **works and looks right**. **Not** production-cutover ready — do not flip DNS.
### Continuation cutover-prep wave (2026-08-09 evening+)
~20 parallel siblings after the miss-check: cutover **tooling/docs/status surfaces** (no live secrets) + re-proofs. Mutations stayed off **A1**; Demo for browser honesty.
#### Executive summary (for parent)
| Question | Answer |
|----------|--------|
| **Product still 10/10?** | **YES** — gates green; forgot-password (goose **041**) closed deferred P1-18; no new Product blockers |
| **Cutover still NO-GO?** | **YES — NO-GO / 6/10** — all 10 ops blockers still require operator runs / live integration proof |
| **What this wave cleared?** | **Code/docs prep only** — migrator dry-run tooling, admin Stripe/mail presence flags, deploy check script, sequenced runbook, UX/docs honesty. **Zero** live SMTP/Stripe/DNS/orphan-run clears |
| **A1 clean?** | **YES** — A1 `/stores`+`/billing` read-only **PASS**; Demo `/stores` honesty **PASS**; gate re-run reported no A1 pollution |
| **Ship public internet today?** | **NO** |
**One-line:** Product **10/10** · cutover tooling **sharper** · production cutover still **NO-GO**. Schema head = goose **041**.
#### Cutover-prep shipped (code/docs — not ops clears)
| Area | Shipped |
|------|--------|
| Forgot-password | Goose **`041_password_reset_tokens`** + self-serve forgot/reset API/UI ([forgot-password.md](forgot-password.md)); admin set-password remains for migration cohort |
| `@legacy.local` tooling | Migrator `-list-legacy-emails` / `-export-legacy-emails` / dry-run patch (A1/live email protection) |
| Role promote | Migrator `-list-member-memberships` + dry-run/confirm `member` → company admin |
| Plan assign | `-assign-missing-plans` requires explicit **`-confirm`** after `-dry-run` |
| Orphan cleanup UX | `/admin/orphan-processed`: report → dry-run → confirm delete |
| Stripe readiness | Admin diagnostics presence flags (`secret_key_set`, `webhook_secret_set`, `mock`, `mock_rejected_in_prod`) — no secrets |
| SMTP readiness | Admin diagnostics mail presence/dry-run flags — `/readyz` stays DB/worker-only |
| Deploy check | `npm run cutover:deploy-check` / `scripts/cutover-deploy-check.mjs` — goose **039+040+041**, `/readyz` worker, `adapter-node` host gates (pin/config/scripts) |
| Runbook | [cutover.md](cutover.md#no-go-blockers-gate-sequenced-dry-runs) sequences all 10 NO-GO blockers dry-run-first |
| Metrics / edge | Scrape topology + RED/sync alert notes; `RATE_LIMIT_REPLICAS` vs edge hard-RPM clarified |
| Migration messaging | API-key reissue OpenAPI/docs; Woo/Shopify/hub reconnect copy aligned |
| Webhooks honesty | **Do not stub** store webhook receive (CSRF-exempt edge); plan copy uses **Full API** (no fake store inbound webhooks) |
| Re-proofs | CSRF `/api/v10` not exempt **PASS**; web **239** / check **0/0** / `go test ./...` **PASS**; i18n **5534** keys × 9 locales, **17/17**; A1+Demo stores/billing honesty **PASS** |
#### Still ops-only NO-GO (unchanged clears)
Same table as [Remaining blockers](#remaining-blockers-production-cutover--ops--true-open-items) — tooling/docs ready does **not** equal run. Live SMTP delivery, live Stripe keys+webhooks, goose+worker+Node on **prod host**, orphan cleanup **once**, edge RPM, metrics **live** scrape targets (examples in `deploy/prometheus/` are CODE DONE), and unmigrated api_keys/blobs/jobs remain open.
#### Product vs Cutover (this wave)
| Lane | Shipped | Still open |
|------|---------|------------|
| **Product** | Forgot-password **041**; plan/webhook honesty copy; reconnect + API-key reissue messaging; CSRF retest | Optional polish only (not scorecard blockers) |
| **Cutover** | Operator tooling, diagnostics flags, deploy check, sequenced runbook, metrics/edge docs | All 10 blockers — **ops must execute** |
**ASSUMPTION:** Raising Cutover to GO requires live SMTP smoke + Stripe signed-webhook proof — Clerk is removed from the product.
**Final answer for parent:** Cutover-prep **shipped** (tooling/docs/status). Product still **10/10**. Cutover still **NO-GO**. **A1 clean**. Do not flip DNS.
### Local cutover-rehearsal wave (2026-08-09 late)
Single-stream ops rehearsal on loopback **API :28471** / **web :28472** (Compose Postgres **:5433**). A1 stayed read-only; Demo OK for opaque forgot-password.
| Check | Result |
|-------|--------|
| `cutover-deploy-check` (prior) | **PASS** — goose **039041**, `/readyz` shape, `adapter-node` (with `HEALTH_BASE_URL=http://127.0.0.1:28471`) |
| Migrator dry-runs (prior) | Legacy email list / promote (A1 skip) / assign-plans dry-run **87 Free****no `-confirm`** |
| Forgot-password route | **Proven**`POST /api/auth/forgot-password` → opaque `{"status":"ok"}` **HTTP 200** (CSRF double-submit) for `demo@descrybe.local` and unknown email; prior UI **404** was down/stale API (route already mounted in `Router`), not a missing handler |
| Optional browser | One attempt hit native form **405** on `/forgot-password` (hydrate / cookie-banner race); **API curl is the proof** |
| `/readyz` this session | **503** `worker heartbeat stale` — expected with API-only (no `dev:worker`); Product assumption #2 |
| Product score | Still **10/10** |
| Cutover | Still **NO-GO** / **9/10** — live SMTP/Stripe (+ prod host mechanics) still **OPS OPEN**; Clerk not a blocker |
| A1 | **Clean** — no A1 mutations this stream |
**One-line:** Local rehearsal unblocked forgot-password (**opaque 200**). Product **10/10**. Cutover **NO-GO**. Do not flip DNS.
### LOCAL/CODE cutover wave (2026-08-09 evening) — exclude SMTP/Stripe (Clerk removed)
~20 parallel siblings closed everything automatable **except** live SMTP and live Stripe. Clerk removed from product. Mutations off **A1**; Demo/local fixture only. Synthesis kept light (no re-run of 20 parallel test suites).
#### Executive table (for parent)
| Question | Answer |
|----------|--------|
| **Product** | **10/10** unchanged |
| **Cutover** | **NO-GO** / **9/10** (raised from **8/10**; Clerk no longer a hard blocker) |
| **Hard OPS OPEN** | **live SMTP** · **live Stripe** only |
| **LOCAL/CODE handled** | Promote/assign (local **87→0**), orphan **0**, goose **042**+worker, adapter-node gates, edge snippets, Prometheus examples, api_keys/blobs inventory, store reconnect admin, session revoke **042**, rehearsal script, diagnostics cutover flags, forgot-pw browser opaque OK |
| **Failed siblings** | **None** material — light gates early **green**; no Product blockers |
| **A1 clean?** | **YES** |
| **Ship public paid today?** | **NO** |
**One-line:** LOCAL/CODE cutover prep **done** · Product **10/10** · cutover **9/10 NO-GO** until live SMTP + live Stripe.
#### Sibling scoreboard (abbrev.)
| Agent | Result |
|-------|--------|
| Assign plans local confirm ([a7a02b5b](a7a02b5b-5cac-480c-b412-7f824c5dca7c)) | **PASS****87→0**; A1 untouched |
| Promote admins ([bd6a6e4e](bd6a6e4e-8276-4fd4-a1cb-ff3ed73341f5)) | **PASS** — A1-safe; Demo already admin |
| Migrator A1 guards ([a8c78f15](a8c78f15-0ad3-4aae-b06f-988b6b4df2f3)) | **PASS** — tests lock skip |
| Orphan cleanup ([41383749](41383749-fb21-4d1b-bf9e-55095c877ed8)) | **PASS** — local **0**; fail-closed |
| Goose/worker ([109c6510](109c6510-a927-42e4-a491-1a52e6cbd911)) + **042** ([5955bd90](5955bd90-2c0f-44d3-a490-ab017a5a13d5)) | **PASS** — deploy-check green; schema **42** |
| Adapter-node ([17c7bdb0](17c7bdb0-25b5-41d6-a61b-07022eecb023)) | **PASS** — pin/scripts; host deploy OPS |
| Edge RPM + snippets ([ef782139](ef782139-a142-4582-b053-ec4dc3c3a9c8), [8ac96d75](8ac96d75-14db-46ef-88fd-c93c2ab39f86)) | **CODE DONE** |
| Metrics examples ([143e0817](143e0817-ffe4-4470-988a-a90342e35c05)) | **CODE DONE** — live targets OPS |
| Store reconnect ([72fb8e54](72fb8e54-ca49-41e7-b69e-380c5c73b08e)) | **PASS** — admin inventory |
| api_keys inventory ([247a9cbb](247a9cbb-0bb8-4230-8df4-e953cc1b7f57)) | **PASS** — no fake migrate |
| Blobs/jobs inventory ([514d1cd5](514d1cd5-4121-4d83-9381-bd6905b2b436)) | **PASS** — diagnostics COUNTs |
| Forgot-pw UI ([98f8dd9d](98f8dd9d-a77b-4552-984d-3efb9446a57d)) | **PASS** — browser opaque success |
| Session revoke **042** ([f17cfa67](f17cfa67-98ec-4ecc-9775-cea9ca2d9b94)) | **PASS** — goose **042** |
| Readyz API-only ([0945d38d](0945d38d-d967-4ad7-a707-3a1114d53bf3)) | **PASS** — clearer 503 reason |
| Rehearsal script ([3e7253b2](3e7253b2-bf35-42ea-b34a-516638297cfd)) | **PASS** — no `-confirm` by default |
| Diagnostics flags ([efab294a](efab294a-4e16-4480-b4fc-bd02d6fc6ec2)) | **PASS** |
| Checklist/docs ([06835bdb](06835bdb-987c-4c5a-8a19-67502f5cdfe2)) | **PASS** — CODE/LOCAL vs 3 OPS |
| Light gates ([87bb7797](87bb7797-60fd-4553-8a08-4387b543b60b)) | Early **green** (no full `./...` re-fanout) |
**Final answer for parent:** Everything except **live SMTP / live Stripe** is **LOCAL/CODE handled**. Product **10/10**. Cutover **9/10 NO-GO**. Clerk removed from product. **A1 clean**. Do not flip DNS.