descrybe/docs/live-pipeline.md

# Live full processing pipeline (Local Demo Co + Green Chat)

**Date:** 2026-08-04  
**Tenant:** Local Demo Co (`ee246275-dec0-4446-9e83-58d0c16c258a`)  
**LLM:** Green Chat `overloaded-local` at `http://192.168.50.181:8767/v1`  
**API:** `http://127.0.0.1:8080`  
**Worker:** `apps/api/bin/worker.exe` with AI configured (dashboard `/integrations/ai` or root `.env` `OPENAI_*`) + `EPREL_ENABLED=true`

## Pipeline (canonical `full`)

```
normalize -> parse_specs -> fill_fields -> eprel -> ai_enhance
```

| Step | Role |
|------|------|
| `normalize` | Flatten mapped/raw keys |
| `parse_specs` | Parse `specifications` into attributes |
| `fill_fields` | Fill missing scalars / standard fields |
| `eprel` | Fetch EU energy label when `eprel_id` present + enricher on |
| `ai_enhance` | Green Chat rewrite of name + description |

## Prerequisites

1. Postgres on `:5433` (`docker compose up -d`)
2. API + worker running; worker **must** inherit:
   - `OPENAI_BASE_URL` / `OPENAI_API_KEY` / `OPENAI_MODEL`
   - `EPREL_ENABLED=true` (otherwise `eprel` soft-skips)
3. Green Chat LAN reachable: `Test-NetConnection 192.168.50.181 -Port 8767`
4. Demo API key: `dk_demo_local_descrybe_test_key_v1`  
   (or session login `demo@descrybe.local` / `DemoPass123!`)

Restart worker after `.env` changes:

```powershell
cd f:\laragon\www\_MY\descrybe-v2
# load root .env into process env (prefer over legacy apps/api/.env), then:
.\apps\api\bin\worker.exe
# or: .\scripts\run-api.ps1 worker
```

## Live run (this pass)

### Sample (3 raw products with real EPREL IDs)

| GTIN | Mapped name | eprel_id |
|------|-------------|----------|
| `8806088787671` | Kuhinjska napa SAMSUNG NK24M3050PS/U1 | `52351` |
| `8059019024967` | Sušilni stroj HOOVER NDEH11A2TCBEXS-S, 11KG | `2155436` |
| `8806088803005` | Vg. pomivalni stroj SAMSUNG DW60M6040BB/EO | `346772` |

### Job

```powershell
$H = @{ Authorization = "Bearer dk_demo_local_descrybe_test_key_v1"; "Content-Type" = "application/json" }
$ids = @(
  "457f844c-6ab3-49ea-96c8-43a7749f8c50",
  "9455d6f0-1f34-435f-925e-b5ce8cdcb45f",
  "e2a815a0-b172-4fcb-b64b-d19682371ef4"
)
$body = @{ raw_product_ids = $ids; processing_type = "full" } | ConvertTo-Json
$job = Invoke-RestMethod -Method POST -Uri http://127.0.0.1:8080/api/v1/process -Headers $H -Body $body
# poll:
Invoke-RestMethod "http://127.0.0.1:8080/api/v1/process/$($job.id)" -Headers $H
```

Dashboard alias: `POST /api/processing/jobs` (same body).

| Field | Result |
|-------|--------|
| Job id | `512ab7f6-1537-47ae-ac3a-6b78616c0a53` |
| Status | **completed** (~6s) |
| Progress | **3/3** |
| Steps | all five **`done`** (normalize -> parse_specs -> fill_fields -> eprel -> ai_enhance) |
| Worker log | `processed=3 failed=0 mode=internal` |

### Field verification (before → after)

| GTIN | Before name (snip) | After `processed_name` | After EPREL | Tokens |
|------|--------------------|------------------------|-------------|--------|
| `8806088787671` | Kuhinjska napa Samsung 60… | **Samsung NK24M3050PS/U Kitchen Hood** | class **D**, label URL set, `eprel_id=52351` | 472 |
| `8059019024967` | Sušilni stroj HOOVER NDE… | **Hoover H-DRY 500 Essential 11kg** | class **E**, label URL set, `eprel_id=2155436` | 586 |
| `8806088803005` | Pomivalni stroj Samsung… | **Samsung DW60M6040BB/EO Built-in Dishwasher** | class **E**, label URL set, `eprel_id=346772` | 382 |

Also confirmed:

- `updated_at` advanced (was ~2026-08-03 22:16Z → 2026-08-04 01:16Z)
- Descriptions rewritten in English via Green Chat (`gpt_response.steps[].raw.model = overloaded-local`)
- Specs → attributes: **47–52** keys including brand / dimensions / `eprel_*`
- `field_sources.eprel = eprel_api`, `name = ai_enhance`
- `ai_provider_mode = internal`

**Verdict: PASS** — full live pipeline against Local Demo Co + Green Chat.

## Issues fixed this pass

| Issue | Fix |
|-------|-----|
| Worker had EPREL enricher **disabled** (`EPREL_ENABLED` unset → skip) | Restarted worker with `EPREL_ENABLED=true`; keep the flag in root `.env` so future worker starts keep EPREL on |
| Empty `eprel_id` keys are common (~17k rows have the key; ~1051 non-empty) | Sample selection used products with **non-empty** `mapped_data.eprel_id` so the eprel step actually hits the EU API |

No API route defects in this run: `POST /api/v1/process` returned 202-shaped pending job; poll to `completed` worked without code changes.

## How to re-run quickly

```powershell
Test-NetConnection 192.168.50.181 -Port 8767
curl.exe -sS http://127.0.0.1:8080/readyz

# Ensure worker log shows:
#   worker: OpenAI enabled base=http://192.168.50.181:8767/v1 model=overloaded-local
#   worker: EPREL enricher enabled

# Pick 3 raw ids with non-empty eprel_id from SQL, then POST processing_type=full as above.
# Poll until status=completed; compare processed_name / processed_description / eprel_* attrs.
```

## Related

- [e2e-feeds-process-export.md](e2e-feeds-process-export.md) — feeds → process → export  
- [e2e-processing.md](e2e-processing.md) — Background Tasks UI / cancel / enhance_only  
- [green-chat-smoke.md](green-chat-smoke.md) — LAN Green Chat wiring  
- [eprel.md](eprel.md) — EPREL env + attribute keys  
- [qa-local-demo.md](qa-local-demo.md) — Local Demo Co checklist