greeneclipse
8580c996c3
Drop one-shot tmp/axe scripts and agent i18n scratch so the Gitea tree is deployable.
126 lines
5.2 KiB
Markdown
126 lines
5.2 KiB
Markdown
# Live full processing pipeline (Local Demo Co + Green Chat)
|
||
|
||
**Date:** 2026-08-04
|
||
**Tenant:** Local Demo Co (`ee246275-dec0-4446-9e83-58d0c16c258a`)
|
||
**LLM:** Green Chat `overloaded-local` at `http://192.168.50.181:8767/v1`
|
||
**API:** `http://127.0.0.1:8080`
|
||
**Worker:** `apps/api/bin/worker.exe` with AI configured (dashboard `/integrations/ai` or root `.env` `OPENAI_*`) + `EPREL_ENABLED=true`
|
||
|
||
## Pipeline (canonical `full`)
|
||
|
||
```
|
||
normalize -> parse_specs -> fill_fields -> eprel -> ai_enhance
|
||
```
|
||
|
||
| Step | Role |
|
||
|------|------|
|
||
| `normalize` | Flatten mapped/raw keys |
|
||
| `parse_specs` | Parse `specifications` into attributes |
|
||
| `fill_fields` | Fill missing scalars / standard fields |
|
||
| `eprel` | Fetch EU energy label when `eprel_id` present + enricher on |
|
||
| `ai_enhance` | Green Chat rewrite of name + description |
|
||
|
||
## Prerequisites
|
||
|
||
1. Postgres on `:5433` (`docker compose up -d`)
|
||
2. API + worker running; worker **must** inherit:
|
||
- `OPENAI_BASE_URL` / `OPENAI_API_KEY` / `OPENAI_MODEL`
|
||
- `EPREL_ENABLED=true` (otherwise `eprel` soft-skips)
|
||
3. Green Chat LAN reachable: `Test-NetConnection 192.168.50.181 -Port 8767`
|
||
4. Demo API key: `dk_demo_local_descrybe_test_key_v1`
|
||
(or session login `demo@descrybe.local` / `DemoPass123!`)
|
||
|
||
Restart worker after `.env` changes:
|
||
|
||
```powershell
|
||
cd f:\laragon\www\_MY\descrybe-v2
|
||
# load root .env into process env (prefer over legacy apps/api/.env), then:
|
||
.\apps\api\bin\worker.exe
|
||
# or: .\scripts\run-api.ps1 worker
|
||
```
|
||
|
||
## Live run (this pass)
|
||
|
||
### Sample (3 raw products with real EPREL IDs)
|
||
|
||
| GTIN | Mapped name | eprel_id |
|
||
|------|-------------|----------|
|
||
| `8806088787671` | Kuhinjska napa SAMSUNG NK24M3050PS/U1 | `52351` |
|
||
| `8059019024967` | Sušilni stroj HOOVER NDEH11A2TCBEXS-S, 11KG | `2155436` |
|
||
| `8806088803005` | Vg. pomivalni stroj SAMSUNG DW60M6040BB/EO | `346772` |
|
||
|
||
### Job
|
||
|
||
```powershell
|
||
$H = @{ Authorization = "Bearer dk_demo_local_descrybe_test_key_v1"; "Content-Type" = "application/json" }
|
||
$ids = @(
|
||
"457f844c-6ab3-49ea-96c8-43a7749f8c50",
|
||
"9455d6f0-1f34-435f-925e-b5ce8cdcb45f",
|
||
"e2a815a0-b172-4fcb-b64b-d19682371ef4"
|
||
)
|
||
$body = @{ raw_product_ids = $ids; processing_type = "full" } | ConvertTo-Json
|
||
$job = Invoke-RestMethod -Method POST -Uri http://127.0.0.1:8080/api/v1/process -Headers $H -Body $body
|
||
# poll:
|
||
Invoke-RestMethod "http://127.0.0.1:8080/api/v1/process/$($job.id)" -Headers $H
|
||
```
|
||
|
||
Dashboard alias: `POST /api/processing/jobs` (same body).
|
||
|
||
| Field | Result |
|
||
|-------|--------|
|
||
| Job id | `512ab7f6-1537-47ae-ac3a-6b78616c0a53` |
|
||
| Status | **completed** (~6s) |
|
||
| Progress | **3/3** |
|
||
| Steps | all five **`done`** (normalize -> parse_specs -> fill_fields -> eprel -> ai_enhance) |
|
||
| Worker log | `processed=3 failed=0 mode=internal` |
|
||
|
||
### Field verification (before → after)
|
||
|
||
| GTIN | Before name (snip) | After `processed_name` | After EPREL | Tokens |
|
||
|------|--------------------|------------------------|-------------|--------|
|
||
| `8806088787671` | Kuhinjska napa Samsung 60… | **Samsung NK24M3050PS/U Kitchen Hood** | class **D**, label URL set, `eprel_id=52351` | 472 |
|
||
| `8059019024967` | Sušilni stroj HOOVER NDE… | **Hoover H-DRY 500 Essential 11kg** | class **E**, label URL set, `eprel_id=2155436` | 586 |
|
||
| `8806088803005` | Pomivalni stroj Samsung… | **Samsung DW60M6040BB/EO Built-in Dishwasher** | class **E**, label URL set, `eprel_id=346772` | 382 |
|
||
|
||
Also confirmed:
|
||
|
||
- `updated_at` advanced (was ~2026-08-03 22:16Z → 2026-08-04 01:16Z)
|
||
- Descriptions rewritten in English via Green Chat (`gpt_response.steps[].raw.model = overloaded-local`)
|
||
- Specs → attributes: **47–52** keys including brand / dimensions / `eprel_*`
|
||
- `field_sources.eprel = eprel_api`, `name = ai_enhance`
|
||
- `ai_provider_mode = internal`
|
||
|
||
**Verdict: PASS** — full live pipeline against Local Demo Co + Green Chat.
|
||
|
||
## Issues fixed this pass
|
||
|
||
| Issue | Fix |
|
||
|-------|-----|
|
||
| Worker had EPREL enricher **disabled** (`EPREL_ENABLED` unset → skip) | Restarted worker with `EPREL_ENABLED=true`; keep the flag in root `.env` so future worker starts keep EPREL on |
|
||
| Empty `eprel_id` keys are common (~17k rows have the key; ~1051 non-empty) | Sample selection used products with **non-empty** `mapped_data.eprel_id` so the eprel step actually hits the EU API |
|
||
|
||
No API route defects in this run: `POST /api/v1/process` returned 202-shaped pending job; poll to `completed` worked without code changes.
|
||
|
||
## How to re-run quickly
|
||
|
||
```powershell
|
||
Test-NetConnection 192.168.50.181 -Port 8767
|
||
curl.exe -sS http://127.0.0.1:8080/readyz
|
||
|
||
# Ensure worker log shows:
|
||
# worker: OpenAI enabled base=http://192.168.50.181:8767/v1 model=overloaded-local
|
||
# worker: EPREL enricher enabled
|
||
|
||
# Pick 3 raw ids with non-empty eprel_id from SQL, then POST processing_type=full as above.
|
||
# Poll until status=completed; compare processed_name / processed_description / eprel_* attrs.
|
||
```
|
||
|
||
## Related
|
||
|
||
- [e2e-feeds-process-export.md](e2e-feeds-process-export.md) — feeds → process → export
|
||
- [e2e-processing.md](e2e-processing.md) — Background Tasks UI / cancel / enhance_only
|
||
- [green-chat-smoke.md](green-chat-smoke.md) — LAN Green Chat wiring
|
||
- [eprel.md](eprel.md) — EPREL env + attribute keys
|
||
- [qa-local-demo.md](qa-local-demo.md) — Local Demo Co checklist
|
||
|