greeneclipse/descrybe
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
593edf35fed40d7798b1715c82459aabb4aa0da5
descrybe/docs/live-pipeline.md
T
greeneclipse 8580c996c3 Initial commit of Descrybe v2 without local scratch artifacts. 
Drop one-shot tmp/axe scripts and agent i18n scratch so the Gitea tree is deployable.
2026-08-09 22:47:43 +02:00

5.2 KiB
Raw Blame History

Live full processing pipeline (Local Demo Co + Green Chat)

Date: 2026-08-04
Tenant: Local Demo Co (ee246275-dec0-4446-9e83-58d0c16c258a)
LLM: Green Chat overloaded-local at http://192.168.50.181:8767/v1
API: http://127.0.0.1:8080
Worker: apps/api/bin/worker.exe with AI configured (dashboard /integrations/ai or root .env OPENAI_*) + EPREL_ENABLED=true

Pipeline (canonical full)

normalize -> parse_specs -> fill_fields -> eprel -> ai_enhance
Step Role
normalize Flatten mapped/raw keys
parse_specs Parse specifications into attributes
fill_fields Fill missing scalars / standard fields
eprel Fetch EU energy label when eprel_id present + enricher on
ai_enhance Green Chat rewrite of name + description

Prerequisites

  1. Postgres on :5433 (docker compose up -d)
  2. API + worker running; worker must inherit:
    • OPENAI_BASE_URL / OPENAI_API_KEY / OPENAI_MODEL
    • EPREL_ENABLED=true (otherwise eprel soft-skips)
  3. Green Chat LAN reachable: Test-NetConnection 192.168.50.181 -Port 8767
  4. Demo API key: dk_demo_local_descrybe_test_key_v1
    (or session login demo@descrybe.local / DemoPass123!)

Restart worker after .env changes:

cd f:\laragon\www\_MY\descrybe-v2
# load root .env into process env (prefer over legacy apps/api/.env), then:
.\apps\api\bin\worker.exe
# or: .\scripts\run-api.ps1 worker

Live run (this pass)

Sample (3 raw products with real EPREL IDs)

GTIN Mapped name eprel_id
8806088787671 Kuhinjska napa SAMSUNG NK24M3050PS/U1 52351
8059019024967 Sušilni stroj HOOVER NDEH11A2TCBEXS-S, 11KG 2155436
8806088803005 Vg. pomivalni stroj SAMSUNG DW60M6040BB/EO 346772

Job

$H = @{ Authorization = "Bearer dk_demo_local_descrybe_test_key_v1"; "Content-Type" = "application/json" }
$ids = @(
  "457f844c-6ab3-49ea-96c8-43a7749f8c50",
  "9455d6f0-1f34-435f-925e-b5ce8cdcb45f",
  "e2a815a0-b172-4fcb-b64b-d19682371ef4"
)
$body = @{ raw_product_ids = $ids; processing_type = "full" } | ConvertTo-Json
$job = Invoke-RestMethod -Method POST -Uri http://127.0.0.1:8080/api/v1/process -Headers $H -Body $body
# poll:
Invoke-RestMethod "http://127.0.0.1:8080/api/v1/process/$($job.id)" -Headers $H

Dashboard alias: POST /api/processing/jobs (same body).

Field Result
Job id 512ab7f6-1537-47ae-ac3a-6b78616c0a53
Status completed (~6s)
Progress 3/3
Steps all five done (normalize -> parse_specs -> fill_fields -> eprel -> ai_enhance)
Worker log processed=3 failed=0 mode=internal

Field verification (before → after)

GTIN Before name (snip) After processed_name After EPREL Tokens
8806088787671 Kuhinjska napa Samsung 60… Samsung NK24M3050PS/U Kitchen Hood class D, label URL set, eprel_id=52351 472
8059019024967 Sušilni stroj HOOVER NDE… Hoover H-DRY 500 Essential 11kg class E, label URL set, eprel_id=2155436 586
8806088803005 Pomivalni stroj Samsung… Samsung DW60M6040BB/EO Built-in Dishwasher class E, label URL set, eprel_id=346772 382

Also confirmed:

  • updated_at advanced (was ~2026-08-03 22:16Z → 2026-08-04 01:16Z)
  • Descriptions rewritten in English via Green Chat (gpt_response.steps[].raw.model = overloaded-local)
  • Specs → attributes: 47–52 keys including brand / dimensions / eprel_*
  • field_sources.eprel = eprel_api, name = ai_enhance
  • ai_provider_mode = internal

Verdict: PASS — full live pipeline against Local Demo Co + Green Chat.

Issues fixed this pass

Issue Fix
Worker had EPREL enricher disabled (EPREL_ENABLED unset → skip) Restarted worker with EPREL_ENABLED=true; keep the flag in root .env so future worker starts keep EPREL on
Empty eprel_id keys are common (~17k rows have the key; ~1051 non-empty) Sample selection used products with non-empty mapped_data.eprel_id so the eprel step actually hits the EU API

No API route defects in this run: POST /api/v1/process returned 202-shaped pending job; poll to completed worked without code changes.

How to re-run quickly

Test-NetConnection 192.168.50.181 -Port 8767
curl.exe -sS http://127.0.0.1:8080/readyz

# Ensure worker log shows:
#   worker: OpenAI enabled base=http://192.168.50.181:8767/v1 model=overloaded-local
#   worker: EPREL enricher enabled

# Pick 3 raw ids with non-empty eprel_id from SQL, then POST processing_type=full as above.
# Poll until status=completed; compare processed_name / processed_description / eprel_* attrs.

Related

Reference in New Issue View Git Blame Copy Permalink