update see calss

2026-08-23 22:03:57 +02:00
parent f3d4fb56ed
commit 8983cfc8a1
32 changed files with 1884 additions and 22 deletions
@@ -0,0 +1,83 @@
+# AI call inspector (Admin → AI calls)
+
+**Admin → AI calls** (`/admin/ai-calls`, platform admin only) shows the exact
+system prompt, user prompt and raw response of every LLM call, per tenant and per
+user.
+
+Before this existed the prompt was only ever in the worker log, truncated to 500
+runes from each end, and `processed_products.gpt_response` kept response metadata
+without the prompt at all — so "what exactly did we send for this job" had no
+answer.
+
+## What is captured
+
+Everything resolved through `aiprovider.Service`, tagged by role:
+
+| role | where it comes from |
+|---|---|
+| `processing` | product enhance (title / description / meta / attrs) |
+| `categorize` | taxonomy pick when the product has no category |
+| `seo_meta` | standalone SEO meta generation |
+| `campaign` | marketing email generation |
+| `support` | support draft assist |
+
+Each row carries company, the user who started the work, job id, raw product id,
+model, provider mode, finish reason, token counts, duration, and the provider error
+when the call failed.
+
+## How it works
+
+- `internal/aiaudit` is a leaf package: context helpers plus the writer. Callers
+  attach detail with `aiaudit.WithCall(ctx, …)`; `ProcessJob` sets company/user/job
+  once and `processOne` narrows it to the product.
+- `aiprovider` wraps every Completer it hands out (`audit.go`). One capture point,
+  so a new caller cannot forget to log. The wrapper forwards
+  `CompleterWithOptions` when the client has it — otherwise every enhance call
+  would silently lose `MaxTokens` / `Temperature` / `ReasoningEffort`.
+- Capture never breaks the call it observes: write failures are swallowed after one
+  log line, and the insert uses `context.WithoutCancel` so a cancelled job still
+  records the call that was in flight.
+
+## Retention
+
+Rows are large and high volume — one per product per language, a few KB each. They
+are a **debugging buffer, not an audit trail**:
+
+- `aiaudit.RetentionDays` = 7.
+- The worker prunes on its retention tick (`aiaudit.CleanupExpired`).
+- Each body is capped at 60k runes.
+- Never build billing or reporting on this table.
+
+For scale: a 25k-product job writes roughly 50–150 MB, which ages out within a
+week.
+
+## API
+
+Platform admin only (`RequirePlatformAdmin`; 401 unauthenticated, 403 non-admin).
+
+```
+GET /api/admin/ai-calls?company_id=&user_id=&job_id=&raw_product_id=&role=&outcome=&q=&limit=&before=
+GET /api/admin/ai-calls/{id}
+```
+
+The list returns lengths plus 240-rune previews — one unfiltered request must never
+stream a tenant's whole prompt corpus. Full bodies come from the detail endpoint,
+one row at a time. `q` searches system prompt, user prompt and response.
+`before` is the RFC3339 cursor returned as `next_before`.
+
+## Reading a row
+
+Start from the outcome, then the prompt:
+
+- `outcome=failed` — the `error` field holds the provider error. If it is a dial or
+  5xx error the model never answered and enhance fell back to supplier copy.
+- `role=processing` with a short `user_length` — the category formula probably did
+  not key. Check that the prompt contains `GPT predloga:` / `Product template:` and
+  a `<name>{` block.
+- Response present but the product still looks like the feed — compare the response
+  against the formula sections; a reply that ignores them is rejected by the
+  formula gate and marked `synthesized`/`refused` (see docs/category-formulas.md).
+
+A job with no rows at all made no LLM calls: it was either hash-skipped
+(`ai_enhance outcome=skip reason=unchanged_hash`), gated by plan/credits, or the
+provider was unset.
@@ -96,4 +96,6 @@ In order of how often it bites:
   stored, so a reprocess tries again rather than caching the bad copy.

 `processing: ai_enhance outcome=… category_uid=… title_formula=… desc_formula=…
-formula_override=…` in the worker log tells you which of these happened.
+formula_override=…` in the worker log tells you which of these happened — and
+**Admin → AI calls** shows the exact prompt and reply for the call itself
+(docs/ai-call-inspector.md).