84 lines
3.5 KiB
Markdown
84 lines
3.5 KiB
Markdown
# AI call inspector (Admin → AI calls)
|
||
|
||
**Admin → AI calls** (`/admin/ai-calls`, platform admin only) shows the exact
|
||
system prompt, user prompt and raw response of every LLM call, per tenant and per
|
||
user.
|
||
|
||
Before this existed the prompt was only ever in the worker log, truncated to 500
|
||
runes from each end, and `processed_products.gpt_response` kept response metadata
|
||
without the prompt at all — so "what exactly did we send for this job" had no
|
||
answer.
|
||
|
||
## What is captured
|
||
|
||
Everything resolved through `aiprovider.Service`, tagged by role:
|
||
|
||
| role | where it comes from |
|
||
|---|---|
|
||
| `processing` | product enhance (title / description / meta / attrs) |
|
||
| `categorize` | taxonomy pick when the product has no category |
|
||
| `seo_meta` | standalone SEO meta generation |
|
||
| `campaign` | marketing email generation |
|
||
| `support` | support draft assist |
|
||
|
||
Each row carries company, the user who started the work, job id, raw product id,
|
||
model, provider mode, finish reason, token counts, duration, and the provider error
|
||
when the call failed.
|
||
|
||
## How it works
|
||
|
||
- `internal/aiaudit` is a leaf package: context helpers plus the writer. Callers
|
||
attach detail with `aiaudit.WithCall(ctx, …)`; `ProcessJob` sets company/user/job
|
||
once and `processOne` narrows it to the product.
|
||
- `aiprovider` wraps every Completer it hands out (`audit.go`). One capture point,
|
||
so a new caller cannot forget to log. The wrapper forwards
|
||
`CompleterWithOptions` when the client has it — otherwise every enhance call
|
||
would silently lose `MaxTokens` / `Temperature` / `ReasoningEffort`.
|
||
- Capture never breaks the call it observes: write failures are swallowed after one
|
||
log line, and the insert uses `context.WithoutCancel` so a cancelled job still
|
||
records the call that was in flight.
|
||
|
||
## Retention
|
||
|
||
Rows are large and high volume — one per product per language, a few KB each. They
|
||
are a **debugging buffer, not an audit trail**:
|
||
|
||
- `aiaudit.RetentionDays` = 7.
|
||
- The worker prunes on its retention tick (`aiaudit.CleanupExpired`).
|
||
- Each body is capped at 60k runes.
|
||
- Never build billing or reporting on this table.
|
||
|
||
For scale: a 25k-product job writes roughly 50–150 MB, which ages out within a
|
||
week.
|
||
|
||
## API
|
||
|
||
Platform admin only (`RequirePlatformAdmin`; 401 unauthenticated, 403 non-admin).
|
||
|
||
```
|
||
GET /api/admin/ai-calls?company_id=&user_id=&job_id=&raw_product_id=&role=&outcome=&q=&limit=&before=
|
||
GET /api/admin/ai-calls/{id}
|
||
```
|
||
|
||
The list returns lengths plus 240-rune previews — one unfiltered request must never
|
||
stream a tenant's whole prompt corpus. Full bodies come from the detail endpoint,
|
||
one row at a time. `q` searches system prompt, user prompt and response.
|
||
`before` is the RFC3339 cursor returned as `next_before`.
|
||
|
||
## Reading a row
|
||
|
||
Start from the outcome, then the prompt:
|
||
|
||
- `outcome=failed` — the `error` field holds the provider error. If it is a dial or
|
||
5xx error the model never answered and enhance fell back to supplier copy.
|
||
- `role=processing` with a short `user_length` — the category formula probably did
|
||
not key. Check that the prompt contains `GPT predloga:` / `Product template:` and
|
||
a `<name>{` block.
|
||
- Response present but the product still looks like the feed — compare the response
|
||
against the formula sections; a reply that ignores them is rejected by the
|
||
formula gate and marked `synthesized`/`refused` (see docs/category-formulas.md).
|
||
|
||
A job with no rows at all made no LLM calls: it was either hash-skipped
|
||
(`ai_enhance outcome=skip reason=unchanged_hash`), gated by plan/credits, or the
|
||
provider was unset.
|