descrybe/docs/ai-call-inspector.md

AI call inspector (Admin → AI calls)

Admin → AI calls (/admin/ai-calls, platform admin only) shows the exact system prompt, user prompt and raw response of every LLM call, per tenant and per user.

Before this existed the prompt was only ever in the worker log, truncated to 500 runes from each end, and processed_products.gpt_response kept response metadata without the prompt at all — so "what exactly did we send for this job" had no answer.

What is captured

Everything resolved through aiprovider.Service, tagged by role:

role	where it comes from
processing	product enhance (title / description / meta / attrs)
categorize	taxonomy pick when the product has no category
seo_meta	standalone SEO meta generation
campaign	marketing email generation
support	support draft assist

Each row carries company, the user who started the work, job id, raw product id, model, provider mode, finish reason, token counts, duration, and the provider error when the call failed.

How it works

• internal/aiaudit is a leaf package: context helpers plus the writer. Callers attach detail with aiaudit.WithCall(ctx, …); ProcessJob sets company/user/job once and processOne narrows it to the product.
• aiprovider wraps every Completer it hands out (audit.go). One capture point, so a new caller cannot forget to log. The wrapper forwards CompleterWithOptions when the client has it — otherwise every enhance call would silently lose MaxTokens / Temperature / ReasoningEffort.
• Capture never breaks the call it observes: write failures are swallowed after one log line, and the insert uses context.WithoutCancel so a cancelled job still records the call that was in flight.

Retention

Rows are large and high volume — one per product per language, a few KB each. They are a debugging buffer, not an audit trail:

• aiaudit.RetentionDays = 7.
• The worker prunes on its retention tick (aiaudit.CleanupExpired).
• Each body is capped at 60k runes.
• Never build billing or reporting on this table.

For scale: a 25k-product job writes roughly 50–150 MB, which ages out within a week.

API

Platform admin only (RequirePlatformAdmin; 401 unauthenticated, 403 non-admin).

GET /api/admin/ai-calls?company_id=&user_id=&job_id=&raw_product_id=&role=&outcome=&q=&limit=&before=
GET /api/admin/ai-calls/{id}

The list returns lengths plus 240-rune previews — one unfiltered request must never stream a tenant's whole prompt corpus. Full bodies come from the detail endpoint, one row at a time. q searches system prompt, user prompt and response. before is the RFC3339 cursor returned as next_before.

Reading a row

Start from the outcome, then the prompt:

• outcome=failed — the error field holds the provider error. If it is a dial or 5xx error the model never answered and enhance fell back to supplier copy.
• role=processing with a short user_length — the category formula probably did not key. Check that the prompt contains GPT predloga: / Product template: and a <name>{ block.
• Response present but the product still looks like the feed — compare the response against the formula sections; a reply that ignores them is rejected by the formula gate and marked synthesized/refused (see docs/category-formulas.md).

A job with no rows at all made no LLM calls: it was either hash-skipped (ai_enhance outcome=skip reason=unchanged_hash), gated by plan/credits, or the provider was unset.