descrybe/docs/ai-call-inspector.md

AI call inspector (Admin → AI calls)

Admin → AI calls (/admin/ai-calls, platform admin only) shows the exact system prompt, user prompt and raw response of every LLM call, per tenant and per user.

Before this existed the prompt was only ever in the worker log, truncated to 500 runes from each end, and processed_products.gpt_response kept response metadata without the prompt at all — so "what exactly did we send for this job" had no answer.

What is captured

Everything resolved through aiprovider.Service, tagged by role:

role	where it comes from
processing	product enhance (title / description / meta / attrs)
categorize	taxonomy pick when the product has no category
seo_meta	standalone SEO meta generation
campaign	marketing email generation
support	support draft assist

Each row carries company, the user who started the work, job id, raw product id, model, provider mode, finish reason, token counts, duration, and the provider error when the call failed.

How it works

• internal/aiaudit is a leaf package: context helpers plus the writer. Callers attach detail with aiaudit.WithCall(ctx, …); ProcessJob sets company/user/job once and processOne narrows it to the product.
• aiprovider wraps every Completer it hands out (audit.go). One capture point, so a new caller cannot forget to log. The wrapper forwards CompleterWithOptions when the client has it — otherwise every enhance call would silently lose MaxTokens / Temperature / ReasoningEffort.
• Capture never breaks the call it observes: write failures are swallowed after one log line, and the insert uses context.WithoutCancel so a cancelled job still records the call that was in flight.

Cost (Admin → AI costs)

/admin/ai-costs answers "what does each user cost me": spend by user, by company and by model, with a grand total across all tenants.

It reads ai_usage_daily, a rollup written next to every captured call — one row per day / company / user / model / role. That is why it can be kept for aiaudit.UsageRetentionDays (3 years) while the prompt bodies expire in a week: a busy tenant adds tens of rows a day, not one per product.

Money is micro-USD integers end to end (cost_micros). Summing millions of fractional-cent calls as floats drifts; integers add up exactly. cost_usd in the API is a display convenience — never re-sum it.

Prices live in ai_model_prices, seeded with the OpenAI GPT-5.6 list prices effective 2026-07-30 (per 1M tokens):

model	input	cached input	output
gpt-5.6-luna	$0.20	$0.02	$1.20
gpt-5.6-terra	$2.00	$0.20	$12.00
gpt-5.6-sol	$5.00	$0.50	$30.00

Update the table when list prices change (UPDATE ai_model_prices …). Recorded cost is never recalculated, so history keeps the rate that applied at the time. aiaudit.defaultModelPrices mirrors these as a code fallback for a model with no row; an unpriced model records usage at zero cost rather than guessing.

Cached prompt tokens are read from usage.prompt_tokens_details.cached_tokens and billed at the cached rate — they are a subset of prompt tokens, so they are discounted, never added on top.

GET /api/admin/ai-costs?days=30&from=&to=&company_id=

History from before capture existed

cmd/backfill-ai-usage recovers cost for products processed before this shipped, reading the provider usage blocks still stored in processed_products.gpt_response.

go run ./cmd/backfill-ai-usage            # dry run
go run ./cmd/backfill-ai-usage -apply

It is best effort, and the limits are real:

• Only rows that still carry a usage block. gpt_response is overwritten on every reprocess and absent on older or failed rows.
• No per-user attribution — processed_products.user_id is null on those rows, so recovered spend lands under "Unattributed" in the by-user view. By-company and the grand total are complete.
• The day is updated_at, the last write to the row, not necessarily when the call was made.
• Cached-token discounts cannot be recovered, so recovered input is priced as fully fresh — an over-estimate, never an under-estimate.

Rows are written with source='backfill', so a re-run replaces its own rows and can never double-count live capture. By default the scan stops at the first day live capture recorded; -before YYYY-MM-DD overrides that.

Retention

Rows are large and high volume — one per product per language, a few KB each. They are a debugging buffer, not an audit trail:

• aiaudit.RetentionDays = 7.
• The worker prunes on its retention tick (aiaudit.CleanupExpired).
• Each body is capped at 60k runes.
• Never build billing or reporting on this table.

For scale: a 25k-product job writes roughly 50–150 MB, which ages out within a week.

API

Platform admin only (RequirePlatformAdmin; 401 unauthenticated, 403 non-admin).

GET /api/admin/ai-calls?company_id=&user_id=&job_id=&raw_product_id=&role=&outcome=&q=&limit=&before=
GET /api/admin/ai-calls/{id}

The list returns lengths plus 240-rune previews — one unfiltered request must never stream a tenant's whole prompt corpus. Full bodies come from the detail endpoint, one row at a time. q searches system prompt, user prompt and response. before is the RFC3339 cursor returned as next_before.

Reading a row

Start from the outcome, then the prompt:

• outcome=failed — the error field holds the provider error. If it is a dial or 5xx error the model never answered and enhance fell back to supplier copy.
• role=processing with a short user_length — the category formula probably did not key. Check that the prompt contains GPT predloga: / Product template: and a <name>{ block.
• Response present but the product still looks like the feed — compare the response against the formula sections; a reply that ignores them is rejected by the formula gate and marked synthesized/refused (see docs/category-formulas.md).

A job with no rows at all made no LLM calls: it was either hash-skipped (ai_enhance outcome=skip reason=unchanged_hash), gated by plan/credits, or the provider was unset.