Category formulas

A category formula is what makes enhance generate copy instead of reshuffling the feed. The feed supplies facts; the formula supplies the shape.

Legacy Descrybe (descrybe-legacy/generator.php) did this by making the category's Prompt column the entire user message — a "GPT predloga" with <name>{…}</name>, <metaDescription>{…}</metaDescription> and an HTML body of <H2>/<p> pairs closing with a technical-specification list. v2 keeps that behaviour and adds structured title_template / description_template columns on top of it.

Two formula sets

	A1 Slovenija (legacy plan)	Everyone else
Source	`scripts/seed/a1-category-prompts.json` (+ `wp_product_categories.sql`)	`apps/api/internal/aiprompts/defaults/default-category-prompts.json`
Applied by	`cmd/seed-a1`, `cmd/repair-category-prompts`, Admin → Sync A1	category create + enhance-time fallback
Language of the formula text	Slovenian (the tenant's own copy)	English
Prompt scaffolding	Slovenian (`GPT predloga:`, `Staro_ime_izdelka:`)	English (`Product template:`, `Old_product_name:`)

Both drive the same code path. The scaffolding language follows the formula's own language (reSlovenianFormulaCue in processing/a1_enhance_prompt.go) so A1 keeps the exact frame generator.php used. Output language is always {{language}} from the company's content language — an English formula still produces Slovenian copy for a Slovenian tenant.

The English defaults

default-category-prompts.json is the A1 seed translated: 116 categories, same naming formulas and body structure, generated by

make seed-defaults      # regenerate
make check-defaults     # fail if stale (CI guard)

Translation is phrase-level over a closed vocabulary (55 title slots, 39 body blocks, 1 meta rule) and the generator fails if any Slovenian text survives, so a new legacy phrase can never ship untranslated.

Where they apply:

Category create — catalog.ApplyDefaultCategoryFormulas fills prompt, title_template and description_template on new categories (API create and CSV taxonomy import). Only empty columns are written, so nothing existing is overwritten.
Enhance-time fallback — processing.withDefaultCategoryFormula supplies the default for a categorised product whose category still has no formula. It is skipped when the product has no category, when the category has its own prompt/formula, or when the tenant authored their own enhance template.

A category outside the seeded taxonomy gets GenericDefaultCategoryPrompt — same structure, no category-specific naming slots.

Proving it locally

cmd/formula-e2e builds the pipeline the way cmd/worker does, runs a real job and checks the output is not a feed copy:

make up                                          # Postgres :5433
cd apps/api && go run ./cmd/mock-llm &            # formula-aware local LLM
make formula-e2e

cmd/mock-llm reads the template out of the prompt and answers in its shape using only the supplied name/description/category/attrs. That proves two things a static stub cannot: the formula reached the model, and a compliant reply survives the pipeline's formula gate.

Useful flags:

go run ./cmd/formula-e2e -company "A1 Slovenija" -gtin 8022068075495 -llm-base http://127.0.0.1:18767/v1
go run ./cmd/formula-e2e -new-tenant -llm-base http://127.0.0.1:18767/v1

-llm-base pins the provider for the run; without it the harness uses whatever platformsettings.ResolveOpenAI returns (admin DB setting first, then env).

What Review compares

Review shows Original vs Enriched for name, description, category, attributes and SEO meta. Each "Original" must come from a source the pipeline never writes, which is less obvious than it sounds:

field	Original comes from	trap
name / description	`original_name` / `original_description`	`name`/`description` from the API are display-preferred and resolve to the ENRICHED copy
category	`feed_category`	`mapped_data.category` is written back by the pipeline; `category_source` says who chose it
attributes	`mapped_data.specifications` (feed specs)	`processed_products.attributes` is written to the same value as `processed_attributes`
		EPREL writes one fact under several names — `product-attributes.ts` folds the aliases so it is listed once
SEO meta	`mapped_data.meta_*`	most feeds have none, so Enriched shows as Added

Attributes are grouped Added / Changed / Unchanged, with unchanged collapsed — a review should lead with what the run actually did. Alias keys are folded into one row: enrichment stores eprel_pdf and eprel_pdf_url, energy_class and eprel_energy_class, which otherwise listed the same fact two or three times. Folding needs a related name AND an identical value, so width and height are never merged just because they match ($lib/attribute-diff, unit-tested).

The product detail endpoint deliberately does not apply the SEO-meta omit rule. That rule keeps meta out of list views for cohorts that do not sell on it, but the detail payload feeds review, and meta the pipeline actually generated has to be reviewable — the panel hides the block itself when both sides are empty.

Anything with no Original is labelled Added and shown as "not in feed", not "Matched" — saying Matched about a field the supplier never sent reads as if the feed had one.

Two things behave this way by design, not by accident:

A1 Slovenija and Platform Demo do not GENERATE SEO meta (processing.CompanyOmitsSEOMetaID). Meta already stored on their rows is still shown — the read path only hides empty values, so Review and Content display what the AI actually produced.
Rows processed before category_source existed cannot say whether their mapped_data.category came from the feed or from an earlier AI run, so Review still treats it as feed-supplied. Reprocessing the product records the real provenance.

Low-confidence categorisation fails the product

The categorize model always answers with some id from the list, so a reply like {"categoryId":"1","confidence":0.08} is the model saying "I do not know". Taking it at face value files a camping chair under Generators and then drives that category's formula, producing confidently wrong copy.

Below processing.DefaultMinCategorizeConfidence (0.75) the product fails with ErrCategoryNotFound and processing stops there — enhance is never called, so no credits and no provider cost are spent writing copy for a category nobody believes in. The same applies when the model returns an id outside the taxonomy.

processing: category rejected uid=1 confidence=0.08 min=0.75 reason=low_confidence
processing: item failed ... err=product category could not be determined:
            best match "1" scored 0.08, below the 0.75 minimum

The product lands in the job as failed with that message on processing_job_products.error, and raw_products.processing_status = 'failed'.

Not gated: a category that came from the feed, vector or a prior run (categorize never runs), a model that reports no confidence at all, and AI-disabled plans. Override the floor per job with StepPolicy.MinCategorizeConfidence.

Why enhance can still return feed copy

In order of how often it bites:

The product has no category. No category → no formula. Roughly 83% of the A1 catalog has no mapped_data.category (see scripts/seed/README.txt); those rely on the categorize step, which needs a working LLM or Pinecone.
The AI provider is unreachable. Enhance falls back to supplier copy and marks field_sources.name = ai_enhance_failed. Check the admin OpenAI setting — it overrides the OPENAI_* env values.
The plan/credits gate. ai_enhance skip reason=entitlement_can_use_ai or insufficient credits.
The model ignored the formula. One retry is issued (descriptionFormulaRetrySuffix / titleRewriteRetrySuffix); if it still fails, the result is marked synthesized/refused and no enhance_input_hash is stored, so a reprocess tries again rather than caching the bad copy.

processing: ai_enhance outcome=… category_uid=… title_formula=… desc_formula=… formula_override=… in the worker log tells you which of these happened — and Admin → AI calls shows the exact prompt and reply for the call itself (docs/ai-call-inspector.md).

8.4 KiB Raw Blame History