# Local LLM tuning (weak / 8k-class models)

Defaults for OpenAI-compatible local servers such as Green Chat Gemma 12B (`overloaded-local`, ~8k context).

## Env

| Variable | Typical local value |
|----------|---------------------|
| `OPENAI_BASE_URL` | `http://<host>:8767/v1` |
| `OPENAI_MODEL` | id from `GET /v1/models` (e.g. `overloaded-local`) |
| `OPENAI_API_KEY` | required non-empty for `Enabled()` |
| `PROCESSING_RPM` | `60` |
| `PROCESSING_MAX_RETRIES` | `3` |

Restart **both** `api` and `worker` after changing `OPENAI_*`. Worker logs should say `OpenAI enabled base=… model=…`, not `heuristic completer`.

## Hardening defaults (code)

| Constant | Value | Where |
|----------|------:|-------|
| `DefaultStructuredTemp` | `0.2` (capped ≤0.3) | `processing/llm_json.go`, `openai.go` |
| `MaxTokensEnhance` | `350` | product title/description |
| `MaxTokensSEO` | `180` | meta title/description |
| `MaxTokensCampaign` | `650` | email JSON |
| `MaxProductDescRunes` | `400` | user context |
| `MaxAttrKeys` / `MaxAttrValueRunes` | `10` / `60` | attrs in enhance prompt |
| `MaxBrandInjectRunes` | `500` | brand kit block |
| `MaxCampaignProducts` | `8` | campaign product list |

## Prompt style

- Short system prompts with **bullet rules** + explicit JSON schema
- **One** few-shot example max
- Brand kit as compact bullets: `Brand:\n- tone: …\n- do: …`
- Product context: category + name + truncated desc + priority attrs only

## JSON reliability

`processing.CompleteJSON`:

1. Call Completer with `max_tokens` + low temperature
2. `StripJSONFences` / isolate `{…}`
3. On parse fail → **one retry** with `INVALID. Reply with ONLY one JSON object…`
4. Call sites fall back to originals/templates instead of storing garbage prose as titles

## Ops tips

- Prefer LAN TCP reachability checks (`Test-NetConnection host -Port 8767`) over ICMP
- Smoke: [ai-full-smoke.md](ai-full-smoke.md), [green-chat-llm.md](green-chat-llm.md)
- CI / no real model: [mock-llm.md](mock-llm.md) (`OPENAI_BASE_URL=http://127.0.0.1:18767/v1`) — includes **translation + processing verification**
- Locale lists + commented `OPENAI_*` / `MOCK_LLM_*` keys: root [`.env.example`](../.env.example)
- Do not commit real keys or LAN secrets