Drop one-shot tmp/axe scripts and agent i18n scratch so the Gitea tree is deployable.
146 lines
6.1 KiB
Markdown
146 lines
6.1 KiB
Markdown
# Support auto-reply (FAQ → AI fallback)
|
||
|
||
End-to-end path: **ticket create → `MatchAutoReply` (FAQ/template) → AI fallback (`TryAutoReplyLLM`) → human queue on failure**.
|
||
|
||
## Status (agent 10 integration)
|
||
|
||
| Gate | Result |
|
||
|------|--------|
|
||
| Migrations `031`–`035` | Applied (`goose` → version **35**) |
|
||
| `go test ./internal/support/... ./internal/httpapi/...` | Pass |
|
||
| `npm run check` (svelte-check) | Pass (0 errors) |
|
||
| `go build ./cmd/api ./cmd/worker` | Pass |
|
||
|
||
**Wiring filled by agent 10:** duplicate `031` migrations renumbered; goose `StatementBegin` around `DO $$`; worker + API poll `ProcessPendingAutoJobs` / `RunAutoJobsLoop` so AI jobs drain.
|
||
|
||
## Flow
|
||
|
||
```
|
||
POST /api/support/tickets
|
||
│
|
||
▼
|
||
Create (tags/SKU/context from 031)
|
||
│
|
||
▼
|
||
MaybeAutoReplyOnCreate
|
||
│
|
||
├─ FAQ enabled + confidence ≥ threshold
|
||
│ → PostMatchedAutoReply (system, labeled)
|
||
│
|
||
├─ miss / below threshold + ai_enabled
|
||
│ → EnqueueAIFallback → support_auto_jobs
|
||
│ → poller: TryAutoReplyLLM → CompleterSupportAI
|
||
│ ├─ draft (internal note) or auto_send (public)
|
||
│ └─ fail / low conf → handoff (needs_human)
|
||
│
|
||
└─ auto off → skipped → human inbox
|
||
```
|
||
|
||
## Configure FAQ / templates
|
||
|
||
1. Platform admin → **`/admin/support/knowledge`** (or APIs below).
|
||
2. Create a **KB article** with `keywords` / `intent_keys` / `category_slugs`, set **`is_published=true`**.
|
||
3. Optionally add a **reply template** (`is_active=true`). Placeholders: `{{subject}}`, `{{category}}`.
|
||
4. Open **auto-config** and set:
|
||
- `enabled: true`
|
||
- `faq_enabled: true`
|
||
- `match_confidence_threshold` (default **0.78**, range 0.50–0.95)
|
||
|
||
APIs (RequirePlatformAdmin):
|
||
|
||
| Method | Path |
|
||
|--------|------|
|
||
| GET/POST | `/api/admin/support/kb/articles` |
|
||
| GET/PATCH/DELETE | `/api/admin/support/kb/articles/{id}` |
|
||
| GET/POST | `/api/admin/support/templates` |
|
||
| GET/PATCH/DELETE | `/api/admin/support/templates/{id}` |
|
||
| GET/PUT | `/api/admin/support/auto-config` |
|
||
|
||
Client: `apps/web/src/lib/support/admin-kb-api.ts`.
|
||
|
||
## Configure admin AI (fallback)
|
||
|
||
1. **`/admin/settings` → AI roles** → role **`support`**.
|
||
2. Set provider / base URL / model / API key; enable the role.
|
||
3. In support auto-config:
|
||
- `ai_enabled: true`
|
||
- `ai_delivery`: `draft` (staff-only note) or `auto_send` (customer-visible, labeled)
|
||
- `ai_confidence_threshold` (default **0.65**)
|
||
- `ai_use_global_support_role: true` (uses platform `ai_roles.support`; no parallel BYOK)
|
||
|
||
Runtime gate: `support.TryAutoReplyLLM` → `CompleterSupportAI` → `aiprovider.ResolveCompleterForRole(..., RoleSupport)`.
|
||
|
||
## Runtime / migrate
|
||
|
||
```bash
|
||
cd apps/api
|
||
make migrate-up # DATABASE_URL from Makefile / .env
|
||
go test ./internal/support/... ./internal/httpapi/...
|
||
go build ./cmd/api ./cmd/worker
|
||
```
|
||
|
||
```bash
|
||
cd apps/web
|
||
npm run check
|
||
```
|
||
|
||
**Processes:** API starts a lightweight `RunAutoJobsLoop` (2s / batch 3). Worker also calls `ProcessPendingAutoJobs` each poll tick. For production, keep the **worker** running so AI jobs are not only processed by the API process.
|
||
|
||
## Manual verification checklist
|
||
|
||
### A. FAQ match → auto reply
|
||
|
||
1. Enable auto-config (`enabled` + `faq_enabled`).
|
||
2. Publish a KB article with distinctive keywords (e.g. `refund policy`).
|
||
3. As a customer, create a ticket whose subject/body includes those keywords.
|
||
4. **Expect:** thread shows a **system** message (`is_auto_reply`, footer “Automated answer from help center”); `auto_reply_status=matched`.
|
||
|
||
### B. Obscure question → AI reply or draft
|
||
|
||
1. Keep FAQ on; enable `ai_enabled`; configure AI role **`support`**.
|
||
2. Create a ticket with nonsense / no keyword overlap.
|
||
3. Wait a few seconds for the job poller (API or worker).
|
||
4. **Expect:**
|
||
- `ai_delivery=draft` → internal AI draft note (`auto_reply_status=ai_draft`); staff can approve at `/admin/support/[id]`.
|
||
- `ai_delivery=auto_send` → public labeled AI reply (`ai_sent`).
|
||
|
||
### C. AI fail → human queue
|
||
|
||
1. Disable the `support` AI role **or** point it at a bad key/model, with `ai_enabled=true`.
|
||
2. Create an unmatched ticket.
|
||
3. **Expect:** job fails / handoff; `auto_reply_status` in `handed_off` / `failed` / `skipped`; ticket appears in staff inbox filters **needs_human** (`flag=needs_human`). No customer-facing fake answer.
|
||
|
||
## Doc index
|
||
|
||
| Doc | Agent | Topic |
|
||
|-----|-------|-------|
|
||
| [01-inventory.md](./01-inventory.md) | 1 | Desk + admin AI inventory |
|
||
| [02-contract.md](./02-contract.md) | 2 | Product contract |
|
||
| [03-kb-auto-reply.md](./03-kb-auto-reply.md) | 3 | FAQ matcher |
|
||
| [04-ai-fallback.md](./04-ai-fallback.md) | 4 | AI assist |
|
||
| [05-ticket-detail.md](./05-ticket-detail.md) | 5 | Rich ticket fields |
|
||
| [06-admin-kb-settings-ui.md](./06-admin-kb-settings-ui.md) | 6 | Admin KB UI |
|
||
| [07-user-support-detail-ui.md](./07-user-support-detail-ui.md) | 7 | Customer UX |
|
||
| [08-staff-auto-ai-ui.md](./08-staff-auto-ai-ui.md) | 8 | Staff drafts / filters |
|
||
| [09-security-perf.md](./09-security-perf.md) | 9 | Rate limits / idempotency |
|
||
|
||
## Schema series
|
||
|
||
| File | Purpose |
|
||
|------|---------|
|
||
| `031_support_ticket_detail.sql` | Tags, categories, activity, auto_* ticket/message cols |
|
||
| `032_support_kb_auto_reply.sql` | KB + templates + `support_auto_config` |
|
||
| `033_support_auto_ai_config.sql` | AI columns on auto-config |
|
||
| `034_support_auto_jobs.sql` | Async AI job queue |
|
||
| `035_support_auto_security_perf.sql` | One public auto-reply per ticket + indexes |
|
||
|
||
## Remaining blockers / caveats
|
||
|
||
1. **Manual E2E against a live LLM was not executed here** — needs a real `support` AI key and browser/session. Use the checklist above.
|
||
2. **CSAT public tokens** remain unwired (pre-existing inventory gap).
|
||
3. **Admin AI role probe** `POST /api/admin/settings/ai-roles/{role}/test` still optional/not mounted (404 skipped in client).
|
||
4. Default **`enabled=false`** / **`ai_enabled=false`** — auto-reply stays off until an admin turns it on (safe default).
|
||
5. If only the API runs and the process is under heavy load, prefer also running **`cmd/worker`** so AI jobs keep draining.
|
||
|
||
No git commit (per task).
|