61 lines
2.0 KiB
Markdown
61 lines
2.0 KiB
Markdown
|
# Jobs, queues, and integrations
|
||
|
|
|
||
|
|
## Processing queue
|
||
|
|
|
||
|
|
ASSUMPTION in code: full River client is deferred. Production MVP uses **Postgres processing_jobs** with FOR UPDATE SKIP LOCKED + pg_notify('processing_jobs').
|
||
|
|
|
||
|
|
- Enqueue: internal/jobs.Queue.EnqueueProcessingJob
|
||
|
|
- Workers: internal/processing.JobSlots.Fill → ClaimNext (count from config / ClampProcessingWorkers)
|
||
|
|
- Process starts also hit HTTP rate limits (RPM per company)
|
||
|
|
|
||
|
|
## Worker loop (what runs)
|
||
|
|
|
||
|
|
From apps/api/cmd/worker:
|
||
|
|
|
||
|
|
1. Fill processing job slots
|
||
|
|
2. ProcessPendingAutoJobs (support AI fallback)
|
||
|
|
3. WooCommerce / Shopify ClaimNextPendingJob + sync
|
||
|
|
4. Periodic: EnqueueDueScheduled (stores), RunDueBillingCycles
|
||
|
|
|
||
|
|
API also runs a light RunAutoJobsLoop for support AI — keep **worker** in production.
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
flowchart TB
|
||
|
|
subgraph ingest [Ingest]
|
||
|
|
FeedURL[Feed URL / CSV]
|
||
|
|
Woo[WooCommerce]
|
||
|
|
Shop[Shopify]
|
||
|
|
end
|
||
|
|
subgraph core [Core]
|
||
|
|
Jobs[(processing_jobs)]
|
||
|
|
Worker[cmd/worker]
|
||
|
|
Catalog[(products)]
|
||
|
|
end
|
||
|
|
subgraph out [Outbound]
|
||
|
|
Export[export feeds CSV/XML]
|
||
|
|
StorePush[Woo/Shopify push]
|
||
|
|
end
|
||
|
|
FeedURL --> Catalog
|
||
|
|
Woo --> Catalog
|
||
|
|
Shop --> Catalog
|
||
|
|
Catalog --> Jobs
|
||
|
|
Jobs --> Worker
|
||
|
|
Worker --> Catalog
|
||
|
|
Catalog --> Export
|
||
|
|
Worker --> StorePush
|
||
|
|
```
|
||
|
|
|
||
|
|
## Integrations (where configured)
|
||
|
|
|
||
|
|
| Integration | Preferred config | Notes |
|
||
|
|
|-------------|------------------|-------|
|
||
|
|
| AI (BYOK / OpenAI-compatible) | Tenant /integrations/ai | Encrypted with APP_ENCRYPTION_KEY; optional process OPENAI_* fallback |
|
||
|
|
| Marketing email | /integrations/email | Separate from platform invite SMTP |
|
||
|
|
| Woo / Shopify / feeds | /stores | Woo secrets at rest; Shopify Admin domain SSRF-hardened |
|
||
|
|
| Stripe / EPREL / feed private-URL allowlist | /admin/settings | Env fallbacks exist; prefer UI |
|
||
|
|
| Platform invite SMTP | Process SMTP_* | See docs/ops-runtime.md |
|
||
|
|
|
||
|
|
Support AI auto-reply jobs: table support_auto_jobs → TryAutoReplyLLM after FAQ miss.
|
||
|
|
|
||
|
|
Sources: apps/api/internal/jobs/river.go, apps/api/cmd/worker/main.go, docs/ops-runtime.md.
|