# Plan — Native Questionnaire Engine, Parts 2–4 (roadmap)

> Status: **ROADMAP, not built.** Part 1 (core engine + sequential builder) is
> **built, deployed, and live** (commits `1dd4afc` schema, `8ad76db` shared
> helpers, `e146587` api+worker runtime; web builder pending publish). This doc
> sketches the next three sub-projects so each can get its own brainstorm →
> design → build cycle. It deliberately reuses the seams Part 1 left.
>
> Read `plans/questionnaire-engine-part1.md` for the foundation and the locked
> decisions. Each part below is **additive** and ships on the same deploy loop
> (shared build → api/worker build → pm2 restart → health → web publish).

## The seams Part 1 already left (what these parts plug into)

- **`start(waId, questionnaireId)`** — a clean entry any trigger/flow invokes.
  Used today by the `start_questionnaire` FlowRule action and (next) by QR.
- **`complete(sessionId, contactId, questionnaire)`** in
  `apps/api/src/questionnaires/questionnaire-engine.service.ts` — currently
  applies `completionTag` and logs a "completed" line. **This is the Part 3
  outcome seam.**
- **`mapToAttribute`** on a question — already writes `contact.attributes[key]`
  (read-merge-write). **This is the Part 2 cohort seam and the Part 4 filter
  source.** Pair it with a `ContactFieldDefinition` (type `select`).
- **`questions: Json` ordered array** — additive shape; new per-question fields
  (branching, AI config) need no migration.
- **`questionnaire_sessions`** (status active/completed/abandoned, `responses`
  history, indexed `[contactId,status]`) — **the Part 4 status source.**
- **FlowRule action union** (`apps/api/src/flows/flow-rules.service.ts`) — the
  extensible trigger/outcome pattern to mirror for Part 3's outcome list.

---

## Part 2 — intake routing / cohort classification

**Goal.** Decide *which* questionnaire (or branch) a contact gets, and tag them
with a cohort: **investor / jobseeker / distributor** (confirm the canonical set
with the operator — fixed vs operator-configurable). Three methods, in rising
complexity:

1. **Keyword (available now, zero new code).** Distinct inbound-keyword
   FlowRules each fire a different `start_questionnaire` action ("invest" →
   investor intake). Document this as the v0 routing; Part 2 only adds the
   nicer methods.
2. **Menu.** A short "router" questionnaire whose first question is a
   `single_choice` ("Who are you?"). The chosen option routes to a cohort +
   sub-flow. Needs **branching** (Part 1 is strictly linear).
3. **AI.** Classify a free-text reply ("Tell us what you're looking for") into a
   cohort with Claude (the `anthropic` connection already exists; use **Haiku**
   for cost). Store the cohort, then route.

**Data-model deltas (additive).**
- Cohort lives in `contact.attributes.contact_type` (a `select`
  `ContactFieldDefinition` seeded with the cohort set) — reuses Part 1's
  `mapToAttribute`, and Segments already filter on attributes (feeds Part 4 for
  free).
- Branching: add an optional per-option target to the Question JSON, e.g.
  `options[].next: { goto?: questionId } | { startQuestionnaireId }`, and/or a
  question-level `next` for non-choice questions. No migration (JSON).

**Runtime.** Make the engine's advance step **branch-aware**: after recording an
answer, if the chosen option/question names a target, follow it (jump to a
question id, or `complete`-then-`start` a sub-questionnaire) instead of linear
`+1`. Add an `ai_classify` question type (or a pre-step) that calls a new
`ClassifierService` (LLM → one of N labels) and writes `contact_type`, then
routes. Keep all of it data-not-exception (a classifier failure falls back to a
menu).

**UI.** Editor gains: per-option routing (pick "next question" / "start
questionnaire"); an AI-classify config (cohort labels + classification prompt +
fallback). Show the branch graph lightly (still linear-first; avoid the
node-graph the operator dislikes).

**Testing.** Pure branch-advance logic (shared, TDD); `ClassifierService` with a
mocked LLM; engine routing (start sub-flow / jump). 

**Open questions.** Fixed vs configurable cohorts? Is AI worth it over a 3-option
menu for v1? Branch model: per-option `goto` vs sub-questionnaire dispatch
(recommend dispatch — simpler, reuses `start`).

---

## Part 3 — outcomes / integrations

**Goal.** On completion (and optionally per-answer), run a configurable list of
outcomes. Part 1 ships exactly one: apply a tag. Part 3 generalizes the
`complete()` seam into an **outcome list** mirroring the FlowRule action union.

**Data-model deltas (additive).** Replace the single `completionTag` with
`questionnaire.outcomes: Json` — an ordered array of:
- `{ kind: "apply_tag", slug }` (what `completionTag` becomes; keep
  `completionTag` working via a back-compat shim or a one-time data migration).
- `{ kind: "notify_agent", to | templateName, ... }` — WhatsApp template to an
  internal agent number (reuse the 01B/reminders agent-notify pattern; resolve
  the destination from config/connection, **not** a hardcoded number — see the
  open 01B "hardcoded test number" item in HANDOFF).
- `{ kind: "create_lead", mapping }` — push answers to LeadRat via
  `LeadRatService` (confirm a `createLead` method exists; reminders only uses
  `reassignLead`/`getLeadStatus` today). Map `responses`/attributes → LeadRat
  fields.
- `{ kind: "email", to, templateName, attachFromAnswer? }` — email a CV/doc.

**Runtime.** `complete()` iterates `outcomes` via a strategy map; each handler is
best-effort, errors logged, never failing completion. The completion event also
becomes the **hand-off point for the future chatbot** (start another flow).

**Integrations / dependencies (flag early).**
- **Email attachment support** must be added to `EmailService` (attach a
  contact-uploaded document — the CV lands via the existing media-download
  pipeline; reference the stored media path). **Blocked on Gmail OAuth2**, which
  HANDOFF lists as still deferred — Part 3's email outcome can't go live until
  that's resolved. Build it behind a feature check.
- Confirm `LeadRatService` surface for lead creation; add it if missing.
- Agent-notify destination: per-questionnaire vs a global setting.

**UI.** An outcomes builder in the editor — mirror `BasicFlowDialog`'s
add/remove action rows (`apps/web/src/routes/Automation.tsx`) with type-specific
config per outcome.

**Testing.** Each outcome handler with mocked services (TagsService /
MessageService / LeadRatService / EmailService); engine runs the full outcome
list on completion (happy + one failing outcome doesn't block the rest).

---

## Part 4 — cohort visibility

**Goal.** Find and segment contacts by **type/cohort** and by **questionnaire
status** ("completed the investor intake", "abandoned mid-flow", "in progress").

**Data-model deltas.** None new. Cohort is `contact.attributes.contact_type`
(Part 2). Status lives in `questionnaire_sessions` (Part 1). Filtering = a join.

**Backend.** Extend the **segment-filter DSL** (`packages/shared/src/contacts/
segment-filter.ts`) + its Prisma query builder with a `questionnaireStatus`
predicate (`{ questionnaireId?, status }` → EXISTS over sessions). Cohort
filtering already works through the attribute filter. Add the matching contacts
API query param.

**UI.** Contacts/Segments filter UI gains "Questionnaire" + "Status" + "Cohort".
Enrich the Part 1 submissions list with status/contact filters. Optionally a
small per-questionnaire dashboard (counts by status, completion rate — numbers
in **DM Sans tabular-nums**).

**Testing.** The new filter predicate (shared, TDD); the contacts/segment query;
the UI filter wiring.

**Open questions.** Should "abandoned" be a first-class, filterable segment
trigger (e.g. re-engagement campaign to everyone who abandoned)? That's a natural
bridge back to Campaigns.

---

## Suggested order & sizing

1. **Part 3 first if outcomes are the urgent business value** (notify agent /
   create lead) — but check the Gmail-OAuth2 / LeadRat-method dependencies up
   front. The `complete()` seam makes it self-contained.
2. **Part 2** unlocks multi-cohort intake; menu-branching is the high-value,
   low-risk slice — defer AI classification to a follow-up unless asked.
3. **Part 4** is the smallest (no new tables) and naturally lands last, once
   cohorts (Part 2) and richer outcomes (Part 3) give it something to filter on.

Each part: brainstorm → write its own `plans/questionnaire-engine-partN.md` →
TDD build → additive migration if any → deploy loop → publish web.