# Agent architecture review (2026-06-11)

Deep-research pass on whether the bot's foundation holds for the planned scope
(listings search/compare/recommend, calculators, 50+ bilingual KB docs, CRM
enrichment, alerts, re-engagement, handoff, voice). 104 research agents,
5 search angles, 11 primary sources fetched, 25 claims adversarially verified
(3 independent skeptics each) → **23 confirmed, 2 refuted and discarded**.
Internals audited alongside (loop, retrieval SQL, memory, prompt growth).

## Verdict

**The foundation is right — do NOT rebuild it.** A single agent + tool loop +
deterministic rails is exactly what current vendor guidance and academic
evidence recommend for a sequential, shared-context conversation like WhatsApp
qualification. What WILL crack at the planned scope is three layers *around*
the loop — the tool set, the monolithic prompt, and lexical-only retrieval —
and all three are cheapest to restructure now, while they're small.

Key verified evidence (all 3-0 unless noted):

- **Single agent is the documented default.** Anthropic (Dec 2024 + Jan 2026):
  "a well-designed single agent with appropriate tools can accomplish far more
  than many developers expect"; add complexity only when it measurably wins.
  Multi-agent costs **3–10× more tokens** (their research system: ~15× chat) —
  fatal to the Haiku/DeepSeek cost goal. Orchestrator/subagent patterns are for
  *parallelizable research*, not conversations.
- **Tool count is the real ceiling — and we're heading straight at it.**
  Degradation starts at **15–20+ tools**; smaller models fail earlier
  (academic evals: DeepSeek-V3.1 optimal at 3–15 tools). The operative rule is
  *ambiguity*, not count: "if a human engineer can't definitively say which
  tool should be used, an AI agent can't be expected to do better."
- **Context rot is architectural.** Recall degrades as the window grows, in
  every model. The 50+ area guides must never enter the prompt wholesale —
  only retrieved excerpts via tools. Compaction + summarized profile (what we
  do) is the named technique for back-and-forth conversation.
- **The prompt should become routed modules, not one growing block.**
  Anthropic's routing workflow + Agent Skills progressive disclosure: classify
  the turn, assemble only the relevant prompt modules. Their own example IS a
  customer-service bot routing query types to different prompts/tools.
  ⚠ Modules must be ordered cache-stably or splitting *breaks prompt caching*
  and costs more.
- **Lexical-only retrieval has three documented failure modes** that compound
  with KB growth: `ts_rank` has no corpus statistics (no IDF — PostgreSQL's
  own docs), vocabulary mismatch (differently-phrased queries never match),
  and complementary blind spots vs vector search. The Postgres-native fix is
  **hybrid: pgvector + existing lexical, fused with Reciprocal Rank Fusion
  (k≈60)** — keep both legs, replace neither.
- **Arabic makes it worse** (2-1 vote — directionally solid, unquantified):
  Postgres has no Arabic stemmer, so Arabic FTS degrades toward exact-match.
  Best verified Arabic-retrieval embeddings (arXiv 2506.06339, Table 2):
  **BGE-M3 (70.99)** and **Multilingual-E5-large (70.31)** — both beat
  Arabic-specialized models; add Qwen3-Embedding to the candidate list (newer
  than the benchmark). Run a small Arabic retrieval eval before committing.

Current internals snapshot (for trajectory): external prompt ~4.3k chars
(~1,060 tokens) and grew ~25% in two days of feature work; 5 tools; loop caps
at 5 rounds; memory = free-text ≤700 chars on the contact.

## What to KEEP (verified as correct)

- Single-agent `runToolLoop` + deterministic pre-prompt audience routing.
- Deterministic rails outside the agent (consent, marker stripping, fallback).
- 30-message window + 4h session gap + summarized durable profile (compaction
  is the right mechanism family for this task shape).
- Dry-run Playground + eval harness — **the harness gates everything below.**

## RESTRUCTURE NOW (cheap while small) — in this order

1. **Modularize the system prompt behind the existing router.** Intent-keyed
   modules (identity/core · lead-capture+temperature · interactive syntax ·
   job-seeker · calculators · area-guide guidance · operator steering),
   assembled per turn; deterministic intent route first (extend the existing
   audience router), an LLM classifier only if evals demand it. Order modules
   so the cache prefix stays stable (core first, volatile last — operator
   steering is already last today).
2. **Fix the tool roadmap before building it: ~8–12 consolidated, namespaced,
   non-overlapping workflow tools** — e.g. ONE `listing_search` that handles
   search/details/compare via parameters (Anthropic's `schedule_event`
   pattern), `crm_lead` + ONE `crm_write` (tags/segments/summaries),
   `kb_search`, `calc_mortgage`. Namespace by service (`crm_*`, `listing_*`,
   `kb_*`) — measured "non-trivial" eval effects, varying by model, so
   re-validate after any model swap. Cap/paginate/truncate tool results.
3. **Conditional tool exposure**: only the routed intent's tool subset enters
   the request (the Tool Search Tool result — up to 85% token reduction AND
   better selection — applied to our own loop as a static per-intent map).
4. **Hybrid retrieval BEFORE the 50-doc KB lands**: pgvector column on
   knowledge_chunks (additive migration), BGE-M3 or mE5-large embeddings
   (self-hostable; benchmark Qwen3-Embedding too), RRF fusion with the
   existing lexical leg, all behind the same `search_knowledge` tool. Do the
   Arabic mini-eval first; the pipeline shapes how the KB pack gets chunked,
   which is why this precedes content import at scale.
5. **Settle the two evidence-gaps empirically in the eval harness** (research
   found NO surviving evidence either way):
   - Structured lead-state (typed slots: budget/area/beds/timeline/finance)
     vs the current free-text profile → A/B as eval scenarios.
   - DeepSeek-V4 vs Haiku tool-calling at 8–12 tools → expand the harness
     with tool-selection assertions and gate the swap on the comparison.

## PREMATURE (explicitly do not build)

- Multi-agent / orchestrator-subagent (3–10× tokens, wrong task shape).
- Replacing the deterministic audience router with an LLM router.
- Embedding infrastructure beyond pgvector-in-existing-Postgres.
- Hot-lead alerts, re-engagement, after-hours awareness as *agent tools* —
  these are deterministic code/n8n jobs **triggered by** agent outputs
  (temperature, timestamps), per the simplest-solution principle.

## Caveats (honest limits of the research)

- Vendor concentration: the topology findings lean on Anthropic's guidance
  (key thresholds independently corroborated by academic evals; specific
  percentages are vendor "up to" benchmarks).
- Questions 3 (memory structure) and 5 (DeepSeek reliability) produced no
  surviving verified evidence — answered empirically via the harness, above.
- No source quantifies the corpus size where lexical-only fails; the argument
  is failure-*modes*, not a size threshold. Both claims about Postgres Arabic
  FTS configuration specifics were REFUTED (unreliable source) — hence the
  mini-eval before budgeting the Arabic work.

## Sources (primary, verified live)

- anthropic.com/research/building-effective-agents
- claude.com/blog/building-multi-agent-systems-when-and-how-to-use-them
- anthropic.com/engineering/multi-agent-research-system
- anthropic.com/engineering/writing-tools-for-agents
- anthropic.com/engineering/advanced-tool-use
- anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
- anthropic.com/engineering/effective-context-engineering-for-ai-agents
- arxiv.org/abs/2506.06339 (Arabic embedding benchmark) · postgresql.org FTS
  docs · paradedb.com + jkatz05.com + supabase.com hybrid-search references
- Academic corroboration: arXiv 2605.24660, 2509.23141 (tool-count
  degradation), 2511.14650 (tool consolidation gains), Chroma 18-model
  context-rot study