Recall, explained: five retrieval arms, one deterministic ranking

Ask most memory systems how they retrieve and the answer is one word: embeddings. Vector similarity is a good signal, but it is one signal, and it misses exact identifiers, misses structure, and has no idea what "last sprint" means. Recall in Illumina treats retrieval as a fusion problem instead.

Four arms, one query

Every recall fans out across four retrieval arms in parallel:

• Semantic — pgvector HNSW search over 384-dimension bge-small-en-v1.5 embeddings. Catches paraphrases and conceptual matches.
• Lexical — BM25 over native PostgreSQL tsvector. Catches the things embeddings fumble: ticket numbers, function names, exact phrases.
• Graph — link expansion through the knowledge graph, pulling in facts connected to the candidates by entity and causal edges.
• Temporal — joins only when the query carries temporal intent. "What changed after the March incident" activates it; "how does auth work" does not.

Each arm returns a ranked candidate list. Reciprocal Rank Fusion merges them with k=60 and a locked arm order, so a fact that ranks well across multiple arms beats a fact that ranks first in one and nowhere else.

A fifth arm joins conditionally. When the query reads as thematic rather than specific — "what have we learned about onboarding", not "what is the retry timeout" — recall also retrieves over community summaries: reports Illumina writes for each cluster it finds in the entity graph. Ten near-duplicate snippets are the wrong answer to a question about a theme, and the community arm is what turns that into one.

Rerank, then score

The fused top 300 candidates go through a cross-encoder — bge-reranker-base, served over Text Embeddings Inference — which reads the query and the candidate together rather than comparing pre-computed vectors. That is where most of the precision comes from.

The final score multiplies normalized relevance by three boosts: recency, temporal proximity, and corroboration. Corroboration matters more than it sounds — a fact backed by more independent sources genuinely ranks higher than a one-off mention. And each boost is designed so that a missing signal is neutral, never distorting: a fact with no temporal data is not penalized on a query with no temporal intent.

Deterministic, on purpose

Run the same recall twice and you get byte-identical output. Final ordering uses a total-order float comparison with a stable id tie-break, so there is no dependence on insertion order, parallel scheduling, or floating-point drift between runs.

If you are building agents on top of a memory system, this is not a nicety. Deterministic retrieval means reproducible agent behavior, debuggable failures, and evals that measure your changes instead of retrieval noise.

{
  "query": "why did we move ingestion off the shared queue",
  "results": [
    {
      "content": "Ingestion moved to an isolated task queue after backfills starved interactive writes",
      "score": 0.9312,
      "corroboration": 3,
      "arms": ["semantic", "lexical", "graph"]
    }
  ],
  "degraded": false
}

Failure is a mode, not an outage

Retrieval infrastructure fails in pieces. If the embedding service is down, recall does not fail with it — it skips the semantic and temporal arms, answers from BM25 plus graph expansion off the top lexical hits, and flags the response as degraded. If the reranker is down instead, the fused RRF ordering stands in with neutral scores. Either way your agent keeps working, and it knows the answer came from a reduced pipeline.

Every arm runs inside the same PostgreSQL database — no separate vector store, no graph engine, no fan-out across services that can half-fail. One query planner, one transaction boundary, one thing to operate.

Recall is available through the REST API, the Python, TypeScript, and Rust SDKs, the CLI, and as an MCP tool your agent can call directly.