> ## Documentation Index
> Fetch the complete documentation index at: https://docs.evermind.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Memory Pipeline
> How EverOS turns raw messages into durable, searchable memory — from ingestion to MemCell extraction to retrieval.
EverOS operates through two tracks: **Memory Construction** (write path) and **Memory Perception** (read path).
## Memory Construction
This track turns raw conversation streams into structured, retrievable knowledge.
Raw messages enter the system through the API. Multi-modal content (images, PDFs, audio, HTML) is parsed and normalized alongside text.
The system identifies shifts in topics or context to segment the conversation into meaningful units.
Specialized prompts and models extract **MemCells** — the atomic memory units: Episodes, Atomic Facts, Foresight, and Profile updates.
MemCells are integrated by theme and participants to form episodes and profiles.
Data is stored with both keyword (BM25) and semantic (vector) indices for robust retrieval.
**Write path in detail:**
```
External message
│
▼
1. Ingestion Multi-modal parser dispatch
│
▼
2. Extraction Calls EverAlgo — boundary detection + MemCell extraction
│
▼
3. Markdown write Atomic: tmp → fsync → rename ✅ returns here
│
┌───┴─────────────────┐
▼ ▼
4a. SQLite audit 4b. LanceDB sync (async cascade daemon)
```
**Consistency guarantee:** The Markdown write is strongly consistent (fsync). LanceDB is eventually consistent — if unavailable, changes buffer in the SQLite `md_change_state` queue and replay on recovery. LanceDB unavailability never blocks a write response.
## Memory Perception
This track handles how agents retrieve and use stored memories. Four methods are available, each with different performance and dependency trade-offs:
* **`keyword`**: BM25 full-text search — fast, exact term matching, no model dependencies
* **`vector`**: ANN embedding similarity — semantic queries, requires an embedding model
* **`hybrid`(recommended)**: BM25 + vector in parallel, hierarchical fusion where atomic facts and episodes compete for the top-N results, followed by LLM rerank — best recall and precision
* **`agentic`**: Multi-round adaptive retrieval — LLM checks sufficiency and generates follow-up queries if needed
**Read path (hybrid):**
```
User query
│
▼
1. BM25 + vector recall Run concurrently, candidate pool assembled
│
▼
2. Hierarchical fusion Episodes and atomic facts compete for top-N
│
▼
3. LLM rerank Final relevance ordering
│
▼
4. (optional) Fetch Read original Markdown for full context
```
All retrieval runs locally within the same process — no network calls to external services. See [Retrieval](/open-source/retrieval) for full method details.
## The EverAlgo Boundary
Memory extraction algorithms live in a separate library, [EverAlgo](https://github.com/EverMind-AI/EverAlgo), not in EverOS itself. It handles:
* Multi-modal parsing (text, image, audio, doc, PDF, HTML, email)
* Episode / AtomicFact / Foresight / Profile extractors
* Case / Skill extractors for agent memory
EverAlgo is **stateless** and **I/O-free** — pure functions that receive messages and return structured MemCells, with no direct access to Markdown files, SQLite, or LanceDB. This boundary keeps the extraction algorithms reusable across product forms (EverOS OSS, EverMind Cloud, and plugins).
Prompts are injected via `PromptSlot` parameters rather than hardcoded, so every extraction stage is configurable without touching algorithm code.