> ## Documentation Index
> Fetch the complete documentation index at: https://docs.evermind.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Wiki

> Turn uploaded documents into a searchable topic library: an LLM-extracted L0→L1→L2 hierarchy, classified into a taxonomy, and persisted as human-readable Markdown.

The Knowledge Wiki turns unstructured documents (Markdown, PDF, DOCX, and more) into a searchable topic library. Upload a file and EverOS extracts a structured topic tree with an LLM, classifies it into a taxonomy, indexes it for keyword + vector search, and keeps the original file for reference.

It is a distinct memory track from conversation memory: the Wiki is reference material you deliberately upload, not experience the agent extracts from talking. All endpoints live under `/api/v1/knowledge`.

## Three-Tier Hierarchy

Instead of slicing every document into fixed-length chunks like classic RAG, the Wiki organizes content into three levels, from broadest to most granular. Each tier maps to its own endpoint, so an agent can navigate top-down and pull detail only when it needs it.

| Tier              | What it is                                     | Endpoint           | Returns                                         |
| ----------------- | ---------------------------------------------- | ------------------ | ----------------------------------------------- |
| **L0 · Category** | Taxonomy bucket (e.g. `Technology`, `Finance`) | `GET /categories`  | `category_id`, `description`, `document_count`  |
| **L1 · Document** | One uploaded file = one document               | `GET /documents`   | `doc_id`, `title`, `category_id`, `topic_count` |
| **L2 · Topic**    | LLM-extracted section with full content        | `GET /topics/{id}` | `content`, `labels`, tree position              |

## Quick Start

<Note>Examples assume EverOS is running on the default port `8000`.</Note>

<CodeGroup>
  ```bash Upload theme={null}
  curl -s -X POST http://127.0.0.1:8000/api/v1/knowledge/documents \
    -F "file=@my-report.pdf" \
    -F "title=Q1 Engineering Report" \
    | jq .data
  # → { "doc_id": "d_a1b2c3d4e5f6", "category_id": "Technology", "topic_count": 8, ... }
  ```

  ```bash Search theme={null}
  curl -s -X POST http://127.0.0.1:8000/api/v1/knowledge/search \
    -H "Content-Type: application/json" \
    -d '{"query": "performance bottleneck", "method": "hybrid"}' \
    | jq '.data.hits[:3] | .[] | {topic_name, score}'
  ```

  ```python Python theme={null}
  from pathlib import Path
  import httpx

  async def upload_document(file_path: str, title: str) -> dict:
      async with httpx.AsyncClient(base_url="http://127.0.0.1:8000") as client:
          with open(file_path, "rb") as f:
              resp = await client.post(
                  "/api/v1/knowledge/documents",
                  files={"file": (Path(file_path).name, f)},
                  data={"title": title},
              )
          resp.raise_for_status()
          return resp.json()["data"]
  ```
</CodeGroup>

Responses use the envelope `{"request_id": "...", "data": {...}}`.

## Storage Layout

Every document is a self-contained directory. Markdown is the single source of truth; SQLite and LanceDB are derived indexes the cascade daemon builds automatically. Even if the indexes are lost, they rebuild fully from the Markdown.

```
<root>/<app>/<project>/knowledge/
├── .taxonomy.md                          ← category definitions (YAML)
├── Technology/
│   └── Q1_Engineering_Report_d_a1b2c3d4e5f6/
│       ├── index.md                      ← document metadata + LLM summary
│       ├── 1_Performance_Analysis.md     ← topic with full content
│       ├── 2_Infrastructure_Costs.md
│       └── _original/                    ← original uploaded file, unchanged
│           └── my-report.pdf
└── Finance/
    └── Budget_Review_d_f6e5d4c3b2a1/
        └── ...
```

| Store    | What it holds                                               | Role                                                |
| -------- | ----------------------------------------------------------- | --------------------------------------------------- |
| Markdown | Document metadata, summaries, topic content, original files | Single source of truth; human-readable and editable |
| SQLite   | Document and topic rows, change queue                       | Structured queries, paginated lists, counts         |
| LanceDB  | Topic vectors, BM25 tokens, scalar fields                   | Search index (rebuildable from Markdown)            |

## Taxonomy

L0 categories are not produced by unsupervised clustering. They come from a predefined taxonomy in `.taxonomy.md` at the knowledge root, and on upload an LLM picks the best-matching category from that list, making classification predictable, auditable, and overridable. EverOS ships with **20 default categories** (Technology, Science, Medical, Finance, Legal, …) plus an `Others` fallback.

* **Editable**: add, rename, or remove categories by editing `.taxonomy.md`. Changes **hot-reload** (read on every upload and category request), so no restart or reindex is needed.
* **Auto or manual**: omit `category_id` on upload to let the LLM choose, or pass one to bypass classification.
* **`Others` fallback**: anything that doesn't match a defined category lands here.
* **Directory follows category**: a `PATCH` that changes `category_id` moves the whole document directory (the `_original/` folder follows).

```yaml .taxonomy.md theme={null}
---
kind: knowledge_taxonomy
categories:
  - id: Technology
    description: Computer science, software engineering, AI/ML.
  - id: InternalOps
    description: Company-specific operational procedures and runbooks.
---
```

## Document Lifecycle

| Operation           | Endpoint                 | Behavior                                                                                                    |
| ------------------- | ------------------------ | ----------------------------------------------------------------------------------------------------------- |
| **Upload**          | `POST /documents`        | Multipart upload → LLM classify (or manual) → extract topic tree → write Markdown → cascade syncs the index |
| **Replace**         | `PUT /documents/{id}`    | Atomic; if extraction fails, the old document is restored from backup                                       |
| **Update metadata** | `PATCH /documents/{id}`  | Edit `title` and `category_id`; a category change moves the directory                                       |
| **Delete**          | `DELETE /documents/{id}` | Idempotent (204 if absent); removes the entire document directory including `_original/`                    |

## Search

Search runs a five-stage pipeline rather than a flat vector lookup:

```
query → embed → keyword (BM25) ┐
                vector (ANN)    ├→ RRF fusion → cross-encoder rerank → top_k
                                ┘
```

Three methods are available via the `method` field:

| Method               | How it works                                                         |
| -------------------- | -------------------------------------------------------------------- |
| `keyword`            | BM25 sparse retrieval over tokenized summary + content               |
| `vector`             | Dense ANN over embedded summary vectors                              |
| `hybrid` *(default)* | Parallel keyword + vector, fused with RRF, then cross-encoder rerank |

All three methods embed the query and apply cross-encoder reranking, with a small category-aware boost (`lambda`, default `0.1`). Results carry their L1 document context (`title`, `summary`) so no second query is needed; set `include_content: true` to inline full topic content (default `false`; drill down via `GET /topics/{id}` instead).

<Warning>
  Knowledge search requires **both** an embedding and a rerank provider; there is no provider-free fallback (by design: no silent degradation). A missing provider returns `500 CONFIGURATION_ERROR` (set `EVEROS_EMBEDDING__*` / `EVEROS_RERANK__*`); a configured provider failing at call time returns `503 EXTERNAL_SERVICE_UNAVAILABLE` (retryable).
</Warning>

Search tuning lives in `[knowledge.search]` (overridable via `EVEROS_KNOWLEDGE__SEARCH__*`): `recall_n` (recall pool per channel, `200`), `rerank_n` (candidates reranked, `50`), `lambda` (category boost, `0.1`), `top_k_cap` (`100`).

## Supported Formats

Text files are accepted natively; binary formats require the multimodal extra (`pip install 'everos[multimodal]'`, which depends on LibreOffice for document conversion).

| Class                 | Formats                                 | Needs `[multimodal]` |
| --------------------- | --------------------------------------- | :------------------: |
| Text                  | `.txt`, `.md`, `.csv`, `.tsv`, `.vtt`   |          No          |
| Documents             | `.pdf`, `.docx`, `.doc`, `.rtf`, `.odt` |          Yes         |
| Spreadsheets / Slides | `.xlsx`, `.xls`, `.pptx`, `.ppt`        |          Yes         |
| Web                   | `.html`, `.htm`, `.eml`                 |          Yes         |
| Images (OCR)          | `.png`, `.jpg`, `.webp`, `.svg`         |          Yes         |
| Audio (transcription) | `.mp3`, `.wav`, `.m4a`, `.flac`         |          Yes         |

See [Multimodal Memory](/open-source/multimodal) for parser configuration.

## Cascade Sync

The cascade daemon watches the knowledge Markdown directory and keeps SQLite + LanceDB in sync: a file write is detected, dispatched by type (`index.md` → document metadata, `N_topic.md` → tokenize + embed + index), and a SHA-256 content digest skips re-embedding unchanged files. Typical latency from file write to search availability is **1–3 seconds**.

## Errors & Multi-Tenancy

Errors return a uniform envelope with a machine-readable `code` and a human-readable `message`:

| HTTP | Code                           | Scenario                                       |
| ---- | ------------------------------ | ---------------------------------------------- |
| 404  | `NOT_FOUND`                    | Document or topic doesn't exist                |
| 409  | `CONFLICT`                     | `doc_id` already exists (use `PUT` to replace) |
| 415  | `UNSUPPORTED_FORMAT`           | File format not parseable                      |
| 422  | `INVALID_INPUT`                | Empty/oversized query, empty title, bad ID     |
| 500  | `CONFIGURATION_ERROR`          | Embedding or rerank provider not configured    |
| 503  | `EXTERNAL_SERVICE_UNAVAILABLE` | Configured provider failing at call time       |
| 503  | `CAPABILITY_UNAVAILABLE`       | `everos[multimodal]` not installed             |

Every endpoint accepts `app_id` and `project_id` (both default to `"default"`). Storage paths, SQLite rows, and LanceDB indexes are fully isolated per tenant pair.

## Design Principles

Three principles run through the Wiki: **progressive disclosure** (navigate L0→L1→L2, pull content on demand), **human-readable and editable** (Markdown is the source of truth), and **determinism** (taxonomy-based classification is predictable and auditable).