Knowledge Wiki

The Knowledge Wiki turns unstructured documents (Markdown, PDF, DOCX, and more) into a searchable topic library. Upload a file and EverOS extracts a structured topic tree with an LLM, classifies it into a taxonomy, indexes it for keyword + vector search, and keeps the original file for reference. It is a distinct memory track from conversation memory: the Wiki is reference material you deliberately upload, not experience the agent extracts from talking. All endpoints live under /api/v1/knowledge.

Three-Tier Hierarchy

Instead of slicing every document into fixed-length chunks like classic RAG, the Wiki organizes content into three levels, from broadest to most granular. Each tier maps to its own endpoint, so an agent can navigate top-down and pull detail only when it needs it.

Tier	What it is	Endpoint	Returns
L0 · Category	Taxonomy bucket (e.g. `Technology`, `Finance`)	`GET /categories`	`category_id`, `description`, `document_count`
L1 · Document	One uploaded file = one document	`GET /documents`	`doc_id`, `title`, `category_id`, `topic_count`
L2 · Topic	LLM-extracted section with full content	`GET /topics/{id}`	`content`, `labels`, tree position

Quick Start

Examples assume EverOS is running on the default port 8000.

curl -s -X POST http://127.0.0.1:8000/api/v1/knowledge/documents \
  -F "file=@my-report.pdf" \
  -F "title=Q1 Engineering Report" \
  | jq .data
# → { "doc_id": "d_a1b2c3d4e5f6", "category_id": "Technology", "topic_count": 8, ... }

curl -s -X POST http://127.0.0.1:8000/api/v1/knowledge/search \
  -H "Content-Type: application/json" \
  -d '{"query": "performance bottleneck", "method": "hybrid"}' \
  | jq '.data.hits[:3] | .[] | {topic_name, score}'

from pathlib import Path
import httpx

async def upload_document(file_path: str, title: str) -> dict:
    async with httpx.AsyncClient(base_url="http://127.0.0.1:8000") as client:
        with open(file_path, "rb") as f:
            resp = await client.post(
                "/api/v1/knowledge/documents",
                files={"file": (Path(file_path).name, f)},
                data={"title": title},
            )
        resp.raise_for_status()
        return resp.json()["data"]

Responses use the envelope {"request_id": "...", "data": {...}}.

Storage Layout

Every document is a self-contained directory. Markdown is the single source of truth; SQLite and LanceDB are derived indexes the cascade daemon builds automatically. Even if the indexes are lost, they rebuild fully from the Markdown.

<root>/<app>/<project>/knowledge/
├── .taxonomy.md                          ← category definitions (YAML)
├── Technology/
│   └── Q1_Engineering_Report_d_a1b2c3d4e5f6/
│       ├── index.md                      ← document metadata + LLM summary
│       ├── 1_Performance_Analysis.md     ← topic with full content
│       ├── 2_Infrastructure_Costs.md
│       └── _original/                    ← original uploaded file, unchanged
│           └── my-report.pdf
└── Finance/
    └── Budget_Review_d_f6e5d4c3b2a1/
        └── ...

Store	What it holds	Role
Markdown	Document metadata, summaries, topic content, original files	Single source of truth; human-readable and editable
SQLite	Document and topic rows, change queue	Structured queries, paginated lists, counts
LanceDB	Topic vectors, BM25 tokens, scalar fields	Search index (rebuildable from Markdown)

Taxonomy

L0 categories are not produced by unsupervised clustering. They come from a predefined taxonomy in .taxonomy.md at the knowledge root, and on upload an LLM picks the best-matching category from that list, making classification predictable, auditable, and overridable. EverOS ships with 20 default categories (Technology, Science, Medical, Finance, Legal, …) plus an Others fallback.

Editable: add, rename, or remove categories by editing .taxonomy.md. Changes hot-reload (read on every upload and category request), so no restart or reindex is needed.
Auto or manual: omit category_id on upload to let the LLM choose, or pass one to bypass classification.
Others fallback: anything that doesn’t match a defined category lands here.
Directory follows category: a PATCH that changes category_id moves the whole document directory (the _original/ folder follows).

.taxonomy.md

---
kind: knowledge_taxonomy
categories:
  - id: Technology
    description: Computer science, software engineering, AI/ML.
  - id: InternalOps
    description: Company-specific operational procedures and runbooks.
---

Document Lifecycle

Operation	Endpoint	Behavior
Upload	`POST /documents`	Multipart upload → LLM classify (or manual) → extract topic tree → write Markdown → cascade syncs the index
Replace	`PUT /documents/{id}`	Atomic; if extraction fails, the old document is restored from backup
Update metadata	`PATCH /documents/{id}`	Edit `title` and `category_id`; a category change moves the directory
Delete	`DELETE /documents/{id}`	Idempotent (204 if absent); removes the entire document directory including `_original/`

Search

Search runs a five-stage pipeline rather than a flat vector lookup:

query → embed → keyword (BM25) ┐
                vector (ANN)    ├→ RRF fusion → cross-encoder rerank → top_k
                                ┘

Three methods are available via the method field:

Method	How it works
`keyword`	BM25 sparse retrieval over tokenized summary + content
`vector`	Dense ANN over embedded summary vectors
`hybrid` (default)	Parallel keyword + vector, fused with RRF, then cross-encoder rerank

All three methods embed the query and apply cross-encoder reranking, with a small category-aware boost (lambda, default 0.1). Results carry their L1 document context (title, summary) so no second query is needed; set include_content: true to inline full topic content (default false; drill down via GET /topics/{id} instead).

Knowledge search requires both an embedding and a rerank provider; there is no provider-free fallback (by design: no silent degradation). A missing provider returns 500 CONFIGURATION_ERROR (set EVEROS_EMBEDDING__* / EVEROS_RERANK__*); a configured provider failing at call time returns 503 EXTERNAL_SERVICE_UNAVAILABLE (retryable).

Search tuning lives in [knowledge.search] (overridable via EVEROS_KNOWLEDGE__SEARCH__*): recall_n (recall pool per channel, 200), rerank_n (candidates reranked, 50), lambda (category boost, 0.1), top_k_cap (100).

Supported Formats

Text files are accepted natively; binary formats require the multimodal extra (pip install 'everos[multimodal]', which depends on LibreOffice for document conversion).

Class	Formats	Needs `[multimodal]`
Text	`.txt`, `.md`, `.csv`, `.tsv`, `.vtt`	No
Documents	`.pdf`, `.docx`, `.doc`, `.rtf`, `.odt`	Yes
Spreadsheets / Slides	`.xlsx`, `.xls`, `.pptx`, `.ppt`	Yes
Web	`.html`, `.htm`, `.eml`	Yes
Images (OCR)	`.png`, `.jpg`, `.webp`, `.svg`	Yes
Audio (transcription)	`.mp3`, `.wav`, `.m4a`, `.flac`	Yes

See Multimodal Memory for parser configuration.

Cascade Sync

The cascade daemon watches the knowledge Markdown directory and keeps SQLite + LanceDB in sync: a file write is detected, dispatched by type (index.md → document metadata, N_topic.md → tokenize + embed + index), and a SHA-256 content digest skips re-embedding unchanged files. Typical latency from file write to search availability is 1–3 seconds.

Errors & Multi-Tenancy

Errors return a uniform envelope with a machine-readable code and a human-readable message:

HTTP	Code	Scenario
404	`NOT_FOUND`	Document or topic doesn’t exist
409	`CONFLICT`	`doc_id` already exists (use `PUT` to replace)
415	`UNSUPPORTED_FORMAT`	File format not parseable
422	`INVALID_INPUT`	Empty/oversized query, empty title, bad ID
500	`CONFIGURATION_ERROR`	Embedding or rerank provider not configured
503	`EXTERNAL_SERVICE_UNAVAILABLE`	Configured provider failing at call time
503	`CAPABILITY_UNAVAILABLE`	`everos[multimodal]` not installed

Every endpoint accepts app_id and project_id (both default to "default"). Storage paths, SQLite rows, and LanceDB indexes are fully isolated per tenant pair.

Design Principles

Three principles run through the Wiki: progressive disclosure (navigate L0→L1→L2, pull content on demand), human-readable and editable (Markdown is the source of truth), and determinism (taxonomy-based classification is predictable and auditable).

Introduction

Getting Started

Core Concepts

Advanced

Knowledge Wiki

Three-Tier Hierarchy

Quick Start

Storage Layout

Taxonomy

Document Lifecycle

Search

Supported Formats

Cascade Sync

Errors & Multi-Tenancy

Design Principles

​Three-Tier Hierarchy

​Quick Start

​Storage Layout

​Taxonomy

​Document Lifecycle

​Search

​Supported Formats

​Cascade Sync

​Errors & Multi-Tenancy

​Design Principles

Three-Tier Hierarchy

Quick Start

Storage Layout

Taxonomy

Document Lifecycle

Search

Supported Formats

Cascade Sync

Errors & Multi-Tenancy

Design Principles