Multimodal Memory

EverOS goes beyond text. You can attach images, PDFs, audio recordings, and other file types to your messages, and EverOS will extract relevant information from them to enrich the memory.

Supported Content Types

Images

JPG, PNG, GIF, WebPMax size: 10 MB

Documents

PDF, DOC, HTML, TXTMax size: 100 MB

Audio & Video

MP3, WAV, MP4, WebMMax size: 500 MB

How It Works

When you include multimodal content in a message, EverOS preprocesses each file into text: OCR for images, transcription for audio, parsing for PDFs and Office documents. It then extracts and stores relevant information as part of the user’s memory, just like text messages. The key difference from text-only messages is that the content field becomes an array of content items instead of a plain string.

Content Item Structure

Each item in the content array has a type field that determines how EverOS processes it:

Type	Description	Required Fields
`text`	Plain text content	`text`
`image`	Image file	`uri`
`audio`	Audio recording	`uri`
`doc`	Document (TXT, DOC)	`uri`
`pdf`	PDF document	`uri`
`html`	HTML content	`uri`
`email`	Email message	`uri`

The uri field contains an object key returned by the upload flow (see below), not a public URL.

Upload with the Python SDK

The Python SDK handles the upload for you: client.upload(path) presigns, uploads the bytes directly to S3, and returns the object_key to reference in a message.

from everos_cloud import EverOS

client = EverOS(api_key="your_api_key")

# 1) Upload the file — returns an object_key (file_type inferred from the extension)
image_key = client.upload("whiteboard.jpg")

# 2) Reference the key in a message's content array
client.add(
    session_id="session_gs_003",
    messages=[
        {
            "sender_id": "user_demo_001",
            "role": "user",
            "content": [
                {"type": "text", "text": "Here is the whiteboard from today's meeting."},
                {"type": "image", "uri": image_key, "name": "whiteboard.jpg", "ext": "jpg"},
            ],
        },
    ],
)

Upload Flow (raw API)

If you are not using the SDK, uploading multimodal data is a 3-step process:

Get a pre-signed upload URL

Call POST /api/v2/object/sign to get a pre-signed S3 upload URL and an objectKey for your file.

curl -X POST https://api.evermind.ai/api/v2/object/sign \
  -H "Authorization: Bearer <api_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "objectList": [
      {
        "fileId": "my_image_001",
        "fileName": "meeting-whiteboard.png",
        "fileType": "image"
      }
    ]
  }'

Response (key fields):

{
  "result": {
    "data": {
      "objectList": [
        {
          "fileId": "my_image_001",
          "objectKey": "982a8db8c67be277/b3303282-2c49-4564-bb1a-6e415b781d0c",
          "objectSignedInfo": {
            "url": "https://s3.amazonaws.com/bucket-name",
            "fields": {
              "key": "982a8db8c67be277/b3303282-...",
              "policy": "eyJjb25k...",
              "x-amz-algorithm": "AWS4-HMAC-SHA256",
              "x-amz-credential": "...",
              "x-amz-signature": "..."
            }
          }
        }
      ]
    }
  }
}

Save the objectKey. You will need it in Step 3.

Upload the file to S3

Use the returned objectSignedInfo.url and fields to upload your file via an S3 POST request.

curl -X POST "https://s3.amazonaws.com/bucket-name" \
  -F "key=982a8db8c67be277/b3303282-2c49-4564-bb1a-6e415b781d0c" \
  -F "policy=eyJjb25k..." \
  -F "x-amz-algorithm=AWS4-HMAC-SHA256" \
  -F "x-amz-credential=..." \
  -F "x-amz-signature=..." \
  -F "file=@meeting-whiteboard.png"

A successful upload returns HTTP 204 No Content.

Add memory with the objectKey

Call POST /api/v2/memory/add with content as an array. Set the uri field to the objectKey from Step 1.

curl -X POST https://api.evermind.ai/api/v2/memory/add \
  -H "Authorization: Bearer <api_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "session_id": "session-1",
    "messages": [
      {
        "sender_id": "user_alice",
        "role": "user",
        "timestamp": 1711900000000,
        "content": [
          {
            "type": "text",
            "text": "Here is the whiteboard from today meeting"
          },
          {
            "type": "image",
            "uri": "982a8db8c67be277/b3303282-2c49-4564-bb1a-6e415b781d0c",
            "name": "meeting-whiteboard.png",
            "ext": "png"
          }
        ]
      }
    ]
  }'

EverOS will process the image, extract relevant information, and store it as part of the user’s memory.

Batch Upload

You can sign multiple files in a single request by passing multiple items in the objectList array (up to 50):

curl -X POST https://api.evermind.ai/api/v2/object/sign \
  -H "Authorization: Bearer <api_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "objectList": [
      {"fileId": "img_001", "fileName": "photo.jpg", "fileType": "image"},
      {"fileId": "doc_001", "fileName": "report.pdf", "fileType": "file"},
      {"fileId": "vid_001", "fileName": "recording.mp4", "fileType": "video"}
    ]
  }'

Then upload each file to its respective pre-signed URL and include all objectKey values in your message content array.

Mixing Text and Files

A single message can contain both text and multiple file attachments. EverOS processes all content items together to build a complete memory (timestamp is a unix millisecond value):

{
  "sender_id": "user_alice",
  "role": "user",
  "timestamp": 1711900000000,
  "content": [
    {"type": "text", "text": "Summary of the Q3 planning session"},
    {"type": "image", "uri": "<objectKey_1>", "name": "whiteboard.png", "ext": "png"},
    {"type": "pdf", "uri": "<objectKey_2>", "name": "q3-plan.pdf", "ext": "pdf"},
    {"type": "audio", "uri": "<objectKey_3>", "name": "recording.mp3", "ext": "mp3"}
  ]
}

Text items have no limit per message. Non-text items (image, audio, doc, etc.) are limited to 10 per message. The total request body must be under 300 KB (file content is stored on S3, not in the request body).

Content Item Reference

Field	Type	Description
`type`	string	Required. One of: `text`, `image`, `audio`, `doc`, `pdf`, `html`, `email`
`text`	string \| null	Content body. For `type: text`, this is the actual text
`uri`	string \| null	Object key from the upload flow (returned by `POST /api/v2/object/sign` or `client.upload()`)
`name`	string \| null	File name
`ext`	string \| null	File extension (e.g., `png`, `mp3`, `pdf`)
`source`	string \| null	Content source (e.g., `google_doc`, `notion`, `confluence`, `zoom`)
`source_info`	object \| null	Source-related traceability metadata
`extras`	object \| null	Type-specific extra fields

Getting started

Core Concepts

Essential Features

Multimodal Memory

Supported Content Types

Images

Documents

Audio & Video

How It Works

Content Item Structure

Upload with the Python SDK

Upload Flow (raw API)

Batch Upload

Mixing Text and Files

Content Item Reference

Next Steps

Cloud Quickstart

API Reference

​Supported Content Types

Images

Documents

Audio & Video

​How It Works

​Content Item Structure

​Upload with the Python SDK

​Upload Flow (raw API)

​Batch Upload

​Mixing Text and Files

​Content Item Reference

​Next Steps

Cloud Quickstart

API Reference

Supported Content Types

How It Works

Content Item Structure

Upload with the Python SDK

Upload Flow (raw API)

Batch Upload

Mixing Text and Files

Content Item Reference

Next Steps