btbytes.com

LLMs as Operating System - Agent Memory

2 min read

By Charles Packer and Sarah Wooders from Letta who wrote the MemGPT: Towards LLMs as Operating Systems paper. Course website

  • Self-editing memory
  • agents framework that allows build and deploy persistent agents as service
  • MemGPT paper introduced the concept of “self-editing memory” for LLMs
  • MemGPT is also a type of agent design inspired by Operating Systems, which has two tiers of memory,..

Chatbots vs Agents

  • agents act autonmously in multi-steps, autonomously.
  • agentic loop (agent state → llm context window → llm inference → agent state)
  • context compilation. put agent state into the context window
  • self editing? memory_replace(s/sarah/charles/)

MemGPT:

  • use LLM to build agent
  • context window of an LLM
  • 0 context window
  • user data
  • message history
  • external data sources
  • tools calls
  • reasoning/chain of thought

Key Ideas behind MemGPT:

  • Self-editing memory
  • Inner Thoughts
  • Every output is a tool
  • Looping via heartbeats

put all the above together makes “MemGPT Agent”. they are self-improving because they can edit their long term memory.

Heartbeat requests next action, thus resulting in a loop.

Agent State = Memories + Tools + Messages


flowchart LR
    subgraph Agent_State["Agent State"]
        A[Agent input] --> B[MemGPT Agent]
        B --> C[Agent output]

        subgraph Internal State
            D[Memories]
            E[Tools]
            F[Messages]
        end

        B --> D
        B --> E
        B --> F
    end

MemGPT helps make decision about how past memories are to be handled when all the past history is too big to fit inside the context window.

MemGPT agent has a special section of context window called the “Core Memory”. Core memory can be customized .. e.g.: one for user, one for agent. Core memory gives the ability to the agent to learn over time. Core memory is archived into archival memory.

Why learn over time? makes the agent more engaging

How do you deal with running out of context window memory? Usually it is dealt by evicting a chunk of chat history by summerizing. the summerization is recursive. But, MemGPT saves the history to persistent memory called recall memory. retrieve using conversation search tool.

External memory Statistics

Agent Memory organization
system prompt
core memory
A/R stats
Summary
chat history

Two tier memory of a MemGPT agent:

  1. In-Context ( core memory )
  2. Out-of-Context ( archival memory, recall memory)

Graph View