LLMs as Operating System - Agent Memory

By Charles Packer and Sarah Wooders from Letta who wrote the MemGPT: Towards LLMs as Operating Systems paper. Course website

• Self-editing memory
• agents framework that allows build and deploy persistent agents as service
• MemGPT paper introduced the concept of “self-editing memory” for LLMs
• MemGPT is also a type of agent design inspired by Operating Systems, which has two tiers of memory,..

Chatbots vs Agents

• agents act autonmously in multi-steps, autonomously.
• agentic loop (agent state → llm context window → llm inference → agent state)
• context compilation. put agent state into the context window
• self editing? memory_replace(s/sarah/charles/)

MemGPT:

• use LLM to build agent
• context window of an LLM
• 0 context window
• user data
• message history
• external data sources
• tools calls
• reasoning/chain of thought

Key Ideas behind MemGPT:

• Self-editing memory
• Inner Thoughts
• Every output is a tool
• Looping via heartbeats

put all the above together makes “MemGPT Agent”. they are self-improving because they can edit their long term memory.

Heartbeat requests next action, thus resulting in a loop.

Agent State = Memories + Tools + Messages

flowchart LR
    subgraph Agent_State["Agent State"]
        A[Agent input] --> B[MemGPT Agent]
        B --> C[Agent output]

        subgraph Internal State
            D[Memories]
            E[Tools]
            F[Messages]
        end

        B --> D
        B --> E
        B --> F
    end

MemGPT helps make decision about how past memories are to be handled when all the past history is too big to fit inside the context window.

MemGPT agent has a special section of context window called the “Core Memory”