Five AI agent harnesses took the same memory test. The results weren’t even close.
Letta introduced a new benchmark called FrontierMemory designed to measure whether AI agents can learn from past conversations and tasks, distinguishing between simple information recall and actual learning capability. Five agent frameworks were tested, with results showing stark disparities: Letta Dream achieved 73.3%, Prime Agent 60%, OpenAI Agents SDK 40%, Claude Managed Agents 33.3%, and Hermes Agent 26.7%. The benchmark addresses limitations in existing metrics, which focus on recall in isolated tasks rather than evaluating continual learning and how agents apply knowledge in unrelated work scenarios.
Key points
- FrontierMemory benchmark tests learning vs. recall in AI agents
- Letta Dream scored highest at 73.3%, others significantly lower
- Existing benchmarks fail to measure practical continual learning
Summary and key points written by AI from the headline and a short excerpt, so they can be incomplete or wrong. The New Stack did the reporting; read the original for the full story.
Read the full story at The New Stack
