MemInsight: Autonomous Memory Augmentation for LLM Agents

Authors: Rana Salama, Jason Cai, Michelle Yuan, Anna Currey, Monica Sunkara, Yi Zhang, Yassine Benajiba

MemInsight is a structured memory augmentation framework designed to enhance the long-term reasoning and adaptability of large language model (LLM) agents. It introduces autonomous memory annotation and retrieval methods that help agents organize and access relevant historical context during inference.

---

🔍 Overview

As LLM agents scale, managing accumulated memory across diverse interactions becomes a major challenge. MemInsight addresses this by:

• Autonomously generating structured memory augmentations
• Prioritizing semantically rich attributes for retrieval
• Improving memory relevance with attribute-based and embedding-based methods
• Boosting response quality in recommendation, QA, and summarization tasks

---

🧩 Key Features

• Attribute Mining: Extracts entity- and conversation-centric attributes from dialogues
• Memory Annotation: Supports both turn-level and session-level augmentation
• Retrieval Methods:
  • Attribute-Based Filtering
  • Embedding-Based Similarity (FAISS)
• Task Support:
  • Conversational Recommendation
  • Question Answering
  • Event Summarization

---

📈 Benchmark Results

MemInsight outperforms traditional memory retrieval methods:

Task	Metric	Improvement
QA (LoCoMo)	Recall@5	+34% over DPR
Conversational Reco.	Persuasiveness	+14%
Event Summarization	G-Eval (Relevance)	Comparable to baseline with less memory

---

📂 Datasets Used

• LLM-REDIAL: Movie recommendation dataset
• LoCoMo: Multi-turn long-context QA and summarization

---

🛠️ Setup

git clone https://github.com/amazon-science/MemInsight

cd meminsight

pip install -r requirements.txt

Run attribute mining and augmentation

python main.py --dataset llm-redial --model claude-3-sonnet

Evaluate

python main.py --task recomm --dataset "dataset_path" --anotations "annotations path"

---

Models Used

• Claude-3 Sonnet / Haiku (Augmentation & Generation)
• LLaMA 3 (Alternative Backbone)
• Mistral (Low-resource variant)
• Titan Text Embedding (FAISS indexing)

---

📊 Evaluation Metrics

• QA: F1, Recall@K
• Movie Recommendation: Recall@K, NDCG, Genre Match, Persuasiveness
• Event Summarization: G-Eval (Relevance, Coherence, Consistency)

---

Paper

This repository implements experiments and methods from the paper: “MemInsight: Autonomous Memory Augmentation for LLM Agents” ACL 2025 Submission (Under Review) 📌 Source code and data samples will be released upon acceptance.

---

Citation

If you use this code or refer to MemInsight in your work, please cite:

@misc{salama2025meminsightautonomousmemoryaugmentation,
  title={MemInsight: Autonomous Memory Augmentation for LLM Agents}, 
  author={Rana Salama and Jason Cai and Michelle Yuan and Anna Currey and Monica Sunkara and Yi Zhang and Yassine Benajiba},
  year={2025},
  eprint={2503.21760},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2503.21760}, 
}

---

Security

See CONTRIBUTING for more information.

License

This library is licensed under the CC-BY-NC 4.0. See LICENSE file.