DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to reproduce AMA's LoCoMo and LongMemEval results with AMA

Run the research pipeline on LoCoMo and LongMemEval_s to check the reported scores.

Short answer: git clone https://github.com/Sherlockwz/AMA.git; then cd AMA; corepack enable pnpm. All steps.

Credentials. Full benchmark runs make many LLM and embedding calls, so the API keys in your environment will incur usage costs.

Who it is forA researcher or student who needs to verify or extend the AMA results for an ACL 2026 paper comparison.

The problemPublished memory benchmark scores are hard to trust without rerunning the original pipeline on the same datasets.

The outcomeYou run the AMA evaluation entry points and produce scores you can compare against the README's reference numbers.

What you need first

  • Python 3.10 or newer
  • Node.js 22.22.3 or newer (24.15 or newer also supported)
  • pnpm 11 or newer
  • An OpenAI-compatible chat-completions endpoint and embedding endpoint, with an API key
  • Git, to clone the repository

Steps

  1. git clone https://github.com/Sherlockwz/AMA.git
  2. cd AMA
  3. corepack enable pnpm
  4. ./scripts/setup.sh
  5. Follow the reproduction guide at docs/reproduction.md to obtain the LoCoMo and LongMemEval_s datasets and place them where the guide says.
  6. Run the evaluation entry points under python/AdaptiveMemory/Core, as described in the reproduction guide.

Prompt for your agent

In the AMA repo at <path-to-AMA>, follow docs/reproduction.md to set up the LoCoMo dataset and run one evaluation entry point from python/AdaptiveMemory/Core. Expected output: an overall LLM Score for the LoCoMo run. Verify that the dataset is placed where the guide specifies before comparing your score to the README's 0.774 reference.

Caveat. The benchmark datasets are not redistributed in the repo, so you must obtain them yourself before any evaluation can run.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for Sherlockwz/AMA and "Reproduce AMA's LoCoMo and LongMemEval results"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer