DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to rerun the LoCoMo and LongMemEval benchmark scores with Memoket Kite

Rebuild the reported benchmark results locally to check KITE's scores before you adopt it.

Short answer: Read the benchmark guide at benchmarks/README.md in the repo; then install the library: python -m pip install memoket-kite; set your key, and if you use another provider also set OPENAI_BASE_URL, following the README. All steps.

Credentials. The benchmark run calls a paid LLM endpoint with your key, so keep the key private and watch the usage on your account.

Who it is forA research engineer evaluating long-term memory options who needs to confirm published LoCoMo and LongMemEval-S scores under a shared protocol before choosing a memory engine.

The problemPublished benchmark numbers are hard to trust without a way to reproduce them under the same model, judge, and dataset revisions.

The outcomeYou get a local run with manifests that record the commit, corpus digest, and scores, which you can verify offline.

What you need first

  • Python with pip available
  • An OpenAI API key, or an OpenAI-compatible provider with OPENAI_BASE_URL set
  • Network access to download the demo_codebook.xml file from GitHub
  • Conversation text you are allowed to store in a local memory file

Steps

  1. Read the benchmark guide at benchmarks/README.md in the repo.
  2. Install the library: python -m pip install memoket-kite
  3. Set your key, and if you use another provider also set OPENAI_BASE_URL, following the README.
  4. Run the provided benchmark scripts named in benchmarks/README.md to rebuild the evaluation from the official datasets.
  5. Check the local manifest and judging manifest, then run the verifier described in docs/reproducibility.md.

Prompt for your agent

Use the memoket-kite repo. Follow benchmarks/README.md to run the LoCoMo evaluation with a configured compatible LLM endpoint. Expected output: a local judging manifest with an overall accuracy and an average reader context. Verify that the manifest's commit and corpus digest match the run before you compare the score with the README's 93.51%.

Caveat. The README notes the benchmark corpora have their own licences in LICENSE-DATA.md, so check those terms before you redistribute any dataset files.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for memoket/memoket-kite and "Rerun the LoCoMo and LongMemEval benchmark scores"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer