Friday, November 20, 2026
Mobile Offer

🎁 You've Got 1 Reward Left

Check if your device is eligible for instant bonuses.

Unlock Now
Survey Cash

🧠 Discover the Simple Money Trick

This quick task could pay you today — no joke.

See It Now
Top Deals

📦 Top Freebies Available Near You

Get hot mobile rewards now. Limited time offers.

Get Started
Game Offer

🎮 Unlock Premium Game Packs

Boost your favorite game with hidden bonuses.

Claim Now
Money Offers

💸 Earn Instantly With This Task

No fees, no waiting — your earnings could be 1 click away.

Start Earning
Crypto Airdrop

🚀 Claim Free Crypto in Seconds

Register & grab real tokens now. Zero investment needed.

Get Tokens
Food Offers

🍔 Get Free Food Coupons

Claim your free fast food deals instantly.

Grab Coupons
VIP Offers

🎉 Join Our VIP Club

Access secret deals and daily giveaways.

Join Now
Mystery Offer

🎁 Mystery Gift Waiting for You

Click to reveal your surprise prize now!

Reveal Gift
App Bonus

📱 Download & Get Bonus

New apps giving out free rewards daily.

Download Now
Exclusive Deals

💎 Exclusive Offers Just for You

Unlock hidden discounts and perks.

Unlock Deals
Movie Offer

🎬 Watch Paid Movies Free

Stream your favorite flicks with no cost.

Watch Now
Prize Offer

🏆 Enter to Win Big Prizes

Join contests and win amazing rewards.

Enter Now
Life Hack

💡 Simple Life Hack to Save Cash

Try this now and watch your savings grow.

Learn More
Top Apps

📲 Top Apps Giving Gifts

Download & get rewards instantly.

Get Gifts
Summer Drinks

🍹 Summer Cocktails Recipes

Make refreshing drinks at home easily.

Get Recipes

Latest Posts

Meta FAIR Self-Improving Memory Hits 56.0 on Continual Learning Bench


Meta FAIR has introduced Self-Improving Memory, a method for AI agents that manage their own memory. The model first discovers its own memory instructions. It then trains those skills into its weights. Co-authors include researchers from the University of Washington and Johns Hopkins University.

TL;DR

  • Method: The base is Kimi-K2.6, a 1T-parameter MoE with 32B active and 256K context. Distillation adds a LoRA adapter (rank 64) on attention projections.
  • Runs on: Training hardware is not disclosed. Kimi-K2.6 serving needs a multi-GPU node; Vast.ai recommends 8x H200. Native INT4 weights exist.
  • Performance: Self-found memory prompts average 56.0 on 3 tasks, versus 44.6 for the best human-written ones.
  • Best: Codebase, 64.3 with optimized prompts, +20.6 over the best hand-written harness.
  • Worst: Spectrum distillation adds only 2.9 points (46.5 vs 43.6 for SFT on generic runs).

What is Self-Improving Memory?

Self-Improving Memory is a 2-stage method from Meta FAIR that teaches an agent model to manage file-based memory. The agent works through a stream of subtasks. Each subtask starts with an empty context. Only a persistent memory directory, edited with file tools, carries over.

Two prompts control that memory. OFFLOAD updates memory after a subtask. LOAD retrieves notes before the next one. Generic versions leave the strategy to the model. A pilot study in the research post found that no single hand-written strategy wins every task.

How does harness optimization work?

In stage 1, a meta agent rewrites the LOAD and OFFLOAD prompts. The search starts from 4 human-written seeds: Summary, Retrieval, Skill and Hypothesis. Each candidate runs the full task stream 5 times. The top 4 by mean score become parents for revision.

For each parent, the meta agent reads trajectories, scores and final memory directories. It then proposes revised prompts. Selection uses a holdout set the meta agent cannot see.

The discovered prompts are concrete. One Database prompt says to read GUARDRAILS.md, then SCHEMA.md, before answering. The best Codebase design uses a quickstart file, an indexed fix registry and separate fix files.

How does harness distillation move the gains into weights?

Stage 2 follows a Discover-Demonstrate-Distill pipeline. The task agent runs train subtasks with the optimized prompts. Every model call is logged. The research team then swaps the optimized prompts back to the generic ones in that history.

The swap works because the behavior stays valid under generic instructions. Examples include keeping QUICKSTART.md, REGISTRY.md and SCHEMA.md files, and verifying join keys before aggregation. The model is fine-tuned with LoRA, rank 64, on attention projection layers. The loss covers only assistant tokens, including reasoning and tool calls. Each task gets equal weight.

The meta agent and task agent are the same Kimi-K2.6 model. So the model is trained on memory skills it discovered itself.

What do the benchmarks show?

The tests use 3 Continual Learning Bench tasks: Database (SQL), Codebase (repo fixes) and Spectrum (radio scans). Test subtasks follow a designed phase shift. So results measure generalization to new instances of the same tasks.

Test score Database Codebase Spectrum Average
No memory 7.5 31.8 21.9 20.4
Compaction 39.8 36.9 34.6 37.1
Generic harness 32.3 33.5 42.5 36.1
Best human-written 47.9 43.7 42.1 44.6
Optimized harness 53.8 64.3 49.9 56.0
SFT on generic trajectories 31.6 37.5 43.6 37.6
SFT on optimized trajectories (generic prompts) 37.7 45.1 46.5 43.1

Source: Figures 4 and 5, Self-Improving Memory blog. Kimi-K2.6 for all rows.

Optimized harnesses beat the best hand-written seed on all 3 tasks. Codebase gained the most, at 20.6 points. Search was noisy: 37 of 44 generated revisions failed to beat the previous best. Database and Spectrum peaked in the first generation. Codebase kept improving through generation 4.

After distillation, every model runs the generic harness. The distilled model averages 43.1 across 10 matched seeds. SFT on generic trajectories reaches 37.6. That is a 5.5-point gain, or 14.6%.

How does it compare with ALMA, MemEvolve and Harness-Zero?

Several recent methods search over agent memory or harnesses. Self-Improving Memory searches memory prompts, then distills them into weights. Benchmarks differ, so headline numbers are not directly comparable.

Feature Self-Improving Memory ALMA MemEvolve Harness-Zero
Org FAIR at Meta, UW, JHU Xiong, Hu, Clune OPPO AI Agent Team, LV-NUS Peking University, Google, HKUST
What is searched LOAD/OFFLOAD memory prompts Memory designs as Python code Memory architecture and contents Full agent harness
Trains weights Yes, LoRA rank 64 No No Yes, LoRA SFT
Models Kimi-K2.6 (agent and meta agent) GPT-5 meta agent, GPT-5-nano/mini task agent GPT-5-mini (plus DeepSeek V3.2, Kimi K2) Qwen3.5-9B student
Benchmarks Continual Learning Bench (3 tasks) ALFWorld, TextWorld, Baba Is AI, MiniHack GAIA, WebWalkerQA, xBench-DeepSearch, TaskCraft SpreadsheetBench Verified, AppWorld, USPTO Retrosynthesis
Headline result 56.0 vs 44.6 best human harness; 43.1 distilled 53.9% vs 41.1% no memory (GPT-5-mini) Up to 17.06% gains 23.3% to 44.3% with harness removed
Code Not released GitHub GitHub GitHub

Harness-Zero is the closest match, but it uses a separate agent to correct the student. Self-Improving Memory skips that step.

Key Takeaways

  • Kimi-K2.6 writes memory prompts that beat human ones: 56.0 vs 44.6.
  • Codebase sees the biggest jump: 64.3, up 20.6 points.
  • Distillation keeps part of the gain: 43.1 vs 37.6 with generic prompts.
  • 37 of 44 meta-agent revisions failed, so evaluation and selection matter.

Check out the blog post, the X thread and the Continual Learning Bench paper. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.

Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us


Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.



Source link

Latest Posts

Don't Miss

Stay in touch

To be updated with all the latest news, offers and special announcements.