Close Menu
NCIJ Network NCIJ Network
    What's Hot

    Do Senate Democrats want to take away Trump’s power to pardon himself?

    November 20, 2026

    German defense minister apologizes for calling US criticism of Ukraine strikes ‘stupid’ – POLITICO

    November 20, 2026

    New NASA Seed Funding Primes Small Businesses for Innovation

    November 20, 2026
    Facebook X (Twitter) Instagram
    Trending
    • Do Senate Democrats want to take away Trump’s power to pardon himself?
    • German defense minister apologizes for calling US criticism of Ukraine strikes ‘stupid’ – POLITICO
    • New NASA Seed Funding Primes Small Businesses for Innovation
    • Birds die in blaze at one of largest foie gras farms in the US
    • Brexit Reset Deal: U.K., EU Agree to Remove Some Barriers
    • EU warns against COP31 becoming a ‘fossil fuel summit’
    • Voters in four out of five UK parliamentary seats back rejoining EU | Andy Burnham
    • This year’s El Niño is unlike any other on record
    • About
      • Our Team
      • Editorial Policy
      • Editorial Independence
      • International Support
    • Trust & Standards
      • AI Usage Policy
      • Conflict of Interest Policy
      • Corrections Policy
      • Ethics Policy
      • Fact-Checking Policy
      • Source Protection
    • Get Involved
      • Guide for Sources
      • Support Independent Journalism
    • Legal
      • Cookie Policy
      • Privacy Policy
      • Terms of Use
    Facebook X (Twitter) Instagram
    NCIJ Network NCIJ Network
    Saturday, November 21
    • Home
    • World
    • Ai
    • Business
    • Politics
    • Health
    • Crypto
    • Science
    • Technology
    • Cybersecurity
    • Defense & Security
    • Economy
    • Energy
    • Europe
    • More
      • Fact Check
      • Investigations
      • Opinion & Analysis
      • Environment
    NCIJ Network NCIJ Network
    Home»Artificial Intelligence

    Meta FAIR Self-Improving Memory Hits 56.0 on Continual Learning Bench

    NCIJ NETWNCIJ NETWORKBy NCIJ NETWNCIJ NETWORKNovember 19, 2026 Artificial Intelligence No Comments4 Mins Read
    Share
    Facebook Twitter LinkedIn Pinterest Email

    Meta FAIR has introduced Self-Improving Memory, a method for AI agents that manage their own memory. The model first discovers its own memory instructions. It then trains those skills into its weights. Co-authors include researchers from the University of Washington and Johns Hopkins University.

    TL;DR

    • Method: The base is Kimi-K2.6, a 1T-parameter MoE with 32B active and 256K context. Distillation adds a LoRA adapter (rank 64) on attention projections.
    • Runs on: Training hardware is not disclosed. Kimi-K2.6 serving needs a multi-GPU node; Vast.ai recommends 8x H200. Native INT4 weights exist.
    • Performance: Self-found memory prompts average 56.0 on 3 tasks, versus 44.6 for the best human-written ones.
    • Best: Codebase, 64.3 with optimized prompts, +20.6 over the best hand-written harness.
    • Worst: Spectrum distillation adds only 2.9 points (46.5 vs 43.6 for SFT on generic runs).

    What is Self-Improving Memory?

    Self-Improving Memory is a 2-stage method from Meta FAIR that teaches an agent model to manage file-based memory. The agent works through a stream of subtasks. Each subtask starts with an empty context. Only a persistent memory directory, edited with file tools, carries over.

    Two prompts control that memory. OFFLOAD updates memory after a subtask. LOAD retrieves notes before the next one. Generic versions leave the strategy to the model. A pilot study in the research post found that no single hand-written strategy wins every task.

    How does harness optimization work?

    In stage 1, a meta agent rewrites the LOAD and OFFLOAD prompts. The search starts from 4 human-written seeds: Summary, Retrieval, Skill and Hypothesis. Each candidate runs the full task stream 5 times. The top 4 by mean score become parents for revision.

    For each parent, the meta agent reads trajectories, scores and final memory directories. It then proposes revised prompts. Selection uses a holdout set the meta agent cannot see.

    The discovered prompts are concrete. One Database prompt says to read GUARDRAILS.md, then SCHEMA.md, before answering. The best Codebase design uses a quickstart file, an indexed fix registry and separate fix files.

    How does harness distillation move the gains into weights?

    Stage 2 follows a Discover-Demonstrate-Distill pipeline. The task agent runs train subtasks with the optimized prompts. Every model call is logged. The research team then swaps the optimized prompts back to the generic ones in that history.

    The swap works because the behavior stays valid under generic instructions. Examples include keeping QUICKSTART.md, REGISTRY.md and SCHEMA.md files, and verifying join keys before aggregation. The model is fine-tuned with LoRA, rank 64, on attention projection layers. The loss covers only assistant tokens, including reasoning and tool calls. Each task gets equal weight.

    The meta agent and task agent are the same Kimi-K2.6 model. So the model is trained on memory skills it discovered itself.

    What do the benchmarks show?

    The tests use 3 Continual Learning Bench tasks: Database (SQL), Codebase (repo fixes) and Spectrum (radio scans). Test subtasks follow a designed phase shift. So results measure generalization to new instances of the same tasks.

    Test score Database Codebase Spectrum Average
    No memory 7.5 31.8 21.9 20.4
    Compaction 39.8 36.9 34.6 37.1
    Generic harness 32.3 33.5 42.5 36.1
    Best human-written 47.9 43.7 42.1 44.6
    Optimized harness 53.8 64.3 49.9 56.0
    SFT on generic trajectories 31.6 37.5 43.6 37.6
    SFT on optimized trajectories (generic prompts) 37.7 45.1 46.5 43.1

    Source: Figures 4 and 5, Self-Improving Memory blog. Kimi-K2.6 for all rows.

    Optimized harnesses beat the best hand-written seed on all 3 tasks. Codebase gained the most, at 20.6 points. Search was noisy: 37 of 44 generated revisions failed to beat the previous best. Database and Spectrum peaked in the first generation. Codebase kept improving through generation 4.

    After distillation, every model runs the generic harness. The distilled model averages 43.1 across 10 matched seeds. SFT on generic trajectories reaches 37.6. That is a 5.5-point gain, or 14.6%.

    Bench Continual fair hits Learning memory Meta selfimproving
    NCIJ NETWNCIJ NETWORK
    • Website

    Keep Reading

    RadixArk Ships DaisyRose-27B, a 27B Agentic Coding Model Trained With Reinforcement Learning Only

    NVIDIA AI Releases Nemotron-Labs-Sigma, a Continuous Diffusion Language Model with 14 checkpoints at 3B/8B

    Russian missile hits Poland as Ukraine’s allies hold military exercise, says Tusk

    Perplexity Launches Orchestrator API for Failover and Model Routing

    13 Best Agent Gateways in 2026 for MCP, A2A and LLM Traffic: Kong, Bifrost, AgentCore, Arcade

    Not If, But How: Responding to Security Incidents in the Age of AI Attackers

    Add A Comment
    Leave A Reply Cancel Reply

    Editors Picks

    Do Senate Democrats want to take away Trump’s power to pardon himself?

    November 20, 2026

    German defense minister apologizes for calling US criticism of Ukraine strikes ‘stupid’ – POLITICO

    November 20, 2026

    New NASA Seed Funding Primes Small Businesses for Innovation

    November 20, 2026

    Birds die in blaze at one of largest foie gras farms in the US

    November 20, 2026
    Latest Posts

    How ‘gentle power’ can preserve traditional biodiversity in Tokyo

    August 21, 2026

    This weight-loss hormone may protect the liver even without weight loss

    August 21, 2026

    Bitcoin Rally Lifts Crypto Stocks as Canaan Jumps 25%

    August 21, 2026

    Subscribe to News

    Get the latest sports news from NewsSite about world, sports and politics.

    NCIJ Network is an independent digital news platform delivering trusted investigative journalism, European and global news, in-depth analysis, and fact-based reporting with accuracy, transparency, and integrity.

    Facebook X (Twitter) Instagram Pinterest YouTube

    Do Senate Democrats want to take away Trump’s power to pardon himself?

    November 20, 2026

    German defense minister apologizes for calling US criticism of Ukraine strikes ‘stupid’ – POLITICO

    November 20, 2026

    New NASA Seed Funding Primes Small Businesses for Innovation

    November 20, 2026

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    Type above and press Enter to search. Press Esc to cancel.