Will the weight of experience help the LLM gain a long-term memory?

This article is part of the PRO Members Communication, and follows "PRO Members of Machine Heart" at the end of the article to read more on the topic.

As large models move from one-time tasks to multi-cycle interactions and long-term tasks, long-term memory becomes an important research direction. Early research extended the use of models for historical information through mechanisms such as long-context, KV Cache, loop state, and external retrieval, but there were still different paths in terms of information perpetuation, state updates, and cross-session replication. In recent years, research has further explored how memory is placed inside models, through updated states, memory modules, and even parameter changes, to directly involve historical information in subsequent computation. Model memory research has also gradually expanded from "preserving more historical information" to "how information enters the model, how it is updated, and how long it can last".

Directory

01. Why did the Big Model start writing memories inside the model?
Why did the long-term memory of a large model begin to point to the inside of the model?...

Why did the big model start writing memories inside the model?

In the course of the development and application of LLM, the limitations of model memory have gradually become apparent and have received increasing attention in academia and industry as the application landscape has evolved towards long-cycle intelligence (Long-Horizon Agents) and lifelong learning (Lifelong Learning).

In exploring the path to greater memory power, previous research has focused on dynamic extraction of information from external non-parametric storage modules such as Rags, through scarcity of attention mechanisms or state-space models (SSMs) to extend the context window (Context Window) and through retrieval-enhanced generation (RAG).

3 Although external storage and long context windows are useful for fact-checking, there are limitations when it comes to deep logical reasoning and implicit knowledge internalisation, prompting the industry to explore the feasibility of directly encoding experiences and storing them in model weight (i.e. internal parameters).

1 Expanded context length is not equivalent to an increase in memory utilization rates, and longer context windows are faced with increasing computational expenses and attenuation problems, and models often fail to make full use of the theoretical capacity they label.

In the field of parametric memory, the industry has begun to introduce the Fast Weights mechanism. During the reasoning phase, the model uses the input context for real-time gradient updates, dynamically adjusting some of the network weights. At the same time, the basic model attempts to maintain a lasting parametric memory state in the backbone network.

1 Researchers at Stanford University, the University of California, San Diego, and other institutions proposed TTT Layers in 2024, which use the weight of small models as a hidden state and are continuously updated when processing inputs to take advantage of longer sequences of information.[1-3]

2 In late 2024, researchers at Google Research proposed Titans, a neural memory module that could be updated when added to reasoning without attention, and adjusted to writing and forgetting based on predicted errors.[1-4]

3 Researchers at institutions such as the University of California, San Diego, and Amazon proposed the MemoryLLM in 2024, which set up fixed-sized memory pools at each layer of the model, replacing some of the old vectors with newly generated memory vectors, to achieve the injection and subsequent call of new knowledge.[1-5]

5 Recently, the Dongjean team at Tsinghua University, working on "Memory For LLM", analysed three-dimensional system models of memory mechanisms, namely how information is stored, when and how it is updated, and how long it can last.[1-1]

1 In storage form, the cyclic state of models such as KV Cache, Mamba and others in focus is dependent on the computational process; specially set memory modules, memory pools, or addressing storage provide relatively clear reading and writing locations.[1-1][1-2]

2 In the update mode, TTT Layers, Titans, etc. attempt to update internal state or memory parameters when processing new inputs, enabling the model to absorb side-by-side information.[1-1][1-3][1-4]

3 On the question of how long information can affect models, KV Cache primarily serves current reasoning; cyclical states, memory modules, and updated parameters attempt to extend historical effects to longer sequences or subsequent interactions.[1-1]

Is the message written in weight even if it is remembered for a long time?

1. By 2024, researchers have begun attempts to make models adjust fast weights when processing inputs, or set up self-renewable neural memory modules and memory pools. Such methods extend the scope of the model using earlier information, but "internal memory" is not the same as modifying the body weight of the baseline model....