Evolutionary Blueprint (5). When learning something new makes AI worse at something old

Series: Evolutionary Blueprint of AI, article 5. raining an AI on new tasks can weaken older skills. Learn why forgetting varies, what brain research can and cannot tell us, and how to choose between retrieval and further training.

Evolutionary Blueprint (5). When learning something new makes AI worse at something old
Illustration generated using ChatGPT. Forgetting causes disappearing territory

Who should read it?

Technology and business leaders, product teams and curious non-specialists.

Methaper

The disappearing territory

Intro

Imagine a company that uses an AI assistant to classify customer requests. It handles billing questions and delivery complaints well enough to help staff sort their queues. The team then fine-tunes the model on examples of product returns. Returns improve, but some billing requests start landing in the wrong queue.

This is a fictional example, not a reported incident. It illustrates a problem that is easy to miss when a team measures only the new capability: an update can improve one task while weakening another. If the company checks returns alone, the release looks successful. Customers whose billing issues are delayed may disagree.

We examine the tension between learning and retention. The brain provides an intriguing research perspective, but neither human memory nor artificial memory comes with a guarantee against forgetting.

New training can interfere with old skills

A neural network has parameters, often called weights, that influence its output. During training, an optimization process adjusts some or all of these weights to improve performance on the training examples. Fine-tuning is additional training that adapts an existing model to a task or domain.

The same weights can contribute to several skills. When training changes them to help with returns, those changes may also affect billing. If performance on earlier tasks falls after learning later ones, researchers call this forgetting. When the loss is severe, it is often called catastrophic forgetting or catastrophic interference. The word describes a measured loss in performance, not proof that every trace of an old skill has been erased.

The sequence matters. A model trained on returns examples alone may receive no reminder that a billing problem should still be classified correctly. If the old examples are mixed into training, the model has a chance to adjust to both. That does not guarantee success: an update may still favor one task, and the chosen examples may not represent all the cases that matter.

The outcome depends on the model, the task sequence, the data, the training method and how performance is tested. Forgetting can be substantial, modest or hard to detect in a particular evaluation. Some changes improve more than one task; others trade one capability for another. It is wrong to assume that every new lesson wipes out everything learned before it.

Yun Luo and colleagues investigated sequential instruction fine-tuning in several language models. They observed losses on tests of domain knowledge, reasoning and reading comprehension, with differences across the models and evaluations they studied. They also found that including general instruction data could reduce some losses. These experiments establish a risk, not a universal forecast for every model or company workflow.

Biology offers a hypothesis about memory, not a cure

Human beings learn new information while retaining much of what they already know, but we also forget, confuse similar experiences and reconstruct memories imperfectly. A theory of biological memory should explain those limits as well as our successes.

In their 1995 complementary learning systems account, James McClelland, Bruce McNaughton and Randall O'Reilly proposed that different learning systems help balance rapid learning with the gradual integration of experience. As O'Reilly and colleagues' review explains, the hippocampus is important for rapidly encoding aspects of particular experiences, while neocortical systems can gradually learn shared structure across experiences. These are interacting systems, not a simple recorder and a separate permanent filing cabinet.

The theory proposes that reactivation of earlier experiences can help integrate newer and older learning. The review discusses evidence of replay during sleep and a proposed role for sharp-wave activity during sleep or quiet waking. How much replay contributes to consolidation, how memories change over time, and the continuing role of the hippocampus remain subjects of research and debate. Consolidation is not a nightly guarantee that a complete recording gets copied into a permanent store.

For an AI team, the useful idea is narrower than "copy the brain": consider how to bring earlier experience back into training while learning something new. Artificial replay is an engineering method inspired by related concerns. It is not evidence that a neural network remembers or sleeps like a person. Nor does preserving a record of past interactions prove that an assistant has a human-like identity.

Retrieval changes the information, not necessarily the model

Return to the support assistant. Classifying incoming requests into queues is one feature. Answering policy questions is a separate, hypothetical feature. Suppose the returns policy for that answering feature changes frequently. The company can consider storing approved policy documents in a searchable system instead of fine-tuning for each policy change. When a question arrives, the application retrieves relevant passages and supplies them to the model along with the question. Whether this approach works well enough depends on evaluation and controls.

This approach is called retrieval-augmented generation RAG. Patrick Lewis and colleagues studied a system that combined a language model with a searchable Wikipedia index and reported improvements on their chosen knowledge-intensive tasks compared with a parametric-only baseline. Their system involved fine-tuning; retrieval and training can be combined, even though they are different operations.

In the company's hypothetical deployment, updating a policy document or placing a retrieved passage in a prompt does not by itself update the language model's weights. The answer can change because the information supplied at request time has changed. Once that passage is no longer in the prompt, the model should not be assumed to have retained it in its weights. A separate training process would be needed to change those weights.

Retrieval is therefore a way to supply changing facts without a weight update. It does not solve catastrophic forgetting during a later fine-tuning run. It also introduces its own failure modes. The system might retrieve an obsolete policy, omit an exception, expose a document the user is not authorized to see, or misinterpret the correct passage. The team still needs document ownership, permissions, effective dates and tests of the full answer, not merely a successful search.

Protect earlier performance when weights must change

Sometimes the team wants a different behavior rather than a newer fact. The support assistant might need to recognize a new category of request or follow a more consistent output format. Fine-tuning may be worth considering, but it should be compared with simpler options such as clearer instructions and examples in the prompt.

If training is justified, one mitigation is replay:

  • Mix suitable examples of earlier tasks with the new training examples

In the fictional support system, that means including representative billing and delivery cases alongside returns. The company must have permission to use those examples and should remove or protect sensitive customer information. Replay consumes storage and training capacity, and a small sample can miss rare but costly failures.

Another approach is to limit changes to parameters judged important for earlier tasks. In James Kirkpatrick and colleagues' elastic weight consolidation study, the method selectively slowed changes to important weights and helped retain performance across sequential handwritten-digit classification and Atari game tasks. This is evidence for a method in those experiments, not a promise that the same settings will protect an enterprise language model. Restricting changes too much can also make the new task harder to learn.

A team can also keep a stable base model and train separate small adapters for particular tasks. An adapter adds task-specific parameters rather than rewriting the whole base model. A routing component selects which adapter, if any, handles each request; ambiguous cases need a defined fallback or human handoff. The base remains available, but an adapter can still change behavior on old tasks. Test each base-and-adapter combination on new and old cases, and test the router end to end on clear and mixed requests. Separate models or adapters also create costs in routing, storage and maintenance.

These approaches do not require rebuilding the model from scratch. Nor is one universally best. The trade-off is between adapting to what changed and preserving what the organization still needs.

Make retention a release condition

The support team can make the trade-off visible before deploying an update. First, save the current model version and record how it performs on representative billing, delivery and returns cases. Keep a separate held-out test set that the team does not use for training. Include rare cases such as a return request that also disputes a charge, with expected handling agreed by the process owner.

After fine-tuning, run the same old-task tests and the new returns tests against both versions. Compare results by category and by the consequences of errors, not only by an overall average. A gain on common returns requests could conceal a drop on disputed charges. Set acceptable regression limits before the release, with stricter limits for mistakes that delay payment resolution or expose customer data.

For example, the owner might decide not to release if reviewers find a consequential regression on disputed billing cases, even when returns classification improves. This is a qualitative illustration, not a prescribed threshold; the owner must define acceptable limits before testing. If the model passes, deploy gradually and monitor cases sent to the wrong queue. If monitoring reveals a consequential regression, pause the rollout and restore the prior tested model and compatible configuration while staff handle affected cases. A rollback needs access to those artifacts and a practiced procedure, not just a note saying it is possible.

The cost is more than compute. Someone must maintain test cases, decide which failures matter and investigate changes in performance. For a low-risk internal draft tool, lighter checks may be reasonable. When an assistant affects customer service, payments or access to private records, the price of overlooking an old skill can exceed the savings from a quick update.

Protect old skills while adding new ones

Sequential training can damage earlier performance, but the amount of forgetting depends on the system and the task. Biological research suggests useful questions about combining new and old experience without establishing that human memory is immune to interference. Consider retrieval for changing information, subject to evaluating its answers and controls; when updating weights, test both the new capability and the older behaviors the business still depends on.

Previous

Article 4: Reward is not the whole lesson

Next

Article 6: Understanding people is not enough for AI safety