Researchers propose a self-distillation fix for ‘catastrophic forgetting’ in LLMs

LLMs tend to lose prior skills when fine-tuned for new tasks. A new self-distillation approach aims to reduce regression and ...

MIT's new fine-tuning method lets LLMs learn new skills without losing old ones

MIT researchers unveil a new fine-tuning method that lets enterprises consolidate their "model zoos" into a single, continuously learning agent.
Science Daily

Novel memristors to overcome AI's 'catastrophic forgetting'

Memristors consume extremely little power and behave similarly to brain cells. Researchers have now introduced novel memristive that offer significant advantages: they are more robust, function across ...
VentureBeat

Researchers find that retraining only small parts of AI models can cut costs and prevent forgetting

Enterprises often find that when they fine-tune models, one effective approach to making a large language model (LLM) fit for purpose and grounded in data is to have the model lose some of its ...
EurekAlert!

Novel memristors to overcome AI’s "catastrophic forgetting"

They consume extremely little power and behave similarly to brain cells: so-called memristors. Researchers from Jülich, led by Ilia Valov, have now introduced novel memristive components in Nature ...