What do you think of MIT's new self learning model method?
https://arxiv.org/abs/2506.10943
Looks like it would give the model a way to permanently incorporate new facts and information that it learns, however it can still suffer from catastrophic forgetting so it's not perfect.