Anonymous
6/20/2025, 11:44:05 PM
No.105655453
>>105655345
To add to that, there are other fundamental mechanisms that are missing which are interesting to think about. Like the issue of catastrophic forgetting for instance is still unsolved, while the brain has the ability to remember something until death even with no more exposures during its lifetime. Meanwhile an LLM needs constant re-viewing of a piece of information in order to not forget it, and why we cannot just simply train an LLM using a curriculum, but need to use huge mixes of random data, which may still have a curriculum but also still needs the random data.
To add to that, there are other fundamental mechanisms that are missing which are interesting to think about. Like the issue of catastrophic forgetting for instance is still unsolved, while the brain has the ability to remember something until death even with no more exposures during its lifetime. Meanwhile an LLM needs constant re-viewing of a piece of information in order to not forget it, and why we cannot just simply train an LLM using a curriculum, but need to use huge mixes of random data, which may still have a curriculum but also still needs the random data.