Anonymous
6/20/2025, 11:30:57 AM
No.105649767
>>105649747
>number of parameters of a model only affect HOW the INITIAL data is transformed in a meaningful output.
not at all, a bigger model also means it has more space to store the knowledge
https://arxiv.org/abs/2205.10770
>We measure the effects of dataset size, learning rate, and model size on memorization, finding that larger language models memorize training data faster across all settings. Surprisingly, we show that larger models can memorize a larger portion of the data before over-fitting and tend to forget less throughout the training process.
>number of parameters of a model only affect HOW the INITIAL data is transformed in a meaningful output.
not at all, a bigger model also means it has more space to store the knowledge
https://arxiv.org/abs/2205.10770
>We measure the effects of dataset size, learning rate, and model size on memorization, finding that larger language models memorize training data faster across all settings. Surprisingly, we show that larger models can memorize a larger portion of the data before over-fitting and tend to forget less throughout the training process.