>>106173093
It might also be that the pretraining phase was more or less standard and most of the damage came from post training and extensive reinforcement learning, although there aren't many details in this regard in the technical report.

It sounds like the 20B model had distilled pretraining (considerably shorter pretraining time), although they don't mention anything like that.