So, are you basically handicapping yourself if you don't get 10/11 books from a single realm of magic? I feel gimped if I don't specialize, especially if I'm splitting books between 2 or 3 schools of magic and getting retorts as well.

The only times I've won, I went with 11 books or 10 books + 1 retort.