>>149928159
>For one thing they're completely linear in comprehension and construction –by design–
By design they're completely nonlinear, and that's kind of their main thing. Without the added nonlinearity the weights just converge to like PCA