Colbert had more numbers than the other late night shows.

It's just a dead medium. Previously they could pull numbers because it was the only way the general public could gain insight into the life of a celebrity (outside tabloid mags).

Social media gives you a parasocial relationship with celebs at every level. A constant drip feed directly into your brain. It made late night obsolete.