I assume most delistings are motivated by money, in one way or another (like not wanting to renew licenses for copyrighted music, not wanting the original game to compete with a worse but more expensive remaster, etc.); and when it comes to Bethesda, they probably think they're doing the world a favor by copy-pasting expansions into base games, as it would never occur to some suit-wearing executive that this shits up the Steam libraries of people who already bought those games and now have one copy of Doom, two copies of Doom II, three copies of Final Doom, and no option to play just the first without downloading everything. At least their intentions were good at best and understandable at worst.
But for some games, I can't imagine a possible reason, except maybe the developer/publisher so abruptly going out of business that no one picked up the distribution rights. Maybe that's what happened to Scratches. But actually, it isn't; I just checked and the listed publisher has a game out this year. So what the fuck.
As for OP's list, at least people who bought the original Dear Esther got the remaster for free. The developers are still very braindead for assuming that the original Source engine game has no value, though.