Because general audiences have no taste.
Look at whats on in your cinema, or whats the highest streamed songs of the year.
The biggest slice of the pie is always gonna be slop. Thats why media sucks. Its no longer about artists making cool stuff, its just a slop factory.To make the big bucks you gotta water down the game so a monkey can play it.
Yes, Baldurs Gate 3 exists and did well, but thats an outlier of the equation. If you take that away there's hardly any single player games that came out recently that have complexity.

And not only are they easier to sell to morons, they are also easier to make if you get rid of the complex. Think about how much work both coming up with the idea of multiple branching paths for one single quest, writing/recording the dialogue, and actually scripting and bug testing it is compared to "yes/sarcastic yes/don't progress" is.
Bethesda will never change. They want the big money. Maybe their next game will have slightly more quests that have 2 different outcomes. But they're never gonna be doing what New Vegas did. It's way easier to just accept they only wanna make money and not buy it unless you wanna eat the slop.