As someone that actually lived through that era, fighting games were often the only meaningful competitive multiplayer game for the late 80s and early 90s. Everything else was just stuff like "Player 1 has a turn/player 2 has a turn" things like SMB, rare competitive side modes in some games like Mario Bros., and maybe Super Mario Kart. Arcades had more options but again, the technology was rudimentary that fighting games were often the most robust option, with the closest being racing games.
By the mid 90s, the fifth gen had arrived, and internet access had started to become more commonplace. Dedicated multiplayer in first person shooters was becoming commonplace, and the FPS market increased exponentially once Goldeneye was released in 1997 for the N64, proving the viability of the format on consoles, where it continues to dominate to this day. Hell, even without the internet, who can forget Pokemania? Anybody that had a link cable automatically became everybody's friend back then. Beyond that, after the turn of the century, other genres continued to find ways to incorporate online multiplayer.

Fighting games aren't growing because they have actual market competition. Everybody would literally rather just play anything else and people actually have options. Why bother with learning a fighting game when you can just hop into Fortnite or Call of Duty and start blasting dudes?