I have a 4080 Super and I rarely use raytracing. Once or twice I've kept it on, but generally it goes like this
>turn on raytracing in new game
>walk around a bit and admire the shiny graphics
>soon after turn it off again to double my FPS and actually play
It's just not worth it. Most cases you're forced to use DLSS, which I try to avoid as much as possible to preserve native res image quality.