Huh, so I decided to try out Dire difficulty just to see how it goes if I give it a honest try and I actually got pretty close to clearing the hardest of the three.
If the rewards were permanent I wouldn't mind hitting my head against the wall a few dozen times until RNG gods favor me and I clear it.