Kaido was a Hakiman truther despite having a devil fruit. The only pure Devil Fruit users are predominantly pre time skip villains.

Hakiman w/ Devil Fruit:
>Joyboy, Soon Luffy

Hakiman w/out Devil Fruit:
>Roger, Garp, Rocks, Shanks

Devil Fruit truthers:
>Admirals, every other villain and Devil fruit user

TL;DR It doesn't matter what actually happens in the story. Luffy could have all the Devil Fruits in the world, but Haki would still be stronger because Kaido and Oda said so.