Just because you use "swordsmanship" doesn't mean you're a PURE swordsman all the time forever.

When going all out, Shanks is not a swordsman.
When going all out, Roger is not a swordsman.
When going all out, Rocks is not a swordsman.

Zoro admitted this when he said that to win, "he might have to step outside of swordsmanship too".