Shanks visited the Gorosei. The Holy Knights have to do with politics

The Holy Knights are like a special battalion unit that you send to a country with a message you either join us or you die.