Yeah I really cannot see the traveler ever beating someone surtalogi or getting on their level. Celestia sealed their powers, yes, but it didn't take their memories, and they're still getting scared by what are essentially jobbers in the grand scheme. Getting a "breakthrough" in swordsmanship from a few days of skirk's training in her SQ (by learning from "great swordsmen" like kaeya and kazuha lmao) also shows they really weren't all that powerful in the past.