I know this is probably just the Mad ghost saying Asgore doesn't know he exists. But could it be possible that when Asgore mentions talking to his flowers, he actually means Mad ghost secretly possesses one and becomes a pseudo flowey?

Hell you could end up talking to Mad ghost possessing a flower and there would be enough similarity in personalities that you wouldn't immediately pick up that you weren't talking to Flowey. If Toby wanted to throw people for a loop making them think "wait Asriel's supposed to be in college and a goatperson".