>>106238153
AFAIK, Google's Gemini scraping respects robots.txt.

Perplexity doesn't respect robots.txt, fakes its user agent if blocked, rotates IP addresses if blocked and so on.