Prior to the rise of call centers, Indians had a reputation for being industrious, education-oriented, and law-abiding (at least somewhat).
Then people all across America start getting these scam calls, usually targeting the elderly, and often getting them constantly.
Imagine people in small towns with no Indians, no associations with indian accents except Apu, getting these calls everyday, to the point that they associate the accents with people trying to steal their money.
And the Indian government did and still does nothing to stop it.