Web Scraping General

FAQ: https://rentry.org/scrapists

> Captcha services
https://2captcha.com/
https://www.capsolver.com/
https://anti-captcha.com/

> Proxies
https://hproxy.com/ (no blacklist) (recommended, owned by friend of /wsg/)
https://infiniteproxies.com/ (no blacklist)
https://www.thunderproxies.com/
http://proxies.fo/ (not recommended)

> Network analysis
https://mitmproxy.org/
https://portswigger.net/burp

> Scraping tools
https://beautiful-soup-4.readthedocs.io/en/latest/
https://www.selenium.dev/documentation/
https://playwright.dev/docs/codegen
https://github.com/lwthiker/curl-impersonate
https://github.com/yifeikong/curl_cffi

> Cool projects by members of our community
doubledouble.top / lucida.to - Free music scraped from spotify
nekohouse.su - Kemonoparty for fanbox/fantia/subscribestar
tv.weboasis.app - Falcon, a goy invite-only pirate streaming service that scrapes video streams from multiple sources

asking for the new telegram edition (i lost my old tg account and the backup signal :/)

oops i messed up... oh well

>>105964580 (OP)
Come post with us on metachan

>>105964728
could i steal a tg link? (or signal <3)
tried finding heavens but i cant
can share pgp if you prefer

Subtle tranny colour scheme

There's nothing to scrape.

>>105965018
This general's tg is @scrapists according to the previous OP >>105917015. That's all I know.

>>105965450
its broken :/

>>105964580 (OP)
would recommned proxycheap as it can be used to scrape .gov domains (at least from my experience).
https://www.proxy-cheap.com/

>>105965830
What are you scraping on .gov space?

>>105965853
In my cuntry there is a government-backed search engine with no authentication that allows you to find someone's real name, address and some other PIE by just knowing the number of their residential record. I wrote a basic scraper that brute-forces those in multiple threads with different proxies. Thanks to that I have obtained names and addresses of all people in my local area who have some share in any regional estate. Sadly, they will soon make it so you need to go through KYC in order to use this website.

>>105966143
Scrap as much info as you can, maybe y ou can sell it to data brokers