anyone here got experience with writing helium/selenium web scraping scripts? my project has several standalone executing scripts that scrape data from different websites. the problem is that now each script has to execute Its own browser instance just to open a website and scrape data from it. what I want to achieve is to open one browser instance on program start and then make my scripts use that one instance instead since this will make the whole thing a lot faster. is there any way to do that? I will be thankful for any sort of advice

Which programming language are you using? I think most PL can use the Seleniums scripts use just one browser instance, but wouldn't many browsers instances be faster? Almost like parallel processing...

>>106127949 (OP)
you can do this in puppeteer.
just get the first tab when browser opens and keep reusing it

>>106127949 (OP)
Couldn't you just
wget your_site | xmllint --xpath your_stuff ?

>>106127949 (OP)
maybe I'm late but I have experience with Selenium, it is generally not advised to share a single browser for an automation task because you might run into conflicts provided they run in parallell

if automation jobs run serially then there's no problem, you just have to open the browser with automation extensions and an open port, then tell your script to connect to that port and voila, I can provide you the chrome command line options I use for this

>>106127949 (OP)
i dont understand how retards use selenium for web scraping. its overkill like you wouldnt believe. just use curl-impersonate if the site has some cancerous ddos protection. if you have to resort to selenium/puppeteer you're doing something wrong

>>106132563
clearly you haven't scraped enough to realize some sites require selenium for scraping

>>106132584
example usecase?

>>106132931
gooning

>>106132563
trvthnvke. encountered my first stubborn website that had some of javascript redirection magic. curl-impersonate bypassed it