SEEKING WORK | Remote
===
I can devote 10-15 hrs per week. I am looking for PoC or RnD kind of work in Data science, Automation or tooling(linux at present)
I am comfortable with Linux, JavaScript, Golang, Java, C and Web Technologies like Css, Html, WebAPIs etc.
If you are reading or building anything from CS technical papers let me know, I would like to be part of it
GH Profile: github dotcom neophyt3
E-Mail: pradeepchauhan91 at gmail dotcom
PS: I am not into web page or web design kind of work
Location: Mumbai, India
Remote: Remote Only
Willing to relocate: No
Technologies:
Nodejs[Graphql, REST apis and tooling], Golang(basic)[web apis, tooling], python, DockerCE, Php Laravel, Mysql, MongoDb, elastic search, nginx
Résumé/CV:
I am an independent developer. I am interested in develoing MVP products. I like linux and like creating apps around it.
Currently I am fulltime working with Nodejs and learning golang during my free time. I mostly create apis and tools and have some experience with ui development with Angularjs1 and have knowledge of reactjs. My past expertise includes Laravel app development, WordPress plugins, Magento 1,9 extension development. I am mostly dependent on docs and even checkout
Github for code examples. I charge for milestones only.
If you like know more please feel free to contact me
Email:pradeepchauhan91 attherate gmail dot com
I can't provide details on any innovations we've done with sites like google, but in general if you want to crawl google you'll want to get "many, many" IP addresses. I've heard of people using services like 2captcha.com but the best way is to obfuscate who you are.
If you can hit Google 60 times per minute per IP before getting blocked and you need to crawl them 1000 times per minute, you need 17 IPs per hour. Randomize headers to look like real people coming from schools, office buildings, etc... Lots of work but possible.
I do it using rotating proxies, stripping cookies between requests, randomly varying the delay between requests, randomly selecting a valid user-agent string, etc. It's a pain in the butt. And to scrape more than I do, faster than I do, would be pretty freaking expensive in terms of time and money.
Note that Google is pretty aggressive about captcha-ing "suspicious" activity and/or throttling responses to suspicious requests. You can easily trigger a captcha with your own manual searching. Just search for something, go to page 10, and repeat maybe 5-20 times and you'll see a captcha challenge.
If Google gets more serious about blocking me then I'll use ML to overcoming their ML (which should be doable because they're always worried about keeping Search consumer-friendly).
I had one client, he paid me 65-70$ for scraping escort sites..
He had it coded from some one in portugese, it was pho scraper..
So i added 7 more sites .. and finally i quit becoz i was not getting private time .. every time there were people arround me
Btw i am from india.. ne1 looking for freelancer (php, nodejs, go, etc..) can get in touch..
GH Profile: github dotcom neophyt3 E-Mail: pradeepchauhan91 at gmail dotcom
PS: I am not into web page or web design kind of work