Hacker Newsnew | past | comments | ask | show | jobs | submit | PatchRequest's commentslogin

Nope, i crawl everything myself.


I manually blacklisted .gov from being crawled on my side. felt like it wasn't worth the potential trouble


Meanwhile you crawl anybody else?

.gov isn't magically any more able to take you to court than a private org


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: