Trivial anti-crawler with Caddy (opens in new tab)
With the internet being crawled to death to feed the AI God, it's becoming seriously annoying to exposed web content on the internet. While , it's absolutely overkill. Instead, I'm taking advantage of git/ so that repository are still cloneable. git.dustri.org { import tls import noindex import compress @unverified { not header Cookie *not_a_crawler=1* not header User-Agent git/* } handle @unverified { header Content-Type text/html respond document.cookie = 'not_a_crawler=1'; w...
Read the original article