# This is an RSS-only post. It's a secret! Read more about RSS Club. The robots.txt for evantravers.com is now generated on site build from darkvisitor.com's API. Pretty simple change. I added this to the helpers block in my middleman's config.rb: helpers do # […] def dark_visitors headers = { Authorization: 'Bearer ' } body = { agent_types: ['AI Assistant', 'AI Data Scraper', 'AI Search Crawler', 'Undocumented AI Agent'] } data = URI.encode_www_form(body) endpoint = URI(' respo...

Read the original article