🤖 Web Crawling Politeness - emschwartz · Scour

local.html: Social Discovery by Browser-Based Crawling

lofihi.fi·18h·

Discuss: Hacker News

🕷️Web Crawling

BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents

research.perplexity.ai·5h

🛡️AI Security

A better way to crawl websites with PHP

freek.dev·9h

🕷️Web Crawling

Ask don't tell: Reducing sycophancy in large language models

arxiv.org·21h

Cloudflare’s new Markdown support shows how the web is evolving for AI agents

thenewstack.io·14h

🕷️Web Crawling

The AI Scraping War: LLM Crawlers Are Breaking the Web

webdecoy.com·1d·

Discuss: r/webdev

💳Content Monetization

Speed of Sites Matters

techrights.org·23h

🚀Web Performance

The Complicators, The Drama Aggregators, and The Avoiders

randsinrepose.com·10h

📰RSS Reading Practices

Zero-Waste Agentic RAG: Designing Caching Architectures to Minimize Latency and LLM Costs at Scale

towardsdatascience.com·1d

💾Prompt Caching

OpenClaw: Setting Up Your First Personal AI Agent

kill-the-newsletter.com·10h

🛡️Open Policy Agent

AI Web Search and Scraping

github.com·1h·

Discuss: Hacker News

🔎Meilisearch

How Google’s Universal Commerce Protocol changes ecommerce SEO

searchengineland.com·12h

💰Search Economics

Parse, Don't Guess

architecture-weekly.com·9h

💾Binary Formats

Architecting and Evaluating an AI-First Search API

research.perplexity.ai·5h

🕷️Web Crawling

Blog/2026-03-02/Copy To Claude

wiki.roshangeorge.dev·18h

💻Claude Code

How fast is fast enough? Rethinking web performance in pragmagical terms

speedcurve.com·1d

🚀Web Performance

Can LLM chat be less prolix?

lesswrong.com·6h

💾Prompt Caching

Blog - Measuring Web Accessibility from Crawl Archives

commoncrawl.org·1d

🕷️Web Crawling

Status Go vs. Status Quo

doc.searls.com·3h

💹Platform Economics

February quick-takes

blog.senko.net·5h

Loading more...