🤖 Web Crawling Politeness - emschwartz · Scour

Efficient Crawling for Scalable Web Data Acquisition (Extended Version)

arxiv.org·18h

🕷️Web Crawling

Introducing Markdown for Agents

blog.cloudflare.com·1d·

Discuss: Hacker News, Hacker News, Hacker News

🕷️Web Crawling

mleddy.blogspot.com·38m·

Discuss: Blogger

Don't interrupt everyone

noathere.org·2h·

Discuss: Hacker News

🪄Prompt Engineering

Hot bots: AI agents create surprise dating accounts for humans

techxplore.com·15h

💳Content Monetization

The Small Web is Tricky to Find

matduggan.com·10h

🕷️Web Crawling

Google releases preview of WebMCP – how AI agents interact with websites

searchengineland.com·2d

💳Content Monetization

Factional Drift: We cluster into factions online

idiallo.com·11h

🛡️Content Moderation

Stop talking to AI, let them talk to each other: The A2A protocol

europedigital.cloud·1d

A Conceptual Framework for Exploration Hacking

lesswrong.com·1d

🕳LLM Vulnerabilities

The Two-slice Team

kill-the-newsletter.com·7h

s3cur3/browsey_http: A browser-imitating HTTP client for scraping websites that resist bot traffic

github.com·1d

🕷️Web Crawling

LASER: An Efficient Target-Aware Segmented Attention Framework for End-to-End Long Sequence Modeling

arxiv.org·18h

📄Semantic Chunking

Terms of Service

felix.dognebula.com·1d·

Discuss: Hacker News

🕷️Web Crawling

The Agentic Pivot: Architectural Transformation of App-to-App Communications in the Fintech Tech…

pub.towardsai.net·7h

💰Web Monetization API

Humanizing vs. Human

blog.gorewood.games·11h

Search Engine Tooling

joshbeckman.org·2d

📊Search Ranking

Skip the Tips 2026-02-13 18:44:05.197291+01

flutterby.com·5h

🔗 Keeping feature creep at bay

yellowduck.be·9h

📇Indexing Strategies

Links 13/02/2026: SUSE Uses Microsoft Internally, MElon's Company Helps Turn Epstein Files Into Child Abuse (After the Pornography Scandals)

techrights.org·13h

🪄Prompt Engineering

Loading more...