Will we soon see navigation apps from major AI companies? At least that would fit the approach of relevant companies that want to launch browsers, social networks, shopping functions, and word processing programs in addition to the core product of the chatbot. But where will the data come from? This concern is currently driving OpenStreetMap (OSM).
According to the project operators, bots are currently harvesting open data on a large scale. These automated visits cost money – for example, due to increased hosting costs. This can serve to build up one’s own services. However, it is also possible that the bots are fetching information from OpenStreetMap that will be used for requests to chatbots. The project has not yet responded to an inquiry for details about the crawlers.
O…
Will we soon see navigation apps from major AI companies? At least that would fit the approach of relevant companies that want to launch browsers, social networks, shopping functions, and word processing programs in addition to the core product of the chatbot. But where will the data come from? This concern is currently driving OpenStreetMap (OSM).
According to the project operators, bots are currently harvesting open data on a large scale. These automated visits cost money – for example, due to increased hosting costs. This can serve to build up one’s own services. However, it is also possible that the bots are fetching information from OpenStreetMap that will be used for requests to chatbots. The project has not yet responded to an inquiry for details about the crawlers.
Open data, open problems
OpenStreetMap is a collaborative project to create free, editable maps. This is comparable to the content of Wikipedia. This is also collected collaboratively and can be viewed by anyone. Navigation services, for example, use the map data. In addition to streets, the data also includes information on sights and traffic infrastructure.
On LinkedIn, the responsible parties write in a plea for help that the bots could endanger the entire project. And they warn that this also endangers other projects. The responsible developer, Grant Slater, is quoted as saying that in recent years he has seen one or a few IPs that have made more than 10,000 requests. Now, in one week, there are 100,000 IPs engaged in coordinated scraping – with each IP making a few requests. This also makes it impossible to block the bots or IPs.
Providers like Cloudflare or Akamai have also recognized that AI bots are becoming an increasing problem. They offer services that are intended to detect and combat bots – before they reach the servers. Captchas are a well-known way to block automated requests.
(emw)
Don’t miss any news – follow us on Facebook, LinkedIn or Mastodon.
This article was originally published in German. It was translated with technical assistance and editorially reviewed before publication.