The Confederation of Open Access Repositories (COAR) has launched a new task force to address increasing disruptions caused by aggressive automated crawlers, often assumed to be AI bots collecting data for generative AI training.
A COAR survey conducted in April 2025 revealed that 90% of survey respondents report frequent interactions with such bots, often weekly, resulting in system slowdowns and service outages. Although some crawlers are benign, a growing number are highly aggressive, straining repository infrastructure and demanding significant time and resources for mitigation.
The surge in bot activity over the past two years has prompted some repositories to consider restricting access entirely. This would affect both human users and legitimate systems, diminishing the broader value of open repositories. Existing mitigation tactics often unintentionally block legitimate services, compounding access issues.
In response, COAR established the “AI Bots and Repositories Task Force” on July 15, 2025. The group brings together technical experts and repository representatives to evaluate sustainable mitigation strategies that preserve open access principles.
• The task force will focus on:
• Articulating the scope of the problem with supporting data
• Documenting existing mitigation approaches
• Reaffirming the importance of allowing legitimate machine access to repositories
• Suggest solutions to address repository issues without hindering legitimate remote access
A final report with recommendations is expected by fall 2025.
Click here to read the original press release.
More News in this Theme
