Open repositories are experiencing increasing disruption from AI bots and other automated crawlers. These bots, which are growing in number and frequency, often exhibit aggressive behavior that can result in system slowdowns and service outages. In response, some repositories have resorted to restricting machine access to their collections. However, these defensive measures have at times inadvertently blocked legitimate services such as scholarly aggregators, indexing platforms, and directories.
To assess the scope and impact of this issue, the Confederation of Open Access Repositories (COAR) conducted a brief survey among its members in April 2025. A total of 66 responses were received from institutions across multiple regions, including 22 from Canada and the United States, 22 from Europe, 9 from Latin America, 6 from Asia, 4 from Australasia, 2 from Africa, and 1 unspecified.
Survey results indicate that over 90% of respondents are encountering AI bots on a regular basis—often more than once per week—with many reporting associated disruptions to repository services. Institutions have adopted a range of mitigation strategies, combining techniques such as rate limiting, firewall configurations, robots.txt rules, and shared whitelists to manage or prevent bot access.
These findings will inform further engagement with COAR members, leading to the formation of a Task Group tasked with developing recommended practices. The goal is to ensure that repositories can remain openly accessible while minimizing the risks posed by disruptive automated activity.
Click here to read the original press release.
