The Confederation of Open Access Repositories (COAR) has just announced the new Dealing with AI Bots website that provides a wealth of information on bots and crawlers impacting the services and operations of open repositories, including mitigation strategies; see the announcement below for more information. This work follows a survey COAR conducted of its members in spring 2025 to better understand the scope and scale of the problem; COAR Executive Director Kathleen Shearer provided a video report of the project for the July 2025 issue of CNI’s Pre-recorded Project Briefing Series, available here: Artificial Intelligence Bots and Repositories—Results and Next Steps from COAR Survey.
My sincere thanks to Kathleen Shearer and all involved for this impactful contribution, and for granting me permission to share it with the CNI community.
-Diane Goldenberg-Hart, CNI
— — —
I am very pleased to announce that the initial outcomes of our work on dealing with bots is now available in the form of a website.
The website provides the background context, conceptual model and a description of various mechanisms for mitigating the impact of bots (including their efficacy). It also provides a mechanism for you to contribute your experiences with various measures.
It is important to note that,
A major conclusion from this work is that there is no “silver bullet” solution to this problem. It is clear that the nature of traffic on the Web has changed, and it seems certain that repositories will continue to deal with a range of bots, both welcome and unwelcome, and that the behaviour of such bots will in many cases be problematic. Repositories will need to walk a fine line between protecting their operations from being overwhelmed by traffic from unscrupulous actors, and maintaining their core mission of providing open access to legitimate users and machines.
This doesn’t represent the end of COAR’s work in this area. We have several AI-related work that we will be pursing in the future as outlined on our new “AI and Repositories” webpage
- Advance COAR’s conceptual model using AI to support discovery of multilingual scholarly research outputs
- Maintain a watching brief on the impact of AI bots on repositories and open science
- Ensure the integrity of the knowledge commons in an AI-mediated world
- Develop recommended practices for repositories to better support machine access
I want to extend my big thanks to the members of the Dealing with Bots Task Group and especially to Paul Walk, who did the majority of the heavy lifting to produce this resource.
Here is the link to the Dealing with AI Bots website.
If you have any questions or comments, please get in touch with me.
Best, Kathleen
Kathleen Shearer
Executive Director, COAR
kathleen.shearer@coar-repositories.org
