Scoring AI for Accessibility: A Rubric-Based Framework
Sarah Cogley
Digital Collections and Repositories Librarian
University at Buffalo, SUNY
Stacy Snyder
Digital Collections Projects and Compliance Librarian; Accessibility Coordinator
University at Buffalo, SUNY
This briefing explores a comprehensive evaluation of generative AI tools for creating alternative text and long descriptions for digital collections at the University at Buffalo Libraries. Prompted by the need to address extensive accessibility remediation as required by the Americans with Disabilities Act Title II regulations, the project team analyzed outputs from three AI tools using a rubric designed to assess three criteria: factual accuracy and correctness, relevance and task completion, and clarity and communication quality. Findings showed limitations of current AI technologies, such as hallucination, omission of key visual elements, and cultural insensitivity. The briefing will discuss challenges, such as how to evaluate tools that are emerging and dynamic, establishing guidelines and best practices for accessible metadata in digital collections that contain diverse content and format types, and how to integrate AI in digital collections workflows in a scalable and sustainable way. The presenters will highlight lessons applicable across institutions, including the importance of cross-unit collaboration with colleagues in user experience and accessibility, and will share the rubric, workflows, and project documentation.
Using Artificial Intelligence to Extract and Understand Cultural Heritage Materials
Paul Gallagher
Associate Dean for Resources and Digital Strategy
Western Michigan University
Much of our cultural legacy is hidden. Despite decades of effort to convert paper documents to electronic form, many issues still impact users’ ability to discover content. Traditional optical character recognition only works so well; handwritten documents need manual transcription, and digital content platforms don’t always provide researchers with meaningful ways to interact with historic content. In this presentation, learn how one library is working with emerging artificial intelligence (AI) models to extract text from heavily degraded documents and historic handwriting, using modern “vibe-based” application development to present cultural legacy items in a new way. Learn what tools are available, how they are best used, and how declining technical barriers will make this more accessible to information professionals. Gain a deeper understanding of powerful uses of AI beyond chat models, and how these approaches may benefit your own organization. Drawing on lessons from a recent project, this session is designed for librarians, archivists, and digital humanities practitioners interested in practical applications of AI for cultural heritage materials—regardless of technical background.
