Stan Gunn
Associate Dean for Information Technology
University of Virginia
Erich Purpur
Science and Engineering Research Librarian
University of Virginia
In January 2026, the University of Virginia Library conducted the first of many proposed hands-on exercises to test four major large language models’ (Anthropic’s Claude, OpenAI’s ChatGPT, Google Gemini, and Microsoft CoPilot) suitability for the transcription and translation of handwritten documents from the University of Virginia Library’s special collections. Thirty-two staff members from multiple disciplines were given standard prompts and rubrics for evaluation, divided into groups, and given an opportunity to select an item to scan and provide to each LLM for transcription, and, if necessary, translation. Time was provided at the end for groups to share their findings and key insights. This initial exercise will be used to calibrate prompts and rubrics for similar events that will be held to continue evaluation of LLMs for processing of collections, and to inform faculty, students, and researchers on the effective use of these tools.
