Vandana Srivastava
AI/Data Science Specialist
University of South Carolina
Libraries hold vast collections of data that document social, cultural, and political history across time and place. Most of these materials are housed in a traditional repository that relies on keyword-based search functionality. While commercial large language model (LLM) chatbots like ChatGPT can respond about such collections, their answers are often unreliable or hallucinatory. These limitations stem from a lack of direct access to library collections, gaps in training from these collections, and the absence of systematic evaluation for accuracy and scholarly quality. The Lighthouse project—a collaboration among librarians, historians, and artificial intelligence (AI) researchers at the University of South Carolina—is an initiative to address these challenges. The team is developing a chatbot trained on eight curated South Carolina civil rights collections using open source LLMs like Llama. To create a trustworthy AI tool, a systematic evaluation of the chatbot’s responses is being done on a question-and-answer dataset prepared by historians and civil rights scholars and further reviewed by experts for accuracy and comprehensiveness. The chatbot is now being introduced to historians for testing. Their feedback informs ongoing improvements, helping ensure the tool supports rigorous research practices while demonstrating the role libraries can play in shaping reliable, accountable AI for scholarship.
https://lighthouse.chat.library.sc.edu/
