Project Briefing: Image Retrieval Benchmark Database: A Report on a CLIR/CNI Exploration Jennifer Trant Executive Director Art Museum Image Consortium (AMICO) The rapid increase in the quantity of visual materials in digital libraries--supported by significant advances in digital imaging technologies--has not been accompanied by a corresponding advance in image retrieval technologies and techniques. Digital Librarians are currently without tools for evaluating either content-based or metadata-based image retrieval systems. They have difficulty assessing existing systems of image access, evaluating proposed changes, or comparing metadata-based to content-based image retrieval. Benchmarking has been proposed as a possible solution to this problem of comparison. CLIR and CNI have supported the exploration of these issues. This session will report on the activities of the CLIR/CNI project to date. We'll review the components of a possible Image Retrieval Benchmarking Service and explore the strategies that might serve to develop the service and a community of researchers whose work would ultimately improve digital library service.