CNI: Coalition for Networked Information

  • About CNI
    • Membership
    • CNI Collaborations
    • Staff
    • Steering Committee
    • CNI Awards
    • History
    • CNI News
  • Program Plan
    • Current Program Plan
    • Program Plan Archive
  • Topics
  • Events & Projects
    • Membership Meetings
    • Workshops & Projects
    • Other Events
    • Event Calendar
  • Resources
    • Publications by CNI Staff
    • Program Plan
    • Pre-Recorded Project Briefing Series
    • Videos & Podcasts
    • Follow CNI
    • Historical Resources
  • Contact Us

Searching Across Time: Issues and Opportunities for Full Text Search of Web Archives Using Nutch

Home / Project Briefing Pages / CNI Fall 2007 Project Briefings / Searching Across Time: Issues and Opportunities for Full Text Search of Web Archives Using Nutch

December 9, 2007

Kris Carpenter Negulescu
Director, Web Archive
Internet Archive

For the past two years, the Internet Archive (IA) has used Nutch/Lucene open source tools to generate full text search indexes of archival Web content for the National Library of Australia, the Bibliothèque Nationale de France, the Library of Congress, and the National Archives and Records Admininstration (NARA). The IA has also produced an experimental search service for consumers of its own historic Web collections, The 20th Century Find, which encompasses content harvested from 1996-1999.

This presentation will review each of these case studies, including lessons learned to date searching Web archives at a scale of 100 million – 1+ billion URLs, and the challenges associated with searching across time, both technically as well as those specific to an end user’s experience of a Web archive. The case studies presented are specific to Nutch/Lucene implementations, but implications for searching archives in general will be the primary focus of this presentation.

http://www.archive.org/
http://lucene.apache.org/nutch/

 

Share this:

  • Click to share on Facebook (Opens in new window)
  • Click to share on Twitter (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)

Filed Under: CNI Fall 2007 Project Briefings
Tagged With: CNI2007fall, Project Briefings & Plenary Sessions

Last updated:  Friday, November 2nd, 2012

 

Contact Us

21 Dupont Circle
Suite 800
Washington, DC, 20036
202.296.5098

Contact us
Copyright © 2023 CNI

  • Copyright Policy
  • Privacy Policy
  • Site map

Keeping up with CNI

CNI-ANNOUNCE is a low-volume electronic forum used for information about the activities and programs of CNI, and events and documents of interest to the CNI community.
Sign up

Follow CNI

  • View cni.org’s profile on Facebook
  • View cni_org’s profile on Twitter
  • LinkedIn
  • YouTube
  • Vimeo

A joint project