Assessing the Academic Networked Environment: Proposals

Statement of Interest and Experience

The University of Washington (UW) has made a substantial investment in both network infrastructure and networked information content and is interested in using assessment tools that can measure the impact of this investment and provide direction for future efforts that best meet the needs of the campus community. While we have accomplished some significant assessment work, we look forward to participating in this project as a way of focusing our efforts using the manual and expertise from the project leaders and institutes, broadening the assessment effort to include other relevant campus units and individuals, and to employ a more diverse set of assessment techniques, especially for qualitative information.

The UW Libraries in collaboration with Computing & Communications has developed what has been called one of the best university information environments available. A rich array of information resources are available through this network and include the library catalog, bibliographic databases, numeric data, reference works, University information, and World-Wide Web connections. The UW is fully networked with connections to all buildings, offices, and most classrooms. The UW was a pioneer in establishing an easy-to-use email system (Pine) that now is used by hundreds of other institutions world-wide and also developed a graphical user interface for databases called Willow which again is used by other institutions. The UW has encouraged use of the network by making available computer hardware in student labs and supporting faculty computers through a faculty workstation initiative. Each faculty, staff and student is entitled to a free electronic mail account and computer space for both departmental and individual Web sites. Dial IP is provided to faculty, students, and staff for off-campus access. UW is the lead institution for library network development to establish a state-funded consortium of higher education sites in Washington. More information on the UW networked environment is available at http://www.washington.edu/home/complib.html .

The UW is recognized as a national leader for its work in using information technology and networked information to enhance teaching and learning. In response to the three-pronged challenge of bringing technology into the service of teaching and learning, the new information literacy, and the creation of community at a large research university, the UW developed a holistic, campus-wide approach called UWired. The primary goal of UWired is to create a networked electronic community in which communication, collaboration, and information technologies become ongoing, integral parts of teaching and learning. More information on UWired is available at http://www.washington.edu/uwired/.

Another area where the institution has made extensive progress in networking is in the Health Sciences. The UW was awarded a five year grant in 1994 (only one of four institutions currently funded) to develop an Integrated Advanced Information Management System (IAIMS) for the Health Sciences by the National Library of Medicine (NLM). IAIMS philosophy is to build and support multidisciplinary distributed information systems which are critically dependent on networking. Of special interest for assessment in the CNI project are the modules on Integrated Databases for Enhancing Academic Learning and Knowledge Resources and Systems Infrastructure. See http://www.hslib.washington.edu/iaims/ for additional information on the IAIMS program at the UW.

SPECIFIC ASSESSMENT AREAS OF INTEREST

The UW is especially interested in the impact of networking on teaching, learning and research. Some questions we hope to address are:

What difference has networking made in the way students are taught and how they learn?

What proportion of courses employ a networking component or project?

How has networking changed the information seeking behavior of students and faculty?

How has networking changed the way students and faculty communicate?

How has networking changed the way research is done?

How has networking changed the use of libraries and library information resources?

How has networking changed the roles of librarians and computer professionals?

INSTITUTIONAL ASSESSMENT EXPERIENCE PARTICULARLY INVOLVING NETWORKING

In addition to data captured by computer logs and help desks, there have been several areas where extensive assessment work has been accomplished. The UW Libraries conducted library use surveys in 1992 and 1995 which included questions on networking and use of networked resources. These surveys established not only an information baseline but also provided direction to library programs and planning. The 1995 survey showed a campus population that was well-connected and used networked resources on a regular basis. Indeed, faculty who used the Libraries regularly (at least weekly) were more likely to use library resources and services through an office computer than actually visit the library. 1995 survey results and a brief analysis are available at http://www.lib.washington.edu/pubservices/survey/.

UWired assessment has focused on teaching and learning and has developed information on networking use among first year students. A separate evaluation team was established and is now working on assessment of upper division linked courses. UWired uses a number of tools to evaluate the effectiveness of its efforts, including pre- and post- testing for computer and information literacy, e-mail monitoring and polling, evaluation surveys, focus groups, and instructor evaluations. The UWired Evaluation Team and UW faculty will be participating in a day-long workshop on evaluating technology and teaching in late February. The workshop will be conducted by Steve Ehrmann of the Flashlight Project. The Flashlight Project seeks to develop evaluation measures and procedures that institutions of higher education can employ to answer questions concerned with technology, teaching, and learning. The project has received funding from the Fund for the Improvement of Post-secondary Education, the Annenberg/CPB Project, and the American Association for Higher Education.

The Office of Educational Assessment (OEA) regularly conducts evaluative research directed toward the improvement of UW educational programs through the analysis of existing institutional records and the collection of original data from students and staff. Staff members are specialists in educational measurement and evaluation and provide technical expertise in a wide range of areas related to data management, statistical analysis, and interpretation. OEA has collaborated on the Libraries, C&C, and UWired assessment programs.

TEAM MEMBERS

Steve Hiller
University of Washington Libraries
Steve Hiller is Head, Science Libraries at the UW. He is co-chair of the Libraries Task Force on Library Services and spearheaded assessment efforts in the Libraries during the past five years especially in coordinating library use surveys. He was a participant in the panel presentation "The User Centered Library and Library Community Analysis: Making It Happen", given at the Association of College and Research Libraries National Conference in March 1995.

Nana Lowell
Office of Educational Assessment
Nana Lowell is Associate Director of the Office of Educational Assessment (OEA), a division of the Office of Undergraduate Education. Dr. Lowell holds a Ph.D. in Educational Psychology and has extensive experience in assessment and evaluation work, including survey research. Her primary responsibilities within OEA are instructional assessment, testing, and data analysis. She has recently taken on leadership of the UWired assessment effort.

Oren Sreebny
Computing & Communications
Oren Sreebny is Assistant Director for Client Services, Computing & Communications (C&C). He oversees the support, training and consulting services for C&C as well as the C&C computer labs. He is active in EDUCOM and other organizations that promote the development of information resources for academic institutions. He actively assesses computing support for faculty and students; an overview of his assessment work can be found in his recent article in "Windows on Computing" available at http://www.washington.edu/cac_docs/windows/issue19/teaching.html. He has also worked in the commercial information industry in user services and database development.

ADDITIONAL CAMPUS RESOURCES

Integrated Advanced Information Management System (IAIMS) and Health Sciences Library staff will assist in assessment of networking within the Health Sciences.

Representatives of the Faculty Council on Educational Technology (FACET) will serve as an advisory group. FACET, a broad-based group of faculty and administrators, is responsible for all matters of policy relating to the introduction of educational technology in instruction. Representatives of the Faculty Council on University Libraries will also serve on the advisory group.

The UWired Faculty Advisory Committee will also provide advice and support to the project. The Committee is comprised of faculty, librarians, and computing professionals engaged in the use of networked technology in teaching and learning.

Several faculty members with expertise in information use assessment have agreed to provide support for the project. They include:

Raya Fidel, Associate Professor, Graduate School of Library and Information Science brings expertise in the fields of information storage and retrieval, and subject analysis. Her research focuses on online searching behavior.

Judith Ramey is Associate Professor in the Department of Technical Communication, College of Engineering. Her research interests are in usability research and user-centered design processes. In 1989, she founded the department's Laboratory for Usability Testing and Evaluation (LUTE), of which she is Director.

---