Assessing the Academic Networked Environment: Proposals

Virginia Polytechnic Institute and State University (Virginia Tech) is a land-grant research university located in Southwest Virginia. With an enrollment exceeding 24,000 students, it is the largest university in the state. Although Virginia Tech has an established reputation for leadership in technology, President Paul Torgersen has recently reaffirmed that role by stating in the 1996-2001 Update to the University Plan, "we must not only anticipate but lead the revolution in information technology that gains momentum daily." We, in Information Systems, regard this statement as our mandate to work with the university community to develop new approaches to teaching and learning, the dissemination of knowledge, our interactions with students and alumni, and our partnerships with our constituencies.

We believe that the assessment of the network environment is vital, but we recognize that the practice of assessment is not yet part of the university culture and sharing assessment results is not yet the norm. We feel that our participation in the CNI project will help establish standard tools and processes for network assessment and also demonstrate the strategic value of a comprehensive, ongoing assessment program. As an example of our progress in this area , the university has recently undertaken a number of ambitious technology initiatives, each of which has a major assessment component. Virginia Tech is also in the middle of a major self-study, in which a major focus is on our use of technology. The following is a brief description of several of the projects in which institutional team members are currently involved, their assessment components, and how they will relate this work to the CNI Project:

Campus Internetwork Development:

Virginia Tech has a long standing committment to maintaining a state of the art campus internetwork. Dramatic increases in the demand to support a variety of network applications coupled with continued pressure to contain costs have led to a complex series of infrastructure modifications. We recognize a requirement to support a spectrum of approaches to utilizing communications technology in the teaching and learning process ranging from Web based delivery to real time, fully interactive, multi-way emulation of an enhanced classroom environment at a distance. Also, we must support the requirements of networked computing research which places ever increasing demands on our network resources both on and off campus. We need the capability to deliver an appropriate Quality of Service when and where required without subjecting our constituents to an inordinate cost burden.

Virginia Tech is currently engaged in deploying a campus backbone infrastructure and building network upgrade which we believe will deliver the capability to meet these increased and heterogeneous requirements at reasonable cost. These upgrades are based on a combination of technologies including asynchronous transfer mode, switched ethernet, fast ethernet, and microsegmentation of shared ethernets. We will implement the methods and procedures described in the McClure/Lopata manual for network benchmarking and traffic flow analysis comparing and contrasting them with our current methods. Our goals are to assess the cost effectiveness of our architecture, to identify areas where further improvements are needed, and to characterize those needs for purposes of accurately designing appropriate solutions.

Faculty Development Initiative (FDI):

The Faculty Development Initiative is a large-scale effort to invest in our faculty by providing them with the opportunity to rethink their teaching and explore the potential of instructional technology for improving the effectiveness of the teaching-learning process. The FDI is part of the Instructional Development Initiative (IDI), which provides resources to improve student access to the network and enhance the technological capabilities of our instructional facilities. The FDI began with three pilot faculty workshops during the Summer of 1993, and by the fall of 1997 almost all of the 1500 faculty at Virginia Tech will have participated in these intensive three to four-day workshops. Sponsored jointly by the Provost and the Vice President for Information Systems, this initiative is providing the opportunity for all faculty over a four-year period to participate in an intensive workshop centered on the integration of instructional technology into the curriculum. The standards described in the McClure/Lopata manual will help our ongoing assessments of the short and long-term effects of these sweeping faculty development efforts. Use of Library Resources in a Networked Environment:

Anecdotal evidence suggests that a growing segment of our library user community turns to networked information resources to meet all or a significant part of their information needs. For example, the electronic reserve system offered by the Libraries gives faculty an opportunity to provide syllabi, lectures, demonstrations, and other class resources to students so they can access them from any networked connection. The Libraries also provide access to a number of networked information services and full content publications as part of the Virtual Library of Virginia (VIVA). Through the Electronic Theses and Dissertations Project (ETD), the University is partnering with other research institutions to provide full content access to these resources for the global community of scholars.

In the library we have seen a decrease in the number of physical items that are put on regular reserve, and a decrease in the actual use of traditional reserve. Use of electronic reserve is growing. In the first full year of VIVA's provision of First Search services, Virginia Tech was a significant user of the database resources provided there. We are also anticipating greater use of the electronic theses and dissertations than that which was noted for the earlier print publications. We are interested in capturing use data about access to information resources in the networked environment, so that we may modify our existing service models, or create new library service models that are responsive to the changing needs of the academic information user.

User Services:

Every student, faculty and staff member at Virginia Tech is issued a personal account (PID), which provides them with e-mail service and access to the Internet. As a front line department, User Services provides help desk support, integration of new computing and network services, short courses, and documentation for Virginia Tech students, faculty, and staff. The definition of "student" has expanded greatly from the traditional 18 to 21 year old dormitory resident to include adult and distance learners who take courses via satellite or wide area network technology. Resources such as digital libraries are available to the K-12 community as well as lifelong learners. As educational systems become more complex, support needs likewise become much more challenging, and yet each user must have the same simple procedure to solve his or her problem. As a result, we must constantly anticipate the support needs of the users in order to guarantee their uninterrupted access to network resources.

We believe that our participation in this project will improve communication across Information systems, resulting in faster, more comprehensive support for our users.We would like to use the McClure/Lopata manual to analyze our help desk service and assure that our support methods are keeping up with the increasingly complex needs of the users. For example, we have already gathered help desk statistics for the past year on the number of questions answered (by phone, walk-in, and e-mail), the response time for each question, the types of users, the types of computers used and 12 broad question categories (PC Internet software, e-mail, student systems, administrative systems, etc.). Other quality control methods are not as quantitative but more labor-intensive. For example, our full-time staff reviews each e-mail answer, and monitors the student consultants' side of phone consultation sessions. We would like to institute a customer call-back plan to better assess the overall quality of our service.

Cyberschool and ACCESS:

Virginia Tech's Cyberschool began in the College of Arts and Sciences as an experimental arena in which interested faculty could transform their courses by utilizing network technology to increase communications and enrich course content, while simultaneously reducing in-class lecture time. Although the design varies for each course, a typical class uses a web site that allows students to communicate with the instructor at their convenience, view and/or download course materials, and participate in synchronous and asynchronous discussion sessions. Encouraged to work at their own pace and form electronic "study groups," students noticed a marked increase in the amount of communication among their peers and with the faculty. Assignments and exams are also administered through the network, and feedback is delivered to the students electronically.

As a result of our experience with Cyberschool, Virginia Tech was awarded a grant from the Sloan Foundation to use asynchronous networked communication to transform a series of first and second-year courses in biology. Selected biology faculty have formed interdisciplinary teams with personnel from Educational Technologies to "scale up" the Cyberschool model from smaller classes (typically 10-25 students) to the much larger classes (200-400) found at this level. In addition, a much more methodical, exhaustive and fine-grained assessment is being conducted for this project. Three to four surveys have been conducted in each of the seven classes under study, and a detailed analysis of class web site traffic is underway. In addition to these quantitative measures, we are also conducting a qualitative study consisting of extensive interviews (video, telephone and e-mail) with students and faculty. Finally, we are participating in a national benefit-cost study of media-based courses, with our focus being the effects on quality of life for faculty who change from traditional to network-based modes of teaching. The procedures in the McClure/Lopata manual will help us standardize our data collection and analysis.

Non-Traditional Self-Study:

In January 1997 Virginia Tech begins the second year of a multi-year, comprehensive assessment, a non-traditional self-study entitled Transforming Virginia Tech in the Information Age, conducted under the auspices of the Southern Association of Colleges and Schools (SACS) for the purpose of re-affirming the university's regional accreditation. Comparing and testing the McClure/Lopata data collection requirements for support services as they relate to the SACS "must statements" in the context of Va Tech's self-study will provide useful information on correlation between the two.

In preparing our response to the CNI's Call for Participation in the Assessing the Networked Environment Project, we discovered a great deal of uncoordinated and overlapping assessment activity within Virginia Tech's Information Systems. While committed -- and indeed, required -- to measure the effectiveness of our use of technology, those people responsible for network infrastructure development, maintenance, and the delivery of network-based information services are not necessarily communicating among themselves. Participation in the project will benefit Virginia Tech by encouraging us to use standard assessment measures, to share and coordinate the results of individual assessment with each other, and to work with our colleagues in the larger academic community.

Institutional Team:

Joanne D. Eustis received her Bachelor of Art in literature, her Master of Library Science and her Master of Arts in literature and film studies degrees from Indiana University. She is currently completing a Doctor of Philosophy degree in Environment Design and Planning, College of Architecture, at Virginia Tech. She joined the staff of the Vice President for Information Systems Office as the Director of Planning and Program Review in 1995. Prior to that, she served for several years as the Interim Director of Virginia Tech's University Libraries. Her service to Virginia Tech spans nearly 20 years, and includes extensive involvement with the governance system culminating in a term as President of the Faculty Senate. As the Director of Planning, she provides leadership for the development of an intraorganizational shared vision and strategic plan.

David Taylor received his B.A. (English) and M.A. (Communications) from the University of Texas at Austin, and his Ed.D. (Instructional Technology) from the University of Houston. He is an assistant professor (professional faculty) and instructional designer in Educational Technologies, a division of Information Systems at Virginia Tech. He serves as a consultant for faculty on video and multimedia instructional development projects, and as an instructor in the Faculty Development Institute. He is currently coordinating assessment activities for two projects: 'A'-TECH, an internal initiative designed to address the unique needs of student-athletes through the use of mentoring and instructional technology; and the ACCESS project, sponsored by the Alfred P. Sloan Foundation.

Jeff Crowder received a Bachelor of Arts degree in economics from the University of Virginia in 1984 and a Master's degree in Instructional Technology from Virginia Tech in 1996. Since 1985, Jeff has worked as a communications systems engineer and consultant for Virginia Tech. His work has centered chiefly around high performance data networks, integrated systems, internetworking, and wide area networking. For several years, Jeff served as the Chief Network Architect for the campus internetwork before moving to a consulting and planning role with the Research and Development group. Most recently, he has lead the effort to develop and implement NETWORK VIRGINIA (a.k.a. The Virginia Broadband Education Network) which delivers advanced, communications services via ATM to education and government statewide.

Eileen Hitchingham has a B.S degree in Chemistry from Chestnut Hill College, a M.S. in Library Science from Western Michigan University, and a Ph.D in Education (Evaluation Research) from Wayne State University. Her experience with networked information systems began with her work as a MEDLINE analyst at Harvard's Countway Library of Medicine. She later introduced online systems and served as Automation Coordinator at Kresge Library, at Oakland University in Michigan. Her administrative experience includes serving as Dean of Libraries at the University of Idaho, Drexel University, and presently at Virginia Tech. She also served on the faculty of the School of Library and Information Studies at Drexel University. Her publications and presentations have focused on networked information services and their users in academic and K-12 environments.

Kimberley Homer received Bachelor of Science and Master of Science degrees in Electrical Engineering from Virginia Tech. She came to work for Virginia Tech in 1987 as a communications engineer, managing the design and construction of the campus cable television system and satellite communication facilities. She has done communication consulting for several higher education projects, including a turnkey satellite downlink/video system for the University of Pennsylvania's Annenberg School. In 1992, she helped start the Blacksburg Electronic Village (BEV) project, designing the user support methodology and creating BEV's first Internet information (Gopher) server. In 1994, she joined the Computing Center's User Services department, where she integrated student consultants in to the help desk function. Under her guidance, the help desk has doubled the number of questions answered and halved the average response time. Kimberley is currently working on a project to provide more formal assistance, through training and networked databases, to the technical support staff in the academic and administrative departments on campus.

---