Information Literacy on Our Campus: An Essential Learning Outcome
The Library and Information Literacy Instruction Program provides leadership for efforts to assess students' information literacy skills at the classroom, course, program, and campus levels.
The UW System is a participant in the Association of American Colleges and Universities Liberal Education and America’s Promise (LEAP) project. From this project, the UW-Madison campus adapted a set of Essential Learning Outcomes. These learning outcomes are used as an assessment framework for departments and programs across campus. Information literacy is included among the "intellectual and practical skills" that students should develop.
The following are examples of assessment activities that Library and Information Literacy Instruction librarians have been engaged in across campus.
The new 2008 Preface outlines university-wide assessment standards, goals, and guidelines for measuring and evaluating student academic achievement. The Plan requires every UW academic program that includes general education goals to have an assessment plan, in order to be able to “construct a case of evidence to evaluate if students are achieving these learning expectations.” It is important that these assessment methods be used on a regular basis at all academic levels in order to receive a full picture of student learning. Each plan must be consistent with the Essential Learning Outcomes, which include "Information Literacy in the area of Intellectual and Practical skills".
According to the Plan, the primary tool used for measuring information literacy at the campus level has been standardized testing. While these efforts have shown that students have achieved an “acceptable level of performance," it is indicated that “[a] more authentic assessment of student learning will provide better information which can be used for program administration and improvement.” A future information literacy assessment is included in the Plan’s calendar for assessment projects scheduled over the next several years (slated for the 2010-11 school year).
Our Students Assessment
The following are examples of Library and Information Literacy Instruction student assessment activities.
An Assessment Study of the Effectiveness of the General Education Communication “A” Requirement at the University of Wisconsin-Madison is an analysis of the self-reported learning of students who had completed the “Comm-A” requirement in comparison with those who, for whatever reason, had not. The students were asked to rate their writing, communication, and information literacy skills in a variety of ways. The results show that students who had taken the Comm-A requirement ranked their information literacy skills dramatically higher than those who had not, in such areas as locating research materials, properly citing the work of others, and understanding the issue of plagiarism. This was true across all five courses that fulfill the Comm-A requirement.
[LINK TO DOC.]
The NSSE Study Report: An Overview of the National Survey of Student Engagement 2008 Results for UW-Madison is a close-up view of freshman and senior undergraduate students at the UW who were surveyed as part of the National Survey of Student Engagement. The NSSE is used to “assess student involvement in practices associated with high levels of learning” at over 750 colleges and universities in the U.S. and Canada. The survey included questions about students' ability to make judgments about the value of information, to synthesize and organize ideas and theories, to apply theories or concepts to practical problems or new situations, and to use the Internet to complete assignments. In each of these areas, UW seniors consistently ranked their abilities in these areas higher than their first-year counterparts. For example, when students were asked questions about their ability to make judgments about the value of information, senior students responded with answers of “Quite a bit” or “Very much” at a rate near 70 percent, whereas first-year students responded with the same answer at 60 percent. Both first-year and senior students ranked their skills highly in these areas, but it is clear that their information literacy skills were impacted in their time at UW-Madison.
The ECAR Study of Undergraduate Students and Information Technology is an investigation of undergraduate students at several schools, including UW-Madison, and their use of IT tools in their personal lives and in their educational careers. The purpose of this study is for college-level educators to get a better “feel” for how IT affects students’ daily lives and how students' skills and knowledge in these areas can be best used for educational purposes. The students were asked to rate their skill levels in the following three areas: “using the Internet to effectively and efficiently search for information;" “evaluating the reliability and credibility of online sources of information," and “understanding the ethical/legal issues surrounding the access and use of digital information.” In all of these areas, students ranked themselves very highly, with almost 80 percent of the students rating themselves at near-“Expert” levels ("Expert" was the highest rating in the scale provided) in their Internet research abilities. In the other two areas, the students’ self-ratings were lower, but still relatively high.
The Project SAILS Test (Standardized Assessment of Information Literacy Skills) was administered to a small group of incoming freshman in 2006 and 2007. The test was used to find out more about the kinds of information literacy skills students have upon arriving on campus, before taking the Communication “A” and “B” courses. Although the sample size was ultimately considered to be too small to reach meaningful conclusions about UW freshman information literacy skills, it was found that there was no significant difference in information literacy skills between UW students and the national benchmark for students at similar institutions, and that UW students tested higher than average than their freshman counterparts in searching and retrieving resources.Librarians used the information collected through SAILS to inform their judgments about areas of information literacy could receive greater emphasis in the library module of the Communication “A” courses.
This article is an account of the development and implementation of required communication courses for undergraduates at the UW. The Communication “A” and “B” courses, which were created in reaction to concerns over the verbal and writing abilities of incoming UW student, was a university-wide effort that resulted in a decentralized model for teaching proper academic communication skills necessary to succeed in every discipline. These necessary skills include information literacy, and it is noted here as a result that “librarians have been treated as information professionals and brought more clearly into the teaching mission of the university.” The authors of the article call for a full-scale assessment of the Communication courses in order to determine the effectiveness of the courses as well as best practices for teaching these skills in the future. Citation: Westphal-Johnson, N. and Fitzpatrick, M. (2002). The role of communication and writing intensive courses in general education: a five-year case study of the University of Wisconsin-Madison. The Journal of General Education 51 (2), 73-102.
- Project SAILS was administered to incoming freshmen to establish an information literacy benchmark for incoming freshmen and to compare UW students to incoming students at similar institutions. (Spring 2007 & Summer 2007)
- Chemistry 346 utilized the Student Assessment of Learning Gains (SALG) which assesses learning as reflected by the student. Information Literacy questions were included; they related to each of the course’s assignments. (Fall 2004 & Spring 2005)
- Life Sciences Communication 100 students take a pre/post survey regarding their information needs and the effectiveness of a library session in meeting those needs for their persuasive-paper assignments. (Each semester)
- Engineering Professional Development 151, Mechanical Engineering 900, Chemical & Biological Engineering, and Interdisciplinary Engineering 413 students take a pre/post survey regarding their information fluency during a library session. (Each semester).
- Communication-A instructors were surveyed about the effectiveness of library instruction sessions in improving their students’ skills as demonstrated in coursework. Changes were made to the library module's curriculum (CLUE tutorial and library instruction session), based on instructor feedback regarding their students’ weaknesses and strengths. (Fall 2007)
- Biocore 304 instructors receive results of a student survey on the library session. Results are examined by the instruction team to inform revisions to the curriculum. (Yearly)
- Biology 151 students are required to complete a tutorial on evaluating Web sites. A Web-based survey is embedded in the tutorial, to enable students to immediately send feedback, questions, etc. Mass emails to students are generated to address comments and needs. (Each semester)
- To fulfill accreditation requirements, second-year medical students are required to submit records of search queries completed in PubMed to instruction librarians. Students have the option of learning skills online, in person, or independently. Librarians analyze the results to assess the effectiveness of each mode of instruction in improving student performance. (Yearly)
- Spanish 266 instructors were surveyed to determine if a library session or a library course page was most effective for students’ assignments. The instructors determined that a hybrid of both approaches was most effective and that good assignment design was integral to student success. (Spring 2008 & Fall 2008)
- Biology 152 instruction coordinators and librarians meet to determine if Chapter 2 Using the Library for Scientific Research (co-written by the librarians and course instructors) needs revisions before being printed in the Biology 152 Lab Manual. (Annually)
- Chemistry 346 students complete a supporting-information document to complete each lab report. See Journal of Chemistry Education article on course revision and components. (Annually)
- A benchmark survey is given to incoming chemistry graduate students in the fall. The survey's results influence the content presented in the first-year organic and inorganic graduate classes. (Annually)
- CP 125: A Wisconsin Experience is a first-year seminar course. Librarians developed a detailed rubric to evaluate ability to craft a research question and evaluate results. (Spring 2012)
- Communication-A Information Literacy Learning Outcomes were updated. To assess students' attainment of these learning outcomes and the effectiveness of our instructional strategies, library staff worked with the University Assessment Council to hold a series of focus groups with Communication A course instructors. Analysis of focus group transcripts showed that students are generally able to access information but struggle to evaluate potential sources' relevance, bias, and appropriateness for the assignment. (Spring 2012)
- Communication-A students completed an online worksheet during the library session. Instruction librarians evaluated responses in each skill area, using a rubric. Results shaped changes in the curriculum (e.g., results showed that we needed to emphasize keyword over natural-language searching, as well as the differences between popular and scholarly sources). (Fall 2007)
- Communication-A students complete an in-class evaluation form at the end of the library session. Instruction librarians review the responses after each class and discuss student comments in planning meetings. The comments and responses after each class have led to changes in instructional strategies, such as providing more hands-on activities and fewer demonstrations. (Each semester)
- Biology 152, Communication-A courses have clicker questions embedded throughout the library-session curriculum. Students’ responses shape the direction of the session, and an analysis of the composite responses shapes the curriculum design. (Annually)
- Biology 152 and Biology 301 students fill out note cards with responses to “one-minute assessment” questions. Students' comments on the most useful things learned and how to improve the session are analyzed and incorporated into the next semester's instruction. (Each semester)
Current Initiatives: Assessment
To improve student learning, Library and Information Literacy Instruction librarians across campus are continuously engaged in systematic assessment of their teaching. Examples of current initiatives are listed below.
The Assessment Plan for General Education at the University of Wisconsin-Madison is a framework for measuring the efficacy of the General Education Requirements (GER) in the courses where such requirements are taught. The “Essential Learning Outcomes,” which include information literacy in the area of “Intellectual and Practical Skills,” provide a framework for student learning in the context of the requirements. Information literacy skills are initially taught in Communication “A” and “B” courses.
According to the Plan, the primary tool used for measuring information literacy skills on campus has been standardized testing. While these efforts have shown that students have achieved an “acceptable level of performance,” it is indicated that “[a] more authentic assessment of student learning will provide better information which can be used for program administration and improvement.” Therefore, a direct assessment of information literacy skills is included in the Plan’s Cycle of Assessment, a calendar for assessment projects scheduled over the next several years (slated for the 2010-11 school year).
The CUWL User Services Coordinating Committee’s Information Literacy Assessment Working Group recently produced a report (October 2008) investigating a variety of commercially available information literacy assessment tools being considered for use in the UW System. They identified three major available tests—iSkills, Project SAILS, and the Information Literacy Test- and studied the merits and drawbacks of each. The group considered several factors, such as suitability for use at a system level; relevance to the information literacy skills being taught at each institution; presence and quality of feedback to test-takers; usability at a variety of levels (classroom level to institution level); practicality of providing the test, given campus schedules; and cost to use the test. Though the group did not ultimately recommend one of these three instruments over the others for use in the UW System, they did find that iSkills would be best used only at an individual-school level. Also, they recommended further investigation, to find out how other schools are using these tools.