Ten Recommendations for Undertaking Institutional Assessment

In 2011, Arum and Roksa published a study that concluded that 30% of the American college students tested made no discernable learning gains over the four years of a college degree (Arum & Roksa, 2011). Institutions naturally wondered, “Are our students learning? What evidence do we have?” Most had no reliable data that could answer those questions. Around the same time, professional programs in Canadian institutions were increasingly trying to assess learning outcomes using reliable measures to meet accreditation requirements and inform program improvement. Government and public discourse has been more frequently focused on asking how well university education has been preparing students for both employment and participation in society. Like others, we were struck by the lack of evidence around student learning at our own institution. At the instigation of our then provost, we responded to a call issued by the Higher Education Quality Council of Ontario (HEQCO) to pull together a group of universities keen on working together to find better ways to measure learning gains. Since 2012, Queen’s University, a mid-size, medical-doctoral institution, has conducted multiple assessment studies, generally for one or both of the following two goals: 1. To gather evidence about how well students develop transferable cognitive skills over time, in order to inform improvement of courses and programs 2. To compare approaches for gathering this evidence, in order to inform long-term sustainable processes. The focus of our assessment has been on our undergraduate programs, particularly in science, engineering, social science and humanities. In this chapter, we will primarily discuss our first Learning Outcomes Assessment Consortium project (LOAC I), but in places we will draw on our experience from a number of other research projects (see summary below). The lessons learned have also leveraged collaborations with other institutions, and have been informed by discussions with groups like the HEQCO Learning Outcomes Assessment Consortium, the Bay View Alliance (Bay View Alliance, n.d.) and the Engineering Change Lab (Engineers Without Borders, n.d.). Collectively, these projects have involved assessing transferable cognitive abilities like critical thinking, problem solving and written communication using two assessment methods: standardized tests (either cross-sectional or longitudinal) and scoring samples of student academic work using consistent criteria (e.g., VALUE rubrics).