THE CHARTS

WHERE ARE STUDENTS LEARNING THE MOST?

A study of 610 universities suggests that smaller schools are best—and finds Canadian universities falling short of their U.S. peers

SANDY FARRAN,TONY KELLER November 24 2008
THE CHARTS

WHERE ARE STUDENTS LEARNING THE MOST?

A study of 610 universities suggests that smaller schools are best—and finds Canadian universities falling short of their U.S. peers

SANDY FARRAN,TONY KELLER November 24 2008

WHERE ARE STUDENTS LEARNING THE MOST?

A study of 610 universities suggests that smaller schools are best—and finds Canadian universities falling short of their U.S. peers

BY SANDY FARRAN AND TONY KELLER •

SANDY FARRAN

TONY KELLER

It’s mid-January, a couple of weeks after the Christmas break, and Mark Woodcroft, a fourth-year biochemistry major at Trent University, is hanging out in the lab with professor Steven Rafferty, his research supervisor and chair of Trent’s chemistry department. Woodcroft is doing what many Canadian undergraduates never get a chance to do: an independent research project under faculty supervision.

So, a reporter asks, what’s your research project about? Woodcroft casts a sly smile at his prof and then launches deadpan into an explanation of the “bioaccumulation of perfluorinated carboxylic acids.” His audience predictably befuddled, Woodcroft stops midsentence. He and Rafferty chuckle in unison. It sounds like a well-rehearsed routine. Not something many 22-year-olds get to cook up with a professor.

“In upper-year courses, the class size is small enough for a professor to know each student by name,” says Woodcroft. “I also know everyone in my program by name. I doubt many students at a larger school can say that.”

Personal contact with faculty members, a sense of community among undergrads, and classes that push students to their intellectual limits—these are all things that many undergraduate students desire. Research suggests that these also promote learning; in the language of the National Survey of Student Engagement, these and other aspects of student engagement are “correlates of quality.” And according to the NSSE Benchmarks of Effective Educational Practice results appearing on the following pages (see pages 102 to 106), undergraduate educational quality at Canadian universities—with only a few exceptions—is below that of American universities.

On the following pages, you will also find

results from the Canadian Undergraduate Survey Consortium, or CUSC, a Canadaonly survey that is much more tilted toward assessing student satisfaction. In 2007, CUSC surveyed first-year students at 32 universities. The answers to two key CUSC student satisfaction questions are featured on page 108. NSSE asked two student satisfaction questions as well; the results for those questions are also published here. You can find results for seven additional CUSC student satisfaction questions on our website, at macleans. ca/oncampus.

While undergraduate student satisfaction remains relatively high at Canadian institutions, the NSSE benchmark results suggest a different story: satisfied or not, many Canadian university campuses are not as engaging and may not be offering as good an educational experience as their American peers. And the problem is particularly pronounced at Canada’s large research universities—the schools educating the overwhelming majority of Canadian undergrads.

The American-based NSSE survey is a tool widely used by universities to analyze, bench-

THE SURVEYS: WHAT THEY ARE, AND HOW THEY WERE DONE

The following pages contain the results from two major student surveys: the National Survey of Student Engagement (NSSE) and the Canadian Undergraduate Survey Consortium (CUSC). The NSSE and CUSC surveys, which were commissioned by the universities, ask more than 150 questions about specific aspects of the undergraduate experience—inside the classroom and beyond—designed to provide universities with data to help them assess programs and services.

The U.S.-based NSSE began in 1999 and is distributed to firstand senior-year students. NSSE is not primarily a student satisfaction survey, but is rather a study of best-educational practices, and an assessment of the degree to which each university follows those best practices. In 2004, 11 Canadian universities participated for the first time in NSSE, with 14,267 students completing the survey. By 2006, that number had grown to approximately 60,000 students at 31 Canadian institutions. Seventeen universities or their affiliates participated in the 2007 NSSE, representing roughly 14,000 students—fewer than in 2006 because most institutions conduct the NSSE survey every two years.

The NSSE results are headlined by the Benchmarks of Effective Educational Practice, created by NSSE to compare performance across all universities—American and Canadian—in five key areas: Level of Academic Challenge, Student-Faculty Interaction, Active and Collaborative Learning, Enriching Educational Experience, and Supportive Campus Environment. Each school's benchmark result was calculated by NSSE, based on student responses to a variety of questions. NSSE also asked two important student satisfaction questions; school-byschool results appear on the following pages.

CUSC was created in 1994; it is a Canadaonly survey, and unlike NSSE, it is in large part a student satisfaction survey. In 2007,

32 universities took part, including two institutions—UBC and the University of New Brunswick—that surveyed multiple campuses. Surveys were sent to a random sample of approximately 1,000 first-year undergrads at each university. Institutions with fewer than 1,000 first-years surveyed the entire cohort. More than 12,700 students responded.

Two CUSC student satisfaction questions are featured in this issue of the magazine.

For the results of seven other CUSC satisfaction questions, visit macleans.ca/oncampus.

READING THE CHARTS

The charts on the accompanying pages list 41 universities, including affiliates, that participated in recent NSSE surveys, as well as 31 university campuses surveyed for the 2007 CUSC.

In each chart, universities are listed in descending order. When displaying NSSE benchmark results, universities are ordered according to their senior-year benchmark scores; for student satisfaction questions, order was determined by the percentage of survey participants who chose the highest level of satisfaction, for example, "excellent."

The NSSE and CUSC surveys include more than 150 questions; we have published those—the five key NSSE benchmarks, plus two satisfaction questions each from NSSE and CUSC—that are the most broad and summative of student experience. NSSE charts include universities taking part in the 2006 or 2007 survey—or both—as well as one institution (Regina) that last conducted the survey in 2005. In each case, we display results from the most recent survey year. No data from first-year students are displayed for Royal Roads University as this institution does not offer first-year courses. No data from senior-year students is included for the University of Ontario Institute of Technology. UOIT took part in the 2006 NSSE, and this relatively new institution (founded in 2003) did not at that time have a senior-year class. Data displayed for the University of Western Ontario does not include results from the three Western affiliates, each of which conducted its own survey.

For a listing of additional CUSC results, as well as data from past NSSE, CUSC and Maclean's surveys, please visit macleans.ca/oncampus and click on "Rankings."

mark and improve their institutional performance. Since 1999, the American-based NSSE (pronounced “Nessie”) has been conducting its survey on a growing number of campuses, and calculating its Benchmarks of Effective Educational Practice for each participating school. Beginning in 2004, a growing number of Canadian universities began to take part in NSSE. The biggest push came from Bob Rae’s 2005 review of postsecondary education in Ontario. Rae called on the province to establish measures for

evaluating quality and publicly reporting on system performance. In his review, Rae asked, “How are we doing? How are others doing? Is there a jurisdiction that does it better?” His conclusion: “We simply don’t know enough about how we are doing or how others are doing.” To this end, Rae recommended that all Ontario universities participate in NSSE. All Ontario universities have done so over the past two years, and most universities in the rest of the country have joined them. Several of the 47 universities that Maclean’s

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Benchmarks of Effective Educational Practice

The NSSE survey asks undergrads nearly 100 questions to assess how engaged they are with their schools, their professors and their peers. Student engagement has been shown to be highly correlated with learning. The benchmarks compare engagement at all universities—American and Canadian—in five key areas. Level of Academic Challenge assesses the intellectual demands on students, measuring such things as the number of assigned readings and written reports, as well as coursework that emphasizes judgment and transforming information into more complex interpretations. Student-Faculty Interaction gauges professors as mentors, measuring how often students meet with faculty to discuss career plans or ideas, or work with them on research projects or other activities outside of class.

Level of Academic Challenge

Mount Allison (49.6/60.9) Royal Roads (n/a/60.1)

Trent (51.8/59.4)

First-year results: «MB Senior-year results:

mmm

Huron (Western) (50/59)

St. Thomas (51.8/58.4) Acadia (51.4/57.4)

Brock (50/57.2)

U N BC (47/56.9)

Brescia (Western) (50.6/56.7) Laurentian (50.7/56.7) Ryerson (52.1/56.5) McMaster (52.4/56.2) Lakehead (50.9/55.9)

UNB (Saint John) (48.1/55.9) Queen's (53.4/55.9)' Nipissing (49.9/55.8)

OCAD (49.1/55.8) U PEI (47.2/55.7)

Wilfrid Laurier (52/55.6) NSSE 2007* (51.6/55.6)

Carleton (50.2/55.5) Victoria (49.6/55.4)

Guelph (48.2/55.3) King's (Western) (50/55)

York (49.7/54.9) Daihousie (48.8/54.8)

Ottawa (48.6/54.5) Waterloo (52.6/54.5)

McGill (51/54.4) Toronto (50.1/54.2)

Concordia (49/54)

UNB (Fredericton) (48.6/53.7)

Saskatchewan (47.4/53.6) Western (48.8/53.6)

Lavai (49.9/53.5)......

Windsor (46.8/53.5) Calgary (48.5/52.9)

UBC (49.9/52.8)......

Lethbridge (45.7/52.6) Alberta (49.3/52)

Regina (45.7/51.1)

Student-Faculty Interaction

First-year results: MM Senior-year results:

Mount Allison (23.6/49.6) Huron (Western) (27/43) Acadia (30.1/42.6)

NSSE 2007* (32.5/40.9) OCAD (28.7/39.9)

UPE I (24.8/38.8)

UNB (Saint J oh n ) (27.3/38.4) St. Thomas (26.8/37.9)

Brescia (Western) (27.2/37.8) U N BC (25.2/37.6)

Brock (22.3/37.3)

Lethbridge (22.4/37.1)

Trent (24.4/36.5)

King's (Western) (26/36) Nipissing (26.7/36)

Laurentian (23.2/35.3)

UNB (Fredericton) (25.5/33.9) Ryerson (25.1/33.3)

Daihousie (23.4/32.8)

Guelph (18.9/32.8)

Queen's (21.8/32.6)

Wilfrid Laurier (22.7/32.5) Saskatchewan (20.4/32) McMaster (23.1/31.8) Windsor (22.1/31.4)

Western (22.5/31.1)

Calgary (20.6/30.9)

Royal Roads (n/a/30.9) Concordia (21.7/30.8) Lakehead (24.2/30.7)

Victoria (21.8/30.7)

Carleton (23.1/30.2)

McGill (20.1/30.2)

Regina (21.8/29.8)......

York (22.4/29.8)

Toronto (19.4/29.1)

Alberta (20.4/29)

Laval (18.9/28.9)

Waterloo (21.1/28.6)

UBC (20/27.4)

Ottawa (18.6/27.2)

Note: UOIT first-year score was 54.2.

*NSSE 2007 represents results from 610 Canadian and American universities.

Note: UOIT first-year score was 27.7. UOIT did not have a senior-year class in 2006. Royal Roads does not have first-year classes. See "Reading the Charts," page 101.

surveys in its annual rankings of Canadian universities have never participated in NSSE; they include Bishop’s University, Cape Breton University, St. Francis Xavier University, Memorial University, Université de Moncton and Université de Sherbrooke.

Most universities on both sides of the bor-

der initially kept their NSSE and CUSC reports confidential or only released selected bits of information; it was only after Maclean’s, backed by the power of provincial access to information laws, began asking for NSSE and CUSC results that all Canadian universities went public.

On the following pages, you will find results for 41 Canadian institutions that participated in NSSE in 2005, 2006 or 2007 NSSE asks first-year and fourth-year undergraduates at participating schools nearly 100 questions about what they have been doing during their university careers. It is not a student satisfac-

tion survey; it asks students to report on the mechanics of their classes, student habits and life at university. The questions—from how often they met outside of class with faculty members to how often they were involved in group work with other students—cover aspects of educational practice that have been shown to promote student engagement, which itself has been shown to promote more and better learning.

For example, faculty-supervised, independent research projects like the one undertaken by Woodcraft would have helped to boost a university’s Student-Faculty Interaction, Enriching Educational Experiences and Level of Academic Challenge benchmark scores.

“What’s important to stress is that NSSE doesn’t directly measure learning outcomes. It measures engagement,” says Ken Norrie of the Higher Education Quality Council of Ontario (HEQCO), which advises government on improving all aspects of post-secondary education, including quality, access and accountability. “If you believe in years of research that engagement is consistently highly correlated with learning outcomes, and we can measure engagement through something like the NSSE survey, then we have a proxy for learning outcomes that you can roughly associate with learning quality.”

So what do the NSSE benchmarks tell us about the undergraduate learning experience at Canadian universities? A good number of Canadian universities—mostly smaller, primarily undergraduate institutions, but including larger institutions such as Ryerson, Queen’s and McMaster—met or exceeded the 2007 NSSE Level of Academic Challenge benchmark average of the results from 610 mostly American universities. The academic challenge benchmark is made up of questions covering areas such as how much time students spent preparing for class, the number of textbooks assigned, number of written papers assigned, and coursework that emphasizes analyzing and synthesizing ideas.

A fair number of Canadian universities— again, mostly smaller institutions—also exceeded the NSSE benchmark for Supportive Campus Environment. The supportive campus environment benchmark focuses on whether the campus provides the support students need to succeed academically and thrive socially, and assesses the quality of students’ relationships with their peers, professors and the administration.

But on the remaining three benchmarks, few Canadian universities met the American standard. A handful of small, primarily undergraduate schools, led by Mount Allison University and Acadia University, are among those that consistently exceeded their American peers. Interestingly, while the University

of Western Ontario did not register aboveaverage scores, two of Western’s affiliated colleges—Huron and Brescia—scored highly in all areas.

The Student-Faculty Interaction benchmark—where no Canadian university exceeded the NSSE first-year benchmark and only three surpassed the fourth-year average—focuses

on the different ways that students interact with faculty members inside and outside of the classroom. Students are asked, for example, whether they have worked with a professor on activities outside of coursework, talked about career plans with a faculty member, received prompt feedback from faculty on their academic performance and worked with

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Benchmarks of Effective Educational Practice

Listed below and on previous and following pages are results for 40 Canadian universities or affiliates that took part in NSSE in 2006 or 2007, as well as one (Regina) whose students completed the survey in 2005. In all cases, results for the most recent survey year are displayed. Active and Collaborative Learning assesses involvement and teamwork, measuring how often students work with classmates, make class presentations, or participate in community-based projects. Enriching Educational Experience recognizes that diversity and complementary learning opportunities enhance academic programs. This includes internships and co-ops, community service, study abroad, as well as a campus environment that promotes contact among students from different backgrounds.

Active and Collaborative Learning

First-year results: Senior-year results: i

Royal Roads (n/a/53.9) Brescia (Western) (36.6/52.5) Brock (35.9/52.5)

Mount Allison (32.4/51.4) Acadia (40.5/51.3)

Ryerson (40/51.2)

OCAD (43.6/50.6)

NSSE 2007* (41/49.9)

UNB (Saint John) (36.1/49.4) Lethbridge (31.8/49.1)

U PE I (36/49.1)

U NBC (35.9/48.6)

Trent (35.4/48.3)

Huron (Western) (35/48)

UNB (Fredericton) (35.2/46.8) MMMMMMM St. Thomas (36.1/46.7)

Wilfrid Laurier (37.7/46.6) Nipissing (38.3/46.2) Guelph (34.1/45.3) Lakehead (38.5/45.2) McMaster (38.6/44.9) Regina (32.6/44.6)

Laval (37.9/44.4).......

King's (Western) (34/44)

Calgary (35.7/43.7) York (34.5/43.7)

Queen's (36.1/43.6) Laurentian (31.1/43.5)

Saskatchewan (31.3/43.3) Concordia (34.8/43.2) Dalhousie (35.1/42.9)

C a r l et ó n (35.2/42.8)

Victoria (32.5/42.8) Windsor (32.1/42.6)

Alberta (33.7/42.1) Ottawa (30.8/41 4)

McGill (34.6/41.2) Western (32.3/40.5)

UBC (34.2/39.7) Waterloo (33.7/38.9) Toronto (29.7/35.6)

Enriching Educational Experience

First-year results: >

* Senior-year results:

Mount Allison (27.3/41.1) Acadia (28.1/40.6)

NSSE 2007* (26.9/39.7)

Huron (Western) (27/39) Queen's (27.5/38.9)

Waterloo (26.8/37.4)

Guelph (24.7/36.9)......

Brescia (Western) (27.2/36.7) McGill (26.8/36.7)

McMaster (25.6/36.1)

Brock (23.3/35.8)

Ryerson (25.3/35.6)

Trent (26.2/34.8)

UNB (Fredericton) (22.9/34.4)

Calgary (24.1/34.3)......

Laurentian (22.6/34.3)

Wilfrid Laurier (25.4/34.1) UBC (25.3/33.9)

Alberta (25/33.7)......

UNB (Saint John) (24.5/33.7) Lethbridge (21.2/33.6) Dalhousie (23.3/33.2)

Western (26.3/33.2)

OCAD (23.7/32.9)

C a r íet on (24.3/32.7)

Victoria (24.1/32.7)

U NBC (25.7/32.6)

Ottawa (22.8/32.6)......

Regina (20.6/32.6)

Windsor (22.9/32.3)

King's (Western) (24/32)

St. Thomas (24.2/31.9) Lakehead (24/31.6)

Laval(21.2/31.6).......

UPEI (22.4/31.3)

Toronto (22.9/31.2)

Royal Roads (n/a/30.9)

York (23.2/30.4) Saskatchewan (20.2/30.3) Concordia (22.8/30)

Nipissing (24.6/29)

NOTE: UOIT first-year score was 41.5.

*NSSE 2007 represents results from 610 Canadian and American universities.

Note: UOIT first-year score was 26.8. UOIT did not have a senior-year class in 2006. Royal Roads does not have first-year classes. See "Reading the Charts," page 101.

a faculty member on a research project.

The overwhelming majority of Canadian students attend large, research-focused universities. Can institutions of such size offer top-level undergraduate experiences, as defined by NSSE? Results from the University of Michigan, a giant public university

that is also one of America’s leading research powerhouses, suggests that it is possible.

What is Michigan doing right in undergraduate education? Earlier this decade, Michigan was one of 20 American universities identified by NSSE as having outperformed on the NSSE benchmarks. A NSSE-

commissioned study visited each of the outperforming campuses to find out what practices were leading to those high NSSE benchmark scores. For example, explaining Michigan’s success on the Student-Faculty Interaction benchmark, the study cited Michigan’s small classes and research opportun-

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Benchmarks of Effective Educational Practice

The Supportive Campus Environment benchmark recognizes that students perform better at schools that support academic and non-academic endeavours, and that cultivate positive relationships among students, faculty and staff. Surprisingly, scores at many schools decline between first and fourth year.

Supportive Campus Environment

First-year results: a

Huron (Western) (56/64)

Brescia (Western) (59.6/63.5)

Mount Allison (59.3/62.7)

Acadia (63.3/60.4)

i Senior-year results:

»mumm**

J|jj||||gjgjg

Royal Roads (n/a/60.1)

U PEI (58.2/59.9)

_

Nipissing (63.5/59.8)

King's (Western) (57/59)

St ' Thomas (58/57.9)

UNB (Saint John) (55.3/57.4) Brock (56.3/57.3)

Guelph (60.6/56.9)

Trent (59.3/56.8)

NSSE 2007* i59 ó/56 7) Queen's (60.7/55.8)

Wilfrid Laurier (59.8/55.3) UNBC (56.2/54.7) Lethbridge (53.9/53.5) McMaster (58.3/53.4) Laurentian (55.9/52.9) Victoria (56/52.8)

Laval (57/52.7)

Western (58.6/52.5)

UNB (Fredericton) (53.9/51.8) Windsor (51.7/51.8)

Concordia (52.5/51.5) Regina (51.8/51.5)

Saskatchewan (51.8/51) Lakehead (56.2/50.8)

Ryerson (55.6/50.3) C a rI eto n (56.5/50.2)

Waterloo (57.5/49.5) Dalhousie (50.9/49.1)

Alberta (53.4/48.6) Calgary (51.2/47.4) York (51.1/46.8)

OCAD (54.1/46.4) McGill (50.9/45.6)

Ottawa (49.4/45.3) UBC (50.8/44.9) Toronto (51.6/44.8)

Note: UOIT first-year score was 59.8. UOIT did not have a senior-year class in 2006. Royal Roads does not have first-year classes. See "Reading the Charts," page 101. *NSSE 2007 represents results from 610 Canadian and American universities.

ities in first year; programs that encourage students and faculty to eat meals together; mentorship programs; extensive email contact between students and faculty; and professors’ offices that are located in residences. On the Level of Academic Challenge bench-

mark, the study pointed to a commitment to excellence that permeates the entire Michigan campus: faculty resistance to grade inflation, introductory courses designed to challenge students’ ability to problem solve, and small classes that encourage active learning

and challenge students to develop critical thinking and independence in carrying out research projects.

NSSE director Alex McCormick says while universities can use NSSE to improve, “these are things that take some intentional effort

CANADIAN UNDERGRADUATE SURVEY CONSORTIUM (CUSC)

Student Satisfaction Results

The CUSC survey is an annual study with a focus on student satisfaction. The 2007 survey, whose results are featured below, canvassed first-year students for their opinions. Participating universities sent an extensive questionnaire to a random sampling of up to 1,000 students, asking questions about everything from academics to support services. In 2007, nearly 13,000 students responded.

Generally, 1 am satisfied with the quality of teaching I have received.

H Strongly Agree (%) || Agree (%)

Nipissing 3 3 60

Trent 33HBIIIHIHHHHÍÍHnMMNNHNMi 62

Wilfrid Laurier 30f 63

Brandon 64

UOIT.............29MBBBB 2 ■ ' - ........57

Winnipeg 29 ■■■■■■■MMHMHHHnHHHI 60

Carleton 26II^HHHMMMHMHMMMNNNMMI 64

Lethbridge 26 66

Ryerson 62

Fraser Valley 67

Mount Saint Vincent 60

UNB (Fredericton) 24HBB1 66

McMaster 65

Saskatchewan 23H^HB ( 65

Concordia 63

UNB (Saint John) 22HHB 69

Saint Mary's 22 ■■■■■■HMMHHHNHHMË!;' 67

Montréal 21 67

UNBC 21 70

UBC (Okanagan) 19HBH69

Brock...........................19 7 2

Victoria 70

Dalhousie 17 70

Regina 17 72

Alberta 15 ■■■■HHHNNNHMMHHMi 70

Ottawa 15HHHÍSMMMHMMMNNHÍ 72

Simon Fraser 1571

Manitoba 171

Windsor 14HBBHHHHHHHHHMMHÍ 69

UBC (Vancouver) 13IIHMHHHMHHHHHHÍ 68

I am satisfied with my decision to attend this university.

I Strongly Agree (%) Agree (%)

Nipissing

Trent

Wilfrid Laurier McMaster

Mount Saint Vincent

Winnipeg

Saskatchewan

UOIT

Victoria

Carleton

UNB (Fredericton)

Lethbridge

Montréal

Ryerson

Saint Mary's

Brandon

Brock

Concordia

Alberta

Dalhousie

UNBC

Regina

Fraser Valley

UNB (Saint John)

Simon Fraser

UBC (Vancouver)

Ottawa

UBC (Okanagan) Calgary Windsor Manitoba

to move the needle on. It’s not quite as simple as stepping on the accelerator in your car.” And while he believes that universities can learn a lot about best practices from one another, he cautions that it’s not always easy to make direct comparisons. Schools that enrol a large number of adults or commuters, for example, are likely to have lower scores because students have less time to spend on campus and, as a result, tend to be less engaged than traditional undergrads living on campus. Yet the same school’s more traditional undergraduate population may be just as engaged as undergrads at other campuses. “There is a robust body of evidence that shows that the vast majority of the variation in individual student scores is within institutions, not between institutions,” says McCormick. “So if you look at all the individual students that are surveyed and look at variation in their responses to the

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Student Satisfaction Results

The NSSE survey is not primarily a student satisfaction survey. The main purpose of NSSE is to assess what students are doing—as shown in the benchmark tables on pages 102 to 106—not to ask for their opinion. However, NSSE includes some satisfaction questions, including one asking students to evaluate their educational experience. Most institutions' scores declined from first to fourth year.

How would you evaluate your entire educational experience at this institution?

Queen's Western Nipissing Guelph Mount Allison Waterloo Wilfrid Laurier Acadia UOIT St. Thomas McMaster King's (Western) NSSE 2007* McGill Huron (Western) Trent Victoria Lakehead OCAD Alberta Brock Carleton Laval Lethbridge Ryerson UPEI Laurentian Brescia (Western) UBC Dalhousie UNB (Fredericton) UNBC Saskatchewan Toronto York Concordia Ottawa UNB (Saint John) Calgary Regina Windsor

41 45 47 45 44 49 49 46 51 48 49 52 48 49 51 55 52 45 53 53 53 57 57 52 57 54 66 50 54 57 59 60 48 55 58 59 58 58 64 55

Huron (Western) Brescia (Western) Mount Allison King's (Western) Guelph Trent UPEI Queen's St. Thomas Acadia Western Brock UNBC Nipissing Waterloo Wilfrid Laurier NSSE 2007* McMaster Lethbridge Victoria McGill Royal Roads Ryerson Alberta Windsor Carleton UNB (Fredericton) Concordia UNB (Saint John) Saskatchewan Toronto Dalhousie Lakehead Lau rentia n Laval Regina UBC OCAD York Calgary Ottawa

29 32 40 41 46 43 50 45 49 50 47 40 47 53 47 52 49 47 52 57 52 49 51 56 53 57 56 55 56 60 48 55 52 53 59 58 52 59 53 53 55

*NSSE 2007 benchmark reflects the overall result for 610 Canadian and American universities.

Note: UOIT did not have a senior-year class in 2006. Royal Roads does not have first-year classes. See "Reading the Charts," page 101.

NSSE items, about 90 per cent of that variation occurs within institutions and only about 10 per cent is between institutions.” As a result, says McCormick, “distilling it down to a number or set of numbers for an institution” may mask variations among departments or faculties at the same university. McCormick says that NSSE needs to find ways “to help institutional leaders look more deeply into varia-

tions within their walls.” In other words, the really interesting story may be one like that of the benchmark scores from Western’s affiliate colleges, which are above those of Western itself.

Norrie of Ontario’s HEQCO sees a similar promise in NSSE. Canadian universities are mostly still in the early stages of drilling down to examine variations among faculties, depart-

ments, courses and even gender and ethnic background. But Norrie says he regularly hears from university administrators who have hit on revealing findings. “When you start doing variations in NSSE results across faculties or departments, and you find some interesting variations, you say, ‘Okay, what’s going on?’ ” says Norrie. “And that gets you into a conversation about what explains the

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Student Satisfaction Results

NSSE is primarily an objective look at life and learning on campus, but it also asks students to answer a few satisfaction questions. In general, senior students are more critical when evaluating their university experience. While the majority of students would choose to return to their alma mater, the number drops—in some cases sharply—for students in their final year as compared to freshmen.

If you could start over, would you go to the institution you are now attending?

FIRST-YEAR STUDENTS

£ Definitely Yes (%) Probably Yes (%)

Western

Guelph

Nipissing

Mount Allison

Waterloo

Wilfrid Laurier

McGill

Laval

St. Thomas OCAD

Huron (Western)

King's (Western)

McMaster

LIOIT

Victoria

Acadia

UBC

NSSE 2007*

Alberta

Trent

Ryerson

Laurentian

Saskatchewan

Carleton

Lakehead

Brock

Lethbridge

UPE I

Brescia (Western)

Concordia

UNB (Fredericton)

Dalhousie

Toronto

UNBC

York

Ottawa

Regina

UNB (Saint John)

Calgary

Windsor

SENIOR-YEAR STUDENTS

Huron (Western) 631

King's (Western) 56

Brescia (Western) 53

Guelph 511

Mount Allison 511

Royal Roads St. Thomas

511 501

Acadia UPE I Wilfrid Laurier Brock

Queen's

Western

451 451

McGil

Trent

441 441

Waterloo 44

Nipissing 43

UNBC NSSE 2007*

McMaster Victoria Laval

Lethbridge

Alberta

361

341

Saskatchewan Concordia OCAD

Ryerson

UBC

331 321

Carleton

Laurentian

Regina

Toronto

Lakehead Windsor UNB (Fredericton) UNB (Saint John)

26

York

Dalhousie

Ottawa

Calgary

I Definitely Yes(%)

Probably Yes(%)

23 34 32 34

34

35

37

38 41

36 38 38 38

38

36

37

41

43

39 39 47

47 45 50

49

48

50

44

45

46

42

49 39

44

45

46

47 45

49 47

50

*NSSE 2007 benchmark reflects the overall result for 610 Canadian and American universities.

Note: UOIT did not have a senior-year class in 2006. Royal Roads does not have first-year classes. See "Reading the Charts," page 101.

variation and the different ways of teaching and learning.”

Phil Wood, associate vice-president of student affairs at McMaster University, has established his own mini-benchmark from a set of 16 NSSE questions that zero in on an area of particular interest to him: student growth and development. Because Wood oversees student services, he’s interested in

figuring out things like: is it beneficial for a student to live in residence? Do students living in residence report higher NSSE engagement scores and higher scores on his mini-benchmark?

In 2005, the University of Toronto—an institution that is in many ways similar to the University of Michigan in terms of its vast size and the quality and breadth of the

graduate and research programs it offers— hired American Tony Chambers to fill a newly created position, associate vice-provost of students. Having worked at post-secondary institutions in both countries, Chambers is often called upon to discuss the uses, and limits, of NSSE, particularly in the Canadian context.

“The systems are considerably and extremely

nuanced, and I think for us to compare what happens in the States to what happens in Canada is sort of a worthless analysis, to be quite honest,” says Chambers. “It gives us a sense of what institutions are doing, for sure, where we can make some decisions at an institutional level, but in terms of systems of education, I don’t think the analysis is worth a whole lot, quite honestly.”

Chambers says that some NSSE questions use terms not in wide currency in Canada, or terms that some students may interpret

differently than their American peers. This could affect the answers offered by Canadians.

Despite its limitations, NSSE is proving to be a valuable diagnostic tool for Canadian universities. Back at Trent, president Bonnie Patterson says NSSE has helped to validate what she and her colleagues already knew: that Trent is a smaller, tighterknit campus, where students experience a good number of small classes, with professors who will probably know them by their first name, and opportunities to do research or independent study with a faculty member.

For Patterson, NSSE’s added value is that it offers comparisons among institutions, and highlights areas needing improvement. Partly in response to its findings, Patterson says that Trent has channelled resources into

five key areas, including library resources and technology in the classroom. “There is always a much longer list of what you can’t do than the list of what you can do,” she says. “Would I have loved to put money into hiring another 15 or 20 faculty members? You bet. You have to find the balance of what makes you successful in student opinion and in satisfaction and what makes you successful in learning outcomes, but at the same time trying to be responsive to them almost from a consumer perspective.”

All Canadian universities struggle with trade-offs: whether to hire more professors or build an athletic complex; upgrade labs, fund new research or offer more undergrad course selection. Like Patterson, university administrators say that surveys such as NSSE and CUSC have helped in that process. “It validates, it informs, it gives us abetter insight into the detail of issues,” says Patterson. “Rather than our own serendipitous, ad hoc examples or anecdotes, it gives you large opinion pieces that we didn’t have before we got into these surveys.” M

macleans.ca/oncampus

Want to see more student survey results?

For additional questions from the CUSC survey of university students, as well as data from past NSSE and CUSC surveys, please visit macleans.ca/oncampus and click on “Rankings.” You can also find college student surveys, covering the opinions of more than 150,000 students at 45 Canadian colleges.

Visit macleans.ca/oncampus and click on “Colleges.”