UNIVERSITY STUDENT ISSUE

WHERE ARE STUDENTS LEARNING THE MOST?

A study of 610 universities suggests that smaller schools are best—and finds Canadian universities falling short of their U.S. peers

SANDY FARRAN,TONY KELLER February 18 2008
UNIVERSITY STUDENT ISSUE

WHERE ARE STUDENTS LEARNING THE MOST?

A study of 610 universities suggests that smaller schools are best—and finds Canadian universities falling short of their U.S. peers

SANDY FARRAN,TONY KELLER February 18 2008

WHERE ARE STUDENTS LEARNING THE MOST?

UNIVERSITY STUDENT ISSUE

A study of 610 universities suggests that smaller schools are best—and finds Canadian universities falling short of their U.S. peers

SANDY FARRAN

TONY KELLER

It’s mid-January, a couple of weeks after the Christmas break, and Mark Woodcroft, a fourth-year biochemistry major at Trent University, is hanging out in the lab with professor Steven Rafferty, his research supervisor and chair of Trent’s chemistry department. Woodcroft is doing what many Canadian undergraduates never get a chance to do: an independent research project under faculty supervision.

So, a reporter asks, what’s your research project about? Woodcroft casts a sly smile at his prof and then launches deadpan into an explanation of the “bioaccumulation of perfluorinated carboxylic acids.” His audience predictably befuddled, Woodcroft stops mid-sentence. He and Rafferty chuckle in unison. It sounds like a well-rehearsed routine. Not something many 22-year-olds get

to cook up with a professor.

“In upper-year courses, the class size is small enough for a professor to know each student by name,” says Woodcroft. “I also know everyone in my program by name. I doubt many students at a larger school can say that.”

Personal contact with faculty members, a sense of community among undergrads and classes that push students to their intellectual limits—these are all things that many undergraduate students desire. Research suggests that these also promote learning; in the language of the National Survey of Student Engagement, these and other aspects of student engagement are “correlates of quality.” And according to the NSSE Benchmarks of Effective Educational Practice results appearing on the following pages (see pages 40 to 44), undergraduate educational quality at Canadian universities—with only a few excep-

tions—is below that of American universities.

On the following pages, you will also find results from the Canadian Undergraduate Survey Consortium, or CUSC, a Canada-only survey that is much more tilted toward assessing student satisfaction. In 2007, CUSC surveyed first-year students at 32 universities. The answers to two key CUSC student satisfaction questions are featured on page 46. NSSE asked two student satisfaction questions as well; the results for those questions are also published here. You can find results for seven additional CUSC student satisfaction questions on our website, at www.macleans.ca/oncampus.

While undergraduate student satisfaction remains relatively high at Canadian institutions, the NSSE benchmark results suggest a different story: satisfied or not, many Canadian university campuses are not as engaging and may not be offering as good an educational experience as their American peers. And the problem is particularly pronounced at Canada’s large research universities—the schools educating the overwhelming major-

ity of Canadian undergrads.

The American-based NSSE survey is a tool widely used by universities to analyze, benchmark and improve their institutional performance. Since 1999, the American-based NSSE (pronounced “Nessie”) has been conducting its survey on a growing number of campuses, and calculating its Benchmarks of Effective Educational Practice for each participating school. Beginning in 2004, a growing number of Canadian universities began to take part in NSSE. The biggest push came from Bob Rae’s 2005 review of post-secondary education in Ontario. Rae called on the province to establish measures for evaluating quality and publicly reporting on system performance. In his review, Rae asked, “How are we doing? How are others doing? Is there a jurisdiction that does it better?” His conclusion: “We simply don’t know enough about how we are doing or how others are doing.” To this end, Rae recommended that all Ontario universities participate in NSSE. All Ontario universities have done so over the past two years, and most universities in the rest of the country have joined them. Several of the 47 universities that Maclean’s surveys in its annual rankings of Canadian universities have never participated in NSSE; they include Bishop’s

THE SURVEYS: WHAT THEY ARE, AND HOW THEY WERE DONE

The following pages contain the results from two major student surveys: the National Survey of Student Engagement (NSSE) and the Canadian Undergraduate Survey Consortium (CUSC). The NSSE and CUSC surveys, which were commissioned by the universities, ask more than 150 questions about specific aspects of the undergraduate experience—inside the classroom and beyond—designed to provide universities with data to help them assess programs and services.

The U.S.-based NSSE began in 1999 and is distributed to firstand senior-year students. NSSE is not primarily a student satisfaction survey, but is rather a study of best-educational practices, and an assessment of the degree to which each university follows those best practices. In 2004, 11 Canadian universities participated for the first time in NSSE, with 14,267 students completing the survey. By 2006, that number had grown to approximately 60,000 students at 31 Canadian institutions. Seventeen universities or their affiliates participated in the 2007 NSSE, representing roughly 14,000 students— fewer than in 2006 because most institutions conduct the NSSE survey every two years.

The NSSE results are headlined by the Benchmarks of Effective Educational Practice, created by NSSE to compare performance across all universities—American and Canadian—in five key areas: Level of Academic

Challenge, Student-Faculty Interaction,

Active and Collaborative Learning, Enriching Educational Experience, and Supportive Campus Environment. Each school's benchmark result was calculated by NSSE, based on student responses to a variety of questions. NSSE also asked two important student satisfaction questions; school-by-school results appear on the following pages.

CUSC was created in 1994; it is a Canadaonly survey, and unlike NSSE, it is in large part a student satisfaction survey. In 2007,

32 universities took part, including two institutions—UBC and the University of New Brunswick—that surveyed multiple campuses. Surveys were sent to a random sample of approximately 1,000 first-year undergrads at each university. Institutions with fewer than 1,000 first-years surveyed the entire cohort. More than 12,700 students responded.

Two CUSC student satisfaction questions are featured in this issue of the magazine.

For the results of seven other CUSC satisfaction questions, visit macleans.ca/oncampus.

READING THE CHARTS

The charts published on the accompanying pages list 41 universities, including affiliates, that participated in recent NSSE surveys, as well as 31 university campuses surveyed for the 2007 CUSC. In each chart, universities are listed in descending order. When displaying the NSSE benchmark charts, universities are listed according to the benchmark scores associated with their senior-year students; for student satisfaction questions, order was determined by the percentage of survey participants who chose the highest level of satisfaction when responding, for example, "excellent."

The NSSE and CUSC surveys include more than 150 questions; we have published those—the five key NSSE benchmarks, plus two satisfaction questions each from NSSE and CUSC—that are the most broad and summative of student experience. NSSE charts include universities taking part in the 2006 or 2007 survey—or both, as well as one institution (Regina) that last conducted the survey in 2005. In each case, we display results from the most recent survey year. No data from first-year students are displayed for Royal Roads University as this institution does not offer first-year courses. Data displayed for the University of Western Ontario do not include results from the three Western affiliates, each of which conducted its own survey.

For a listing of additional CUSC results, as well as data from past NSSE, CUSC and Maclean's surveys, please visit www.macleans.ca/oncampus and click on "Rankings."

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Benchmarks of Effective Educational Practice

The NSSE survey asks undergrads nearly 100 questions to assess how engaged they are with their schools, their professors and their peers. Student engagement has been shown to be highly correlated with learning. The benchmarks compare engagement at all universities—American and Canadian—in five key areas. "Level of Academic Challenge" assesses the intellectual demands on students, measuring such things as the number of assigned readings and written reports, as well as coursework that emphasizes judgment and transforming information into more complex interpretations. "Student-Faculty Interaction" gauges professors as mentors, measuring how often students meet with faculty to discuss career plans or ideas, or work with them on research projects or other activities outside of class.

Level of Academic Challenge First-year results: **M* *m* Senior-year

emm Mount Allison um.......................... Royal Roads Trent (51.8/59.4).................... Huron (Western) ................. St. Thomas (51.8/58.4)........................................ Acadia (51.4/57.4) Brock ........................... UNBC SM«BÍ¿aMÍÍMÍ Brescia (Western) Laurentian n Ryerson (52.1/56.5) McMaster (52.4/56.2) Lakehead (50.9/55.9) UNB (Saint John) (48.1/55.9) Queen's m Nipissing (49.9/55.8) OCAD ............ U PEÍ (47.2/55.7)...................... Wilfrid Laurier (52/55.6) NSSE 2007* —vTytffrrni Carleton (50.2/55.5) Victoria ... Guelph ....... King's (Western) MH York .............. Dalhousie (48.8/54.8) Ottawa (48.6/54.5) Waterloo .............. McGill (51/54.4) Toronto (50.1/54.2) Concordia (49/54) UNB (Fredericton) (48.6/53.7) Saskatchewan (47.4/53.6) Western aaMSKA..................... Laval (49.9/53.5)..................... Windsor (46.8/53.5) Calgary (48.5/52.9) UBC (49.9/52.8)....................... Lethbridge (45.7/52.6) Alberta ...................... Regina ....... Manitoba

Manitoba Refused to make this information

universities Student-Faculty Interaction First-year results: **(» Senior-year

results: Mount Allison (23.6/49.6) Huron (Western) (27/43) Acadia (30.1/42.6) NSSE 2007* (32.5/40.9) OCAD (28.7/39.9) U PEÍ (24.8/38'8) UNB (Saint John) (27.3/38.4) StT Thomas (26.8/37.9) Brescia (Western) (27.2/37.8) Ü NBC (25.2/37.6) Brock (22.3/37.3) Lethbridge (22.4/37.1) Trent (24.4/36.5) King's (Western) (26/36) Nipissing (26.7/36) Laurentian (23.2/35.3) UNB (Fredericton) (25.5/33.9) Ryerson (25.1/33.3) Dalhousie (23.4/32.8) Guelph (18.9/32.8) Queen's (21.8/32.6) Wilfrid Laurier (22.7/32.5) Saskatchewan (20.4/32) McMaster (23.1/31.8) Windsor (22.1/31.4) Western (22.5/31.1) Calgary (20.6/30.9) Royal Roads (n/a/30.9) Concordia (21.7/30.8) Lakehead (24.2/30.7) Victoria (21.8/30.7) Carleton (23.1/30.2) McGill (20.1/30.2) Regina (21.8/29.8) York (22.4/29.8) Toronto (19.4/29.1) Alberta (20.4/29) Laval (18.9/28.9) Waterloo (21.1/28.6) UBC (20/27.4) Ottawa (18.6/27.2)

SCORE Refused to make this information

public BENCHMARK SCORE 30 45 Manitoba BENCHMARK 60 *NSSE 2007 represents results from 610 Canadian and American

public University, Cape Breton University, St. Francis Xavier University, Memorial University, Université de Moncton and Université de

Sherbrooke. Most universities on both sides of the border initially kept their NSSE and CUSC reports confidential or only released selected bits of information; it was only after

Maclean’s, backed by the power of provincial access to information laws, began asking for NSSE and CUSC results that the majority of Canadian universities began to go public. We began asking for the latest survey results four months ago; the only ranked university that failed to make public all of its NSSE and CUSC information in time for

publication was the University of

Manitoba. On the following pages, you will find results for 41 Canadian institutions that participated in NSSE in 2005, 2006 or 2007 NSSE asks first-year and fourth-year undergraduates at participating schools nearly 100 questions about what they have been doing during their university careers. It is not a

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Benchmarks of Effective Educational Practice

Listed below and on the previous and following page are results for 40 Canadian universities or affiliates that took part in NSSE in 2006 or 2007, as well as one (Regina) whose students completed the survey in 2005. In all cases, results for the most recent survey year are displayed. Active and Collaborative Learning assesses involvement and teamwork, measuring how often students work with classmates, make class presentations, or participate in community-based projects. Enriching Educational Experience recognizes that diversity and complementary learning opportunities enhance academic programs. This includes internships and co-ops, community service, study abroad, as well as a campus environment that promotes contact among students from different backgrounds.

Active and Collaborative Learning First-year results: s» Senior-year results:

Royal Roads (n/a/53.9) Brescia (Western) (36.6/52.5) Brock (35.9/52.5) Mount Allison (32.4/51.4) Acadia (40.5/51.3) Ryerson (40/51.2) OCAD (43.6/50.6) ÑSSE 2007* * (4!/49 9) UNB (Saint John) (36.1/49.4) Lethbridge (31.8/49.1) UPEI (36/49.1) UNBC (35.9/48.6) Trent (35.4/48.3) Huron (Western) (35/48) UNB (Fredericton) (35.2/46.8) St. Thomas (36.1/46.7) Wilfrid Laurier (37.7/46.6) Nipissing (38.3/46.2) Guelph (34.1/45.3) Lakehead (38.5/45.2) McMaster (38.6/44.9) Regina (32.6/44.6) Laval (37.9/44.4)...... King's (Western) (34/44) Calgary (35.7/43.7) York (34.5/43.7) Queen's (36.1/43.6) Laurentian (31.1/43.5) Saskatchewan (31.3/43.3) Concordia (34.8/43.2) Dalhousie (35.1/42.9) Carleton (35.2/42.8) Victoria (32.5/42.8) Windsor (32.1/42.6) Alberta (33.7/42.1) Ottawa (30.8/41.4) McGill (34.6/41.2) Western (32.3/40.5) UBC (34.2/39.7) Waterloo (33.7/38.9) Toronto (29.7/35.6) Manitoba

Refused to make this information public

BENCHMARKSCORE 25 40 55 *NSSE 2007 represents results from 610 Canadian and American universities

Enriching Educational Experience First-year results: Senior-year results:

Mount Allison (27.3/41.1) Acadia (28.1/40.6)...... NSSE 2007* (26.9/39.7) ....... Huron (Western) (27/39) Queen's (27.5/38.9)...... Waterloo (26.8/37.4) Guelph (24.7/36.9)...... Brescia (Western) (27.2/36.7) McGill (26.8/36.7)...... McMaster (25.6/36.1) Brock (23.3/35.8)................................* Ryerson (25.3/35.6) Trent (26.2/34.8) U N B (Fredericton) (22.9/34.4) Calgary (24.1/34.3) Laurentian (22.6/34.3) Wilfrid Laurier (25.4/34.1) UBC (25.3/33.9) Alberta (25/33.7) UNB (Saint John) (24.5/33.7) Lethbridge (21.2/33.6) Dalhousie (23.3/33.2) Western (26.3/33.2) OCAD (23.7/32.9) Carleton (24.3/32.7) Victoria (24.1/32.7)...... UNBC (25.7/32.6) Ottawa (22.8/32.6) Regina (20.6/32.6) Windsor (22.9/32.3) King's (Western) (24/32) St. Thomas (24.2/3Ï.9) Lakehead (24/31.6) Laval(21.2/31.6) UPEI (22.4/31.3) Toronto (22.9/31.2) Royal Roads (n/a/30.9) York (23.2/30.4) Saskatchewan (20.2/30.3) Concordia (22.8/30) Nipissing (24.6/29) Manitoba

Refused to make this information public

BENCHMARK SCORE

satisfaction survey; it asks students to report on the mechanics of their classes, student habits and life at university. The questions— from how often they met outside of class with faculty members to how often they were involved in group work with other students—cover aspects of educational practice that have been shown to promote student

engagement, which itself has been shown to promote more and better learning.

For example, faculty-supervised, independent research projects like the one undertaken by Woodcraft would have helped to boost a university’s Student-Faculty Interaction, Enriching Educational Experiences and Level of Academic Challenge benchmark scores.

“What’s important to stress is that NSSE doesn’t directly measure learning outcomes. It measures engagement,” says Ken Norrie of the Higher Education Quality Council of Ontario (HEQCO), which advises government on improving all aspects of post-secondary education, including quality, access and accountability. “If you believe in years

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Benchmarks of Effective Educational Practice

The Supportive Campus Environment benchmark recognizes that students perform better at schools that support academic and non-academic endeavours, and that cultivate positive relationships among students, faculty and staff. Surprisingly, scores at many schools decline between first and fourth year.

Supportive Campus Environment

First-year results: «**» Senior-year results:

Huron (Western) (56/64) Brescia (Western) (59.6/63.5) Mount Allison (59.3/62.7) Acadia (63.3760.4) Royal Roads (n/a/60.1) UPEI (58.2/59.9) Nipissing (63.5/59.8) King's (Western) (57/59) St. Thomas (58/57.9) UNB (Saint John) (55.3/57.4) Brock (56.3/57.3) Guelph (60.6/56.9)...... Trent (59.3/56.8) NSSE 2007* (59 6/56 7) Queen's (60.7/55.8) Wilfrid Laurier (59.8/55.3) UNBC (56.2/54.7) Lethbridge (53.9/53.5) McMaster (58.3/53.4) Laurentian (55.9/52.9) Victoria (56/52.8) Laval (57/52.7) Western (58.6/52.5) UN B (Fredericton) (53.9/51.8) Windsor (51.7/51.8) Concordia (52.5/51.5) Regina (51.8/51.5) Saskatchewan (51.8/51) Lakehead (56.2/50.6) Ryerson (55.6/50.3) Carleton (56.5/50.2) Waterloo (57.5/49.5) Dalhousie (50.9/49.1) Alberta (53.4/48.6) Calgary (51.2/47.4) York (51.1/46.8) OCAD (54.1/46.4) McGill (50.9/45.6) Ottawa (49.4/45.3)....... UBC (50.8/44.9) Toronto (51.6/44.8) Manitoba

Refused to make this information public

BENCHMARKSCORE 35 50 *NSSE 2007 represents results from 610 Canadian and American universities

of research that engagement is consistently highly correlated with learning outcomes, and we can measure engagement through something like the NSSE survey, then we have a proxy for learning outcomes which you can roughly associate with learning quality.”

So what do the NSSE benchmarks tell us

about the undergraduate learning experience at Canadian universities? A good number of Canadian universities—mostly smaller, primarily undergraduate institutions, but including larger institutions such as Ryerson, Queen’s and McMaster—met or exceeded the 2007 NSSE Level of Academic Challenge

benchmark average of the results from 610 mostly American universities. The academic challenge benchmark is made up of questions covering areas such as how much time students spent preparing for class, the number of textbooks assigned, number of written papers assigned, and coursework that empha-

CANADIAN UNDERGRADUATE SURVEY CONSORTIUM (CUSC)

Student Satisfaction Results

The CUSC survey is an annual study, with a focus on student satisfaction. The 2007 survey, whose results are featured below, canvassed first-year students for their opinions. Participating universities sent an extensive questionnaire to a random sampling of up to 1,000 students, asking questions about everything from academics to support services. In 2007, nearly 13,000 students responded.

Generally, I am satisfied with the quality of teaching I have received. I Strongly Agree (%) Agree (%)

Nipissing Trent Wilfrid Laurier Brandon UOIT Winnipeg Carleton 26 Lethbridge Ryerson Fraser VaÍley Mount Saint Vincent UNB (Fredericton) McMaster Saskatchewan Concordia UNB (Saint John) Saint Mary's Montréal UNBC...... UBC (Okanagan) Brock Victoria 19 Dalhousie Regina 17 Alberta Ottawa Simon Fraser Windsor UBC (Vancouver) Calgary Manitoba

Refused to make this information public

I am satisfied with my decision to attend this university. I Strongly Agree (%) Agree (%)

Nipissing Trent Wilfrid Laurier 51 McMaster Mount Saint Vincent Winnipeg 46 Saskatchewan UOIT..........................................44BBIBBBBI Victoria Carleton 41 UNB (Fredericton) 41 Lethbridge Montréal Ryerson Saint Mary!s Brandon Brock Concordia Alberta 35 Dalhousie UNBC Regina Fraser Valley UNB (Saint John) Simon Fraser UBC (Vancouver) Ottawa UBC (Okanagan) Calgary Windsor Manitoba

sizes analyzing and synthesizing ideas.

A fair number of Canadian universities— again, mostly smaller institutions—also exceeded the NSSE benchmark for Supportive Campus Environment. The supportive campus environment benchmark focuses on whether the campus provides the support students need to succeed academically and thrive socially, and assesses the quality of students’ relationships with their peers, professors and the administration.

But on the remaining three benchmarks, few Canadian universities met the American standard. A handful of small, primarily undergraduate schools, led by Mount Allison University and Acadia University, are among those that consistently exceeded their American peers. Interestingly, while the University of Western Ontario did not register above-average scores, two of Western’s affiliated colleges—Huron and Bre-

scia-scored highly in all areas.

The Student-Faculty Interaction benchmark—where no Canadian university exceeded the NSSE first-year benchmark and only three surpassed the fourth-year average—focuses on the different ways that students interact with faculty members inside and outside of the classroom. Students are asked, for example, whether they have worked with a professor on activities outside of coursework, talked about career plans with a faculty member, received prompt feedback from faculty on their academic performance and worked with a faculty member on a research project.

The overwhelming majority of Canadian students attend large, research-focused universities. Can institutions of such size offer top-level undergraduate experiences, as defined by NSSE? Results from the University of Michigan, a giant public university that is also one of America’s leading research

powerhouses, suggests that it is possible.

What is Michigan doing right in undergraduate education? Earlier this decade, Michigan was one of 20 American universities identified by NSSE as having outperformed on the NSSE benchmarks. A NSSEcommissioned study visited each of the outperforming campuses to find out what practices were leading to those high NSSE benchmark scores. For example, explaining Michigan’s success on the Student-Faculty Interaction benchmark, the study cited Michigan’s small classes and research opportunities in first-year; programs that encourage students and faculty to eat meals together; mentorship programs; extensive email contact between students and faculty and professors’ offices that are located in residences. On the Level of Academic Challenge benchmark, the study pointed to a commitment to excellence that permeates the entire Mich-

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Student Satisfaction Results

The NSSE survey is not primarily a student satisfaction survey. The main purpose of NSSE is to assess what students are doing—as shown in the benchmark tables on pages 40, 42 and 44—not to ask for their opinion. However, NSSE includes some satisfaction questions, including one asking students to evaluate their educational experience. Most institutions' scores declined from first to fourth year.

How would you evaluate your entire educational experience at this institution?

FIRST-YEAR STUDENTS

ä Excellent (%) Good (%)

Queen's 53 Western 47 Nipissing 45 Guelph 431 Mount Allison 43 Waterloo 411 Wilfrid Laurier 40 Acadia 391 St. Thomas 38! McMaster 371 King's (Western) 361 NSSE 2007* 34 McGill 331 Huron (Western) 321 Trent 311 Victoria 31 Lakehead 291 OCAD 293 Alberta 281 Brock 28! Carleton 281 Laval 27! Lethbridge 271 Ryerson 271 UPE I 261 Laurentian 251 Brescia (Western) 241 UBC 24 Dalhousie 231 UNB (Fredericton) 23 UNBC 23! Saskatchewan 231 Toronto 22! York 201 Concordia 191 Ottawa 191 UNB (Saint John) 181 Calgary Regina Windsor 16 Manitoba

Refused to make this information public

*NSSE 2007 benchmark reflects the overall result for 610 Canadian and American universities

SENIOR-YEAR STUDENTS

i Excellent(%) ÜÍGood(%)

Huron (Western) 63! Brescia (Western) 57 Mount Allison 53 King's (Western) 511 Guelph Trent UPEI 41 Queen's 41 St. Thomas 411 Acadia 401 Western 38! Brock 371 UNBC 37! Nipissing 361 Waterloo 361 Wilfrid Laurier 361 NSSE 2007* 34 McMaster 341 Lethbridge 32! Victoria 291 McGill 261 Royal Roads 261 Ryerson 24! Alberta 231 Windsor 23! Carleton 221 UNB (Fredericton) 221 Concordia 21 UNB (Saint John) 21 Saskatchewan 21 Toronto 211 Dalhousie 201 Lakehead 20! Laurentian 20! Laval 201 Regina 20 UBC OCAD York 181 Calgary Ottawa 11 Manitoba

Refused to make this information public

igan campus: faculty resistance to grade inflation, introductory courses designed to challenge students’ ability to problem solve, and small classes that encourage active learning and challenge students to develop critical thinking and independence in carrying out research projects.

NSSE director Alex McCormick says while universities can use NSSE to improve, “these are things that take some intentional effort

to move the needle on. It’s not quite as simple as stepping on the accelerator in your car.” And while he believes that universities can learn a lot about best practices from one another, he cautions that it’s not always easy to make direct comparisons. Schools that enrol a large number of adults or commuters, for example, are likely to have lower scores because students have less time to spend on campus and, as a result, tend to be

less engaged than traditional undergrads living on campus. Yet the same school’s more traditional undergraduate population may be just as engaged as undergrads at other campuses. “There is a robust body of evidence that shows that the vast majority of the variation in individual student scores is within institutions, not between institutions,” says McCormick. “So if you look at all the individual students that are surveyed and

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Student Satisfaction Results

NSSE is primarily an objective look at life and learning on campus, but it also asks students to answer a few satisfaction questions. In general, senior students are more critical when evaluating their university experience. While the majority of students would choose to return to their alma mater, the number drops—in some cases sharply—for students in their final year as compared to freshmen.

If you could start over, would you go to the institution you are now attending?

FIRST-YEAR STUDENTS

Queen's 601 Western 601 Guelph 541 Nipissing 541 Mount Allison 531 Waterloo 531 Wilfrid Laurier 531 McGill 521 Laval 501 St. Thomas 491 OCAD 461 Huron (Western) 451 King's (Western) 451 McMaster 451 Victoria 451 Acadia 431 UBC 431 NSSE 2007* 431 Alberta 421 Trent 421 Ryerson 411 Laurentian 401 Saskatchewan 401 Carleton 391 Lakehead 391 Brock 381 Lethbridge 381 UPEI 381 Brescia (Western) 371 Concordia 371 UNB (Fredericton) 371 Dalhousie 341 Toronto 341 UNBC 331 York 321 Ottawa 301 Regina 291 UNB (Saint John) 271 Calgary 261 Windsor 261 Manitoba I

Refused to make this information public

*NSSE 2007 benchmark reflects the overall result for 610 Canadian and American universities

Huron (Western) 631 King's (Western) 561 Brescia (Western) 531 Guelph 511 Mount Allison 511 Royal Roads 511 St. Thomas 501 Acadia 461 UPEI 461 Wilfrid Laurier 461 Brock 451 Queen's 451 Western 451 McGill 441 Trent 441 Waterloo 441 Nipissing 431 UNBC 431 NSSE 2007* 421 McMaster 411 Victoria 391 Laval 381 Lethbridge 361 Alberta 341 Saskatchewan 341 Concordia 331 OCAD 331 Ryerson 331 UBC 321 Carleton 301 Laurentian 301 Regina 291 Toronto 291 Lakehead 281 Windsor 281 UNB (Fredericton) 261 UNB (Saint John) 26~ York 24~ Dalhousie 21 Ottawa 19~ Calgary 16~ Manitoba I

Refused to make this information public

look at variation in their responses to the NSSE items, about 90 per cent of that variation occurs within institutions and only about 10 per cent is between institutions.” As a result, says McCormick, “distilling it down to a number or set of numbers for an institution” may mask variations among departments or faculties at the same university. McCormick says that NSSE needs to find ways “to help institutional leaders look more

deeply into variations within their walls.” In other words, the really interesting story may be one like that of the benchmark scores from Western’s affiliate colleges, which are above those of Western itself.

Norrie of Ontario’s HEQCO sees a similar promise in NSSE. Canadian universities are mostly still in the early stages of drilling down to examine variations among faculties, departments, courses and even gender and

ethnic background. But Norrie says he regularly hears from university administrators who have hit on revealing findings. “When you start doing variations in NSSE results across faculties or departments, and you find some interesting variations, you say, ‘Okay, what’s going on?’ ” says Norrie. “And that gets you into a conversation about what explains the variation and the different ways of teaching and learning.”

Phil Wood, associate vicepresident of student affairs at McMaster University, has established his own minibenchmark from a set of 16 NSSE questions that zero in on an area of particular interest to him: student growth and development.

Because Wood oversees student services, he’s interested in figuring out things like: is it beneficial for a student to live in residence? Do students living in residence report higher NSSE engagement scores and higher scores on his mini-benchmark?

In 2005, the University of Toronto—an institution that is in many ways similar to the University of Michigan in terms of its vast size and the quality and breadth of the graduate and research programs it offers— hired American Tony Chambers to fill a newly created position, associate vice-provost of students. Having worked at post-secondary institutions in both countries, Chambers is often called upon to discuss the uses, and limits, of NSSE, particularly in the Canadian context.

“The systems are considerably and extremely nuanced and I think for us to compare what happens in the States to what happens in Canada is sort of a worthless analysis, to be quite honest,” says Chambers. “It gives us a sense of what institutions are doing, for sure, where we can make some decisions at an institutional level, but in terms of systems of education, I don’t think the analysis is worth a whole lot, quite honestly.”

Chambers says that some NSSE questions use terms not in wide currency in Canada, or terms that some students may interpret differently than their American peers. This could affect the answers offered by Canadians.

Despite its limitations, NSSE is proving to be a valuable diagnostic tool for Canadian universities. Back at Trent, president Bonnie Patterson says NSSE has helped to validate what she and her colleagues already knew: that Trent is a smaller, tighter-knit campus, where students experience a good number of small classes, with professors who will probably know them by their first name, and opportunities to do research or independent study with a faculty member.

For Patterson, NSSE’s value-added is that it offers comparisons among institutions, and highlights areas needing improvement. Partly in response to its findings, Patterson says that Trent has channelled resources into five key areas, including library resources and technology in the classroom. “There is always a

much longer list of what you can’t do than the list of what you can do,” she says. “Would I have loved to put money into hiring another 15 or 20 faculty members? You bet. You have to find the balance of what makes you suc-

cessful in student opinion and in satisfaction and what makes you successful in learning outcomes, but at the same time trying to be responsive to them almost from a consumer perspective.”

All Canadian universities struggle with trade-offs: whether to hire more professors or build an athletic complex; upgrade labs, fund new research or offer more undergrad course selection. Like Patterson, university administrators say that surveys such as NSSE and CUSC have helped in that process. “It validates, it informs, it gives us a better insight into the detail of issues,” says Patterson. “Rather than our own serendipitous, ad hoc examples or anecdotes, it gives you large opinion pieces that we didn’t have before we got into these surveys.” M

macleans.ca/oncampus Want to see more student survey results? For seven additional CUSC questions, as well as data from past NSSE and CUSC surveys, please visit us online. Also on our website: view Maclean's exclusive university rankings data, use our Personalized University Ranking Tool and search our database of over 10,000 scholarships in our Scholarship Finder. All at macleans.ca/oncampus.