Measuring young minds

A national exam stirs a debate over testing

VICTOR DWYER February 10 1997

Measuring young minds

A national exam stirs a debate over testing

VICTOR DWYER February 10 1997

Measuring young minds


A national exam stirs a debate over testing


It is a community whose job is to teach—and grade—the nation’s students. So it seemed appropriate last week when public school educators across Canada received a report card of their own. And as with any exam results, the final marks on a nationwide science test of 13-and 16-year-olds produced both jubilation and disappointment.

“This is very gratifying,” said Paul Webster, a science teacher at Western Canada High School in Calgary, upon hearing that Alberta students performed exceptionally well on the test of 37,500 students, part of the School Achievement Indicators Program (SAIP) of the Council of Ministers of Education. “I think it reflects both the commitment of our teachers and the hard work of our students.” But in a handful of other provinces—particularly Ontario, the only one in which scores lagged behind the national average for students of both ages—the reaction was decidedly more glum. There was no ducking from Joan Green, chief executive officer of the Ontario Educational Quality and Accountability Office, which was launched last year with a mandate to develop provincewide tests of student achievement as early as Grade 3. “Let’s face it,” she said, “we simply did not do as well as we would have liked to.”

But amid reaction to specific scores, the results reignited a broader debate about the merits of standardized tests, their effects on teaching and learning—and about what they really prove. Critics say that while such examinations provide one broad indicator of educational achievement, their attempt to measure all students with a single yardstick creates a false impression of how well schools are preparing pupils for the real world. Others welcomed the results as a vital check on the quality of public education. In fact, some argued that the SAIP test did not go far enough.

Like the Third International Mathematics and Science Study of a half-million 13-year-olds in 41 countries, whose results were released last November, the SAIP test was designed to measure the success of educational systems—

not individual pupils. Said Cynthia Bled, chairwoman of the Ottawabased Committee for Canada Wide Standards in Education: “Every student, everywhere in the country, should have better feedback about what they are learning in our schools.”

The science test, conducted in the spring of 1996, follows similar surveys in mathematics, held in 1993, and reading and writing, in 1994. Last week’s results were based on a random sample of students, broken into two groups. One performed a written assessment consisting of multiple-choice and short-answer questions. The other conducted hands-on scientific experiments. In both

cases, tasks were assigned at five levels of increasing difficulty.

The national team of educators who designed the test determined that most 13-year-olds should be able to achieve at least Level 2 on the scale, and most 16-year-olds Level 3 or better. In the written portion, the only part that produced province-by-province results, students at Level 2 were asked, for example, to classify substances according to their physical properties, and to describe how the movement and tilt of the Earth affects cycles such as years, days and seasons. Students at Level 3 were expected to know that some forms of life are unicellular and others multicellular, and to use

chemical properties to classify substances.

In the test’s practical component, 93 per cent of 13-year-olds nationwide, and 65 per cent of 16-year-olds, performed at the levels expected by the panel. In the written portion, roughly 70 per cent of both age groups did. But provincial scores showed a number of regional differences. Among 13-year-olds, most in Grade 8, students in Alberta, Prince Edward Island and Saskatchewan performed significantly ahead of the national average. Among 16-year-olds, the only province to do so was Alberta. Lagging significantly behind the nation were students of both ages in Ontario, and 16-year-olds in Newfoundland.

But beyond such regional differences, the tests also revealed another, more far-reaching concern. As a nation, Canada is producing only tiny numbers of students able to tackle the most complex scientific problems. While a national panel of teachers, parents and business leaders had expected 73 per cent of 13-year-olds to perform at Level 3 on the written component, only 63 per cent did so. By age 16, a scant 3.4 per cent were able to conquer the hardest

questions—those at Levels 4 and 5. That was far fewer than the 13.3 per cent expected.

Such numbers are alarming, say many observers, because they reflect the country’s inability to compete in the global race to produce cutting-edge scientific thinkers. A similar warning was sounded when Canadian students were outpaced by many other countries in the recent international math and science study. In that test, Canada placed roughly in the middle of the pack, far behind such Asian powerhouses as Singapore, Korea and Japan. And only nine per cent scored in the top tenth of that international class—compared with 30 per cent in top-rated Singapore. Those numbers carry a warning. “It is particularly in math and science areas that companies are running into skills shortages,” says David Stewart-Patterson, senior associate for policy and communications at the Business Council on National Issues. “If they can’t find skilled people here, and they can find them somewhere else, that is where they will put their money.” In the international results, Alberta, along with British Columbia, finished well ahead of the Canadian average in both subjects. And Alberta’s repeat success in last week’s tests is no accident, provincial authorities say. In a program launched about a decade ago, the province completely revamped its approach to teaching science and three other core subjects: language arts, mathematics and social studies. Along with explicit curriculum guidelines, which spell out what students should be able to do and know at various grade levels, the ministry of education introduced provincewide tests in grades 3, 6 and 9, as well as high-school graduating exams. “We now regard the assessment side of things as a vital part of the learning loop,” says Jim Brackenbury, acting director of student evaluation for the Alberta ministry. “It’s all part of making the standards and expectations as clear as possible.” That drive to state expectations clearly—and to ensure they are met—appears to be spreading. The other three Western provinces, Quebec and New Brunswick now have some form of standardized exams. And next month, Ontario will launch a

series of examinations of students at four different grade levels.

The first of those will measure the reading, writing and mathematical skills of every Grade 3 student in the province. Following a one-day training session, teachers will conduct comprehensive, two-week tests, and then forward the results to Green’s quality and accountability office. There, a team of 1,000 teachers I will mark the examinait tions. In June, every g child’s report card will 1 include his or her score. “ Those exams will include writing samples and hands-on activities in addition to more traditional, standardized questions. Says Green: ‘We want to ensure that the tests will measure skills that apply to the real world.”

A long-held criticism of standardized tests is that they have not attempted to measure such things. And many critics argue that the secret of such educational Wunderkinder as Korea—which placed among the top three countries in both the math and science parts of the international tests—is a school system that stresses hard facts over creativity and imagination. Beginning as early as Grade 3, it is common for young Koreans to attend after-school hagwons— cram schools that emphasize the memorization and regurgitation of facts. Such classes are the start of a decade of preparation for the intensely competitive annual College Scholastic Aptitude Test, a

9.5-hour marathon that includes hundreds of questions, and determines who will gain acceptance to the country’s elite universities.

Still, even as some test designers begin to include more practical components, some educators continue to worry about the effect of such exams on other aspects of teaching and learning. “When you focus too much on standardized tests, you run the risk of gearing your curriculum to them,” says Maureen Morris, president of The Canadian Teachers’ Federation. We should be directing what goes on in the classroom to what is new, what is relevant, and what meets the needs of individual students and individual schools.” Others agree. “There are a number of ways to measure a school’s success—its relationship with the community, its responsiveness to parents,” says Richard Gauthier, director of the Frenchlanguage Education Policy and Programs Branch at the Ontario Ministry of Education and Training.

“Academic performance is an important measure, but it gives us only one indication of how the system is doing.”

In fact, many educators argue that acrossthe-board testing can often lead to real changes at the classroom level. In Ontario,

Green’s office has already commissioned a study into the educational backgrounds of math and science teachers in those jurisdictions that scored well in the recent international tests, with the aim of building on their success. When the 1994 Canadian tests in reading and writing showed francophone students in several provinces to be scoring lower than anglophones, the ministries of education in Ontario, Manitoba and New Brunswick launched a joint project, scheduled to be completed as early as June, to get to the bottom of it.

And in the four Atlantic provinces, which scored at or below the national average in the SAIP math tests in 1993, the joint Atlantic Provinces Education Foundation has since worked to develop an, improved, common math curriculum for grades 1 to 12, now being piloted across the region. It mirrors a high-school physics and chemistry curriculum launched last fall, and follows the publication last year of The Essential Graduation Learnings, a document that outlines shared expectations for high-school graduates in all four provinces. “Our students are facing a national competition for university spots and an international job market,” says foundation member Tom Rich. “This is an attempt to define standards that ensure we are offering the best education, and the clearest expectations that we possibly can.”

As the pressure mounts on graduates themselves, it is almost certain that educators will continue to be pressed for clear, easy-to-read evidence that they are doing their own jobs well. At the Committee for Canada Wide Standards, Bled insists the next step is to offer every high-school student in the country the chance to take a national final exam. “Our graduates need the kind of document they can take anywhere inside or outside the country,” says Bled. “They should be able to prove to anybody out there just how much they know.” Short of that ambitious goal, teachers like Calgary’s Webster were happy last week simply to take a few minutes to bask in the latest indication of their success—and that of their students. “At the end of the day, the goal of every teacher is to turn kids on to learning,” said Webster. “It’s nice just to be told we’re doing something right.”

With MICHAEL SNIDER in Chonju, South Korea