Trends in PCAP student assessment

The Pan-Canadian Assessment Program (PCAP) is a student achievement assessment that is administered every 3 years to Grade 8 students across Canada. The Council of Ministers of Education, Canada (CMEC) is responsible for preparing these standardized tests, then collating data from the assessments. Roughly 30,000 Grade 8 students participate in each Reading, Mathematics, and Science assessment. In Canada there were about 360,000 Grade 8 students in 2019; in New Brunswick, there were about 7500 Gr 8 students (Statistics Canada Table 37-10-0007-01). That’s a sample size of about 8%; a good sample size, provided students are selected at random. The charts below show the results for each province and the average performance for all provinces and territories (Canada); territorial results aren’t shown as numbers of students are quite small. There are separate charts with results of Reading, Mathematics, and Science assessments. These show results for each 3-year assessment (2007, 2010, 2013, 2016, 2019) by province.

How are assessment scores used? From the PCAP 2019 Assessment Framework document: “The actual results from students’ assessments are called “raw scores.” The raw scores are converted to a scale, which has a range of 0 to 1000. These raw scores are standardized, providing a common measurement so that meaningful comparisons can be made of scores obtained from different populations over time and on different versions of a test. The standardized scale used for PCAP assessments places scores on a normal distribution with a midpoint or mean of 500 and a standard deviation of 100. The scale midpoint of 500 is equal to the pan-Canadian average for each subject in the baseline year [The baseline year is the year in which the domain was the major domain assessed (2007 and 2016 for reading, 2010 and 2019 for mathematics, and 2013 for science)]. The majority of students in Canada—about two thirds—will score between 400 and 600, or within one standard deviation of the mean. This mean can then be used as a reference point that allows the comparison of Canada-wide results.” Scores are also weighted so that provinces with greater student populations are given more weight in constructing the national average scores.

For the first set of charts, I’m not providing a breakdown of results by gender or language of instruction although CMEC reports do have those data. Some data from New Brunswick showing results for anglophone and francophone school systems are shown further down on this page. There seems to be a national trend, which also appears in NB, where students in francophone schools tend to have higher scores in mathematics than anglophones.  Each area (e.g. Mathematics) can broken down into various sub-groups (geometry, data management) but I’m not showing those here. If you examine the PCAP reports for each year, you’ll see that standard errors are provided to show statistically significant differences between different provinces for each year. Here, I’m more interested in changes over time, and, while PCAP results do provide some analyses showing changes over time, they’re not extensive. Therefore, I can’t do more than take a look at trends over time.

Now let’s look at student assessment by the language of the school system. In New Brunswick, there are anglophone and francophone schools with separate province-wide administrative systems. The following charts show Reading, Mathematics and Science for the two language groups in New Brunswick.

Comment: For the Maritime provinces there’s been a slow but generally steady upward improvement in test scores over time in Reading (a measure of literacy), but note the lower scores in 2013 for a number of provinces (including NB and NS). Science scores show a similar trend. For Mathematics, 2019 saw either a lower score, or a score similar to those of 2016.  Science results followed a pattern similar to that for Reading. Although there’s quite a bit of year-to-year variation for most Maritime provinces, the trend is upward and that’s a good thing. Note, however, that some provinces are quite a bit better than NB or NS. Quebec, for example, is quite strong in Mathematics; Ontario is strong in Reading; and Alberta does well in all domains.

The strength of Quebec students in Mathematics is consistent with the performance of francophone students in other provinces. Perhaps there’s a different approach in Mathematics instruction there that is worth emulating.

We can pull out the PCAP assessment data for New Brunswick and surrounding provinces (Nova Scotia, Prince Edward Island, and Quebec). Apart from Mathematics, where Quebec out-performs, the assessments and trends are similar.

There’s good evidence that family income has an impact on ‘outcomes’ (overall well-being, student assessments) of the children. A report by Statistics Canada illustrates this nicely: “The size of the association between income and child outcomes varies with developmental domain. Thus, for example, income has particularly strong associations with cognitive outcomes (e.g., Peabody Picture Vocabulary Test (PPVT) scores or math and reading scores) and behavioural outcomes (e.g., hours spent watching television). Physical health outcomes also have quite consistent positive associations with family income”

It’s no secret that residents of New Brunswick and surrounding provinces generally have the lowest median family incomes in Canada. Perhaps that is a critical factor controlling student achievement results. Look at these trends in median income from Statistics Canada Table: 11-10-0009-01 (formerly CANSIM 111-0009):

Compared to the median income for Canada, families in New Brunswick (and other Maritime provinces) are generally poorer. Further, if you look at the ‘average’ median income for Alberta, British Columbia and Ontario, Maritime families are quite a bit lower (BC, ON and AB Reading and Science scores tend to be higher than those NB).

If school assessment scores are a concern, then given the relationship between income and child outcomes, perhaps New Brunswick should be providing more financial assistance to low income residents (as Federal governments have done) and/or invest more dollars in pre-K, kindergarten, and primary school resources. Perhaps that would be a better policy approach than tax cuts, or abolishing French Immersion.

Data sources:

Pan-Canadian Assessment Program : https://www.cmec.ca/538/PCAP-13_2007.html

Income and the Outcomes of Children. 2006. Shelley Phipps and Lynn Lethbridge, Statistics Canada, Business and Labour Market Analysis. Analytical Studies Branch Research Paper Series 11F0019 No. 281  ISSN: 1205-9153  ISBN: 0-662-42899-4.