What national monitoring can tell us about student and school characteristics associated with mathematics achievement

Introduction and purpose of the study:

New Zealand, as indicated by latest international assessment results, is one the countries with the widest spread of achievement in mathematical literacy. In other words, the variability of student mathematics scores within a school is high while the variability in scores across schools is relatively low. Moreover, Māori and Pasifika students, and students from lower socio-economic backgrounds are overrepresented in groups with lower achievement outcomes.
The purpose of the study described in this paper was to investigate and better understand the effects of student and school characteristics on students’ mathematics and statistics achievement across New Zealand. The study used data from the 2013 National Monitoring Study of Student Achievement (NMSSA).
In this study, we tried to answer the following questions:
a. To what extent do schools vary from one another in terms of mathematics and statistics scores?
b. Which student-level factors can explain the differences in students’ scores within a particular school?
c. Which school-level factors can explain the mathematics and statistics score differences between schools?

The NMSSA study and sample:
The 2013 NMSSA programme for the mathematics and statistics learning area of the New Zealand Curriculum was designed to assess and understand student achievement and progress at Year 4 and Year 8. The analysed sample was comprised of 2087 Year 4 and 2088 Year 8 students from 199 schools (100 Year 4, 99 Year 8). Contextual and background information data was gathered by means of student, teacher and principal questionnaires.
Student achievement in mathematics was assessed using a combination of group-administered and individual tasks. Among the student characteristics captured were gender, ethnicity, The NZDep2013 Index (proxy for socio-economic status), special education needs status, attitude to mathematics, amount of English spoken at home, and number of schools attended. Among the school characteristics captured were school type, school size, and school decile.

Methodology:
This study is the first to apply Hierarchical Linear Modelling (HLM) methods to examine the extent to which student-level and school-level factors may be differentially associated with NMSSA mathematics and statistics achievement for Year4 and Year 8 students. HLMs (also called multilevel models) were purposefully selected due to their accommodation of the nested structure of the data (i.e., students within schools). Following a typical approach to an HLM analysis; One-way ANOVA with Random Effects Model (empty model), Random Coefficients Model (student model), Means as Outcomes Model (school model) and Random Intercepts and Slopes Model (final model) were run for each year level.

Findings and Discussion:
The paper will describe the amount of explained variance within/between schools and report the significance of student and school-level predictors and the size of their related effects. Results will also be compared with New Zealand Programme for International Student Assessment (PISA) 2012 results where Mathematics was the major domain of assessment.
The findings of this study are expected to be of interest to researcher and policy makers. Moreover, the techniques used in the study will inform similar analyses done in the future using NMSSA data for other learning areas. The study might also inform the design of the NMSSA study going forward into future cycles of data collection.
Keywords: HLM, NMSSA, Mathematics Achievement