Monitoring Educational Quality Internationally:
Recent Findings from TIMSS and PIRLS 2011
| Ina VS Mullis | mullis@bc.edu
TIMSS & PIRLS International Study Center - Boston College
Session Organizer |
|
| Michael Oliver Martin | martinas@bc.edu
Boston College
Presenter |
|
| Hans Wagemaker | HWagemaker@fakecies.com
International Association for the Evaluation of Educational Achievement (IEA), The Netherlands
Presenter |
This panel session will present trend results recently released from IEA's TIMSS and PIRLS 2011 international assessments. In particular, in 2011 the four-year trend cycle of the TIMSS mathematics and science assessments and the five-year cycle of the PIRLS reading assessment came together, providing a unique opportunity for reading, mathematics, and science achievement data, accompanied by information from parents, schools, and teachers. TIMSS and PIRLS 2011 continue the series of international assessments in mathematics, science, and reading conducted by the International Association for the Evaluation of Educational Achievement (IEA). IEA pioneered international comparative assessments of educational achievement in the 1960s to gain a deeper understanding of the effects of policies and practices across countries' different systems of education. TIMSS and PIRLS are directed by IEA's TIMSS & PIRLS International Study Center at Boston College.
TIMSS is an international assessment of mathematics and science at the fourth and eighth grades that has been conducted every four years since 1995. Nationally representative samples of students in 63 countries and 14 benchmarking entities (regional jurisdictions of countries, such as states) participated in TIMSS 2011. Countries and benchmarking participants could elect to participate in the fourth grade assessment, the eighth grade assessment, or both. Fifty-two countries and seven benchmarking entities participated in the fourth grade assessment, and 45 countries and 14 benchmarking entities participated in the eighth grade assessment. In total, more than 600,000 students participated in TIMSS 2011.
PIRLS is an international assessment of reading comprehension at the fourth grade that has been conducted every five years since 2001. Nationally representative samples of students in 49 countries participated in PIRLS and pre PIRLS 2011. Forty-five countries assessed fourth grade students, and some countries participated in one or more of the other available options initiated in 2011 to permit wider participation at the end of the primary school cycle. Four countries assessed their sixth grade students, and three countries participated in prePIRLS, a less difficult version of PIRLS inaugurated in 2011 to be a stepping stone to PIRLS. In total, approximately 325,000 students participated in PIRLS 2011, including countries assessing students at more than one grade, benchmarking assessments, and prePIRLS.
Taken together, the TIMSS and PIRLS assessments constitute an internationally accepted definition of the goals of reading, mathematics, and science education. The TIMSS assessments of mathematics and science are based on comprehensive frameworks developed collaboratively with the participating countries, each organized around two dimensions: a content dimension specifying the domains or subject matter to be assessed within mathematics or science, and a cognitive dimension specifying the domains or thinking processes expected of students as they engage with the content. In PIRLS, students responded to questions designed to measure their reading comprehension across two overarching purposes for reading: reading for literary experience, and reading to acquire and use information. Within the two reading purposes, assessment items addressed a range of comprehension processes.
To monitor progress toward the goals, the assessments are administered regularly – every four years for TIMSS and every five years for PIRLS. Representative samples of students in the participating countries take the assessments, which are developed collaboratively through joint item-writing workshops. Given the frameworks' broad coverage goals, both the mathematics and science assessment pools of items were necessarily large, each with a total of approximately 400 assessment items. The assessment was comprised of 10 reading passages (5 literary and 5 informational) with each 800-1,000 word passage accompanied by 12-15 items, for a total of 135 items.
The TIMSS and PIRLS results are reported on achievement scales with a range of 0-1,000 (although student performance typically ranges between 300 and 700). Across TIMSS and PIRLS 2011, there are separate scales for reading, mathematics, and science at the fourth grade and mathematics and science at the eighth grade. In addition, there are approximately a dozen background scales measuring home, school, and classroom contexts for learning.
Drs. Ina V.S. Mullis and Michael O. Martin, Executive Directors of the TIMSS & PIRLS International Study Center, will highlight the results of the TIMSS and PIRLS 2011 assessments. Dr. Hans Wagemaker, IEA Executive Director, will discuss the expanding role and importance of international assessments.
More specifically, Dr. Ina Mullis will summarize the latest TIMSS 2011 mathematics and science achievement results of fourth and eighth grade students in countries around the world, and describe trends over the five TIMSS assessments since 1995. For PIRLS, she will present the reading achievement results for 2011 and over time since 2001 and 2006. To illustrate high quality educational outcomes, Dr. Mullis will describe the achievements of students reaching the top TIMSS and PIRLS International Benchmarks and show results for high-performing countries. She will conclude by providing some examples of how countries use TIMSS and PIRLS data for system-level monitoring in a global context to inform policy. Essentially, countries compare trends in student achievement to that of other countries and implement reforms as necessary to address weaknesses or inequities among groups.
Dr. Michael Martin will present the contextual data for teaching and learning in mathematics, science, and reading collected by the TIMSS and PIRLS 2011 questionnaires for parents, principals, teachers, and students. He will begin by describing the TIMSS and PIRLS innovative approach for creating and reporting policy relevant background scales measuring home, school, and classroom environments supportive of learning. The scale results will be discussed in relation to student achievement in mathematics, science, and reading. For example, in every country an early start was associated with higher achievement on TIMSS and PIRLS. That is, students had higher achievement if their parents had engaged them in early literacy and numeracy activities. Likewise, parents' reports of their children's ability to do early literacy and numeracy tasks upon starting primary school was related to higher achievement, as was attending preprimary education. The data also show that achievement in mathematics, science, and reading is higher among students attending well-resourced schools that have safe and orderly environments, emphasize academic success, and engage students in learning.
Dr. Hans Wagemaker will reflect on the increasing prominence of quality issues in national, regional and global educational contexts. He will briefly outline IEA's strategy for addressing current concerns about assessing educational quality as central to the global development agenda. Dr. Wagemaker will discuss the role of IEA's international assessments in responding to, and shaping the conversation related to a greater focus on measuring educational outcomes. For example, to meet the needs of a range of countries where most children in the fourth grade are still developing fundamental reading skills, IEA has developed prePIRLS as a stepping stone to participating in PIRLS. prePIRLS reflects the same conception of reading as PIRLS, except it is less difficult and can be given at the fourth, fifth, or sixth grade.
|
