Using the TIMSS and PIRLS 2011 International Databases
for Secondary Analysis
Time: 1pm-4pm, March 10th Room: Oke Alley
Description of Session Edit Description of Session
(a) Summary
International large-scale assessments (LSA) collect an enormous amount of data, which result in datasets containing a wealth of information for secondary analysts. In addition to achievement data from students, they also collect data from teachers, school principals and parents in the hope that this information will assist in our collective understanding of education systems and their improvement. Unfortunately, much of LSA data remains underutilized by policy makers as well as researchers, in part due to the fact that proper analysis requires the use of survey methods that are generally unfamiliar to non-technical researchers (e.g. plausible values, replicate and sampling weights). The intent of this course is to provide educational researchers with a set of tools to assist them in unraveling the complexities of two of the largest and most important LSA studies currently taking place: Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) 2011.
This workshop has three main objectives.
The first one is to provide an overview of the objectives and theoretical frameworks of two of the LSAs conducted by the International Association for the Evaluation of Educational Achievement (IEA). These are the latest cycles of the PIRLS and TIMSS, both conducted at the same time in 2011. The first part of the workshop will briefly review the documentation of these two studies: International Reports, Encyclopedias, Technical Reports, User Guides, and Theoretical Frameworks. The databases will be released in February 2013, just before the CIES pre-conference workshop.
The second objective of the workshop is to review the studies’ methodological complexities (survey and sample design), implications for analysis and the appropriate analysis methods. An overview of the upcoming (later in 2013) combined TIMSS and PIRLS 2011 data files will be provided. This dataset will permit researchers to compare the relative effectiveness of schooling across reading literacy, math and science for 4th grade students in 37 educational systems. Detailed discussion of the complexities of this combined dataset and the value of such kind of analysis will be discussed in detail.
The third objective of this course is to train the participants in analyzing the data from the TIMSS and PIRLS 2011 taking into account the studies’ complexities and design issues. To achieve this goal the course provides hands-on training on analyzing data from these studies using software (provided for free by the course organizers) that handles all issues related with the analysis of large-scale assessment data. A new version of the software, which will be released in early 2013, will be presented, introducing both improvements and new functionalities. Among these new functionalities is the use of plausible values achievement scores as explanatory variables.
Experts in the field of international LSAs will conduct the course and will be on-hand for questions about the latest cycles of the studies as well as general questions concerning the administration of LSAs and analysis of their data.
This course is intended for graduate students, emerging researchers and researchers at all levels who are interested in issues concerning international LSAs, as well as secondary data analysis in reading, mathematics, and science curriculum. Participants are required to bring laptops (Windows PC) with SPSS 14 or higher and have a basic working knowledge in inferential statistics.
The training has several instructional goals. After completing the course, the participants will be able to:
• Comprehend the conceptual and theoretical underpinnings of LSAs;
• Understand the sampling design and data structure of the TIMSS and PIRLS 2011 databases, and their implication for secondary analysis;
• Retrieve the TIMSS and PIRLS databases, international reports, technical documentation and questionnaires from the IEA’s website;
• Understand the necessity of applying specific statistical tools to analyze data from LSAs;Prepare TIMSS and PIRLS data for analysis using the software provided (IDB-Analyzer);
• Utilize IDB-Analyzer to carry out cross-national analysis of TIMSS and PIRLS data 2011.
The syllabus/planned workshop activities are as follow:
Hour 1 – Introduction to the PIRLS and TIMSS 2011studies, questionnaires and data files
Hour 2 – Introduction to statistical complexities of using PIRLS and TIMSS data
Hour 3 – Overview of the IEA IDB Analyzer
Hour 4 – Work through guided practice exercise, analysis, Q & A
Note: While a considerable amount of information will be reviewed in a short period of time, the instructors for this course have adopted a set of successful techniques that they have employed in similar trainings. The last hour will be dedicated to a structured data analysis assignment. During this time participants are encouraged to ask specific questions they may have on the topics covered.
(b) A list of the names and institutional affiliations of the instructional staff
1. Director/Session Organizer: Andres Sandoval, PhD
Position: Head of the Research and Analysis Unit
Institution: IEA Data Processing and Research Center
e-mail: andres.sandoval@iea-dpc.de
2. Co-Instructor: Plamen Mirazchiyski, PhD
Position: Researcher
Institution: IEA Data Processing and Research Center
e-mail: plamen.mirazchiyski@iea-dpc.de |
|
