Create a set of procedures and tools for complex analysis of educational tests

Methods based on classical test theory (CTT) dominate in processing data from educational tests administered in the Czech Republic. These methods have limited capacity in more complex analytical tasks. The proposed set of procedures and tools will allow a choice between CTT methods, item response theory (IRT) methods, and nonIRT advanced methods. The methods implemented in the set of procedures and tools will be useful not only for complex analysis of administered tests but also for the development of new admission, final or other educational tests that meet high quality standards. Examples of methods implemented in the set of procedures and tools include:

• methods to calculate total score from a combination of different item types (e.g., multiple choice and ordinally scored items)
• methods to identify problematic test items
• methods to detect between-group differences
• test equating methods
• methods to analyze text difficulty in language tests
• computer-adaptive testing methods

Support the use of proposed procedures and tools in the analysis of existing admission, school leaving, and other educational tests, and in the development of new tests

The project reacts to a nation-wide need for high-quality educational tests that do not contain questionable items, are equally fair to different gender, social, and other groups of students, and comparable across different versions and years. The project therefore puts emphasis not only on developing advanced procedures and tools but also on their implementation in real educational assessment, most importantly in high-stakes exams and tests. The following activities are planned to support the use of proposed procedures and tools in routine processes of test development, validation and data analysis:

• Regular meetings with the application guarantor
• Adding new options and functions to the set of procedures and tools based on the feedback from the application guarantor and other users
• Workshop for the application guarantor focusing on using the set of procedures and tools in the processes of test development and data analysis

Support informed decision-making at the level of schools, regional authorities, and state administration

Educational tests can provide valuable data on strengths and weaknesses of individual students, groups of students (e.g., based on their gender, region, school type), classes, or schools. However, the use of these data is rather limited. An important aim of the project is to propose, in collaboration with school leaders and the application guarantor, procedures supporting the use of data from educational tests in the process of reflecting on school instruction and the quality of education. To do this, the project will offer:

• Model reports from educational tests with verbal interpretations of achieved results
• Tools for graphical representation of achieved results, between-group
  differences, and other information
• Workshop for school leaders aimed at interpreting reports from educational tests
• Workshop for professionals demonstrating new possibilities of data analysis and data reporting.