Create a set of procedures and tools for complex analysis of educational tests

Methods based on classical test theory (CTT) dominate in processing data from educational tests administered in the Czech Republic. These methods have limited capacity in more complex analytical tasks. The proposed set of procedures and tools allows a choice between CTT methods, item response theory (IRT) methods, and nonIRT advanced methods. The methods implemented in the set of procedures and tools are useful not only for complex analysis of administered tests but also for the development of new admission, final or other educational tests that meet high quality standards. Examples of methods implemented in the set of procedures and tools include:

• methods to calculate total score from a combination of different item types (e.g., multiple choice and ordinally scored items)
• methods to identify problematic test items
• methods to detect between-group differences
• methods to analyze text difficulty in language tests
• computer-adaptive testing methods

Support the use of proposed procedures and tools in the analysis of existing admission, school leaving, and other educational tests, and in the development of new tests

The project reacted to a nation-wide need for high-quality educational tests that do not contain questionable items, are equally fair to different gender, social, and other groups of students, and comparable across different versions and years. The project therefore put emphasis not only on developing advanced procedures and tools but also on their implementation in real educational assessments, most importantly in high-stakes exams and tests. The following activities supported the use of proposed procedures and tools in routine processes of test development, validation, and data analysis:

• Regular meetings with the application guarantor
• Adding new options and functions to the set of procedures and tools based on the feedback from the application guarantor and other users
• Workshop for the application guarantor with a focus on demonstrating the use of the proposed set of procedures and tools in the processes of test development and data analysis
• Workshop for professionals demonstrating new possibilities of data analysis and data reporting
• Presentation of project results on national and international conferences 

Support informed decision-making at the level of schools, regional authorities, and state administration

Educational tests can provide valuable data on strengths and weaknesses of individual students, groups of students (e.g., based on their gender, region, school type), classes, or schools. However, the use of these data is rather limited. An important aim of the project was to propose, in collaboration with school leaders and the application guarantor, procedures to support the use of data from educational tests in the process of reflecting on school instruction and the quality of education. To fulfill this aim, the project offered:

• Model reports from educational tests with verbal interpretations of achieved results
• Tools for graphical representation of achieved results, between-group
  differences, and other information
• Workshop for school leaders aimed at interpreting reports from educational tests
• Workshop for professionals demonstrating new possibilities of data analysis and data reporting.