Project
EduTest
Advances in educational assessment:
Analytical support for educational test development
Results
Set of procedures and tools for complex analysis of educational tests
The set of procedures and tools enables rapid analysis of data from educational tests, including detailed analysis of between-group differences (e.g., by gender or school type), and textual analysis of item wording. The set of procedures and tools consists of several interactive modules and interactive applications tailored for different types of analyses. It further includes a set of text documents that describe the interactive tools, explain their functions, and demonstrate their use by providing practical data analysis examples. The complete set of procedures and tools is also available at https://edutest.cs.cas.cz/.
EduTest Item Analysis
The interactive module for the analysis of item data from maturita exams and secondary school admission tests produced by the Centre for the Evaluation of Educational Achievement (CERMAT) is available in the Modules section of the ShinyItemAnalysis interactive application. Publicly available item data from a selected test can be uploaded to the module. The user must also prepare and upload metadata (table in .xlsx format) describing test item format and response options, as defined in publicly available test booklets and correct solution keys. The module offers data analysis based on both classical test theory and item response theory.
Open module
Open methods description and user guide (in Czech)
Download sample metadata for the maturita exam test Mathematics 2019
EduTest Text Analysis
The interactive module for predicting item difficulty from item wording is available in the Modules section of the ShinyItemAnalysis interactive application. The module is designed to complement and improve item difficulty predictions done by content experts during the test development phase. The module specifically focuses on reading comprehension items in tests of English as a foreign language. Once the item wording is entered, the module calculates the item’s textual features and estimates its difficulty based on these features. In addition to analysing items entered by the user, the module offers a demonstration of the available analyses on three sample items already loaded in the module.
Open module
Open methods description and user guide (in Czech)
EduTest Maturita
The application is destined for secondary schools. It enables interactive analyses of student results in maturita exams in a particular school against the background of results achieved by students in selected school types in the Czech Republic. It offers data filtering options according to several criteria, summary reports in textual and graphical formats, and a detailed analysis of the school’s strengths and weaknesses. The application uses publicly available anonymized nationwide item data of student performance in maturita tests in the Czech language and literature, English, and mathematics. To perform the analyses, the school must upload its students’ data, which it receives from CERMAT. For demonstration purposes, a similar application was created, which contains pre-loaded sample data of a fictitious secondary school.
Open the application for secondary schools
Open the application with pre-loaded sample data of a fictitious school
Open methods description and user guide (in Czech)
EduTest JPZ
The application is similar to EduTest Maturita but destined for analyses of student performance in secondary school admission tests (JPZ). It enables interactive analyses of results achieved by students of a particular basic school against the background of results achieved by all students in the Czech Republic. The application uses publicly available anonymized nationwide item data of student performance in secondary school admission tests. To perform the analyses, the school must upload its students’ data, which it receives from CERMAT. Simulated sample data of students from a fictitious basic school can be used to try out the application.
Open the application for basic schools
Download sample data of a fictitious school
EduTest CAT
The application provides a tool for users to practice for educational tests using the principles of computerized adaptive testing (CAT). Currently, it contains items from Maturita exam tests of mathematics administered in 2019 and 2022. In addition to using it as a training tool for Maturita examinations, the application can be employed to demonstrate the process of computerized adaptive testing via graphical visualizations included directly in the main interface. Username: catlogin1 and password: catpassword1 must be entered to access the application.
Open the application
Open methods description and user guide (in Czech)
Project presentations at conferences
Martinková, P., Hladká, A., & Netík, J. (2021, April). Psychometrická analýza interaktivně a v R: Co je nového v ShinyItemAnalysis [Psychometric analysis interactively and in R: What’s new in ShinyItemAnalysis] [Paper presentation]. Psychologická diagnostika Brno, online.
https://psychodiagnostika.fss.muni.cz/program/prispevky/id44
Martinková, P. (2021, July). Computational aspects of psychometrics taught with R and Shiny [Paper presentation]. UseR!2021, online.
Dlouhá, J., Štěpánek, L., & Martinková, P. (2022, July). Item difficulty prediction using computational psychometrics and linguistic algorithms [Poster presentation]. International Meeting of the Psychometric Society, Bologna, Italy.
Martinková, P., Potužníková, E., & Vařejková, M. (2022, September). Repeated covariate equating to provide comparable scores from matura exam in case of non-equivalent groups [Paper presentation]. European Conference on Educational Research, online.
https://eera-ecer.de/ecer-programmes/conference/27/contribution/53048/
Martinková, P., Potužníková, E., & Netík, J. (2022, September). Interaktivní nástroj pro podporu vyhodnocování dat ze standardizovaných testů [An interactive tool for supporting analyses of data from standardized tests] [Paper presentation]. Annual conference of Czech Educational Research Association, Babice, Czech Republic.
https://capv.cz/konference-capv-2022/
Štěpánek, L., & Netík, J. (2022, November). Testujeme testy: Přijímačky a maturity z pohledu psychometrie a data science [Testing the tests: Psychometrics and data science views of school admission and maturita tests] [Paper presentation]. The Week of the Czech Academy of Sciences, Praha, Czech Republic.
https://www.cs.cas.cz/news/2022-11-02-Den-otevrenych-dveri/cs
Netík, J., & Martinková, P. (2023, July). Enhancing Psychometrics with Interactive ShinyItemAnalysis Modules [Poster presentation]. European Congress of Methodology, Gent, Belgium. https://eam2023.ugent.be/
Vařejková, M., Martinková, P., & Potužníková, E. (2023, July). A simulation study of repeated covariate equating [Poster presentation]. European Congress of Methodology, Gent, Belgium. https://eam2023.ugent.be/
Štěpánek, L., Dlouhá, J. & Martinková, P. (2023, July). Machine-learning prediction of test item difficulty using item text wordings: Comparison of algorithms and domain experts‘ predictive performance [Paper presentation]. European Congress of Methodology, Gent, Belgium. https://eam2023.ugent.be/
Martinková, P., Vařejková, M., & Potužníková, E. (2023, September). Obtaining comparable scores from multiple test forms in case of non-equivalent groups via repeated covariate equating [Paper presentation]. Frontiers in Educational Measurement, Oslo, Norway. https://www.uv.uio.no/cemo/english/about/news-events-and-publications/events/conferences/fremo2023/
Martinková, P. (2023, November). Data science pro analýzu přijímacích a maturitních testů [Data science for the analysis of school admission and maturita tests] [Paper presentation]. The Week of the Czech Academy of Sciences, Praha, Czech Republic.
https://www.cs.cas.cz/news/2023-11-08-Den-otevrenych-dveri/cs