An Analysis of the Online Final Examination Items in the Mathematics Course Using the Rasch Measurement Model

Authors

  • Tommy Tanu Wijaya (Scopus Author ID : 57218281226), Beijing Normal University, Beijing
  • Jiajie Yan Beijing Normal University, Beijing
  • In In Mutmainah SMPN33 Bandung, Indonesia
  • M. Afrilianto Institut Keguruan dan Ilmu Pendidikan Siliwangi
  • Tina Rosyana Institut Keguruan dan Ilmu Pendidikan Siliwangi
  • Xinxin Li Guangxi Normal University, Guilin

DOI:

https://doi.org/10.30983/educative.v7i2.6001

Abstract

Abstract. The National Examination in Indonesia has been abolished since 2020. Hence, the Indonesian Junior High School (SMP) makes its final examination items for the 9th-grade, and from the results, the school determines success in the course. The final examination has an important role and significant value in making decisions about students' graduation. Therefore, this study is aimed to analyze the Online Final Examination items in one of the public junior high schools in Bandung. The sample was 234 students in grade 9 using their mathematics examination tests, comprising 20 multiple-choice items with 4 options (A-D), while the data processing used Winsteps software with the Rasch modeling technique. Subsequently, the Rasch model results showed an acceptable people separation statistic of 1.54 and poor person reliability at 0.7<0.8. The item separation statistics was in a good category at 2.21, while the reliability at 0.83 was excellent. Although 4 online final examination items were in the fit category, 16 were good and capable of dividing students according to their abilities. They also provided very detailed data about the quality of the items and the ability of each grade 9 student. Since each test item is included in the fit category, this study contributes information on preparing and analyzing the Online Final Examination to teachers.

References

Bai, C., Chi, W., & Qian, X. (2014). China Economic Review Do college entrance examination scores predict undergraduate GPAs ? A tale of two universities. China Economic Review, 30, 632–647. https://doi.org/10.1016/j.chieco.2013.08.005

Boone, W. J., Yale, M. S., & Staver, J. R. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4

Engelhard, G., & Wang, J. (2020). Developing a concept map for rasch measurement theory. Springer Proceedings in Mathematics and Statistics, 322, 19–29. https://doi.org/10.1007/978-3-030-43469-4_2

Faradillah, A., & Febriani, L. (2021). Mathematical Trauma Students’ Junior High School Based on Grade and Gender. Infinity Journal, 10(1), 53. https://doi.org/10.22460/infinity.v10i1.p53-68

Haspel, R. L., Lin, Y., Fisher, P., Ali, A., & Parks, E. (2014). Development of a validated exam to assess physician transfusion medicine knowledge. Transfusion, 54(5), 1225–1230. https://doi.org/10.1111/trf.12425

Hohne, H. H. (1949). the Prediction of Academic Success. Australian Journal of Psychology, 1(1), 38–42. https://doi.org/10.1080/00049534908256014

Huang, L., Huang, F., Oon, P. T., & Mak, M. C. K. (2019). Constructs evaluation of student attitudes towards science. Eurasia Journal of Mathematics, Science and Technology Education, 15(12). https://doi.org/10.29333/ejmste/109168

Jennings, N. B., Slack, M. K., Mollon, L. E., & Warholak, T. L. (2016). Measurement characteristics of a concept classification exam using multiple case examples: A Rasch analysis. Currents in Pharmacy Teaching and Learning, 8(1), 31–38. https://doi.org/10.1016/j.cptl.2015.09.010

Karlin, O., & Karlin, S. (2018). Making Better Tests with the Rasch Measurement Model. InSight: A Journal of Scholarly Teaching, 13(1), 76–100. https://doi.org/10.46504/14201805ka

Locastro, V. (1990). The English in Japanese university entrance examinations: a sociocultural analysis. World Englishes, 9(3), 343–354. https://doi.org/10.1111/j.1467-971X.1990.tb00271.x

Manullang, S. O., Satria, E., Krisnadwipayana, U., & Hatta, U. B. (2020). The Review of the International Voices on the Responses of the Worldwide School Closures Policy Searching during Covid-19 Pandemic. 5(2), 1–13.

Mokshein, S. E., Ishak, H., & Ahmad, H. (2019). The use of rasch measurement model in English testing. Cakrawala Pendidikan, 38(1), 16–32. https://doi.org/10.21831/cp.v38i1.22750

Nopiah, Z. M., Ismail, N. A., Othman, H., Asshaari, I., Razali, N., Osman, M. H., & Jamalluddin, M. H. (2011). Identification of student achievement and academic profile in the linear algebra course: An analysis using the Rasch model. 2011 3rd International Congress on Engineering Education: Rethinking Engineering Education, The Way Forward, ICEED 2011, 197–202. https://doi.org/10.1109/ICEED.2011.6235389

Othman, H., Asshaari, I., Bahaludin, H., Nopiah, Z. M., & Ismail, N. A. (2012). Application of Rasch Measurement Model in Reliability and Quality Evaluation of Examination Paper for Engineering Mathematics Courses. Procedia - Social and Behavioral Sciences, 60(2009), 163–171. https://doi.org/10.1016/j.sbspro.2012.09.363

Purwanto, A., Pramono, R., Asbari, M., Santoso, P. B., Wijayanti, L. M., Hyun, C. C., & S, P. R. (2020). Studi Eksploratif Dampak Pandemi COVID-19 Terhadap Proses Pembelajaran Online di Sekolah Dasar. EduPsyCouns Journal, 2(1).

Putra, P., Liriwati, F. Y., & Tahrim, T. (2020). The Students Learning from Home Experience during Covid-19 School Closures Policy in Indonesia. Jurnal Iqra’ : Kajian Ilmu Pendidikan, 5(2), 30–42.

Putra, Z. H., Witri, G., & Sari, I. K. (2020). Prospective elementary teachers’ perspectives on online mathematics learning during coronavirus outbreak. Journal of Physics: Conference Series, 1655(1). https://doi.org/10.1088/1742-6596/1655/1/012057

Retnawati, H., & Wulandari, N. F. (2019). The development of students’ mathematical literacy proficiency. Problems of Education in the 21st Century, 77(4), 502–514. https://doi.org/10.33225/pec/19.77.502

Sanz, A. G., & Pavón, A. S. (2015). Toward implementing computer-assisted foreign language assessment in the official Spanish University Entrance Examination. Proceedings of the 2015 EUROCALL Conference, Padova, Italy, 215–220.

Shamsuddin, H., & Khairani, A. Z. (2019). Proceedings of the Regional Conference on Science, Technology and Social Sciences (RCSTSS 2016). In Proceedings of the Regional Conference on Science, Technology and Social Sciences (RCSTSS 2016). Springer Singapore. https://doi.org/10.1007/978-981-13-0203-9

Tavakol, M., & Dennick, R. (2013). Psychometric evaluation of a knowledge based examination using Rasch analysis: An illustrative guide: AMEE Guide No. 72. Medical Teacher, 35(1), e838–e848. https://doi.org/10.3109/0142159X.2012.737488

Weller, J. A., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W. J., & Peters, E. (2013). Development and Testing of an Abbreviated Numeracy Scale: A Rasch Analysis Approach. Journal of Behavioral Decision Making, 26(2), 198–212. https://doi.org/10.1002/bdm.1751

Wijaya, T. T., Ying, Z., Purnama, A., & Hermita, N. (2020). Indonesian students’ learning attitude towards online learning during the coronavirus pandemic. Psychology, Evaluation, and Technology in Educational Research, 3(1), 17–25. https://doi.org/10.33292/petier.v3i1.56

Wijaya, T. T., Ying, Z., & Suan, L. (2020). Gender and Self-regulated Learning During COVID-19 Pandemic in Indonesia. Jurnal Basicedu, 4(3), 725–732. https://doi.org/10.31004/basicedu.v4i3.422

Wind, S. A., & Gale, J. D. (2015). Diagnostic Opportunities Using Rasch Measurement in the Context of a Misconceptions-Based Physical Science Assessment. Science Education, 99(4), 721–741. https://doi.org/10.1002/sce.21172

Wu, X., Zhou, Y., & Mo, Z. (2020). A Comparative Study on the comprehensive difficulty of Junior High School National Examination. Journal On Education, 2(4), 352–366.

Yang, S. C., Tsou, M. Y., Chen, E. T., Chan, K. H., & Chang, K. Y. (2011). Statistical item analysis of the examination in anesthesiology for medical students using the Rasch model. Journal of the Chinese Medical Association, 74(3), 125–129. https://doi.org/10.1016/j.jcma.2011.01.027

Downloads

Submitted

2022-11-01

Accepted

2023-01-11

Published

2022-12-31

Issue

Vol. 7 No. 2 (2022): December 2022

Section

Articles

License

Authors who publish with this journal agree to the following terms:
1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).