Abstract:This study evaluates a 40-item mathematics placement examination administered to 198 students using a multi-method framework combining Classical Test Theory, machine learning, and unsupervised clustering. Classical Test Theory analysis reveals that 55% of items achieve excellent discrimination ($D \geq 0.40$) while 30% demonstrate poor discrimination ($D < 0.20$) requiring replacement. Question 6 (Graph Interpretation) emerges as the examination’s most powerful discriminator, achieving perfect discrimination ($D = 1.000$), highest ANOVA F-statistic ($F = 4609.1$), and maximum Random Forest feature importance (0.206), accounting for 20.6% of predictiv…
Abstract:This study evaluates a 40-item mathematics placement examination administered to 198 students using a multi-method framework combining Classical Test Theory, machine learning, and unsupervised clustering. Classical Test Theory analysis reveals that 55% of items achieve excellent discrimination ($D \geq 0.40$) while 30% demonstrate poor discrimination ($D < 0.20$) requiring replacement. Question 6 (Graph Interpretation) emerges as the examination’s most powerful discriminator, achieving perfect discrimination ($D = 1.000$), highest ANOVA F-statistic ($F = 4609.1$), and maximum Random Forest feature importance (0.206), accounting for 20.6% of predictive power. Machine learning algorithms demonstrate exceptional performance, with Random Forest and Gradient Boosting achieving 97.5% and 96.0% cross-validation accuracy. K-means clustering identifies a natural binary competency structure with a boundary at 42.5%, diverging from the institutional threshold of 55% and suggesting potential overclassification into remedial categories. The two-cluster solution exhibits exceptional stability (bootstrap ARI = 0.855) with perfect lower-cluster purity. Convergent evidence across methods supports specific refinements: replace poorly discriminating items, implement a two-stage assessment, and integrate Random Forest predictions with transparency mechanisms. These findings demonstrate that multi-method integration provides a robust empirical foundation for evidence-based mathematics placement optimization.
| Comments: | 28 pages, 8 table, 4figures, NAM conference |
| Subjects: | Machine Learning (cs.LG) |
| MSC classes: | 97C70, 62P25, 62H30, 68T05 |
| Cite as: | arXiv:2511.04667 [cs.LG] |
| (or arXiv:2511.04667v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2511.04667 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Julian Allagan [view email] [v1] Thu, 6 Nov 2025 18:53:07 UTC (31 KB)