Dhaka University is contemplating a significant change by proposing separate Science Unit (Ka unit) question sets tailored for Bangla-medium and English-medium applicants to align with their respective curricula. The divergence in study materials between Bangla-medium and English-medium students, including madrasa students, often creates challenges for the latter group when preparing for public university admission exams. The initiative by DU to acknowledge and address these educational background differences is commendable and promotes wider access to higher education.
However, while inclusivity is vital, it is imperative that any high-stakes admission exam ensures fairness through scientific validation. As a Professor of Psychology and a psychometrics researcher, the focus is on the necessity for a valid, reliable, comparable, and fair assessment system, rather than advocating for or against separate question sets.
A crucial consideration is whether the two sets of examinations effectively measure the same underlying abilities. Without establishing construct equivalence, ensuring that the assessment is consistent and meaningful for all test takers, making scores comparable post-assessment becomes challenging.
Another key aspect is the method of merit ranking. If separate question sets are employed, determining whether applicants will be ranked separately based on their curriculum or if scores will be combined into a unified merit list requires a transparent scientific rationale to support the chosen approach.
Internationally recognized examinations like the SAT, GRE, and TOEFL have successfully managed similar challenges for years by utilizing advanced psychometric techniques such as Item Response Theory (IRT), test equating, common anchor items, differential item functioning (DIF) analyses, and measurement invariance studies to maintain score comparability across different versions of the same exam.
The critical factor is ensuring that separate question sets, if implemented, can accurately measure the same abilities with comparable precision. If proven feasible, separate question sets can enhance inclusivity without compromising fairness. However, without proper scientific validation, the reform may unintentionally erode public trust.
For multiple-choice-based exams, incorporating common anchor items in both sets is a practical approach to establish a statistical link for comparing question difficulty levels. Using appropriate IRT models and conducting test equating are crucial steps to convert raw scores onto a standardized reporting scale.
Furthermore, conducting DIF analyses on each item to prevent bias towards any specific curriculum and performing measurement invariance assessments to ensure score consistency across diverse educational backgrounds are essential. If written components are included, additional measures like detailed scoring rubrics, assessor training, and inter-rater reliability checks are necessary safeguards.
To bolster confidence in the proposed reform, Dhaka University should consider conducting a pilot study before full implementation, creating a calibrated item bank, designing exams with common anchor items, assembling a psychometric team for equating analyses, publishing a comprehensive technical report on reliability, validity, and equating outcomes, and seeking guidance from a panel of psychometricians, educational measurement specialists, statisticians, and subject matter experts.
Entrance exams play a pivotal role in shaping educational opportunities for numerous talented individuals annually. Therefore, decisions of such significance should be substantiated by scientific evidence to ensure that assessments are objective and fair for all applicants. Through meticulous psychometric planning and transparent reporting, Dhaka University can showcase that inclusivity and fairness are not conflicting ideals but rather complementary goals.
Dr Md. Kamal Uddin, a Professor in the Department of Psychology at Dhaka University, President of the Bangladesh Psychometric Association, and Secretary-General of the Bangladesh Psychological Association, shared these views in the article.
