The Difference Between Normalized Gain g and Effect Size Cohen’s d for Measuring the Improvement of Student’s Scientific Literacy

As an undergraduate from Physics Education, I began teaching of Biology at the secondary school on 22 July 2018 until 30 June 2019 when I acceded to come back at primary school, both Islamic Madrasah. Teaching at the Islamic Madrasah is a hassle because I should consider my perspective on Islam in teaching. However, teaching at the Islamic Madrasah is not and should not be considered a burden or chore that just needs to be done. It is a crucial part of moslem scholar, as we all want to do scientifically sound research and we should all strive to be effective teachers. Through teaching, we are responsible for the education of the next generation of islamic peoples, who will use their own unique ideas and skill sets to advance their society. Teaching, in general, should not be seen as a hassle in scholar, but rather as a skill to be developed and a responsibility to be taken seriously. Teaching does not have to decrease research productivity, it can greatly enhance research if we allow it to. One of my evidence about this statement is my experience and work. After a year devoted to spruce up the teaching of Biology, I produced a series of work on scientific literacy related Biology, that continues my undergraduate thesis, which was related Physics. In these works, I wrote about my experiences teaching Biology in Islamic Madrasah. Then, I became think to reconsider my method on measuring student learning. Measuring student learning is a complicated but necessary task for understanding the student’s improvement and effectiveness of instruction. I have curious about the the difference between normalized gain g and effect size Cohen’s d for measuring the improvement of student’s scientific literacy. I used normalized gain g in my undergraduate thesis nor my first work on Biology Education, then used effect size Cohen’s d on my latest work on scientific literacy in teaching of Biology. I see need reasons for using one or both of them, to be explained in any writings on educational research. So, in this work I investigate about my curiousity. My investigation focused on the implications on claims about student learning that result from choosing between one of two metrics. The metrics are normalized gain g, which is the most common method used in Physics Education Research (PER), and effect size Cohen’s d, which is broadly used in Discipline-Based Education Research (DBER) including Biology Education Research (BER). Data for the analyses came from the research about scientific literacy on Physics and Biology Education from courses at institutions across Indonesia. The results showed that the two metrics lead to different inferences about student learning. First, normalized gain g being biased in favor of populations with higher pretest means. Second, effect size Cohen’s d may mitigate the limitations of these metric for measuring the learning of high or low pretest populations of students by accounting for the distribution of tests scores. Third, by comparing the two metrics across all data, effect size Cohen’s d is larger than normalized gain g in these cases for the same size change in the means. This work reveals that the bias in normalized gaing can harm efforts to improve student’s scientific literacy by misrepresenting the efficacy of teaching practices across populations of students and across institutions. This work, also, recommends use effect size Cohen’s d for measuring student learning, based on reliability statistical method for calculating student learning. In addition, using effect size Cohen’s d would allow scholars to use their work in subsequent studies and meta-analyses, align with the practices of the larger education research community, nor facilitating more cross-disciplinary conversations and collaborations as well.