Hot on the heals of my last post, reporting on work which did not provide support for the previous finding that men and women perform differentially on different types of assessed tasks, I bring you a very interesting finding from work done at the University of Hull (with Ross Galloway from the University of Edinburgh, and me). David Sands from Hull and two of his students came to the OU on Thursday and gave a presentation which is to be repeated at the GIREP-EPEC conference in Poland next week.
We are seeking to investigate whether findings from use of the well-established force concept inventory (FCI) (Hestenes et al, 1992), are replicated when the questions are asked as free text rather than multiple choice questions. Free text versions of the questions have been trialed at Hull and Edinburgh, and the next step is to attempt to write automatically marked versions of these.
However, the interesting finding for now is that whilst in general students perform in a similar way on the free text and multiple choice version of the FCI, there are some variations in the detail. In particular, whilst men outperform women in the MCQ version of the FCI (Bates etc al., 2013) it seems that the gender difference may be reduced or even reversed with the free text version. We don’t have enough responses yet to be sure, but watch this space!
Bates, S., Donnelly, R., MacPhee, C., Sands, D., Birch, M., & Walet, N. R. (2013). Gender differences in conceptual understanding of Newtonian mechanics: a UK cross-institution comparison. European Journal of Physics, 34(2), 421-434
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141-158.
The students in my Biology 12 class discovered this for themselves while discussing tests. However, they had believed it to be due to the individual learning types and abilities represented by the various members of the class. However, thinking back, I can now see that-for the most part-it was a gender-split. This is a very challenging set of information, and I would like to pose a challenge of my own, to standardized test-writers. The bulk of most standardized tests is composed of solely multiple choice questions. Could this be having an effect-ever so slight-on performance? Perhaps the system has been set up-however unintentionally-to prefer the male mind, which would then lead on to a higher percent of scientists, doctors, and the like entering into career paths that require higher initial grades.
Thanks for your insightful comment Sarah. I agree that this represents a danger in the use of multiple choice questions for standardized tests. Some of the literature surrounding gender responses to different question types is quite old, but yet there seems to be so much work still to do!