Cultural bias on the IELTS examination - Freimuth, Hilda (Rhodes University, Faculty of Education, Centre for Higher Education Research, Teaching and Learning, 2014)
The study reported in this thesis investigated Emirati students' claims related to experiences of cultural bias of the reading component of the IELTS examination through a critical realist lens. Critical realism posits a layered reality which allows for the conceptualization of experiences as emerging from the interplay of events and mechanisms found in two other realms of reality "“ the actual and the real. Experiences, therefore, have a different ontological status than the events and the causal mechanisms to which they are attributed. Social realism was used to further explore the depth of the realm of the real through Archer's construct of analytical dualism. This allowed for the placement of generative mechanisms into three domains: structure, culture, and agency. There were two parts to this investigation: a content analysis and a focus group study. The first part of the content analysis consisted of analyzing 60 reading passages from 20 IELTS examinations for a number of cultural categories. These included such things as cultural objects, social roles, idiomatic expressions, traditions and festivals, superstitions and beliefs, and political and historical settings. The second part of the content analysis focused on the question types and syntactical structure of the 5 different IELTS examinations that the focus groups students sat. All three components of the analysis "“ the cultural content, question types, and syntactical structure "“ were conducted at the level of the actual. Findings indicated that on average, an IELTS examination contained 14 cultural references of various kinds. Only 4% of all geographical references pertained to the Middle East with the biggest share being western locations. The most common question types were matching questions, fill in the blank questions, and yes/no questions with more than 50% of all questions requiring some form of higher order thinking due to text reinterpretation. The study also found that the question types were not consistently distributed over the examinations with each consisting of a different variety of questions and some even having repetitive question types on one reading examination. The second part of the study was the focus groups. Here, 21 Emirati students sat 5 different IELTS examinations. Upon test completion, these students underwent a semistructured interview to relate their experiences of the test. These experiences, at the level of the empirical, all shared 7 ideas: reading is hard, the questions are too difficult, the passages are too long and difficult, the topics are unfamiliar, the topics are not interesting, the vocabulary is too difficult, and there is not enough time. When the processes of retroduction and abduction were applied to both the content analysis and these common experiences, numerous structures and discourses at the level of the real were identified as having contributed to the emergence of the feeling of bias at the level of the empirical. These structures included such things as the students' school system (eg. curriculum, assessment, instructors etc.), religion, literacy practices, and home. In the cultural domain, a number of discourses were found to contribute to the experiences at the level of the empirical. Amongst these were the 'Unimportance of Reading', the culture of 'Obedience', the rejection of the 'un-Islamic', and the students' sense of 'Entitlement'.
Keywords: International English Language Testing System -- Evaluation; Language and culture -- United Arab Emirates; English language -- Study and teaching -- Foreign speakers; English language -- Examinations; Critical realism