Tuesday, March 17, 2009

Qualitative Study Critique (by Angela)

1.Title: How English as a Second Language Graduate Students Perceive Online Learning from Perspectives of Second Language Acquisition and Culturally Responsive Pedagogy

2.Summary: Employing an interview method, this qualitative study is to explore the effects of online learning on ESL graduate students’ English language improvement and how cultural diversity has impacted such students’ online learning attitudes. Three researchers conducted individual in-depth face-to-face interview on 7 international ESL graduate students (from native countries of China, Korea, Japan, the Philippines, Thailand, and Russia) who have taken online courses through a major Western university. By asking 8 open-ended questions with follow-up questions, the researchers tried to ascertain the following perceptions:
  • participants’ likes and dislikes about their online learning experiences
  • the affects (positive and/or negative affects of online learning on English acquisition
  • the affects of online learning on individual learning styles
  • the affect of online learning on individual attitude, motivation, and anxiety toward learning
  • how cultural differences affect online learning in comparison to face-to-face class experiences

Data analysis shows the following results and discoveries:
  • Online learning environment helps improve participants’ writing and reading skills, providing with more opportunity in vocabulary-building
  • Online learning environment shows no sign for participants to improve the speaking and listening skills, due to lack of audio and visual materials.
  • Online learning environment causes participants confusion due to vernacular and acronym used by native English speakers
  • Participants viewed language and culture as important issues in online learning environment. This makes them choose not to take more than one course at one time.
  • The attitude toward online learning tended to be more positive with the increased language proficiency, number of online courses taken, and more time spent in the United States.
  • Participants perceived challenges regarding culturally related difficulty with time management, lack of technological trust and/or experience, content and nature of online conversations.

3.Critique: This study shows relevance to my research interests. However, in general, this study fails to reach several criteria for a good qualitative study. Some of them hinder the readability, while others may affect its internal and external validity.

Readability Issues:
  • lack of themes – In the result section, the researchers merely described the whole findings in the first paragraph, without any themes developed. It seems to me that the researchers explained specific findings in detail in the following several paragraphs. This makes reader difficult to grab the general concept of the study.
  • lack of visualization representativeness – Since this study is lack of theme and there were totally 7 interviewees, I do think there is necessity for presenting some data visually (e.g., tables for participants’ responses toward each questions to not only demonstrate overall perspectives but also highlight outliers )

Validity Issues:
  • lack of triangulation – The researchers did not mention much in methodology section concerning the data sources. It seems that the interviewees’ responses are the only source, which affects its internal validity.
  • how to analyze / code data is unknown – The researchers did not address how to code and analyze the data (interview transcript) in any section of this study. From what was presented in the result section, the researchers seemed only directly quote relevant responses. Nor did the researchers mention if they had members check the transcript.
  • lack of tacit knowledge – The tacit knowledge is invisible in this study. For one thing, the researchers did not describe how they dealt with the data in detail in methodology section. On the other hand, even though there are quotes from interviewees, the researchers did not mention the specific tacit knowledge like those unarticulated behavior or facial expressions.
  • lack of interpretation from researchers – Throughout the study, I can hardly perceive researchers’ voice, but the interviewees’.
  • participants representativeness – Notice that most of the participants are Asian people. This makes me doubt the participants representativeness. However, I think of some plausible explanation for this: maybe this is due to researchers’ consideration that most ESL learners are Asian people. Still, this calls for more detail.

On the other hand, this study is doing well in terms of: 1) employing lots of quotes from interviewees, which precisely demonstrated interviewees’ perspective; 2) the process for data collection and analysis is simple and iterative. The latter one is a little ambiguous, though, since the researchers did not explicitly address this. However, it might be reasonable to suppose that the researchers employed similar procedures. For one thing, this will be more convenient, economical, and time-saving. Secondly, if there were difference, the researchers would have explicitly described that.

With regards to the writing style, this study is highly descriptive, rather than narrative. I was wondering, though, if this is one of the reasons that the researchers’ voice is lost. Nevertheless, in this way the researchers seem to maintain their neutrality in this study.

1 comment:

  1. Everytime I read another blog, I learn about another way to communicate effectively using technology - thanks for the labels, it is a great idea! Your summary is great....