Qualitative Study Critique (by Angela)

1.Title: How English as a Second Language Graduate Students Perceive Online Learning from Perspectives of Second Language Acquisition and Culturally Responsive Pedagogy

2.Summary: Employing an interview method, this qualitative study is to explore the effects of online learning on ESL graduate students’ English language improvement and how cultural diversity has impacted such students’ online learning attitudes. Three researchers conducted individual in-depth face-to-face interview on 7 international ESL graduate students (from native countries of China, Korea, Japan, the Philippines, Thailand, and Russia) who have taken online courses through a major Western university. By asking 8 open-ended questions with follow-up questions, the researchers tried to ascertain the following perceptions:

• participants’ likes and dislikes about their online learning experiences
• the affects (positive and/or negative affects of online learning on English acquisition
• the affects of online learning on individual learning styles
• the affect of online learning on individual attitude, motivation, and anxiety toward learning
• how cultural differences affect online learning in comparison to face-to-face class experiences
  

Data analysis shows the following results and discoveries:

• Online learning environment helps improve participants’ writing and reading skills, providing with more opportunity in vocabulary-building
• Online learning environment shows no sign for participants to improve the speaking and listening skills, due to lack of audio and visual materials.
• Online learning environment causes participants confusion due to vernacular and acronym used by native English speakers
• Participants viewed language and culture as important issues in online learning environment. This makes them choose not to take more than one course at one time.
• The attitude toward online learning tended to be more positive with the increased language proficiency, number of online courses taken, and more time spent in the United States.
• Participants perceived challenges regarding culturally related difficulty with time management, lack of technological trust and/or experience, content and nature of online conversations.

3.Critique: This study shows relevance to my research interests. However, in general, this study fails to reach several criteria for a good qualitative study. Some of them hinder the readability, while others may affect its internal and external validity.

Readability Issues:

• lack of themes – In the result section, the researchers merely described the whole findings in the first paragraph, without any themes developed. It seems to me that the researchers explained specific findings in detail in the following several paragraphs. This makes reader difficult to grab the general concept of the study.
• lack of visualization representativeness – Since this study is lack of theme and there were totally 7 interviewees, I do think there is necessity for presenting some data visually (e.g., tables for participants’ responses toward each questions to not only demonstrate overall perspectives but also highlight outliers )

Validity Issues:

• lack of triangulation – The researchers did not mention much in methodology section concerning the data sources. It seems that the interviewees’ responses are the only source, which affects its internal validity.
• how to analyze / code data is unknown – The researchers did not address how to code and analyze the data (interview transcript) in any section of this study. From what was presented in the result section, the researchers seemed only directly quote relevant responses. Nor did the researchers mention if they had members check the transcript.
• lack of tacit knowledge – The tacit knowledge is invisible in this study. For one thing, the researchers did not describe how they dealt with the data in detail in methodology section. On the other hand, even though there are quotes from interviewees, the researchers did not mention the specific tacit knowledge like those unarticulated behavior or facial expressions.
• lack of interpretation from researchers – Throughout the study, I can hardly perceive researchers’ voice, but the interviewees’.
• participants representativeness – Notice that most of the participants are Asian people. This makes me doubt the participants representativeness. However, I think of some plausible explanation for this: maybe this is due to researchers’ consideration that most ESL learners are Asian people. Still, this calls for more detail.

On the other hand, this study is doing well in terms of: 1) employing lots of quotes from interviewees, which precisely demonstrated interviewees’ perspective; 2) the process for data collection and analysis is simple and iterative. The latter one is a little ambiguous, though, since the researchers did not explicitly address this. However, it might be reasonable to suppose that the researchers employed similar procedures. For one thing, this will be more convenient, economical, and time-saving. Secondly, if there were difference, the researchers would have explicitly described that.

With regards to the writing style, this study is highly descriptive, rather than narrative. I was wondering, though, if this is one of the reasons that the researchers’ voice is lost. Nevertheless, in this way the researchers seem to maintain their neutrality in this study.

Assignment 1.3: "An Investigation of Communicative Competence of ESL Students Using Electronic Discussion Boards"

Authors/Date: Shenghua Zha, Paul Kelly, MeeAeng Ko Park, and Gail Fitzgerald (2006)

Descriptors: ESL learning, electronic discussion boards, communicative competence, peer interaction

Research Design: This is an expanded case study (qualitative), with experimental research design (quantitative). The intervention included a one-week training period, followed by three communication and writing activities in an electronic discussion board. Each activity lasted two weeks consecutively. The three activities, whose guideline for design is: 1) language learning occurs through meaningful social and cultural interactions; and 2) ESL instruction should provide students with authentic tasks and audience to interact socially and negotiate meanings, are listed as follows:

1. Activity 1— Creating Clubs

2. Activity 2— Recommending a Holiday Menu

3. Activity 3— Planning a Party

Type of Research: Qualitative and Quantitative

Problem: According the authors, abundant studies have been conducted to investigate advantages of language learning using peer social interactions in CMC. Students in those studies are involved in either collaborative learning tasks (Belz & Kinginger, 2002; Chun, 1994; Lam, 2000; Singhal, 1998; Warschauer, 1996) or class/group discussions with individual assignments (Beauvois, 1992; Beauvois, 1994; Kern, 1995). However, none of these findings compared the effect of collaborative versus individual CMC learning tasks on students’ communicative competence. Thus, the problem of the research is that what the effect of collaborative versus individual CMC learning tasks on ESL students’ communicative competence is.

Purpose:

1. To examine the patterns of K–12 ESL students’ communicative competence through peer interaction in collaborative versus individual learning tasks in CMC, with particular attention to appropriate use of language for social purposes.

2. To investigate improvements in communicative competence within CMC environments.

Questions:

1. What is these ESL students’ qualitative improvement in their use of English through the three consecutive activities?

2. What are the changes of children’s communicative competence measured by the ESL Standards in the three CMC activities?

Participants: 28 elementary-aged (grade 2-5) ESL students (956 messages)

Research Site: In seven ESL classrooms in six primary schools in a suburban area in a mid-western metropolitan U.S. city

Hypothesis: No obvious hypotheses due to the fact that it is a case study. However, with reading through this research, I kind of perceive that the author implicitly had the hypothesis that “meaning was more important than the structure of language and that the primary goal of language learning should be the development of communicative skills (Krashen & Terrell, 1983)” (P.1)

Coding nodes (in qualitative analysis): With the national ESL standards for PreK-12 students, nineteen competence indicators from four areas were used. The four areas are: (P.5-7)

1. use of English to participate in social interaction

2. use of written English for personal expression and enjoyment

3. use of learning strategies to extend communicative competence

4. use of appropriate English variety, register, and genre according to audience, purpose, and settings

Variables (in quantitative analysis): frequencies of children’s use of language coded under specific indicators in the ESL Standards

Result: Changes were found in children’s use of language for social purposes and appropriate use of language in different social and cultural settings

Recommendations:

1. design of online discussion activities

2. peer assistance in language learning

Download the Article