On Tuesday I went to part of a Research Forum hosted by the Intellect research group, which is part of CREET. I heard a really interesting talk by Joseph Hopkins of Universitat Oberta de Catalunya. Joe is visiting the OU until the end of July.
Joe’s talk was titled:
A method for analysing online interaction in synchronous audio-graphic conferencing for language learning
The talk was about a project where groups of students from the Open University of Catalunya used the OU’s Flashmeeting audio- and video-conferencing software for collaborative language learning activities. This is work-in progress for Joe’s PhD research (which Regine Hampel, from the OU’s Department of Languages, is supervising).
I was particularly interested in this project because we (Frances, Helen, Judith, Hazel and I) have started some research with T175 students and tutors (and also Judith’s tutor group from T209) using Elluminate.
Anyway, back to Joe’s talk. Here is the abstract:
This paper will focus on the development of a methodological approach aimed at analyzing learner interaction in synchronous audio-graphic (SAC) environments. The raw data consisted of digital video recordings of small groups of learners engaged in collaborative speaking tasks on FlashMeeting, a SAC tool developed by the Knowledge Media Institute (KMi) of the Open University. An approach focusing on “critical incidents” (Tripp, 1993) was utilized to identify instances of interaction viewed as beneficial to second language acquisition, such as negotiation of meaning (Long, 1983; Varonis & Gass, 1985), negative feedback (Long, 1996), and scaffolding from more competent peers. In order to ensure a degree of objectivity, each recording was viewed by two observers, who noted critical incidents in the learners’ spoken conversations following an established protocol. Subsequently, using the observation data, the digital video recordings were coded using Atlas.ti, a qualitative data analysis software package. The procedure used for observer training and standardization will be presented, along with the method utilized to account for other forms of verbal and non-verbal communication (e.g., text chat, vote feature, emoticons).
Below is a summary based on the notes I made during the talk. I imagine the slides will be made available soon.
The research was carried out with 146 distance learners (93 females and 53 males) at UOC, who were studying English as a foreign language. This was a compulsory course for students across many different subject areas. The age range of the students was typical of UOC, with an average age of 37.
The software used was Flashmeeting, which includes voice communication, video, text chat, voting, emoticons etc. Turntaking is managed via a button which allows an individual to ‘broadcast’ or ‘stop broadcasting’. Other participants can virtually ‘raise their hands’ if they wish to speak. The software then gives each of them the floor in the order in which they raised their hands (as I understand it). For users who have webcams, the person speaking is shown in a largish video window with a reasonable refresh rate, while other participants are shown in smaller windows with low refresh rates (e.g. once every 30 seconds).
For the research (and as part of his teaching) Joe developed ‘tutorless’ speaking activities for groups of students to carry out. Students self-formed into groups of four (by communication via the VLE). The groups were offered a practice session with a tutor, but only 8 of the 40 groups opted for this.
The speaking activities were an assessed part of the course, and were linked to what students were studying at the time. One was about managing time: students groups were asked to come up with an ordered list of advice points about time management. The other was about survival: students came up with an ordered list of items which would be important for survival if they were stranded on a remote island.
The students were given a script to guide them through the activity, and were asked to appoint a moderator (to facilitate the online session) and a spokesperson (to be responsible for creating an account of the session afterwards). The sessions were all recorded using a facility that Flashmeeting provides. Tutors reviwed the recordings afterwards and gave marks and feedback to students.
The research aimed to investigate:
- What were students’ perceptions? Were they satisfied with the experience? Was it useful?
- Was there evidence of learning taking place? Could this be deduced from the recordings of the sessions?
Sources of data were:
- An online survey to students
- Semi-structured interviews with students and tutors
- Metadata provided by the Flashmeeting software
- Recordings (from the Flashmeeting software) of the sessions
In the survey, students were asked:
(1) whether they had found the sessions useful
(2) whether they had learned new aspects of English
For question (1) 65% of respondents totally agreed that they did, and 28% agreed (not totally!). For question (2) 28% of students totally agreed and 48% agreed.
The following procedures were used for analysing the observation data:
Tutors viewed the recordings for their groups, and made notes on e.g. the roles students played, and the use of chat, voting etc. Tutors decided on grades for students, together with an ordering of group members’ relative performance.
The tutors, and Joe as primary researcher, viewed the recordings a second time, but now looking for ‘critical incidents’ (Tripp, 1993) which triggered the negotiation of learning. These were categorised using predetermined codes, for example to indicate:
- Checking for confirmation of understanding;
- Inappropriate responses;
- Statements of non-understanding.
The tutors were trained for carrying out this coding exercise, and there were practice sessions. The recording for each group session was coded independently by the students’ tutor and by Joe. This was followed by a negotiation stage. After three such rounds, the coding agreement between Joe and the tutors was about 85%. The observation data was then loaded into the Atlas-ti qualitative data analysis package for further analysis. This is ongoing, but Joe reported some initial findings.
There were frequent problems for students with the audio quality. These technical problems accounted for many of the difficulties students had in understanding each other. There were several examples of negotiation of meaning – although this did not happen all that often. Students used text chat for confirmation of spelling etc.
There were particular difficulties because of the software’s arrangements for turntaking – whereby one speaker at a time has control over audio transmission. This means that other participants cannot intervene to give help, or to briefly comment, because they do not have the facility to speak until the current speaker finishes, and their turn (in the sequence of raised hands) arrives.
This results in various undesirable phenomena, such as serial monologues, overlapping exchanges, phantom adjacencies, topic abandonment, questions/requests which are ignored. These phenomena are remarkably similar to those which can occur with text chat, or even in discussion forums. They arise because the turntaking/adjacency conventions are not like those in face-to-face speech.
Joe described the interactions as perhaps quasi-synchronous rather than synchronous. He commented that the design features of the software had effects on the interactions between students. A further example of this was the very limited use of the text chat facility, probably because it is hidden under a tab, so isn’t easily visible to students.
The talk was followed by a short period for questions and discussion. Joe had already raised aspects, other than the software itself, that might affect students’ interactions – for example, the task they were asked to carry out. Other aspects raised by audience members included:
- Would there be changes over time, as students gained experience?
- Were there examples of breakdown and repair of communication?
- Do participants who do not take a very active role also learn a lot?
- Is the video interaction important to students?
- Would it be helpful for students to watch the recording afterwards?
- Would software such as Skype make the interactions more fluid?