On this page
The development of fOCUS arose from a recognition of the difficulties of providing behavioural science students with adequate training in the observation and analysis of behaviour, and direct experience of experimental research techniques. These opportunities are difficult (in some respects impossible) to deliver using traditional media and are particularly problematic in a distance teaching context, yet are an important part of the curriculum in the discipline.
Analogue video and audio tape cassettes, while being a valuable aid to training in this area, have major disadvantages for close and detailed analysis of behaviour. It is frustratingly time-consuming and inefficient to attempt to accurately identify, locate and replay target segments on a tape cassette. It is not possible to then mark-up and annotate such segments. The only equipment that allows these to be done is prohibitively expensive for use by students, and technically difficult to use. These activities are vital components of systematic observation and analysis methodology. In fOCUS, we have developed a way of overcoming these limitations, using digital A/V files within an easy-to-use multimedia environment for carrying out all these forms of analysis.
fOCUS capitalises on the new possibilities afforded by digital media and was developed at the OU headquarters in Milton Keynes, England. The development team included OU psychology academics and an INSTILL lecturer specialising in interactive media, a project officer, a BBC television producer, a graphic designer and programmers contracted to the Interactive Media Group of the BBC/OU Production Centre.
This work was funded by the OU Office for Technology Development and the media production and programming by the OU School of Education Media Group.
Building on the success of early versions of fOCUS, the Observation Skills in Psychology project was funded by the Higher Education Funding Council for England (HEFCE) and the Department for Employment and Learning (DEL) under the Fund for the Development of Teaching and Learning (FDTL) from January 2002 to July 2007 to improve the quality and cost-effectiveness of training in systematic observation and assessment skills in undergraduate psychology and postgraduate clinical psychology courses.
The project was particularly timely because although the skills of systematic observation and assessment of behaviour are central components of both undergraduate psychology and postgraduate clinical psychology courses, they have become increasingly difficult to teach effectively using traditional approaches given the increase in student/trainee numbers.
The projects’ outputs (see the Teaching and Learning area of this web site) which have been widely evaluated, offer a solution praised by both students and academics.
During the final ‘transferability’ phase of the project (January 2006 - July 2007) the project concentrated on further developing the outputs to support other disciplines and uses.
"Many fields of study involve extracting information from dynamic, real-world events," says Project Director, John Oates. "Fields as varied as vulcanology, musicology, animal behaviour and theatre studies can usefully employ video for teaching and research. The accurate and systematic analysis of video material is a key process. By bringing events into the learners’ immediate environments, direct contact with otherwise inaccessible experiences is made possible."
Publication of an article entitled "Teaching of Research Methods in undergraduate psychology courses: a survey of provision in HE institutions and colleges in the UK" in the journal ‘Psychology Learning and Teaching’.
Launch of the ‘Attachment’ and ‘Child Development Assets 1’ CD-ROMs.
A further grant of £25,000 awarded through the Fund for the Development of Teaching and Learning for transfer work.
Paper presented to the ‘Measuring Behaviour’ conference, Wageningen, The Netherlands in September 2005.
Production, piloting and refinement of an ‘Observation Methods’ CD-Rom containing two learning units ("Coding" and "Reliability and Validity" aimed primarily at undergraduate psychology courses continued. Evaluation partners during this year included Aston University, Liverpool John Moores University, Northumbria University, Portsmouth University, Sunderland University, and The University of the West of England, Bristol.
Two clinical learning units (‘Clinical Note-taking’ and ‘Opening and Closing Sessions’) comprising CD-Roms and unit guides and two DVDs to support postgraduate clinical psychology training were completed. These materials developed for clinical courses departed markedly from those outlined in the project plan and reflect the application of observation skills in practical work with patients. They were the subject to extensive consultation during their development both within and beyond the project consortium. Pilot partners included Birmingham University, Leeds University, Leicester University, Newcastle University and Sheffield University.
These changes were viewed very positively by the project team and the sector and considerably increased the potential value of the project outcomes both in terms of the short and the long term applications of the developed materials.
In addition feedback was obtained from attendees at the following presentations/events:
| 24th - 25th February | FDTL Phases 4 and 5 Conference, Manchester |
| 31st March and 2nd April | BPS quinquennial conference: Learning objects: a new approach to teaching observation skills |
| 6th April | Cascade event: University of Central Lancashire |
| 11th April | Observation skills: Accelerating the curve from learning to doing. Bristol University. |
| 13th April | Cascade event: Liverpool Hope University |
| 15th April | Group annual Conference 2005). University of Leicester. "Videoed Supervision: The Camera Doesn’t Lie" |
| 23rd May | Learning and Teaching Technologies Committee. Open University |
| 23rd June | ICT Awareness Day. Faculty of Health & Social Care, The Open University |
| 30th Aug and 2nd Sept | Measuring Behaviour 2005 conference. ‘fOCUS: a tool for teaching observational skills to undergraduate psychology and postgraduate clinical psychology students.’ Netherlands |
| 23rd Sept | Open University Business School. National conference. "Sharing Opening Learning" |
| 7th - 9th Nov | Clinical: group of trainer’s conference. Manchester |
The ‘Methods’ CD-Rom was piloted at King Alfred’s College Winchester, Leeds Metropolitan University, Oxford Brookes University, Oxford University and West Suffolk College. A total of 183 students were involved. Following feedback from these institutions, the content was modified and a new version ('Observation Methods’) produced.
Work on the clinical materials continued throughout the year.
The evaluation strategy aimed to gather both qualitative and quantitative data from staff and students to help determine whether the materials were ‘fit for purpose’ and to assess impact on student learning.
Information from staff was gathered using an evaluation template followed by a meeting (on 25th May 2004 at the British Psychological Society Offices in London) to discuss the points raised. Feedback from teachers of psychology was gathered via a poster presentation and demonstration of fOCUS II and the Phase 1 materials at the PLAT2004 conference at the University of Strathclyde, Glasgow, in April 2004.
An evaluation report was presented to a project team meeting held on 12th July 2004 at the Open University and areas identified for development were discussed and agreed. Overall, feedback suggested that the materials were suitable and fit for purpose.
The most significant changes related to the Familiarisation and Reflective Exercise and Evaluation units:
It was further agreed to provide more support for tutors including:
In addition, throughout the year, the fOCUS II software was improved to accommodate feedback from partners regarding a variety of technical and presentational issues. (e.g. the text background colour was altered from blue to white; a facility allowing users to choose the location to which their data will be saved was added).
Feedback from the Quality assurance checking by the OU Centre for Educational Software showed no problematic technical flaws. It identified a small number of minor improvements, primarily to the textual design in help and text files to improve readability. Working with professional editors developing the hypertext design style for maximum on-screen readability of text files, improvements were implemented in the materials distributed for phase 3.
Questionnaire surveys
Questionnaire surveys of all clinical and undergraduate psychology departments in England.
BPS conference:
Feedback was obtained on the then current version of fOCUS through a demonstration at the Learning and Teaching Subject Network launch of the FDTL phase 4 psychology projects at the BPS conference (March 2003).
Leeds Metropolitan University:
During the Spring semester, 65 students and 4 staff were involved in evaluating fOCUS (CDOR Child Development Observation Resource) in a face to face teaching environment at Leeds Metropolitan University. This was the first time fOCUS had been used with larger numbers of students. Qualitative and quantitative data were collected from questionnaires, discussion and a coursework assessment; a report of the findings was presented to the project team at a consortium meeting on 6 th June and provided some key insights which guided the development of fOCUS II . For example:
Findings from the surveys and feedback from the conference and pilot study were of great value in directing content for the materials produced by the project and in identifying elements of the fOCUS software which needed changing to meet the demands of wider usage. These software changes led to a new generation of the software known as fOCUS II.
Changes included:
In response to the findings of surveys of sector needs development of the clinical and undergraduate materials followed two rather different paths. The undergraduate materials were developed along the lines of those envisaged for both sectors at the outset.
Prior to the project, fOCUS had already undergone extensive systematic formative feedback evaluations to direct the programming and design of the application. At various stages, different categories of potential users and testers with differing levels of prior knowledge and expertise have been used. On a regular basis, copies of working versions have been distributed widely within the Open University and elsewhere for more informal feedback.
The types of evaluations completed and feedback opportunities (including conference and seminar presentations) are detailed below.
End-user testing in OU Courses:
A version of fOCUS was produced in collaboration with the BBC for the new Open University introductory psychology course DSE212 Exploring Psychology. This was used by more than 2,500 students and tutors in the first presentation of the course in 2002, and was well-received. It is now used annually in the course, by more than 3,500 students and tutors. In this large-scale use, the software has proven itself as robust and user-friendly, offering a valued learning resource.
Other versions were produced in collaboration with the OU Learning and Teaching Solutions unit for use in the Masters course ED840 Child Development in Families, Schools and Society and the third-level undergraduate course EK310 Research with Children and Young People.
European Academic Software Award:
The report submitted for this award can be accessed following the link European Academic Software Award 2000.
Open University Institute of Educational Technology and the Open University Knowledge Media Institute, ‘work in progress’ seminar:
The audience of multi-media developers provided feedback on the conceptual design of the application.
Paper presented at the SURF conference, Rotterdam, Netherlands:
Oates, J. (2000) ‘fOCUS: a CD-ROM based application for developing systematic observation skills’.
Higher Education Funding Council’s Quality Assurance Agency External Subject Review:
fOCUS was included in the British Higher Education Funding Council’s Quality Assurance Agency External Subject Review of psychology teaching in the Open University, conducted during 1999, which resulted in an overall rating of ‘excellent’ for the quality of provision. The review report commented that they were "impressed by the commitment of staff to introducing IT in a manner that is sensitive to the student profile".
Computer Assisted Learning Research Group CALRG99 Conference:
Educational media specialists, offered feedback on the pedagogic design of the application.
8th European Conference for Research on Learning and Instruction (EARLI) in Göteborg, Sweden:
International experts in both developmental psychology and instructional design gave detailed comments and feedback on issues of functionality and pedagogic design.
Paper presented at the Cip98 Computers in Psychology conference, York, England :
Gove, J., Oates, J. & Littleton, K. (1998) ‘The Child Observation Resource: a tool for learning by doing’.
A structured evaluation:
Involving 40 students from a range of socio-economic, geographical and educational backgrounds taking the Open University ED209 Child Development course was completed. The students were sent a copy of the software on CD and an evaluation questionnaire. The evaluation questionnaire comprised two instruments. The first was a carefully structured set of activities which prompted the student to use each of the key features of fOCUS in turn, and to write comments about their experience of using each of these features. The second evaluation instrument was the 'User interface rating tool' (Reeves & Harmon, 1993; see http://mime1.marc.gatech.edu/MM_Tools).
Between 1997 and 1979, CDOR-1 (functional version) was evaluated:
This involved implementing the prototyping demonstrator as a fully functional application, written in MacroMedia Director™. Three forms of evaluation were conducted during this stage:
i. 1997 Ed-Media Conference at the University of Calgary, Canada. Extensive formative comments were gathered following presentation of a paper and demonstration of CDOR-1. ( Hall, J. L. & Oates, J. (1997). ‘Proficiency and Practice with Observation: a child development theory and methods tool’)
ii. A proforma-based evaluation was carried out, with the evaluators being asked to carry out a series of defined tasks and to report on the user-interface and functionality on a series of rating scales and open-ended questions. The ‘User Interface Rating Tool’, Reeves & Harmon, 1993 (see http://mime1.marc.gatech.edu/MM_Tools) was used.
iii. A panel of naïve users, with no experience of observational methods, completed a series of evaluation tasks in a similar format to those used in ii above.
Feedback from these three sources was summarised as a series of design and development recommendations for each aspect of the fOCUS functionality.
Seminar day:
This period involved writing a demonstrator version of fOCUS, in Apple MediaTool™ to illustrate the main areas of functionality. This was used as the basis for a seminar day in which members of academic staff in the Faculty of Education and Language Studies at the Open University, along with an external consultant, ‘brainstormed’ the possible modifications and extensions of the proposed product. A rich set of constructive ideas for development was generated.
In presenting extracts from detailed qualitative data, it is difficult to do justice to the information and tempting to make a simple selection of positive excerpts. In view of this the first sentence of each respondent’s ‘general comments’ is presented. These give a fair and balanced picture of the range of views expressed by the students:
"Overall I think the CD is an excellent observation resource medium, even someone with only the basics of computing could use it."
"Most of programme is easy to use and functions well."
"With the use of digital video cameras one can make own recordings and add to the collection."
"I think the programme is on the whole easy to use."
"I enjoyed working through the fOCUS package and would like to think that future courses could be supplemented by packages such as this."
"The programme generally is good and has great potential."
"Very easy, fun even."
"Whilst novice computer users may find the interface simple to use (though I doubt it), it was a constant source of frustration to me as a windows user."
"Very instinctive and user friendly although a ‘print’ facility for help would be desirable."
"I was baffled by the two coding functions."
"Useful practice, especially for carrying out observations of your own."
"I think this is an excellent resource and will be valuable for many."
Following this evaluation, the ratings data have been statistically analysed and the qualitative data content analysed. The recommendations produced have informed the further development of fOCUS.
Independent academic and technical reviews have been received. Responses have been consistently favourable. Among the positive comments have been the following:
"An interesting and powerful tool."
"I found the system friendly and easy to use."
"The quality of videos supplied is really good and they are perfect illustrations of the different behaviours."
"This is cutting edge technology. The video marking tool represents state of the art usage of media to enhance learning."
"I loved playing with the scoring system. It is easy and pleasant to use…a wonderful way to teach observation."
"I’ve been impressed by the presentation and design."
"A very exciting project."
