video record
Media not available in the Digital Archive
This programme comprises a discussion on educational tests and marking systems. The problems of error inherent in marking systems and the work of the NFER attainment tests in this regard feature la...rgely in the programme.
Metadata describing this Open University video programme
Module code and title: E341, Methods of educational enquiry: an empirical approach
Item code: E341; 03
First transmission date: 14-04-1973
Published: 1973
Rights Statement:
Restrictions on use:
Duration: 00:23:42
+ Show more...
Producer: Vic Lockwood
Contributors: Desmond Nuttall; Philip Williams; Alan Willmott
Publisher: BBC Open University
Keyword(s): Educational tests; Error distribution; Essay marking; Marking systems; NFER attainment tests; Reliability; Results variations; Standard deviation
Footage description: A shot of two sequences depicting two forms of measurements a class of pupils about to sit an examination, and a scientist using a balance to weigh chemicals. Prof. Phillip Williams then introduces another sequence in which the weight of a cannon ball is measured repeatedly and recorded. The variation in the weight recorded, even though the same scale and object are used on each occasion, is used to reflect the sort of error that can occur in results. Prof. Williams then elaborates on what the programme will contain. He then introduces another sequence which reflects variation in its results at different times. Shot of a girl doing an arithmetic test for the second time. After scoring a different mark this time Prof. Williams then points out reasons why this state of affairs does occur. These can be one of a number of reasons, pointing out that the marker can be a source of error. This point is further emphasised when two teachers assess and award marks for a fourteen year old boy's essay. Shot of two teachers, one of them reading the boy's essay entitled 'What I would like to be when I grow up'. The other teacher assesses the work and awards a higher mark for it than the teacher who read it. Points of merit and demerit in the essay evolve from a discussion that ensues. Prof. Williams then talks about marker differences using the last sequence as an example. He then lists the four main sources of error before a demonstration is given about how to estimate an error. An experiment in which twenty five markers are involved is used to estimate the amount of error present in a result. The marks (as given by the twenty five markers) are represented on a graph and the mean score taken as a guide to the child's performance. The scores of twenty five markers is represented on a graph and compared to that in which two thousand markers took part. Alan Willmott expresses the difficulty involved in determining the extent to which error affects the scores a child might achieve on a test and then explains how this is overcome by estimating the size of the error distribution from known characteristics of the test. Prof. Williams sums up the last sequence, pointing out that the task of determining the spread of error scores present in a test administered as a problem. He cites situations when the results of such tests please those undertaking it and gives the reasons for this. Dr. David Nuttall refers to an experiment of reading test and uses a range of scores given. He points out that the spread of error does not give all the information to evaluate the test. He uses a graph which shows the spread of error scores and total scores of the test range to show that the importance of a fixed amount of error increases as the spread of total test scores decreases. Prof. Williams reiterates that the important is the relation between the spread of error score to the spread of scores on a test. He then introduces Alan Willmott to examine the ratio factor. Alan Willmott explains what the ratio represents He also points out how a scale of reliability can be obtained by use of the ratio. He uses a scale to show reliable and unreliable tests and by the use of various graphs shows how the reliability of tests and the spread of error are closely related. Prof. Williams poses the question on the significance of the reliability of a test when comparison between individuals is being carried out. Dr. Desmond Nuttall represents scores of two examinees on a graph. The error of the two scores is not represented. By attempting to ascertain the real difference, if any, between the two candidates, Dr. Nuttall makes two guesses at the reliability of the test. The figures are represented on the graph and there is an overlap between the scores (using the first figure) thereby making it difficult to assess the difference between the two candidates. The other assumed figure for reliability of test when represented on the graph does not result in an overlapping of the spread of error, In this instance the real difference between the two can be assessed. Dr. Nuttall then gives the reason for preparing tests with small error bands Prof. Williams talks about what we need to know when measuring the reading attainment of a child aged 10, and how this information can be used to estimate the band of error scores associated with the child's performance on a given test occasion. Prof. Williams then talks about the standard deviation of error curve, and how it can be calculated for each test. He then refers to the NFER attainment tests and explains the various graphs to ascertain the candidate's true score. He gives the conditions which make the calculation of the true score possible. Prof. Williams then points to the dangers that are inherent in error bands when used to measure a Child's true performance. He supports its use, however, but with caution. A matter of sensitivity on the part of the research worker in being aware of this error will account for appropriate inferences to be made from the research, Prof. William added.
Master spool number: 6HT/70753
Production number: 00525_6011
Videofinder number: 3895
Available to public: no