FAST: AEQ

Assessment Experience Questionnaire (AEQ)

The structure of the AEQ | Recording the AEQ outcomes | Interpreting the AEQ outcomes

Download a pdf version (47kb) of this page.

The theoretical foundation of the FAST project is that there are 11 conditions under which assessment best supports student learning. The principal evaluation tool of the project, the Assessment Experience Questionnaire (AEQ) was initially developed as a prototype (Gibbs and Simpson 2003), based on the 11 conditions, to provide quick and easy evidence from students about the extent to which students’ experiences of assessment meet the conditions under which assessment best supports their learning. Discovering why is likely to require more detailed follow-up using other analytical tools.The AEQ can be used in the first-stage evaluation of a single module or to compare a suite of modules and, as with all the FAST tools, it can be adapted to suit your own needs and the type of assessment that is to be evaluated.

1. The structure of the AEQ

The AEQ contains six sections, each of which relates to particular framework conditions as shown in Table 1.

Table 1 The FAST Conditions evaluated by the different sections of the AEQ

Section 1	Amount/Quantity and distribution of student effort
Conditions	Assessed tasks capture sufficient student time and effort These tasks distribute student effort evenly across time and topics
Section 2	Quality and level of student effort/assignments and learning
Conditions	These tasks engage students in productive learning activity Assessment communicates clear and high expectations to students
Section 3	Quantity and timing of feedback
Conditions	Feedback is sufficient, frequent and adequately detailed Feedback is provided quickly enough to be useful
Section 4	Quality of feedback
Conditions	Feedback focuses on learning rather than marks Feedback is linked to the purpose of the assignment and to criteria Feedback is understandable to students
Section 5	Student response to feedback/What you do with the feedback
Conditions	Feedback is received by students and attended to (i.e. they read it) Feedback is acted upon by students to improve their work or their learning
Section 6	The examination and learning (relevant only if a module contains an exam)
Conditions	The tasks (i.e. revision process and exam) engage students in productive learning activity Assessment communicates clear and high expectations to students

Each section of the AEQ contains six statements and students are invited to express the extent of their agreement or disagreement with each statement using a 5-point scale in which the third point represents a neutral position, i.e. the student neither agrees nor disagrees with the statement. There is also an open-ended section at the end which allows students to comment on other ways, not captured by the AEQ, in which assessment may have affected their learning.

2. Recording the AEQ outcomes

The outcomes of the AEQ have been recorded in different ways by different practitioners during the development of the project: by calculating the mean response value or by percentages of responses. Use the scoring sheet and the notes below to help you interpret results.

Calculating the mean response value

In the original version of the AEQ the individual responses to each question were assigned a numerical value on the scale of 1-5, the most desirable response being allocated 5, the least desirable 1 and the neutral position 3. Thus a response strongly agreeing with the statement ‘Tackling the assignments really makes me think’ (Condition 3, above) would be assigned a value of 5 whereas strongly agreeing with the statement ‘The assignments are not very challenging’ (also Condition 3) would be assigned a value of 1. A mean (average) value for all the responses can be calculated, which is a measure of student satisfaction with respect to a particular Condition see, e.g. Tables 2a and 2b, each demonstrating 20 student responses:

Table 2a: Responses to the statement ‘Doing the exam brought things together for me’

Response (value)	Strongly disagree (1)	Disagree (2)	? (3)	Agree (4)	Strongly agree (5)	Mean response
No. students responding	2	3	6	7	2	3.2

Table 2b: Responses to the statement ‘Preparing for the exam was mainly a matter of memorising’

Response (value)	Strongly disagree (5)	Disagree (4)	? (3)	Agree (2)	Strongly agree (1)	Mean response
No. students responding	3	6	2	6	3	3.0

The higher the mean response value in each case, the more closely Condition 3 is assumed to be met. The mean responses for the 6 statements in each section can be added to give a value out of 30 which is a ‘broad-brush’ measure of the extent to which the pair of Conditions evaluated in each section are being met.

A variant in this approach involves the same values being assigned to the same type of response, regardless of the desirability of the response, e.g. ‘strongly agree’ is always valued as 5. In this instance a low average response value to the statement in Table 2b might indicate satisfaction with respect to Condition 3 (reverse scoring). Note that you cannot add the individual scores to give a section score in this instance.

Percentages of responses

The similarity in the mean response values for the two statements shown in Table 2 masks marked differences between the two sets of student responses. In Table 2a nearly twice as many students agreed with the statement as disagreed and the average response value was strongly influenced by the number of students ‘sitting on the fence’. In Table 2b the student responses were polarised, with as many agreeing as disagreeing with the statement. Expressing the outcomes in terms of percentages, as well (Table 3), not only aids the diagnoses of weaknesses, it also makes comparisons easier between modules with differently-sized student populations. However, when student populations are small a difference in opinion involving one student can lead to a marked difference in the percentage recorded.

Table 3: Proportion of students responding to two AEQ statements, expressed as percentages

	Strongly disagree	Disagree	?	Agree	Strongly agree	Mean response
Doing the exam brought things together for me	10%	15%	30%	35%	10%	3.2
Preparing for the exam was mainly a matter of memorising	15%	30%	10%	30%	15%	3.0

3. Interpreting the AEQ outcomes

It is up to you to decide from students’ responses, within the context of your investigation, where you feel there is cause for concern. The AEQ responses are only a rough measure of students’ satisfaction with the assessment experience and provide pointers to areas that may require further follow-up, not precise diagnoses of weaknesses. Students may respond within the context of the assignment or module being evaluated but experience has shown that some students may respond more generally in terms of their experience across the range of modules they are studying concurrently, or that they studied in a previous semester/year if the AEQ is administered at an early stage of a module. Furthermore, external factors such as time-constraints (e.g. domestic responsibilities, part-time jobs etc.) may influence students’ study patterns (see AEQ Section 1 in Table 1).

Because of these factors we also recommend caution when re-using the AEQ to evaluate the impact of changes introduced following initial use of the tool, especially if the student population that experiencing the changes is different from the population assessed initially. The follow-up tools not only offer more precise diagnoses of areas of weakness but also yield responses that are either more objective or can be qualified by students and so are more reliable when measuring the impact of changes.

The structure of the AEQ | Recording the AEQ outcomes | Interpreting the AEQ outcomes