Study design. Verbalisation and subsequent protocol analysis is routinely used in design studies to investigate the thinking process during designing. Each session lasted around two hours, including instructions and warm-up activity, with one hour being dedicated to the main design task. The sessions were video and audio recorded. A facilitator was present to prompt participants to talk aloud if necessary (see picture right).
Botswana protocol study. A total of 30 participants were chosen from an initial pool of 70, according to demographic screening including tribal background and family situation, and course performance. The study was conducted in the participants’ chosen language (Setswana or Kalanga). The studies were run with pairs of participants, leading to 13 usable sessions (target of 10).
UK protocol study. Seven participants volunteered for the constrained study, leading to 7 usable sessions (target of 10). Participants in the UK took part in the study as singletons. Identifying UK participants proved to be challenging.
Analysis.We have developed a coding scheme and associated visual notation that builds on and expands Valkenburg and Dorst’s (1998) descriptive analysis method of the flow of design activity in design teams. Their work builds on Schön’s framework of reflective practice (1983). Our new approach augments the original one to clearly describe the elements of a frame using the Frame Signature Matrix analytical tool. Using this new approach to code a design protocol provides a detailed visual overview of design activity in an entire design session. This representation clearly differentiates activity in the problem space from activity that focuses in the solution space, which allows us to study design flow as well as patterns of design behaviour.
Findings. The protocol study revealed a different flow of design activity in Botswana pairs and UK designers (diagram right). Problem-solution co-evolution is observed in novice interaction designers in the UK and Botswana. In problem-solution co-evolution the problem space is explored before a solution is suggested. In the majority of Botswana protocols we could see a new type of co-evolution, which we termed solution-problem co-evolution. Solution-problem co-evolution uses ‘off the shelf’’ solutions to structure the problem space and develop a solution further. Although we can’t directly compare patterns of design behaviour in individuals in the UK and pairs in Botswana, we could observe recurrent themes and predominant patterns in each setting.
In Botswana, participants immediately suggest a solution when they are given a design problem. Participants generate adaptations of simple, practical, off the shelf designs. They design solutions they generate are not very detailed but they produce multiple outline solutions. Participants often change direction, one moment talking about one solution and the next about another solution. Botswana pairs use analogy to help set their design direction and they use design methods to represent design solutions.
In the UK, participants explore the problem space before they suggest a solution. They focus on developing a single solution in more detail. Designers show a pattern of rapid move-reflect-move when developing the detail of a solution for long periods. Participants use design methods to create the overall design direction/design concept and to devise solutions. They use analogy to help set their design direction.