What is multimodality? Language has traditionally been the focus for the development of young children’s literacy, but a multimodal perspective broadens that view and regards learning as involving more than words. Multimodal approaches to communication and meaning-making take into account the whole range of ‘modes’ that young children encounter in a variety of texts (such as words, images and sounds in printed and electronic media and in face-to-face interaction) and the range and combinations of modes they use to make and express meaning (such as gesture, gaze, facial expression, movement, image, music, sound effects and language).
New insights: Multimodality offers a fundamentally different perspective on communication in that it does not assume that language always plays a central role in interaction. This change in perspective can offer radically new insights into understandings of communicative and learning processes – particularly in the current era where digital technologies have fundamentally changed the kinds of representational tools and modes that are used in knowledge construction and exchange.
Multimodality and social semiotics: The approach to multimodality used in this study is rooted in Michael Halliday’s (1978) theories of Social Semiosis. Halliday’s work drew attention to the interdependent relationship between language and social context, and to how communicative events are shaped by both social and linguistic processes. According to Halliday’s theory, the meaning of any utterance depends on an understanding of the range of utterances which are possible in a given social context. Hallidayan researchers ask themselves questions such as ‘Out of the range of possible utterances, why did the speaker/writer choose that grammatical structure and those particular words in that particular social context?’ This aspect of choice in text-making is also central to multimodality, but rather than focusing exclusively on the linguistic mode, multimodal analysis explores how diverse semiotic modes are used in the design of a semiotic product or communicative event (Kress and Van Leeuwen, 2001).
