Pioneering English educator and researcher Gunther Kress presents a semiotic theory of literacy to challenge the traditional linguistic theories that have dominated previous scholarship but that fail to address the multimodal realities of today.
Here I will outline some elements of such a theory of literacy; it cannot be complete, but it may provide some useful tools. This theory, as I said, cannot be a linguistic theory. The modes which occur, together with the language-modes of speech and writing, on pages or screens, are constituted on different principles to those of language; their materiality is different; and the work that cultures have done with them has differed also. The theoretical change is from linguistics to semiotics—from a theory that accounted for language alone to a theory that can account equally well for gesture, speech, image, writing, 3D objects, colour, music and no doubt others. …
In the era of the screen and of multimodality some fundamental changes are inevitable as far as forms, functions and uses of writing are concerned. Maybe first and foremost there is the question of how the modes of image and writing appear together, how they are designed to appear together and how they are to be read together. There is the question then—a real question—in what direction writing is likely to move: will it move back towards speech-like forms, and become mere transcription of speech again, or will it move back in the direction of its image origins? And there is the old question of the resources of the mode of writing … .
The current landscape of communication can be characterised by the metaphor of the move from telling the world to showing the world. The metaphor points to a profound change in the act of reading, which can be characterised by the phrases `reading as interpreting’ and `reading as ordering’. The metaphor and the two phrases allow us to explore the questions that reading poses—narrowly as `getting meaning from a written text’, and widely as `making sense of the world around me’—through a new lens. Both senses of reading rest on the idea of reading as sign-making. The signs that are made by readers in their reading draw on what there is to be read. They draw on the shape of the cultural world of representation, and on the reader’s prior training in how and what to read. New forms of reading, when texts show the world rather than tell the world have consequences for the relations between makers and remakers of meaning (writers and readers, image-makers and viewers). In this it is important to focus on materiality, on the materiality of the bodily senses that are engaged in reading—hearing (as in speech), sight (as in reading and viewing), touch (as in the feel of Braille)—and on the materiality of the means for making the representations that are to be `read’—graphic stuff such as letters or ideograms, sound as in speech, movement as in gesture.
Some things are common to `reading’ across time, across cultures, across space, namely those which derive from the way in which our bodies place us in the world, ranging from the physiology of vision to the structure of the organs which we use for speech and hearing, to the organisation of the brain and its inherent capacities for memory, for instance. At the same time, many things are not common across cultures, times, places. Some things which seem part of our ‘nature’ are shaped by culture in important ways, such as the training of memory for instance. Forms of learning may have as much to do with human culture as with human nature. Above all, the shape of what there is to read has its effects on `reading’. Reading practices, and the understanding of what reading is, develop in the constant interaction between the shape of what there is to read and the socially located reader and their human nature.
Immediately, there are the differences between alphabetic writing and logographic or pictographic writing: the one orienting readers towards sound, the other towards meaning. Script systems range from those which attempt to represent sounds graphically as letters, as the alphabet does, to those which attempt to represent meaning as images, as do, in various forms, logographic and pictographic scripts. Even within alphabetic writing there are deep differences in the use of `lettered representation’ over different periods and in different cultures. As it happens, we are in a period where vast changes are taking place in this respect. In the Western, alphabetically oriented world, the change is one where image is ever more insistently appearing with or even instead of writing.
No one theory can deal with everything necessary for a full understanding of reading. In my approach, a semiotic one, I focus on the `how’, the `what with’ and the `why’ of representation and communication: how, in what way, with what material and cultural resources, do we make the signs that represent our interests? That focus is one slice of the pie, so to speak. Ethnographers of writing and reading want to know where and when and for what purposes reading happens; what the environments are and look like in which reading happens. Researchers coming from media sociology want to know the shape of the whole media-field in which reading and the book have their contemporary uses and functions, a place in which `reading’ vies with all other media for the user’s time, energy and attention. Closely related are the interests of those who ask about power, about exclusions and inclusions, and about domination through texts. Others ask about reading from the point of view of leisure and pleasure: given that there is such a vast range of media, what uses are being made of reading for entertainment, for fun, for relaxation, but also for ostensibly more serious purposes? Yet others ask questions which come from phonetics and phonology; yet others focus on more strictly psychological issues such as memory, recognition, retention and so on.
… [T]he increasingly and insistently more multimodal forms of contemporary texts make it essential to rethink our notions of what reading is.