Paul Elliott, The Eye, the Brain, the Screen: What Neuroscience Can Teach Film Theory, Excursions, Vol. 1, Issue 1 (June 2010), 1-16 [1]
There are strong indications that among the loose federation of sciences dealing with knowledge and cognition…there is a slowly growing conviction that…a radical paradigmatic or epistemic shift is rapidly developing. At the very centre of this emerging view is the belief that the proper units of knowledge are primarily concrete, embodied, incorporated, lived. [2]
Throughout the last 10 years of his life, the philosopher Gilles Deleuze became increasingly interested in using the brain and the brain sciences as models and tools with which to understand philosophy and its related discourses, most noticeably film and the visual arts. In his Cinema 2: The Time-image for instance, he declared that the brain was at once “the screen, the film stock and the camera”, [3] and in an interview published as “The Brain is the Screen” he declared that “I don’t believe that linguistics and psychoanalysis offer a great deal to the cinema. On the contrary, the biology of the brain – molecular biology – does”. [4]
It is crucial to note here that Deleuze makes the distinction between psychoanalysis and neurobiology, the study of the mind and the study of the materiality of the brain. For the poststructuralist philosopher, the workings of electrical synapses, neural networks, and action potentials offered a form of objective correlative for notions such as the rhizome, molecularity, and the body without organs. It also presented a way out of the bifurcated en passe of a psychology that had for so long viewed the unconscious as being structured like a language. Deleuze and Guattari examine new movements in neuroanatomy in the first chapter of their 1980 work A Thousand Plateaus; for them, the brain stood as a model of thought as well as its progenitor:
The discontinuity between cells, the role of the axons, the functioning of the synapses, the existence of synaptic microfissures, the leap each message makes across those fissures, make the brain a multiplicity immersed in its plane of consistency [PAGE 2] or neuroglia, a whole uncertain, probalistic system....many people have a tree growing in their heads, but the brain itself is much more a grass than a tree. [5]
Deleuze then not only saw neurobiology as offering a method of examining the perceptual and cognitive processes, but as a conceptual and analogical model for the exposition of ontology and philosophical method. When he talks of the brain he uses it both as a biological entity and as a heuristic device that can aid in understanding disciplines that might otherwise seem unconnected to science, like art and film theory. When Deleuze claimed that “the brain is the screen” he was being both literal and metaphorical, borrowing from neuroscience and biology in an act of theoretical cross-fertilization and, in turn, mirroring what we could assert were paradigmatic epistemological shifts in neurobiological thinking.[6]
It is in this spirit that the present paper asks: what can neuroscience teach film theory? How can recent ideas about how the brain works provide us with both practical and metaphorical models with which to re-define and re-codify issues such as spectatorship and the processes inherent in watching a film? It is surprising for instance, given that film theory’s main perceptual object is vision, that the various advances in our knowledge of optics arising from the development of a range of scanning techniques in the 1980s and ‘90s, have not found their way into theories of cinema or its spectatorship. In many ways our conceptions of what it means to view a film are still dependent, to a very large extent, on the perceptual sciences heralding from the nineteenth century. Recent developments in neurobiology, most particularly through the work of writers such as Antonio Damasio, Richard Cytowic, Simon Baron Cohen, and Giacomo Rizolatti, have all suggested neurological ideas that have a marked effect on how we perceive and orientate ourselves in the world. These not only question the traditionally held binaries of body/mind, thought/feeling, self/other, reason/emotion and so on, but also encourage disciplines such as film and visual culture to draw up new models with which to base their theories on.
Like many areas, recent developments in neuroscience have shown a propensity to challenge the dominance of vision and its role in the construction of the fixed ontological self. The conscious ego, that was the mainstay of many film theories (the self that sits in the cinema theatre, that watches a movie and empathises, identifies and internalises what it sees) is being gradually eroded by a knowledge of [PAGE 3] perception that suggests our conscious brains are far less important than was initially thought, and that the processes of vision are inextricably linked to our other sensual modalities. In a recent essay on the area of illusory perception, for example, Patricia Pisters draws on neuroscientific research to illuminate the relationship between perception and awareness. Linking Hugo Munsterberg with contemporary neurophilosophy, Pisters explores the different levels of visual processing that are inevitably employed in the cinema experience.[7] She declares that the dividing line between what is consciously perceived and what is not is difficult to determine, and uses scientific insights gained through MRI scanning to better understand the images of memory and perception in Neil Burger’s The Illusionist. Although Pisters uses neurobiology here only to examine the diegetic reality of a film, her essay is noticeable for its willingness to challenge the accepted modes of specular involvement. The notion that vision may be a contestable field (a notion that has long been accepted in neuroscience and optics) is relatively new to film theory but it is one that, of course, has a major impact.
This paper examines three areas of recent neuroscientific research, all concerned with vision, and asks how they affect our conception of that very basic of filmic concepts, the spectator. How do developments in optical thinking shape and challenge our notion of the filmic processes and the ways in which we understand moving images? Some of the research that will be cited here is contentious and all of it exists as work in progress but, as a film theorist, my main interest is not to uncover objective scientific truths but to explore various models and suggestions that come from a variety of different disciplines and to ask how they can be utilised to better understand my own subject area. I look for objective truth in neuroscience in the same way that we have looked for objective truth in Freudian psychoanalysis for over 100 years, that is to say, not at all; what interests me more is how different disciplines approach the same subject, how their language, models and notions can be translated from one field to another and then back again.
The exercise at the heart of this paper then is one of conceptual harvesting, with neuroscience being the main source of inquiry. What emerges from this exercise is a spectator who is far more [PAGE 4] fractured and evanescent than once thought, whose sense of self is not fixed and stable but schizophrenic and centrifugal; whose gaze is not based in simple perspectivalism but in a synaesthetic virtuality that constantly negotiates between all the senses and their respective memories. The spectator that begins to appear is more unconscious than conscious, their knowledge based not in cognition but in sub-cognition, and the value of the film lying not only with narrative and character but with that sense, that inexpressible feeling, you get when the credits roll – as if somehow your body knows something your mind does not.
Imagine yourself walking home alone, around midnight, in whatever metropolis it is that you still walk home in, and realizing all of a sudden that somebody is persistently following you not far behind. In commonsense discourse, this is what happens: Your brain detects the threat; conjures up a few response options; selects one; acts on it; thus reduces or eliminates the risk.[8]
So begins Antonio Damasio’s chapter on ‘the body-minded brain’ in his book Descartes’ Error. For Damasio, the brain is inherently embodied; phylogenetically programmed to perform a kind of feedback loop between sense and thought, the brain continually registers specific somatic affects and sends out related neural messages, mapping the body’s state and monitoring its changes. Damasio complicates the body-brain dyad that has formed the basis of much of Western ontology (Descartes’ error) by adding a third, indeterminate term - the emotional and feeling somatic self - that exists somewhere between, or rather out of, the two. He admits to a body proper and a cognitive brain, to flesh and thought as it were, but the self that we know, that we understand, happens between these two – what we are, he asserts, is neither all flesh nor all mind, but a constantly changing state that is shaped out of synaptic firings, biochemical reactions and biological interruptions. However, most of us, unaware of the complexity of these subconscious processes, merely experience our self within this.
On the midnight walk home, for example, we might sense someone behind us. Our perceptions [PAGE 5] may tell us that we are being followed and our bodies and brains react; the biological chain of events that is set in motion involves the whole organism, as Damasio states:
Your brain has detected a threat, namely the person following you, and initiates several complicated chains of biochemical and neural reactions. Some of the lines in this internal screenplay are written in the body proper, some are written in the brain itself. Yet you do not neatly differentiate between what goes on in your brain and what goes on in your body...You will be aware that you are in danger, that you are alarmed and that you should walk faster...[9]
In his other books Damasio links this idea with the emotions; again, here, the body and brain form one homeostatic whole that resonates with happiness, sadness, fear or terror, complicating the Cartesian duality and its ontological ground.[10] The brain both interprets and feeds back into the body, causing the state to either continue or transform into something else; this is a world away from the lifeless mass of flesh that is sometimes seen as being merely the support system for an all powerful brain. The body with its perceptive organs, biochemistry and capacity for liminality drives the brain every bit as much as the brain drives the body. Even perception can be viewed, he suggests, as both cognitive and physical, as the muscles around the eye (in sight for example) adjust to the correct focus as well as the brain sensing their micro-movement. The clearest example of Descartes’ error, in this sense, would not be the Cogito but his conception of sight in Dioptrics, where the disembodied eye (note: singular; binocularity had not yet begun to occur in optics) of an ox can be used to understand the processes inherent in vision.[11] It would take the physiology of modernity to understand the role that living muscles, tendons and nerves play in sight, and the hermeneutic framework of neural networks and synapses in the late twentieth century to understand the role of the lived body.
The body and the brain, for Damasio, cannot be separated, and the same, presumably, would go [PAGE 6] for the body of the cinema spectator. In Damasio’s conception of the neuro-somatic organism, we see reflections of the Deleuzio-Guattarian heuristic framework that was highlighted earlier: the connections between the flesh and the mind are indeed revealed to be more grass than tree and the perceptual processes are discussed within the physical framework that they are inevitably rooted in. As Damasio is at pains to point out throughout his texts, this is a recent direction in neuroscience, a discipline that has traditionally privileged thought over feeling and that has, more recently, viewed the mind as the more important software that runs on the hardware of the body.
Damasio’s ideas on the importance of the lived body in the processes of perception have been underlined, in recent years, by an increasing interest in the subject of synaesthesia. Contemporary neurobiology however has markedly different conceptions of this condition than those that preceded it. In a famous poem written in 1871, Rimbaud speaks of assigning a specific colour to each of the vowels.[12] The poem paints a fairly typical picture of nineteenth century notions of the synaesthetic experience: the heightened awareness of the poet uncovering the normally invisible perceptual links that exist between the different senses, creating a kind of hallucinogenic melange of sensations that reveal the hidden correspondences between colours and sounds, letters and images, eye and body; the term ‘correspondences’ was also used, of course, by Baudelaire in his poem of the same name.
An essay by Robert Robertson on Sergei Eisenstein’s interest in synaesthesia gives us some indication of how the condition was thought of in the nineteenth and early twentieth centuries:
The evidence of Eisenstein’s interest in synaesthesia and the interaction of the senses is present in his library. According to Hakan Lovgren, in the 1930s he began to acquire books by the Russian symbolist poets and writers…A marked interest in alchemy and other occult and hermetic traditions was characteristic of artists in the symbolist movement.[13]
Synaesthesia was then seen as being commensurate with Blavatskian theosophy, with Giordano Bruno, with symbolist aesthetics and with drug experimentation. It also, as the writing of Eisenstein suggests, [PAGE 7] was seen to refer to the synchronization of the senses- the twinning of colour, sound and movement for instance through montage. This paints the synaesthetic experience as being somehow beyond the everyday, a kind of sixth sense that could be achieved through meditation or mescaline.
More recently however, definitions of synaesthesia have distanced themselves from such gnostically based experiences. Synaesthetes are being taken seriously not so much as possessors of a sixth sense but as, what Richard Cytowic calls, “cognitive fossils”, examples of individuals whose cognitive and perceptual processes have retained some of the phylogenetic imprints of our own evolution. Cytowic suggests that synaesthetes can provide us with snapshots of how our brains work before cognition orders our world and constructs the reality that we are so used to.
In his book The Man Who Tasted Shapes, Cytowic outlines the experimental processes that Michael, Cytowic’s subject, undertook. Michael was a gustatory-tactile synaesthete who literally felt different shapes in his hands depending on what he tasted or smelt.
“Are you comfy?” I asked. Michael was strapped down on the couch, his head nestled inside the helmet with its electrical devices protruding two feet in all directions. Tons of nuclear, electronic, and computer equipment hummed around him in the darkened room. He looked like a cyborg. I was adjusting the strap on the black anaesthesia mask that covered his face. His eyes peered at me from beneath the helmet. “I’m not sure how I look,” Michael mumbled through the mask, “but you were right. I feel like the Bride of Frankenstein”[14]
The process that Michael was undertaking here was a CBF or Cerebral Blood Flow scan that, as the name suggests, was designed to show the levels of blood flow to, and thus activity of, the different parts of the brain during various actions and thought processes. During the hour of the test, Michael was asked to inhale a number of pre-selected smells in order to, firstly trigger his synaesthesia and, then, to monitor the resulting brain activity; activity that was then placed in comparison to non-synaesthetic [PAGE 8] readings.
What was interesting and crucial about Cytowic’s study was that Michael’s bout of synaesthesia did not produce an increase in blood flow to any measurable part of the cerebral cortex, in fact it showed a decrease. As the subject felt the shape of the smells that were administered to him, his brain showed less activity than when he was resting; sometimes dropping to levels that would normally be considered symptomatic of illness or damage in a normal perceptual subject. Cytowic concluded from this that the synaesthetic experience was not contained in the cerebral cortex at all but in the limbic system, the part of the brain that was inaccessible to the CBR scan.[15]
The limbic system is concerned with memory, with emotion and with spatial awareness. As Susan Greenfield states, it especially governs aggression and sexual behaviour and, thus, is often referred to as the reptilian or animal brain.[16] By suggesting that the synaesthetic experience lies in the limbic system, Cytowic assumes that we might all be party to synaesthesia but that the higher, cortical processes might obscure it somehow, creating order and reality out of the chaos of multi-modal perception. We might feel the effects of such perception without it ever entering into our consciousness at all, a memory might be triggered without our knowledge, a taste may be evoked by a sight, a smell by a sound, a sensation of touch by a word or an image and so on.
Some studies have suggested that synaesthesia is merely a part of normal neurological development, that babies up to about 4 months experience sensory input in an undifferentiated way and that sounds trigger auditory, visual and tactile experiences. Using neuroimaging techniques, the brain patterning of young children was tested and the results suggested that, unlike most adults, infants experience the world synaesthetically, never being able to fully distinguish between their senses at all and living in a reality of holistic perception.[17]
Further studies have also stated the possibility that the universality of synaesthesia (both in [PAGE 9] terms of individual human development and in terms of species evolution) has contributed to the widespread use of metaphorical speech. Poetry, religious symbolism, abstraction, art and scientific models could all be a product of the primal tendency to experience the world in a synaesthetic way – with all senses contributing to the perceptual wholeness of existence. The sharpness of a lemon, the blueness of depression and the hardness of a maths equation could all be the products of our more basic perceptual processes, processes whose lack of logic and reason are usually tidied up by cognitive thought. Processes that, until now, have been largely ignored by disciplines such as film theory.[18]
The image we receive from studies such as Cytowic’s is of a brain that is constantly filtering out the multi-modality of experience. That whilst you are reading this, you are able to understand the words only after your cognitive processes have filtered out your body’s desire to conjure up all of the sensual inputs and memories that it has access to. It would be hard, after all, to concentrate on this page if you were experiencing the memory trace of the smell of this morning’s bacon, or you were too aware of the feel of the seat on the back of your legs, or of the way the words on the page remind you of that summer you spent reading philosophy on holiday. Multi-modal perception however does leave its trace on the body and the mind. It is in the feeling of disgust one experiences when witnessing a distressing scene, the vague sense of unease one gets when certain images are presented to you, the recognition of images of taste, and all the other non-cognitive situations that we can all relate to.
The synaesthetic links between vision and the other senses, and vision and memory impact greatly upon our notion of what constitutes the process of cinema spectatorship. Cytowic’s thesis, that we might all be experiencing synaesthesia below the level of our consciousness, means that, when we watch a film for instance, our minds can be having one experience whereas our bodies and our limbic systems could be having another, altogether different one; one that is rooted in our past, in our sensual memory and in synaesthesia. It also means that the image of the spectator as the detached voyeur who merely witnesses visual images before him or herself has to be re-drawn to accommodate the experiences of the body and the animal brain.
Cytowic’s conception of the limbic brain, and its importance to vision and understanding, [PAGE 10] suggests that we could perhaps be subject to many more perceptual processes than we might be aware of. Our attachment to those around us might be as much subconscious as conscious and the dividing line between our own physical selves and that of others may be more difficult to discern than we once thought. One area of neurobiology that suggests this, is research dealing with mirror neurons. Mirror neurons suggest that we understand those around us not through identification, empathy and sympathy but through neurological processes that evoke the movements of others within ourselves – that when we see someone eating, the same neurons are fired in our brains as if we ourselves were eating. Identification becomes a secondary process of this rather than the other way round. We must remember that this is occurring on a neurobiological level and we may not even be aware that what we are responding to is governed by what, or more rightly who, we are looking at. Mirror neurons knit us into those around us without us being conscious of it, they allow us to understand other people not through abstract reasoning or categorical imperatives but because our bodies imitate those we are looking at. Our brains and bodies simulate that which is presented to us, creating a kind of virtual copy that allows us to feel as others feel.[19]
Mirror neurons were first discovered in the ventral premotor area of the frontal lobes of monkeys, the area known as F5.[20] The neurons in this area largely govern the link between vision and movement, especially “walking, turning the head, bending the torso and moving the arms.” [21] However, as Rizzolatti, Fadiga, Gallese and Fogassi found, neurons that would fire when these activities were being conducted would also fire when the subject was watching these activities performed by another.[22] Further studies revealed that not only did the gestures and actions result in neurons being activated but that this depended, to a large extent, on context and intention: miming grasping without actually doing [PAGE 11] so, a picture of a hand or (in monkeys) a movie of a hand would not result in the same effect.[23] What is interesting and vital to an understanding of how mirror neurons can inform film theory is that, in humans, movies do produce results. In a study by Fogassi et al films of hand actions were witnessed by a series of subjects and the resulting mirror neurons were activated; suggesting that for humans, at least, cinema is as real as life.[24]
A study to test the presence and importance of mirror neurons was conducted in the University of Rome by Salvatore Aglioti. Aglioti’s subjects were shown videos of needles being pushed into the hands of volunteers on screen and the resulting neurological activity was measured using transcranial magnetic stimulation. At the same time the excitability of the subject’s own hand was measured as well as a muscle that had no role in moving it. The results showed that subjects experienced a reaction in their own bodies that corresponded to the images they witnessed on screen, as Marco Iacoboni outlines:
The decreased excitability during the observation of pain was also specific to the muscle penetrated by the needles. Neighbouring muscles in the hand did not change their excitability. Furthermore, subjects were asked after the experiment to rate the intensity of the pain felt by the individuals observed in the videos. Aglioti and colleagues found that the lower the motor excitability in subject’s muscles during the experiment, the higher they rated pain.[25]
As Iacoboni outlines, this radically alters our notions of what it means to see and understand the pain of other people; rather than relying on notions such as empathy and sympathy, such research suggests that we could actually be experiencing (in small but significant ways) the pain (and supposedly also the pleasures) of those we witness on screen. How much more involved does this make the spectator? How much more embodied does this suggest the cinema experience is? How much more real is the pain of Marion Crane in Psycho or the victims of Saw’s John Kramer when we not only see but, to an extent, fully simulate the neurological firings of their pain? Again, recent neurobiology suggests that our bodies are far more important to sight and to the self then we might traditionally have thought.[PAGE 12]
Recently, film theory has taken up some of these ideas; the work of Laura Marks, Vivian Sobchack, Barbara Kennedy, Tarja Lane and others have begun to look at the area of synaesthetic multi-modal spectatorship and what it can offer us in terms of understanding, what Clifford Geertz might have called, the full thickness of the filmic experience. Such theories do not negate the importance of vision nor do they suggest that more traditional tools of film theory should be abandoned, instead what they represent is an attempt to bring cinema studies more in line with epistemic and scientific shifts in how we think about vision; ultimately their work is an attempt to address the kinds of embodiment I have been outlining here and to re-define and re-present the conceptual image of the spectator. As Thomas Elsaesser has recently stated:
Over the past ten years, we have discarded one type of theory, gradually switching to another, as yet to be defined, paradigm. Rather than continue to think about the cinema as an ocular-specular phenomenon, whose indexical realism we either celebrated or whose illusionism we excoriated scholars now tend to regard the cinema as an immersive perceptual event. Body and sound-space, somatic, kinetic and affective sensations have become its default values, and not the eye, the look and ocular verification.[26]
The image of cinema as an ‘immersive perceptual event’ is in complete accord with recent ideas heralding from neuroscience. Interestingly we see similar moves in areas such as architecture with Juhani Pallassmaa, in anthropology with the work of David Howes and others, in psychoanalysis through the work of Didier Anzieu, in gender politics with Luce Irigaray and a whole range of other works by other writers that all assert the importance of the body as a perceptual system, and how it can be used to counter what has been called the hegemony of the eye that characterised modernity. Film theory has begun to examine the basic element of its discipline: what it means to see, and has begun to question the specificity of sight as a perceptual event. This has not only meant that there is a proliferation of exciting and highly original studies, but that we might be going someway to describing [PAGE 13] what most cinema spectators know are the true experiences [PAGE 14] of film – excitement, thrills, chills, nausea and tear jerking.
So, neuroscience then can offer film theory not only research that deals specifically with the visual processes, but models and images with which to challenge our most fundamental of ideas, and a point of view that breaks free from the traditions of psychoanalysis and semiotics. Deleuze may well have been wrong, however: the brain may not be revealed to be the screen, the screen may include the body as well.