<< In Defense of the Humanities Again and Again and Again | Front Page | Apocalypse 4: Sacrifice, the Great Availability >>
Thursday, June 09, 2011
Inference at the Movies
Having given notice of David Bordwell’s article on common sense film theory, I now want to take a look at one of his topics: inference. Inference is important because it’s how the mind gets from here to there. Here might be transducing the array of light impinging on the retinas or it might be sitting back and thinking about “S → NP + VP”. There might be Aunt Betty’s gotten her hair dyed or it might be Uncle Noam sure took linguistics on a wild goose chase. Whatever it is, inference is the mind’s bridge from here to there.
It’s not clear to me whether or not Bordwell and I disagree on inference, at least not at this level of detail. I offer my commentary by way of, well, commentary. These are thoughts prompted by Bordwell’s remarks. Nothing more, nothing less.
The Gibson Challenge
As Bordwell tells it, and I agree, the “New Look” psychologists “held that the stimuli hitting our sense organs were noisy, incomplete, and ambiguous; we needed higher-level faculties to sort them out.” We simply can’t make sense of the world with “bottom-up” strategies that simply assemble sense data into percepts of intelligible objects and actions. We need “top-down” strategies that drive the interpretation of sense data with prior knowledge of and expectations of the world.
Bordwell came to grips with these ideas in his 1985 Narration in the Fiction Film (NiFF). But, Bordwell goes on, there were problems – aren’t there always?
Theoretically, however, NiFF ran into problems in the role it assigned to inference. At the time of writing NiFF, I was aware of the writings of J. J. Gibson and his insistence that perception evolved in environments very different from the impoverished information that New Look theorists assumed triggered perception. In the three-dimensional world in which creatures like us live, the stimuli are not typically partial or degraded; they are in fact quite rich, even redundant. Moving through space, we register an optic flow that specifies the layout of surfaces quite precisely.
Ah, Gibson, fascinating thinker, important ideas.
I first became away of Gibson as something of a rogue psychologist going against the grain of whatever was the mainstream back then – just when that ‘then’ was, that’s a bit vague. Yet, in some ways, he was a deeply conservative ally of, ugh, of all things, the behaviorists. The behaviorists would have nothing to do with mental processes, and neither would Gibson. If you couldn’t observe it, it wasn’t real. Can’t see mental processes.
The difference, however, is that behaviorists were, for the most part, learning theorists and were interested in and actions. Not perceptions. They studied actions and their contingent relationship to stimuli and rewards, good old stimulus and response. But Gibson wasn’t a bell-ringer and a shocker, he wasn’t interested in what animals, or humans, avoided or approached. He was interested seeing and hearing and touching and smelling.
And, as Bordwell notes, he argued that the natural world – as opposed to the psychological laboratory – was richly structured. Hence, Gibson argued, it wasn’t necessary for us to do all this top-down inferencing and information processing. All we had to do was simply ‘take in’ (my phrase) the rich structure that’s already out there in the world.
It’s the nature of that taking in that’s vague. Gibson sometimes wrote as though there was nothing at all in the mind that was doing this taking in. It somehow just happened, like magic. It’s one thing to argue against the highly logical and inference-heavy kinds of processes proposed by the cognitivists, by AI investigators entirely. It’s another thing entirely to imply that there’s nothing at all in the mind but, well, magic.
The mind’s got to have something that “latches onto” things in the external world and thereby recognizes them. Just what those somethings are – for there’s likely more than one type – that’s under investigation, with many proposals pending. It’s quite possible that none of them will survive another quarter century of investigation.
Story and Plot
But let’s get back to Bordwell, who’s now introduced the distinction between fabula (story) and syuzhet (plot):
Clearly the distinction is useful as an analytical tool, to study how a narrative can “deform” its underlying story for aesthetic purposes. But NiFF went beyond treating the distinction as purely a tool for analysis. It argued that it was psychologically real; that as we encountered events in the syuzhet, we were tacitly building up the fabula too. The process is a bit like double-entry bookkeeping, with the viewer keeping track not only of what is happening each moment on the screen but also slotting that into the chronological pattern of fabula events. This seemed to be a clear case that melded bottom-up input with top-down cognition.
Not having read NiFF, I don’t know the details of Bordwell’s proposals, and such details are important. But let me make a general observation or two.
As Bordwell certainly knows, the point of distinguishing between story and plot is that story events are not always narrated in temporal order. Flashbacks are common enough, and some stories are narrated in highly convoluted order, such as that in Tristram Shandy, the locus classicus for the distinction. But let’s consider a very simple case, that of story-telling in oral societies. In these societies the really important stories, the myths, are told time and again. They’re unchanging and everyone knows them. That is to saw, everyone already has the story in their mind when the teller starts it off.
What’s to infer? No much, it would seem. Not much.
And, while that’s not the world Bordwell’s dealing with, our world is not so different. For one thing, we sometimes see movies where we already know what’s going to happen, and in considerable detail. Shakespeare plays get turned into movies often enough, and I rather suspect that most people in the audience have read the play before they entered the theatre (or popped the DVD in their player). Beyond such cases, most films are made in familiar genres and the number of distinctly different plots is limited; whether it’s seven or 36 doesn’t much matter. When we see a movie we’re assimilating what we’re seeing to patterns we’ve got somehow stored away in our heads. Many of those pending accounts of mental machinery are accounts of just how we organize these familiar patterns.
These observations seem broadly consistent with where Bordwell goes next:
Unfortunately, some people argued, [all this inference is] psychologically implausible. Eventually I had to agree. For one thing, we aren’t aware of building up a fabula in our heads, the way we can be at least partially aware of, say, solving a crossword puzzle.
Notice that last example, the one that’s NOT like moving watching: solving a crossword puzzle. That surely does involve conscious inference. And we generally don’t think stories through in that manner.
Bordwell goes on:
For another, we can’t access it easily; try stopping a film or video and reciting the entire chain of events leading up to the moment of pause. Worse, try at the end of the movie to grasp mentally the entire fabula you’ve purportedly worked out. Chances are you can’t do it. Given that our memories are reconstructive rather than photographic, creating an accurate fabula is extremely difficult.
These observations, which I accept, seem to offer difficulties for the view I’ve offered, that, in effect, we already know the story when the movie starts. Pretty much. If we already know it, then we should be able to rattle it off, at any point, pretty much. No?
Not quite. It’s one thing to be able to know a story well enough to recognize it when you hear it; it’s another thing entirely to be able to rattle it off yourself. Just why this is the case, that’s not obvious. But it seems true enough. And this asymmetry of recognition over production starts early. A three year old can’t tell much of a story, but can be quite picky about the details of stories you tell or read to her.
Further, the story we’re watching on the screen may differ in many details from the generic story we hold in our heads. To the extent that we rely on what we already know, that existing knowledge may distort our sense of the story we’re watching, and this is independent of any difficulties imposed by attempting to recount the story ourselves, in whole or in part.
Time and Meaning
Let’s continue with Bordwell:
I think that NiFF made the valid point that our understanding of narrative is often inferential, and we do flesh out what we’re given. But I now think that the inference-making takes place in a very narrow window of time, and it leaves few tangible traces.
First, in some ways it seems to me that “flesh out” is slightly misleading. For it suggests that we’re presented bare bones and we inferentially supply the flesh. The opposite seems just as likely, that we’re presented with flesh and we supply the skeleton that holds it all together. The inferencing we do is not so much a matter of elaboration as it is of holding together, bringing into coherence.
That we must do so “in a very narrow window of time” is, I believe, a critical observation. The movie must have ‘popped’ into shape, at least provisionally, by the time the end credits have started to roll. We can continue to mull things over after that, and have realizations about this or that, but we must have a sense of coherence and satisfaction by the time the story has ended.
If not, then the film has not worked, at least not for us. Though, it may be the case that it pulls together upon a second, or even a third viewing. For that matter, it’s possible for one to learn to like a film by thinking about it, over time, by discussing it with others, and so forth. But that, that’s way beyond a consideration of what happens in real time while our sensorium is coupled to the images on the screen the and sounds coming from the speakers.
This is quite different from the processing of discovering a meaning in a film through the process of thinking and reading and writing about it – to which Bordwell has devoted a book, Making Meaning. That process most certainly does involve conscious inference, lots of it, and it is unbounded by time. A critic can spend hours and hours over weeks and months and even years thinking and writing about a film. The relationship between conclusions about meaning that arise though this process and what happens in the mind from, say, ten minutes before the movie starts through ten minutes after it ends, that’s somewhat mysterious.
Quite possibly there’s little relationship at all between them. To the extent that that is so – and I fear it’s true to an embarrassingly large extent – the study of how we understand films is still in its infancy.
We’ve got a lot of work to do.
EDIT: Addendum
Consider the last clause of this statement: “the inference-making takes place in a very narrow window of time, and it leaves few tangible traces.” Few tangible traces. What if it left NO tangible traces?
I’m thinking about some experiments that Robert Ornstein reported in On the experience of time (Penguin 1969). Ornstein was interested in our subjective sense of duration. Why does one 10-minute experience seem like it was over in a flash while another feels like it took a million years? Through a series of ingenious experiments he arrived at the notion that our sense of duration was a function of how much information we recorded about what happened in that interval. If we recorded a lot of information, the interval felt long; just a little information, short.
In his most ingenious experiment he presented his subjects with a stimulus consisting of a meaningless line drawing of modest complexity. He had them look at it for some period of time and to be prepared to describe it (I believe). Then he asked them to estimate the duration of the interval in which they’d examined the image. Some time later he gave them a simple way of recoding the image, telling them it was the word “man” written in script over its mirror image. He asked for a duration estimate again. The estimate went down considerably.
That prompted me to the following speculation: Someone’s listening to a piece of music, or watching a movie, or reading a book. They’re thinking furiously during the process. And then, at the end, everything but everything snaps into place. It all makes sense; it all takes the form of a familiar coherent pattern. In consequence, it takes up almost no storage space.
Few tangible traces. No tangible traces.
Does that mean that the interval of listening, watching, or reading drops out of time altogether?
Comments
The perception of time in viewing Shakespeare is something about which I’ve done a lot of thinking. I talk about it a little (you may remember) in Comeuppance, in the last real chapter on King Lear and Oliver Twist. I think it’s an obvious and interesting issue in The WInter’s Tale, where we are supposed to forget why we know, and also why we know that Hermione is dead (her spirit comes to Antigonus in a dream, makes an accurate prophecy to him about his own future, and names Perdita there, so that the living back in Sicilia— Hermione and Paulina—cannot possibly know this is her name), so that when she turns out not to be dead, the fact that she was no longer quite registers for us. Well, in movies cheat cuts work the same way.
Wasn’t Gibson wrong about the richness of visual perception? I love Gibson, but hasn’t it been demonstrated that actually visual perception is very poor, and that the brain enhances the very rough information the rods and cones deliver? I’d have thought movies proved this.
I’ve also written a lot about how we can perceive film and ignore cuts and the like, and here I think Boardwell may miss something as well. During visual saccades we go partially blind, while lower level neural processors stabilize our eyes on the kind of object we must be looking for since we just darted them elsewhere. This is an instantaneous and unperceived moment of blindness. I think the fact that cuts don’t disorient us provided pretty good evidence, before the fact, for how saccades work. I haven’t read NiFF either, but I once dug into this a little, and I don’t think he talks about that.
For my Kantian theory of the perception of film, which seems in line with Boardwell in a bunch of ways, see “Proximity and Power: Shakespearean and Cinematic Space,” Theatre Journal 39 (October 1987), 277-293.
On Gibson, I believe his last major statement on vision was a 1979 book. We’ve learned a lot since then. In any event, much depends on details: just what do you mean by poor, and enhances details?
Back in the classical days of AI (up through, say, the late 70s or so) a few laboratories built mobile robots that would move about the laboratory, guided by artificial vision. The system used a lot of horsepower, which wasn’t physically on the mobile platform, and the software would laboriously construct a representation of the world in 3D space based on inputs from video cameras. They weren’t very good at navigating about.
The sometime in the 1980s along comes Rodney Brooks, who constructed relatively small robots, that relied only on modest onboard computing, simple sensors, neural net learning, and no complex construction of 3D visual space. His bots were much better navigators, despite the lack of horsepower and visual complexity. That seems Gibsonian in spirit.
I really like your point about the asymmetry of recognition over production. We’d need a better test for whether viewers are reconstructing fabula. I’m guessing that the popularity of _Memento_ (for example) is a good indication that we’re doing it: evidence is not our ability to recite the fabula, but simply our ability to enjoy the film. (Compare perhaps a language one is learning: it may well be possible to make good moves within it without being able yet to explain why they work.)
Yeah, if we weren’t ‘reconstructing’ the fabula in some way, then movie would simply fail. It wouldn’t make sense.
BTW, over the years I’ve noticed that, on occasion, even the best reviewers will make mistakes about what happened in a film. They’ll get something wrong, something of a sort that could easily be corrected by another viewing, not something of a more speculative nature. Yet it’s clear from the review that they pretty much ‘got’ the film. It made sense and gave them pleasure, or not.