Conscious percepts depend strongly on past events. Expectations, primes, and prior experiences all shape the percept we have at any moment in time. Yet does this imply that conscious experience should be viewed as extended in time—as “flowing”—instead of as just happening now

Lamme, Victor

English

Gutenberg Open

Predictive Coding Is Unconscious, so 
that Consciousness Happens Now
A Reply to Lucia Melloni
Victor Lamme
Conscious  percepts  depend strongly on past events.  Expectations,  primes,  and
prior experiences all shape the percept we have at any moment in time. Yet does
this imply that conscious experience should be viewed as extended in time—as
“flowing”—instead of as just happening now?
Keywords
Bayesian framework | Inference | Predictive coding | Snapshot vision | Spatial in-
tegration | Stream of consciousness | Temporal integration
Author
Victor Lamme
Victorlamme@gmail.com   
Universiteit van Amsterdam
Amsterdam, Netherlands
Commentator
Lucia Melloni
lucia.melloni@brain.mpg.de   
Max Planck Institute for Brain 
Research
Frankfurt a. M., Germany
Editors
Thomas Metzinger
metzinger@uni-mainz.de   
Johannes Gutenberg-Universität
Mainz, Germany
Jennifer M. Windt
jennifer.windt@monash.edu   
Monash University
Melbourne, Australia
1 To infer or to integrate, that is the 
question
In her commentary, Lucia Melloni argues that
consciousness unfolds in time: there is a stream
of consciousness. What I see now is intricately
linked to what I have seen before. And what I
see now is what I expect to see—much along
the  lines  of  predictive  coding.  A  full  under-
standing  of  consciousness  should  not  neglect
this point. There is even a stronger claim that
somehow the process of inference over time is
crucial to understanding consciousness.
I  appreciate  the  boldness  of  linking  the
framework of Bayesian predictive coding to specific
stages in the process of generating consciousness:
One  promising  framework  within  which
the influence of previous experience can
be  understood  is  the  Bayesian  frame-
work. When applied to perception, each
mathematically-formulated  ingredient  of
this framework can be assigned a percep-
Lamme, V. (2015). Predictive Coding Is Unconscious, so that Consciousness Happens Now - A Reply to Lucia Melloni.
In T. Metzinger & J. M. Windt (Eds). Open MIND: 22(R). Frankfurt am Main: MIND Group. doi: 10.15502/9783958571105 1 | 4
www.open-mind.net
tual  counterpart,  with  previous  experi-
ence  referring  to  the  prior,  the  current
moment  referring  to  the  likelihood,  un-
conscious  inference  referring  to  Bayes
rule (which combines the prior with the
likelihood  in  an  optimal  way),  and  the
result—our perception—referring to (the
peak of) the posterior distribution. (Mel-
loni this collection, p. 4) 
To my knowledge,  this  is  the  first  time this
has  been  so  explicitly  laid  out—writers  on
predictive coding thus far have always stayed
a little vague on where exactly consciousness
sits in the Bayesian framework.
Yet at the same time, there is the sug-
gestion of long temporal range integration be-
ing the key ingredient of consciousness: 
Event-objects of the conscious mind thus
per definition unfold in time and we also
act in time: we make use of current and
previous input to figure out the most ap-
propriate  response  predicting  their  con-
sequences. There is thus a continuum of
interdependencies along the time dimen-
sion whereby every past moment is integ-
rated with the present and projected into
the future, giving rise to the flow of con-
sciousness. The same way we have been
thinking  about  the  integration  of  mul-
tiple  sources  of  information  within  a
given moment of time, such as multiple
features of a single object, there is thus
integration across time. (Melloni this col-
lection, pp. 7-8) 
This makes intuitive sense, particularly in the
case of moving objects, such as the tennis ball
Melloni uses as an example. Indeed it is hard
—if  not  impossible—to  pinpoint  the  exact
now of conscious experience of such a ball.1 
Yet the two points seem contradictory. In
the  Bayesian  predictive  coding  framework,
consciousness is the  result  of the unconscious
inferential processes. Previous knowledge and
1 Although some have argued that consciousness unfolds in time as a succes-
sion of static frames, more or less like the single frames of a movie—even at
specific frequencies, namely 10Hz and 40Hz (Van Rullen & Koch 2003).
experience (the priors) play an important role,
but they are combined with current input to
produce the posterior, which is conscious sen-
sation.  In the second account,  however,  con-
sciousness  seems  to  be  something  that  is
stretched  out  over  time,  so  that  both  prior
and posterior are smelted into a “flow” of con-
sciousness.  I  find  it  hard  to  reconcile  these
two views. 
2 The latency of visual consciousness is 
variable
Melloni discusses some impressive experiments
that show the crucial importance of prior in-
formation  and  expectation  in  shaping  or
simply altering conscious experience (and her
example, figure 1, is enlightening and flatter-
ing at the same time). In all these cases, how-
ever,  consciousness  is  portrayed  as  the  out-
come or result of an otherwise unconscious in-
ferential process. The result may come earlier
or later, as in the experiment on letter prim-
ing that Melloni describes, resulting in earlier
(200ms) or  later  (300ms) electrophysiological
correlates  of  conscious  recognition  depending
on the presence or absence of appropriate pri-
ors. Further experiments are discussed, show-
ing  that  neural  correlates  of  consciousness
may shift (neural) location, depending on ex-
pectation and priors. Yet still, the end result
—consciousness—occurs at the end of  a cas-
cade  of  neural  operations.  Consciousness,  in
this account, may occur at variable moments
and locations, but moments they are.
These results complement earlier findings
that the latency of recurrent processing—and
hence the emergence of a conscious sensation
—may vary.  Super et al. (2001) showed that
degrading  stimulus  quality  may  increase  the
latency of recurrent signals to V1 in the mon-
key visual cortex (see figure 5c of Supèr et al.
2001), and that this affects the latency of be-
havioral  responses  of  animals  that  are  con-
sciously reporting the presence or absence of
the stimuli. Latency of recurrent signals may
also vary spontaneously between trials, which
correlates with the latency of memory-guided
—but  not  reflexive—saccades  to  the  targets
Lamme, V. (2015). Predictive Coding Is Unconscious, so that Consciousness Happens Now - A Reply to Lucia Melloni.
In T. Metzinger & J. M. Windt (Eds). Open MIND: 22(R). Frankfurt am Main: MIND Group. doi: 10.15502/9783958571105 2 | 4
www.open-mind.net
that elicit these recurrent signals (Supèr et al.
2004).  In  humans,  the  latencies  of  electro-
physiological correlates of recurrent processing
also  vary,  either  spontaneously  or  depending
on stimulus properties  (Jolij et  al. 2011),  or
depending on the IQ of the subject (Jolij et
al. 2007). Likewise, this has consequences for
the latency of conscious sensations. The Jolij
2011 study, for example, found that variations
in  the  latency  of  recurrent  EEG signals  co-
vary with variations in subjective simultaneity
of  the  stimuli  evoking  these  signals.2 These
results  invariably  imply  that  consciousness
arises  at  a  particular  moment in  time.  That
moment may vary from stimulus to stimulus,
from  trial  to  trial,  from  person  to  person,
from prior to prior. But nothing is flowing or
stretched out over time. 
3 Consciousness is not streaming, but 
taking snapshots
One may argue that these findings are all ob-
tained  with  stimuli  that  are  presented  de
novo,  using  the  classic  stimulus-onset
paradigms. In normal vision, things don’t sud-
denly appear out of nowhere. Or do they? We
naturally  make  about  three  saccadic  eye
movements per second, and each time the eye
lands on a “new” scene which is—from a ret-
inotopic  point  of  view—radically  different
from  the  previous  one.  In  between,  we  are
blind due to saccadic  suppression.  Moreover,
little information seems to be transferred from
one view to the next,  although some (atten-
ded)  neural  representations  seem  to  be
remapped across saccades (see  Bays & Husain
2007,  for  an  overview  of  trans-saccadic
memory  and  neural  remapping).  Such  a
remapping may allow for a more efficient sac-
cade from one object to the next, when both
were already present before the first  saccade
was  made.  The  predictive  coding  framework
seems  to  re-emerge  in  this  context:  objects
that were present or attended on a first fixa-
2 For this reason, I don’t quite understand why Melloni suggests that I
am claiming that consciousness arises at a particular and fixed mo-
ment in time. My claim is only that it comes after feedforward pro-
cessing,  and as  soon as  recurrent  processing  emerges—which may
vary.
tion form a sort of prior for the representation
that is built during the second fixation (which
may then arise more rapidly).
Melloni further claims that previous ex-
perience has different effects on what is per-
ceived now depending on the temporal inter-
val  between  prior  and  current  experience.
Bistable  percepts  show hysteresis  or  adapta-
tion depending on these temporal intervals, or
depending on whether the previous experience
was conscious or not. But again, I fail to see
how these findings support the idea that con-
sciousness is stretched out over time instead of
just happening now.3
So  I  appreciate  the  importance  of  the
predictive coding framework. Previous experi-
ence plays a very important role in the con-
scious  sensations  we have,  and the why and
how of  this  is  extremely  important  for  fully
understanding vision. But these contributions
are unconscious. Consciousness happens now,
and its  neural  correlates  are likewise  limited
in  time.  Consciousness  of  the  past  we  call
memory.
Acknowledgements
This work was supported by an advanced in-
vestigator grant from the ERC.
3 Of course there are visual percepts that are more or less defined by
their temporal sequence, the prime example being motion. But this
does not imply that the perception of motion is flowing. The first
thing the brain does in detecting motion is to convert the flow of
motion into a discrete and momentary signal, indistinguishable from
how the brain represents other features such as orientation, color, or
shape. As a result we see motion now, and instantaneously, which is
also  crucial  for  our  survival:  perceiving  something  moving  in  the
shadows of a bush (e.g., a snake) needs to be translated into action
as soon as possible (e.g., running away). No time for any flow there. 
Lamme, V. (2015). Predictive Coding Is Unconscious, so that Consciousness Happens Now - A Reply to Lucia Melloni.
In T. Metzinger & J. M. Windt (Eds). Open MIND: 22(R). Frankfurt am Main: MIND Group. doi: 10.15502/9783958571105 3 | 4
www.open-mind.net
References
Bays, P. M. & Husain, M. (2007). Spatial remapping of
the visual world across saccades. Neuroreport, 18 (12),
1207-1213. 10.1097/WNR.0b013e328244e6c3
Jolij, J., Huisman, D., Scholte, H. S., Hamel, R., Kemner,
C. & Lamme, V. A. F. (2007). Processing speed in re-
current visual networks correlates with general intelli-
gence. Neuroreport, 18 (1), 39-43.
10.1097/01.wnr.0000236863.46952.a6
Jolij, J., Scholte, H. S., Van Gaal, S., Hodgson, T. L. &
Lamme,  V.  A.  F.  (2011).  Act  quickly,  decide  later:
Long-latency visual processing underlies perceptual de-
cisions but not reflexive behavior. Journal of Cognitive
Neuroscience, 23 (12), 3734-3745.
10.1162/jocn_a_00034
Melloni, L. (2015). Consciousness as inference in time. In
T.  Metzinger  &  J.  M.  Windt  (Eds.)  Open  MIND.
Frankfurt a. M., GER: MIND Group. 
Supèr, H., Spekreijse, H. & Lamme, V. A. F. (2001). Two
distinct modes of sensory processing observed in mon-
key primary visual cortex (V1).  Nature Neuroscience,
4, 304-310. 10.1038/85170
Supèr, H., Van der Togt, C., Spekreijse, H. & Lamme, V.
A .F. (2004). Correspondence of presaccadic activity in
the  monkey  primary visual  cortex with  saccadic  eye
movements.  Proceedings  of  the National  Academy of
Sciences  of  the  United  States  of  America,  101 (9),
3230-3235. 10.1073/pnas.0400433101
Van Rullen, R. & Koch, C. (2003). Is perception discrete
or  continuous?  Trends  in  Cognitive  Sciences,  7 (5),
207-213. 10.1016/S1364-6613(03)00095-0
Lamme, V. (2015). Predictive Coding Is Unconscious, so that Consciousness Happens Now - A Reply to Lucia Melloni.
In T. Metzinger & J. M. Windt (Eds). Open MIND: 22(R). Frankfurt am Main: MIND Group. doi: 10.15502/9783958571105 4 | 4


Predictive coding is unconscious, so that consciousness happens now : a reply to Lucia Melloni

http://publications.ub.uni-mainz.de/opus/volltexte/2016/55183/pdf/55183.pdf

Predictive coding is unconscious, so that consciousness happens now : a reply to Lucia Melloni

Abstract

Similar works

Full text

Available Versions

Gutenberg Open