Model: Predictive Processing and Meditation

I've gotten my hands on a copy of Surfing Uncertainty, after being turned onto it by Scott's review, and have been reading through it with an eye toward, "How can this theory be applied toward meditation?"

The book's primary thesis is that the brain is best understood as a prediction organ:

To deal rapidly and fluently with an uncertain and noisy world, brains like ours have become masters of prediction—surfing the waves of noisy and ambiguous sensory stimulation by, in effect, trying to stay just ahead of them. A skilled surfer stays 'in the pocket': close to, yet just ahead of the place where the wave is breaking. This provides power and, when the wave breaks, it does not catch her. The brain's task is not dissimilar. By constantly attempting to predict the incoming sensory signal we become able—in ways we shall soon explore in detail—to learn about the world around us and to engage that world in thought and action.

The impetus for such a theory comes from, well, first consider the plight of a baby, here captured masterfully by William James:

The baby, assailed by eyes, ears, nose, skin, and entrails at once, feels it all as one great blooming, buzzing confusion.

The idea is that, to a baby, the world isn't "the world", but instead appears as an undifferentiated mass of sensory data. Just noise without meaning, a chaotic, fluxing, experiential soup. The task of learning, then, is to make sense of the senses, to squeeze the signal out of this noise, to create "the world."

[E]veryone who comes into contact with a child is a teacher who incessantly describes the world to him, until the moment when the child is capable of perceiving the world as it is described.

—Carlos Castaneda, Journey to Ixtlan

Remind you of anything? Learning an explanatory model from data is the entire point of statistics. The question that naturally arises, then, is "Might the brain be a statistical machine?" An observation which Wikipedia helpfully informs me dates back to at least 1860s "with the work of Hermann Helmholtz in experimental psychology [who modeled] the brain's ability to extract perceptual information from sensory data […] in terms of probabilistic estimation."

Might, then, the brain be a statistical machine?

I've the vague impression that there's good evidence for "yes" across a bunch of disciplines but reviewing said evidence sounds tedious so I'm just going to give you my off-the-cuff take, you know, as a highly qualified, professional web surfer.

When it comes to teaching machines to learn, our best-in-class methods are "statistical machinery"-approaches, often possessing a theory-core at least adjacent to Bayesian math voodoo. The argument for "yes", then, is that of structural realism: just as we ought to regard our best physical theories as true, we similarly ought to believe that our best theories of machine perception reveal something fundamental about ourselves.

That said, the more I study this, the more I feel compelled to slam the brakes on this sort of exuberance. The model is gorgeous, totally seductive: neatly tying together computational models, statistics, cognitive science, etc. all with this beautiful bow, Bayes' theorem but IMO, like any gorgeous woman, she drives her suitors absolutely mad, just bonkers. You can get a sense of this in Surfing Uncertainty —certain sections read like, "HEY, look at how shiny my hammer is! And look at all of these nails! Nails, everywhere! Thunk, thunk, thunk! Ah, the serene melody of my hammer!"

There is a potential parallel here with animal magnetism, the 18th century pseudoscience that explained living things in terms of an invisible natural force, which developed out of an excess of enthusiasm for the then new science of electromagnetism. (I can't blame them: radio still seems to me like magic.) I worry that this predictive processing model is less the truth-about-things and instead a similar fad, except stemming from the ebullience around machine learning, and that I'm about to layer even more speculation on top of it.

What does that imply for meditators?

Monks, a monk who has six qualities could shatter Himalaya, the king of mountains, let alone this wretched ignorance! What six? It’s when a monk is skilled in entering immersion, skilled in remaining in immersion, skilled in emerging from immersion, skilled in gladdening the mind for immersion, skilled in the meditation subjects for immersion, and skilled in projecting the mind purified by immersion. A monk who has these six qualities could shatter Himalaya, the king of mountains, let alone this wretched ignorance!

—Buddha, the Himalaya sutta


So the idea in the PP framework is that the brain walks around with a kind of intuitive model of the world and is each moment again trying to predict the experience of the senses. Whatever is accurately predicted gets filtered out and surpressed, as it is already understood and thus can be safely ignored, and only the error (the surprise) gets passed up the hierarchy to the next layer of mind to be digested and integrated into a new, better predictive model. In essence the brain is saying, "I only care about error, surprise, what I didn't predict."

The experience of trying to focus on the breath is a good example of this: it becomes progressively more subtle and slippery the longer you hold attention on it, more ephemeral, like it is fading away. It is like listening to a song on loop: it soon fades into the background, invisible, the mind has sucked all the signal out of it, is now predicting it, no need to pay any attention to it.

The flip side of this is the experience of a sudden noise or anything unexpected, really. Subjectively it feels very "loud", especially when in the midst of meditation where it's contrasted with a very subtle attention tuned into keeping aware of this very weak signal, the breath.

This is empirically backed, e.g. "Surprise is found to yield robust performance at predicting human gaze".

So, experience fading away when you pay stable attention to it, that remind you of anything? This is the simplification of attention that unfolds with the jhanas! It's the "wet vehicle" to cessation!

There is also a cute sense in which maybe this is how no-self unfolds: you turn attention back onto yourself, become familiar with your patterns, and thus able to predict them, there is no longer any need to continue to inject them into conscious awareness. Maybe this is what Ken McLeod is talking about in Wake Up to Your Life with abandoning patterned behavior.

Layers of mind

There is this model of meditative progress that I associate most strongly with Bill Hamilton and his lineage (Kenneth Folk, Daniel Ingram) where one penetrates and pulls progressively earlier and less refined layers of mind into consciousness. Here he is in Saints and Psychopaths:

By focusing the mind in a profound examination of the present moment, processes of the mind which were not accessible to normal consciousness become conscious. Unconscious processes become conscious processes. Enlightenment is a particularly good term for this process. It is like turning a light on in a dark room so that which was unseen becomes seen.

The predictive processing model is very compatible with this idea. I don't know that much about machine learning but my understanding of deep learning models is that they layer neural nets in such a way that early layers recognize structure and pattern and then feed that more refined output into later layers.

PP assumes something similar, that the mind functions as a hierarchal model where early layers recognize basic structure and feed that into later structures, progressively refining sensory experience into something meaningful.

It seems to me that one thing we are doing in meditation is developing conscious models of these earlier layers of mind. Like, in meditation, one first aims to set aside very loud signals, via seclusion one insulates oneself from surprising sensory information and through non-doing one additionally lets go of the gross movements of the self. Then, one strains to keep a subtle object like the breath in awareness.

This all serves to turn up the gain of attention and unearth the previous unnoticed subtle fluxing of the background which, with repetition and exposure, one begins to recognize structure in until viola! now you're conscious of an earlier layer of mind, able to recognize a new, earlier, subtler phase of experience at the point of sensory contact. As one progresses this deepens into an understanding of dependent origination and how experience is fabricated into the standard world we inhabit.


Hobson and Friston (2009, section 4.2.1) further speculate that the sleep state offers an opportunity for the brain to engage in ’post—synaptic pruning’—removing redundant or low-strength connections so as to reduce the complexity of the generative model itself. The idea here (more on this in chapters 8 and 9) is that reducing prediction error while awake and alert sometimes results in models that, although able to capture the sensory patterns, are nevertheless overly complex. Such models effectively treat too much of the signal as data and not enough as noise. They thus ’overfit’ the specific data and (thereby) fail to generalize to new situations.

Sleep, thanks to the altered balances just described, provides an opportunity to remedy this. During sleep, the brain’s model is insulated from further sensory testing but can still be improved by simplification and streamlining. This is because the quantity that is minimized by the brain is actually (as we will see in chapter 9) prediction error plus model complexity. During sleep, precise prediction errors are not gen— erated, so the balance shifts towards the reduction of model complex- ity. Sleep may thus allow the brain to engage in synaptic pruning so as to improve (make more powerful and generalizable) the knowledge enshrined in the generative model (see Tononi 8: Cirelli, 2006; Gilestro, Tononi, & Cirelli, 2009; Friston 8: Penny, 2011).13

—Andy Clark, Surfing Uncertainty

Perhaps some meditative states function somewhat similarly, allowing for a semi-online, conscious tuning of predictive models (lucid dreaming).

Stabilizing awareness enabling long-range gamma synchronization

Mindfulness = keeping attention at bodily contact = higher fidelity prediction of sensory experience = fewer prediction errors driving reactivity = less suffering

Less prediction error = less interrupts in awareness = long range gamma synchronization of previously disparate mental networks = experience of more integrated present moment, ultimate reality, eternal "now"

The process by which the salience of a stimulus event captures attention and drives the response of higher cognitive systems (memories, tho ughts, plans and expectations) appears to correspond closely with the dynamics of attachment/aversion and mental proliferation described in Buddhist psychologies of mindfulness. If this correspondence is valid then the non-reactivity or nonattachment to the contents of awareness that is cultivated in mindfulness must correspond to a heightened level o f matching between interoceptive predictions and the felt state of the body as well as to a reduction in the salience response of the dAI (as does the sense reality in the case of hypnosis). On this account a suitable awareness of bodily sensations, such as breathing, acts as a gateway to unfolding states of mindfulness. An important consequence of the proces s described here will be the longer duration of current generative models (with perceiv ed slowing of time; Naish, 2007; Ott, 2013) due to reduced destabilisation by prediction error signals. Thus it is expected that time dilation will form an essential part of the cognitive phenomenology accompanying the interoceptive representations, within the insula and across the highest available levels of model representation hierarchies, which are now released from the ordinary disruption by the salience and central executive networks.


Paradoxically the outcomes of maintaining a single focus of awareness without disruption for extended periods during concentrative meditation may be explained in a similar way. The static postures, rhythmic breathing (pranayama) and repetitive mantras found in Yoga meditation may all be expected to establish a conti nuous predictable pattern of interoceptive inputs while practices such as body scan meditation focus awareness specifically on interoceptive inputs, the felt state of the body. T hese practices may be expected to diminish interoceptive prediction mismatch in dAI and vAI. In turn this will result in diminished salience system activity and reduced constraints by bottom-up prediction errors from sensory and motor inputs and reduced activity within the execut ive control system. This allows the emergence of active integrated networks of high level multimodal or amodal representations prohibited by everyday modes of brain activity.


Longer periods of undisturbed awareness of meditation practice do cor respond to reduced mind wandering and thus reduced activity in the DMN (Farb et al., 2007; Hölzel et al., 2007; Brewer et al., 2011) but this appears to be an effect rather than a cause of changes in meditative experience. Rather, longer periods of synchronized activity across the representational elements of generative models, particularly at the higher multimodal and amodal levels of such representational hierarchies, will spontaneously occur due to the absence of the influence of the bottom-up prediction error signals that focused attention and executive control facilitate. […]

As the time periods during which high level generative models can be maintaine d is extended, it is reasonable to predict that the scope (i.e. the number, diversity and leve ls) of the representational units that may be incorporated in these integrated hierarchical model s is also expanded. As it is these generative models which provide the contents of con sciousness in moments of absorption, not only the sense of reality of that moment but th e scope of the reality able to be encompassed in the experience itself may be greatly expanded beyond that which is possible within the constraints imposed by the ordi nary operation of the SN and CEN. If so it is to be expected that these states will be accompa nied by a qualitatively unique “cognitive phenomenology” (see Bayne and Montague, 2011), whic h may well correspond to descriptions in the analytic discourse which has gr own up amongst observers within related traditions.

There is now a wide body of evidence that synchronization in fast frequencies of the EEG (gamma-band >30Hz) plays an essential role in the t emporal binding of separated cell assemblies throughout the cortex, coding for divers e representational features, into unified representational states (Singer and Gray, 1995; Eng el and Singer 2001). While initial research identified gamma synchronization as a dete rminant of perceptual feature binding, this role has since been extended to include the bi nding of representational elements in a wide range of cognitive processes (Senkowski, Schne ider, Foxe and Engel, 2008; Uhlhaas et al., 2009). Therefore it is expected that the ex panded generative models predicted in episodes of absorption will be accompanied by exten sive and intense patterns of EEG gamma-band synchronization which support them.

There is now evidence from a number of studies of Buddhist meditation practices in advanced practitioners that such techniques elicit specific patterns of synchronized gamma activity (Lehmann et al., 2001: Lutz et al., 2004; Cahn, Delorme and Polich, 2010; Berkovich-Ohana, Glicksohn, and Goldstein, 2012). I n particular Lutz et al., (2004) reported levels of synchronized gamma activity during medita tion in advanced Buddhist meditators that were the highest known to be recorded in the a bsence of pathology.

"A unified theory of hypnosis and meditation states: the interoceptive predictive coding approach"

In a sample of n = 38 meditators and n = 38 non-meditators, meditators showed longer duration of subjective nowness. This effect was associated with individual mindfulness levels. It is concluded that the subjective now can be longer for meditators than for non-meditators, and individual levels of mindfulness may convey this effect.

"How long is now for mindfulness meditators?"

Visual cortex as experiential model of attention

If you have questions, perspective, doubt, or a simple longing for general camraderie, you can communicate with me directly by emailing robert at 99theses dot com. Don't hesitate. Your correspondence is personally enriching.