Transient Chaos Unites Narratives, Hippocampus and Deep Learning
Good stories are attractive because they feed hippocampus with chaos. Hippocampus employs chaos to recollect memories of who and what we are. Deep neural networks can model chaos in hippocampus. Chaos improves performance of deep neural networks.
Narratives Are Powered by Chaos
From a poet and philosopher of early romanticism to modern psychologists, narrative scholars have been pointing out that a link exists between narratives and chaos. Let’s have a look at some examples.
Novalis, an early German Romantic poet and theorist who greatly influenced later Romantic thought, wrote about transient chaos centuries before scientists coined the term.
“The world of fairy tales is the absolutely opposite world to the world of truth (history) — and for this reason so remarkably similar to it — as chaos is to completed creation. (On idylls). In the world of the future everything is just as it is in the former world — and yet everything is utterly different. The world of the future is rational chaos — chaos suffused with itself — inside and outside of itself — chaos or ∞,” Novalis explained his view.
“Interest in narrative within the human sciences is comparable to interest in chaos within the natural sciences. In their respective ways, theories on narrative and theories on chaos are aimed at appreciating the dynamics of complex, multi-dimensional systems which otherwise resist our attempts to predict, measure, and control them,” William L. Randall, professor of Gerontology at St. Thomas University, wrote. His first career was as a minister for 11 years with the United Church of Canada. A graduate of Harvard University, Princeton Seminary, and the University of Toronto, he has helped to pioneer an approach to the study of aging known as narrative gerontology.
Jerome Bruner was the American psychologist and educator who developed theories on perception, learning, memory, and other aspects of cognition in young children that had a strong influence on the American educational system and helped launch the field of cognitive psychology. Bruner in his words reiterated the thought of Novalis about chaos under veil of order, “Stories render the unexpected less surprising, less uncanny: they domesticate unexpectedness, give it a sheen of ordinariness.”
Both Bruner and Randall supported the idea that people tell and retell their lives as narratives.
“We seem to have no other way of describing “lived time” save in the form of a narrative, Bruner stated, “the mimesis between life so-called and narrative is a two-way affair: that is to say, just as art imitates life in Aristotle’s sense, so, in Oscar Wilde’s, life imitates art. Narrative imitates life, life imitates narrative. “Life” in this sense is the same kind of construction of the human imagination as “a narrative” is. It is constructed by human beings through active ratiocination, by the same kind of ratiocination through which we construct narratives.”
“Our sense of self is dependent upon our memories, yet what we remember is dependent on the self, or selves, for whom it holds meaning and is judged worthy of retaining. The sorts of memories that are at issue here — “episodic” memories or “autobiographical” memories — invariably have a narrative dimension They take the form of little narratives that we tell ourselves, and others, about events in our lives that for some reason we hang onto as pertinent to our identity. In short, such memories are stories, and taken together, they constitute “the story of my life”: the internalized, evolving myth that rumbles through our minds as inseparable from who we see ourselves to be,” Randall wrote.
Hippocampus Encodes and Recollects Narratives
Hippocampus is an extension of cerebral cortex situated into temporal lobe area of the brain. It is a vulnerable and plastic structure that gets affected by a variety of stimuli. It is one of the markers of cognitive decline and diagnosis of Alzheimer’s disease. Drugs that are able to cause slow down of hippocampal atrophy or reversal are actively being sought.
Hippocampus plays a pivotal role in memory and learning. In particular it encodes and recollects our autobiographical memories in the form of episodes hence the name episodic memory. As Randall put it, such episodes are stories, which taken together constitute the narrative of our life.
“The last decade has seen dramatic technological and conceptual changes in research on episodic memory and the brain. New technologies, and increased use of more naturalistic observations, have enabled investigators to delve deeply into the structures that mediate episodic memory, particularly the hippocampus, and to track functional and structural interactions among brain regions that support it. Conceptually, episodic memory is increasingly being viewed as subject to lifelong transformations that are reflected in the neural substrates that mediate it. In keeping with this dynamic perspective, research on episodic memory (and the hippocampus) has infiltrated domains, from perception to language and from empathy to problem solving, that were once considered outside its boundaries,” a group of researchers from the University of Toronto, Duke University and the University of Arizona summed up in their paper in 2017.
The recent progress in neuroscience allowed researchers to observe with fMRI imaging how narratives are dynamically encoded and recollected by the human hippocampus. Branka Milojevic et al. shared their findings, “Narratives also provide a more general context, unrestricted by space and time, that can be used to organize memories into networks of related events. For this reason, narratives are ideally suited to engage neural mechanisms underlying episodic memory formation. In this study, participants watched a movie with two interleaved narratives while their brain activity was monitored using fMRI. We show that the hippocampus, which is involved in formation of spatiotemporal contexts in episodic memory, also represents gradually diverging narrative contexts as well as narrative elements, such as people and locations.”
Hippocampus Uses Chaos to Encode and Recollect Narratives
Episodic memory is dynamic, with significant transformations occurring throughout the memory’s lifetime. The mechanism of its dynamic encoding and recollection remains unclear. A decade ago three groups of researchers from Japan proposed their models of the chaotic nature of hippocampal encoding and recollection of episodic memory.
“We construct a mathematical model for the dynamic behavior of hippocampus. The model is described by the skew product transformation in terms of chaotic dynamics and contracting dynamics. In the contracting subspace, fractal objects are generated. We show that such fractal objects are characterized by a code of a temporal sequence generated by chaotic dynamics,” Ichiro Tsuda et al. wrote in their paper. Tsuda also presented a hypothesis “that the hippocampus plays a role in the formation of self-identity via interactions with the lateral prefrontal cortex.”
“In this paper, we have proposed the chaotic complex valued associative memory which is able to recall the multi-value patterns dynamically. The proposed model is based on the complex-valued associative memory and the chaotic associative memory, and the proposed model is constructed of the chaotic complex-valued neuron models,” Masao Nakada and Yuko Osana wrote in their symposium paper.
“To realize mutual association function, we propose a hippocampus-neocortex model with multi-layered chaotic neural network,” Takashi Kuremoto et al. stated in their paper.
Deep Learning Learns Chaos from Brain
Recently the interest in chaos began to grow in the area of deep learning. Researchers in AI started to point out that chaos may both explain success of deep neural networks and further improve their performance.
“We find that the transient, finite depth evolution in the chaotic regime underlies the origins of exponential expressivity in deep random networks,” Ben Poole et al. from Stanford and Google Brain concluded in their paper “Exponential expressivity in deep neural networks through transient chaos” published in 2016.
“Chaos theory can be used to demonstrate the irregular activities in the brain. Although chaotic system has unpredictable long-term behavior. Instead of disorder, chaos can be generated by simple deterministic system. Chaotic attractor becomes attracted and has small disturbance stability. This feature makes the chaos system have a certain ability to remember as the human brain does. And this is also just the way artificial neural network does,” Guoming Chen et al. from Guangdong universities elaborated in their paper published in 2016, “A large number of biological experiments have shown that the brain systems have bifurcations, chaos and strange attractor dynamics behavior. However, is there any similar kinetic behavior in neural network as some kind of nonlinear system? This motivates us to integrating the chaotic maps to the network.”
“Generally, one of the methods of utilizing chaos in deep learning is to use them instead of random values. In other words, chaotic maps substitute random values to provide chaotic behaviors for deep learning algorithms. So, we employ chaotic maps whenever there is a need for a random value,” Guoming Chen et al. reported, “This work integrates chaotic into the recently proposed multilayer neural network algorithm called deep learning. The effects of different chaotic on improving the performance of deep learning are investigated. The results demonstrate that chaotic maps are able to improve the performance of deep learning.”
Rose Yu et al. from University of Southern California and California Institute of Technology in their paper published in 2017 presented “Tensor-RNN, a novel RNN architecture for multivariate forecasting in chaotic dynamical systems.” They wrote, “Our proposed architecture captures highly nonlinear dynamic behavior by using high-order Markov states and transition functions.”
“In a work of art, chaos must shimmer through the veil of order.”
― Novalis
