Thinking Spaces

This is a very brief outline of the decades long story of struggle for understanding of the true fundamental structure of human intelligence - the geometry of human thought. Recently, with the rise of the AI technology, the stakes in this struggle became unprecedentedly high. If the scientist who predicted cognitive maps “in rats and men” more than 70 years ago was right any artificial general intelligence (AGI) developed from current golden standard models in machine learning will inevitably become evil.

The first Russian Nobel Prize winner, physiologist Ivan Pavlov discovered classical conditioning and thus provided a major impact in psychology specifically on the development of behaviorism. Ironically, closer to the end of his life Pavlov arrived to the conclusion that stimulus-response is not the only (and maybe even not the most important) mechanism that directs animal behavior.

When Pavlov learned about experiments of Edward Thorndike with cats in puzzle boxes he wrote in his paper Psychology as a Science, “In experiments with conditioned stimuli, the animal determines the relationship of individual environmental objects to itself. In Thorndike’s experiments, the animal becomes acquainted with the relation of external things among themselves, with their connections. Therefore, it is the knowledge of the world. This is the embryo, the germ of science.”

Probably, it was the first clear formulation of the idea of mental representations. When Pavlov’s own students first time heard the new idea of their master, they thought that they heard him wrong and turned to a stenographer to check with her records. When they discovered the same heresy in notes, they accused the stenographer of making mistakes.

The paper cited above was written by Pavlov in 1933 (three years before his death) but it was published for the first time only in 1975 in a collection of Unpublished and Little-known Materials of I.P.Pavlov.

It took 15 years from Pavlov’s idea before Edward Tolman coined the term cognitive map to define mental representations, “We believe that in the course of learning, something like a field map of the environment gets established in the rat’s brain . . . The stimuli . . . are usually worked over . . . into a tentative, cognitive-like map of the environment. And it is this tentative map, indicating routes and paths and environmental relationships, which finally determines what responses, if any, the animal will finally release.”

Tolman arrived at the above conclusion following a series of very clever experiments with rats in labyrinths. In his landmark paper Cognitive Maps in Rats and Men he also proposed that cognitive maps in humans exist not only to position them in physical space, but within a broader network of casual, social and emotional relationships.

The idea that cognitive maps have a broader area of application than just navigation in physical spaces coupled together with the lack at that time of any technique to identify a cognitive map or its components led to Tollman’s theory falling out of use under heavy critique from behaviorists, led by Clark Hull.

When decades later John O’Keefe observed behavior of certain neurons in rodent hippocampus that resembled him a process of formation of a cognitive map he got passion and persistence to dig deeper because “possessing such a map would surely furnish an animal with a very handy cognitive device enabling it to move around environments in a creative, flexible way rather than being reduced to the simple rigid stimulus-response routes envisaged by the behaviourists,” as he put it in his Nobel Prize lecture decades later.

O’Keefe and Lynn Nadel discovered place cells in hippocampus in support of the theory of Tolman. It their landmark book Hippocampus as a Cognitive Map published in 1978, thirty years after the paper by Tolman, they also extended cognitive-mapping system in humans beyond navigation in physical space and proposed that it could function as a deep structure for language.

In 2014 O’Keefe together with May-Britt and Edvard Moser was awarded a Nobel Prize for the discovery of the GPS system of the brain in rodents. In his Nobel Prize lecture O’Keefe stressed out that “a similar spatial system exists in humans which additionally provides the
basis for human episodic memory.”

Moser who together with his wife discovered grid cells continues to explore the application of cognitive maps in humans beyond the physical space. In 2018 he teamed up with a multidisciplinary group of scientists to offer a new proposal—Humans think using their brain’s navigation system.

“By connecting all these previous discoveries, we came to the assumption that the brain stores a mental map, regardless of whether we are thinking about a real space or the space between dimensions of our thoughts. Our train of thought can be considered a path through the spaces of our thoughts, along different mental dimensions,” Jacob Bellmund, the first author of the publication, explains.

This piece gives a good overview of the geometry of thought theory by Peter Gärdenfors, another co-author of the above paper.

“In his book Conceptual Spaces, from 2000, he wrote, “It has long been a common prejudice in cognitive science that the brain is either a Turing machine working with symbols or a connectionist system using neural networks.” In Krakow, Gärdenfors pushed against that prejudice. In his talk, “The Geometry of Thinking,” he suggested that humans are able to do things that today’s powerful computers can’t do—like learn language quickly and generalize from particulars with ease (to see, in other words, without much training, that lions and tigers are four-legged felines)—because we, unlike our computers, represent information in geometrical space.”

“The common viewpoint that a neuron consists of a unique and centralized excitable element which sums all incoming signals was questioned through the proposed new types of experiments. A new realization for the computational scheme of a neuron was presented, indicating that a neuron consists of several independent threshold units. Each sub-cellular threshold unit sums the incoming signals from a given confined direction with its given threshold,” Israeli scientists reported in their recent paper on their discovery of direction dependent computations in a single neuron debunking a 100 year old viewpoint on the neuronal firing principle.

"We reached this conclusion using a new experimental setup, but in principle these results could have been discovered using technology that has existed since the 1980s," said lead researcher Ido Kanter, "the belief that has been rooted in the scientific world for 100 years resulted in this delay of several decades."

It may look like now the theory of human thinking envisaged by Pavlov, coined by Tolman and further developed and verified by their many followers is slowly but steadily approaching the tipping point of broad recognition but Clark Hull became the most cited psychologist of the XXth century in the XXIst century.

“Simple rigid stimulus-response routes” prove to be still firmly rooted in the scientific world. Furthermore, the deep learning and reinforcement learning methods which are rapidly expanding today in machine learning are also deeply rooted in the hullian behaviorism.

I don’t know if it will be necessary to redo the math underlying artificial neural networks to reflect the true, geometrical structure of human thinking. Current DNNs have walked a long way from their biologically inspired prototypes. Changing their underlying algorithms according to the new insights won’t, most probably, improve their performance. One thing, however, is obvious - if the theory of geometric thinking is right the level of AGI won’t be achieved on the basis of the existing models because they are sub-intelligent at the core.

The race for AGI with methods deeply rooted in hullian behaviorism will more and more resemble the misfortunate attempt of Dr. Frankenstein to create an artificial man. Given the proliferation of modern AI methods and the hype about their superhuman performance the consequences of their fundamental sub-intelligent nature may become much more devastating than in Frankenstein’s case.

Fortunately, Frankenstein’s AGI doesn’t exist yet. However, human general intelligence does exist. It’s overtraining with “simple rigid stimulus-response” methods has a significant negative impact, according to Tolman. As he put it in his landmark paper over 70 years ago, “Over and over again men are blinded by too violent motivations and too intense frustrations into blind and unintelligent and in the end desperately dangerous hates of outsiders. And the expression of these their displaced hates ranges all the way from discrimination against minorities to world conflagrations. What is the name of Heaven and Psychology can we do about it? My only answer is to preach again the virtues of reason-of, that is, broad cognitive maps.”

Now, seventy years later, we can do more than preach, can we? I hope, yes we can: Desire to Play. Scientists use video games to enhance hippocampus in humans.

The stakes are also rising: Dehumanization. Shrinking Brain and Hippocampus.

UPD 8:25 Moscow time: In a paper published today, a group of researchers led by Ido Kanter who discovered geometrically asynchronous spiking of neurons described above introduced a new method of machine learning that outperforms current state of the art methods.

Photo by David Cassolato from Pexels