The Natural Method of Learning

Photo by Alex Knight from Pexels

Section One. The Natural Method of Learning

The natural method of learning was discovered at the end of the nineteenth century. Since then it has been rediscovered many times yet it never attracted the interest of mainstream science. It performs computations with “topological constructive objects” instead of numbers. Therefore, it’s hard to embrace it as technology. Yet there seems to be no shortcut to building artificial general intelligence but for understanding the method used by nature. Let’s begin by scrutinizing the natural method’s discovery and rediscoveries in order to grasp its essence.

References:

1. V. A. Uspenski, A. L. Semyonov (1993) Kolmogorov’s Algorithms or Machines

Chapter One. Cat Scientists

In this chapter, we discuss how Pavlov’s dogs and Thorndike’s cats defined the mainstream of psychology, neuroscience, and artificial intelligence for the Twentieth century and defied the discovery of the method of understanding.

An animal and a scientist learn to understand the world around them with no need for reinforcement. The first Russian Nobel laureate Ivan Pavlov arrived at such a conclusion in 1933 and shared it in a paper that wasn’t published for 40 years after it was written.

The reason why that paper was neglected even by Pavlov’s most committed pupils was that it declared a blatant heresy against the preaching of their worshipped teacher, Ivan Petrovich Pavlov himself.

One can’t avoid seeing a deep irony in the fact that in the same year when Pavlov suggested that there was another way of learning, in 1933 the two most influential American scientists in the area of psychology of learning, Edward Thorndike and B. F. Skinner both adopted Pavlov’s term ‘reinforcement’ to describe the major underlying process of any learning. The final drop of irony was added by Pavlov citing Thorndike’s puzzle-box experiments with cats as evidence supporting the idea that animals and humans learn by using the method of science i.e. by connecting external objects in the environment without direct interaction with them.

Pavlov coined the term reinforcement in 1903 to describe the establishment and strengthening of the connection (association) between unconditioned and conditioned stimuli. In his renowned experiments on dogs, he simultaneously gave food to a dog (unconditioned stimulus) and rang a bell (stimulus he wanted to condition). Sometimes he only rang the bell.

He measured the secretion of gastric juice as the dog’s response to stimuli. When the dog had begun to secrete gastric juice in response only to the bell the connection was considered established successfully. However, if the bell wasn’t reinforced periodically by food, the connection fell apart.

We can think about food in this experiment as a stimulus when seen and sniffed and also as a reward when eaten with a sensation of pleasure. Will the connection between conditional and unconditional stimuli be established if dogs in Pavlov’s experiments when a bell is ringing just will see food but not actually eat it?

Edward Thorndike was locking cats in boxes with several levers, buttons, and bars inside. An animal could open the box’s door by either pulling a lever or pushing a button or pressing a bar. There was only one proper action that was leading to a successful escape.

Containment in a cage was an unpleasant stimulus that invoked a cat’s response. When the cat by sheer accident, as Thorndike believed, performed the proper action for the first time, the animal experienced the pleasure of freedom as a reward shortly afterward.

However, the connection between a stimulus and a response was established before but not in parallel or after the reinforcing sensation of pleasure. Thorndike called that behavior of cats finding a new association between a stimulus and a response “the method of trial and error, with accidental success.”

Pavlov defined it as an association an animal establishes between external stimuli but not in relation to itself.

Thorndike focused on the process of reinforcement that, in his opinion, took place each time the animal was rewarded shortly after the repetition of a successful response.

Pavlov focused on the fundamental differences between reinforcement and impulses governing the trial and error method of Thorndike’s cats. In trial and error “the formation of an association, on the one hand, and the maintenance of the necessary motor activity of the animal, as well as the necessary tone of the cortex, on the other, are separated from each other,” he wrote pointing out that in the process of reinforcement “they are fused.”

Thorndike of 1898 believed that “the method of trial and error, with accidental success,” observed in his experiments had been antiquated and replaced by faster and better ways of learning in humans.

He even suggested that “this animal-like method of learning” might explain “the slow progress of primitive man, the long time between stone age and iron age, for instance.”

Pavlov of 1933 had another view. “In Thorndike’s experiments, the animal becomes familiar with the relations of external things among themselves, with their connections.” He wrote. “Therefore, it is the knowledge of the world. This is the embryo, the germ of science.”

Pavlov clearly stated that the only difference between the way of learning demonstrated by Torndike’s cats and the scientific method was in the fact that science was more and more relying on old already existing knowledge in its search for new.

“I would not advocate this animal-like method of learning in place of the later ones unless it does the same work better.” Thorndike wrote.

Thirty-five years later Pavlov found the most pivotal learning work that only the animal-like method discovered by Thorndike could do. “Each new association of external things is the addition of knowledge, and the use of this knowledge is what we call understanding. It is impossible to imagine understanding something in any other way. How can one understand something without knowing different associations, i.e. connections of external objects!” He wrote.

Thorndike suggested in his 1989 paper that the method of learning discovered by him will be useful for understanding how children learn. Jean Piaget and Jerome Brunner proved that he was right by introducing the same method as the way how babies learn.

He also predicted that the method will have an impact on anthropology. Claude Levi-Strauss, the father of modern anthropology, rediscovered that method as ‘the science of concrete.’

However, as in the case of Pavlov, only reinforcement-based methods survived from his discoveries and were widely adopted in mainstream psychology, neuroscience, and later artificial intelligence.

That was another irony that, probably, the most important discoveries of Pavlov and Thorndike didn’t survive but the method of animal-like learning was since rediscovered many times. One of the most accurate descriptions of that method was left to us by Nobel laureates in physics Max Planck and Albert Einstein.

Planck described the general framework of how children and scientists create and update their picture of the world. Einstein described his own way of thinking. We’ll discuss their findings in the next chapter.

Summary

The approach of Thorndike, Skinner, and early Pavlov has been prevailing in mainstream psychology and neuroscience for more than a century. However, the brightest scientific minds in other fields kept rediscovering that ‘animal-like method of learning’ throughout the Twentieth century. The quest for artificial general intelligence should reinvigorate the interest in the method of learning practiced in the clearest form by animals, savages, babies, and scientists because only this method can lead to understanding — the underlying quality of any intelligence.

References:

1. Ivan Pavlov (1933) Psychology as a Science. Unpublished and Little-known Materials of I.P. Pavlov (1975)
2. Edward Thorndike (1898) Animal intelligence: An experimental study of the associative processes in animals. Monograph Supplement №8
3. B.F. Skinner (1933) The rate of establishment of a discrimination. Journal of General Psychology.
4. Edward Thorndike (1933) A theory of the action of the after-effects of a connection upon it. Psychological Review
5. Ivan Pavlov (1903) The Experimental Psychology and Psychopathology of Animals
6. Claude Levi-Strauss (1962) The Savage Mind
7. Jean Piaget (1947) The Psychology of Intelligence
8. Jerome Brunner (1964) The Course of Cognitive Growth
9. Max Plank (1942) The Meaning and Limits of Exact Science
10. Jacques S. Hadamard (1954) Albert Einstein’s testimonial for An Essay on the Psychology of Invention in the Mathematical Field

Chapter Two. Real Scientists

In this chapter, the man who invented quantum physics defines the way of thinking of the man who coined the theory of relativity. With some help from a prominent linguist and Aristotle, we are extending this definition to another way of learning.

As described in the previous chapter, humans and other animals can be trained with reinforcement to recognize regularities known to others. Regularities learned this way should be regularly reinforced. Furthermore, it’s impossible to learn to understand a new regularity this way, because understanding requires the “animal-like method of trial and error, with accidental success,” discovered by Thorndike but understood as the primary method of science only by Pavlov 35 years later. It is another way.

Another way of learning is the way of discovery of regularities anew. Some would even claim that we are rather inventing or creating than discovering new regularities because those regularities don’t exist in the reality given to us in our sensations before we indeed create them in our minds and prove by trial and error that they objectively work. According to Max Planck, babies and true scientists practice another way of learning in its most pure form.

Indeed it’s impossible to discover anything new in modern science without the existing knowledge accumulated in the relevant field but such knowledge is a supportive foundation but not the cutting edge of the process of scientific discovery. Let Albert Einstein explain to us how another way of learning works.

Einstein described such a process as a “combinatory play” with “visual and motor” “physical entities” that continues until “the mentioned associative play is sufficiently established and can be reproduced at will.”

Only after the establishment of a new regularity through the play of rearranging images “conventional words or other signs have to be sought for laboriously,” according to Einstein.

Interestingly, unambiguous definitions of Aristotelian logic were too always preceded by metaphors which, according to Aristotle, allowed people to understand unknown things through their resemblance with things which they understood. Furthermore, metaphor, according to Aristotle, “makes people see things” by presenting them “as in a state of activity.”

So what’s the connection between Einstein’s sensory-motor way of thinking and Aristotelian view on metaphors if usually, we think of metaphor as something purely linguistic?

However, from the modern theory of conceptual metaphor, introduced by George Lakoff, we learn that “the metaphor is not just a matter of language, but of thought and reason. The language is secondary. The mapping is primary.” Conceptual metaphor, according to Lakoff, is the ontological mapping of one concept to another.

In its turn, an object, a building block of the objective world around us is just “a synthesis of different sensory impressions achieved with the help of the unifying concept of a thing,” as Max Planck put it. In other words, it means that a thing itself is a concept. If so then Einstein’s play with “physical entities” is nothing else but an exercise of mapping of a new emerging concept to existing nonverbal concepts.

The reason why Einstein was playing with “physical things” in his mind is also very well explained by Max Planck. The task of science, according to him, is to introduce “order and regularity into the wealth of heterogeneous experiences conveyed by the various fields of the sense world.”

The “sense world” of Plank consists of “the impressions we receive in life from the outside world directly through our sense organs, the eyes, ears, etc.” Therefore, the “sense world” supplies science with “absolutely the most certain” facts about the outside world.

“Under closer examination, this task (of introducing order and regularity) proves to be fully consistent with the task which we are habitually performing in our lives ever since our earliest infancy, in order to find our way and place in our environment,” Planck continues re-establishing the connection between the scientific way of thinking of people like Albert Einstein and the conventional empiricism of all people.

Summary

Objects surrounding us in the ‘real’ world of our sensations are nothing but concepts unifying different sensations. Conceptual metaphors i.e. regularities in relations between these concepts are established with another way of learning. Such regularities are the laws of Nature, which true scientists, babies, and curious people alike are seeking to invent (discovering) all their lives.

References:

1. Edward Thorndike (1898) Animal intelligence: An experimental study of the associative processes in animals. Monograph Supplement №8
2. Max Plank, The Meaning and Limits of Exact Science
3. Jacques S. Hadamard, Albert Einstein’s testimonial for An Essay on the Psychology of Invention in the Mathematical Field
4. Aristotle, Rhetoric
5. George Lakoff, The Contemporary Theory of Metaphor

Chapter Three. Infant Scientists

In this chapter modern researchers in the area of child development provide experimental evidence supporting the genius insight of Max Planck about the motivation and the method of learning of the tiny scientists, human infants.

“What, then, does the child think as he makes these discoveries? First of all, he wonders. This feeling of wonderment is the source and inexhaustible fountain-head of his desire for knowledge. It drives the child irresistibly on to solve the mystery, and if in his attempt he encounters a causal relationship, he will not tire of repeating the same experiment ten times, a hundred times, in order to taste the thrill of discovery over and over again. Thus, by a process of incessant labor from day to day, the child eventually develops his world picture, to the degree needed by him in practical life,” Max Planck wrote so temperamentally about the motivation of the child to learn because he experienced the same wonderment, excitement, and thrill of discovery in spite of the fact that he had grown up long ago. He was a scientist.

Of course, Planck’s considerations about a child’s motivation were purely speculative, but it was speculation of one of the brightest scientific minds in the entire history of humankind. It is not a surprise therefore that observations of the behavior of toddlers in some cleverly designed experiments by modern scientists entirely support the views of Planck.

According to researchers from Johns Hopkins University, Aimee E. Stahl and Lisa Feigenson infants begin to use the concept of an object in their first year. Furthermore, eleven months old toddlers are already familiar with the core regularities governing the behavior of objects. They have their objective picture of the world that they share with other humans and animals.

When toddlers observe something that looks like violations of those core regularities they begin to explore the violating object by elaborating and testing hypotheses like real scientists. Object’s behavior that makes toddlers wonder attracts their attention and induces learning while they pay no attention to objects which behave in a regular way.

For instance, if it seems that a toy car drove through a solid wall a toddler will probe the solidity of both the car and the wall by dropping the car and pushing the wall. Maybe either the car or the wall, or both were not solid this time?

Stahl and Feigenson support the idea that infants inherit the core knowledge such as the concept of object and regularities rather than acquire them at the earliest stage of development as Planck proposed. Planck’s view supported by the concept of sensorimotor intelligence by Jean Piaget appears to be more consistent than the introduction of the factor of inheritance. Anyway, the findings of Stahl and Feugenson provide strong supportive evidence to the idea of Plank that wonderment drives learning in young children.

Another team of researchers, Paul Muentener from Tufts University and Laura Schultz from Massachusetts Institute of Technology also demonstrated in their experiments that toddlers actively search for hidden causes of spontaneously occurring events but don’t pay attention to events with observable or clearly demonstrated causes.

According to Muentener and Shultz, two years olds infer that spontaneous events have hidden causes, hypothesize about cause candidates, and test their hypotheses in practice.

In their further research scientists found out that toddlers infer not only certain hidden causes of spontaneously occurring events but also causes which induce such events with only probable regularity.

Those findings suggest that toddlers have a dynamic rather than a static picture of the objective world. The probabilistic approach enables toddlers to dynamically update their picture of the world whenever its regularities change.

Summary

Infants are driven to learn only by observations that make them wonder. They don’t pay attention to already known regularities although they can readily use them to attain known results. Stochasticity (or chaos) doesn’t prevent them from inferring causes to events that occur seemingly spontaneously. That suggests that they invent core regularities of their world-picture by themselves in the process of animal-like learning discovered by Edward Thorndike.

References:

1. Max Plank (1942) The Meaning and Limits of Exact Science
2. Aimee E. Stahl, Lisa Feigenson (2018) Violations of Core Knowledge Shape Early Learning
3. Aimee E. Stahl, Lisa Feigenson (2015) Observing the unexpected enhances infants’ learning and exploration
4. Smithsonian Magazine (2015) Like Tiny Scientists, Babies Learn Best By Focusing on Surprising Objects
5. Paul Muentener, Laura Schultz (2014) Toddlers infer unobserved causes for spontaneous events
6. Yang Wu, Paul Muentener, Laura Schultz (2015) The Invisible Hand: Toddlers Connect Probabilistic Events With Agentive Causes
7. Edward Thorndike (1898) Animal intelligence: An experimental study of the associative processes in animals. Monograph Supplement №8

Chapter Four. Savage Scientists

In this chapter, we will see how savage tribes before their close contact with modern humans as well as our prehistoric ancestors were using empiricism and logic to build and keep up to date their picture of the world — the gigantic foundation of modern science.

Claude Levi-Strauss, a French social anthropologist often known as “the father of modern anthropology”, in his book The Savage Mind describes in great detail the ways how savage tribes meticulously classify the world around them. With his deep insight derived from observations of the savage life and the structural analysis of ancient myths, he draws a picture of the overwhelming scientific project undertaken by our distant ancestors in learning from nature around them.

Like cat scientists in the experiments of Edward Thorndike, savage scientists in the descriptions cited by Levi-Strauss were relentlessly classifying external objects through relationships between them.

“Any classification is superior to chaos and even a classification at the level of sensible properties is a step towards rational ordering. It is legitimate, in classifying fruits into relatively heavy and relatively light, to begin by separating the apples from the pears even though shape, colour and taste are unconnected with weight and volume.”

“This preoccupation with exhaustive observation and the systematic cataloguing of relations and connections can sometimes lead to scientifically valid results.” He wrote. “The Blackfoot Indians for instance were able to prognosticate the approach of spring by the state of development of the foetus of bison which they took from the uterus of females killed in hunting. These successes cannot of course be isolated from the numerous other associations of the same kind which science condemns as illusory.”

Indeed, savage scientists established many illusory connections between external objects. They used their knowledge and imagination to find the best possible candidate for the cause of the effect that made them wonder in the same way as infant scientists were doing in the experiments described in the previous chapter.

Levi-Strauss made his best in trying to establish a clear distinction between magic practiced by primitive cultures and modern science but he finally arrived at the conclusion that learning directly from nature, as savage scientists did, laid at the core of modern science as well. However, as Albert Einstein described it, modern scientists play with illusory associations introspectively in their minds until they arrive at a regularity that satisfies them and can be logically explained.

Interestingly, John Maynard Keynes, better known for his ideas that fundamentally changed the theory and practice of macroeconomics and the economic policies of governments, in his essay Newton, the Man named Newton “the last of the magicians.”

“Anyone who has ever attempted pure scientific or philosophical thought knows how one can hold a problem momentarily in one’s mind and apply all one’s powers of concentration to piercing through it, and how it will dissolve and escape and you find that what you are surveying is a blank. I believe that Newton could hold a problem in his mind for hours and days and weeks until it surrendered to him its secret. Then being a supreme mathematical technician he could dress it up, how you will, for purposes of exposition.” He described the magical thinking process of Newton in a way very much resembling the words of Einstein.

Magic isn’t proto-science, it’s “a metaphorical expression” of science, as Levi-Strauss put it. “Both science and magic however require the same sort of mental operations and they differ not so much in kind as in the different types of phenomena to which they are applied.”

Summary

Savage scientists practice learning directly from nature in the same way as cat scientists, infant scientists, and real scientists do. They apply empiricism and logic to everything that makes them wonder to derive order out of chaos.

References

1. Claude Levi-Strauss (1962) The Savage Mind
2. Edward Thorndike (1898) Animal intelligence: An experimental study of the associative processes in animals. Monograph Supplement №8
3. Jacques S. Hadamard, Albert Einstein’s testimonial for An Essay on the Psychology of Invention in the Mathematical Field
4. John Maynard Keynes (1942) Newton, the Man

Chapter Five. Objective Reality

In this Chapter, a physicist Max Plank and a psychologist Jean Piaget explain that the only objective reality accessible to us is a conceptual space created by the power of our intelligence.

Max Planck proposed to use investigating “the most primitive world picture, the naive world picture of the child” as “the best start toward a correct understanding of the scientific world picture.” His description of the way the child creates his world picture is so accurate that I won’t even try to rephrase him. I’ll just point your attention to the similarity between Plank’s description and Pavlov’s notion about the scientific (or animal-like as per Thorndike) method of learning. The child learns by classifying and connecting external stimuli with each other. The scientist learns in the same way. So did Thorndike’s cats.

“As soon as the child begins to think, he begins to form his world picture. For this purpose, he directs his attention toward the impressions which he receives through his sense organs. He tries to classify them, and in this endeavor he makes all kinds of discoveries, such as, for instance, that there is a certain orderly interrelation between the inherently different impressions conveyed by the senses of sight, touch and hearing. If you give the child a toy, let us say, a rattle, he will find that the tactile sensation is always accompanied by a corresponding visual sensation, and as he moves the rattle back and forth, he also perceives a certain regular auditory sensation.”

Planck clearly depicted the way how the structure of the child’s world picture changes in the process of learning.

Sensations used to be the only components of the original sense world of the child but eventually, the concept of an object emerges. The object becomes the dominant element of the new picture, and the tactile, visual, and auditory sensations become its attributes.

Yet the sensations associated with objects are private and vary from one individual to another. The transition from the sense world to the objective world becomes complete only after “the concept of an objectively valid regularity” is introduced. Objectively valid regularities make the world of objects truly objective — the same for all humans.

The concept of an object plays a key role also in the theory of Jean Piaget on the development of understanding in children. Piaget, a Swiss psychologist who is thought by many to have been the major figure in 20th-century developmental psychology, proposed that the emergence of understanding in the child begins with the stage of sensory-motor intelligence that operates in the sense world as per the explanation of Plank.

“A world without objects is a universe without systematic differentiation between subjective and external realities,” he wrote resonating with Plank’s ideas. A world picture embracing concepts of an object and space, according to him, makes possible the advancement of the child’s thought to the level of conceptual or operational intelligence.

Furthermore, the objective world picture of the child expands spatially to reach out into spaces that were never seen before or even can’t be seen at all. The objective world is getting populated not only with “real entities” of sensory-motor intelligence but also with imaginary ones. Imaginary objects become objective because they obey the regularities governing the objective world, but not because their reality is given to us in sensations.

“Certainly, sensori-motor intelligence lies at the source of thought, and continues to affect it throughout life through perceptions and practical sets,” wrote Piaget.

He proposed to distinguish practical and deliberate types of intelligence. “In the first the question appears in the form of a simple need, the hypothesis as a sensori-motor random trial, and the testing process as a mere series of failures or successes. In the second, the need is reflected in the question, trial-and-error is internalised as a search for hypotheses and the testing process anticipates the sanction of experience by means of an “awareness of relations”, which is sufficient to discard false hypotheses and to retain true ones.”

As Thorndike before, he failed to see that the practical type of intelligence (the natural way of learning) works in the organic cohesion with the deliberate one as it was described by Einstein in the previous Chapter. The father of modern Anthropology Claude Levi-Strauss also made the same mistake by distinguishing “the science of the concrete” of primitive people and modern science as two entirely separate ways of attacking the same problems. I guess it was a kind of anthropocentric blindness that prevented them from seeing the intrinsic value of the animal-like learning method to all other methods of learning and, especially, creativity.

Yet Piaget managed to establish the equivalence between his concept of “assimilatory schemata” and the concept of cognitive maps, proposed by American psychologist Edward Tolman. “This double property of general validity and meaning belonging to the structures considered by Tolman is a fairly good indication that he is concerned with what we call assimilatory schemata,” he wrote, paving us the way to conceptual spaces that underlie the natural method of learning.

The Swedish philosopher and cognitive scientist Peter Gärdenfors introduced the theory of conceptual spaces as a framework for knowledge representation “that is based on using geometrical structures rather than symbols or connections between neurons.” Later he linked his theory to the neurophysiological research in the area of cognitive maps represented in the brain by the system of grid cells and place cells. In the joint paper with Jacob Bellmund and Christian Doeller from the Max Planck Institute for Human Cognitive and Brain Sciences, and Edvard Mozer who was awarded the Nobel Prize in 2014 for the discovery of that system they wrote, “we propose cognitive spaces as a primary representational format for information processing in the brain.”

However, they stopped one step short from the idea of Max Planck himself that the objective reality as we see, hear, feel, smell, and taste it is itself a conceptual space. Albeit it is the most fundamental, the most exact, and the most important conceptual space of all.

Summary

Humans create, update, and navigate the objective reality as a space of concepts that either integrate different sensations into concrete objects or create abstract concepts with the power of imagination. If a concept obeys the laws of nature known to us, it belongs to the objective reality no matter if it’s sensual or imaginary. Cats, scientists, babies, and savages play with imaginary object concepts in the conceptual space of objective reality in order to invent new “objectively valid regularities.”

References:

1. Max Plank (1942) The Meaning and Limits of Exact Science
2. Ivan Pavlov (1933) Psychology as a Science. Unpublished and Little-known Materials of I.P. Pavlov (1975)
3. Jean Piaget (1960) The Psychology of Intelligence
4. Edward Thorndike (1898) Animal intelligence: An experimental study of the associative processes in animals. Monograph Supplement №8
5. Claude Levi-Strauss (1962) The Savage Mind
6. Edward Tolman (1948) Cognitive Maps in Rats and Men
7. Peter Gärdenfors (2004) Conceptual Spaces as a Framework for Knowledge Representation
8. Jacob L. S. Bellmund, Peter Gärdenfors, Edvard I. Moser, Christian F. Doeller (2018) Navigating cognition: Spatial codes for human thinking

Science learns from nature. Infant scientists do the same. Many scientists consider natural learning slow and obsolete but in infants, it’s really fast. It’s the foundation of all artificial learning. Can we stay human without it? Can we build a near-human AI without it?