Broad Speculations

The article is primarily about something called backpropagation as a technique for improving neural nets. At the The article says:

Models developed independently by Kording in 2001, and more recently by Blake Richards of McGill University and the Quebec Artificial Intelligence Institute and his colleagues, have shown that pyramidal neurons could form the basic units of a deep learning network by doing both forward and backward computations simultaneously. The key is in the separation of the signals entering the neuron for forward-going inference and for backward-flowing errors, which could be handled in the model by the basal and apical dendrites, respectively. Information for both signals can be encoded in the spikes of electrical activity that the neuron sends down its axon as an output”.

The issue is whether something like backpropagation is actually happening in brains, especial in the pyramidal neurons. Even the researchers discussed in the Quanta article think they still need to explain something. Unfortunately, it doesn’t seem likely the neurons are actually running backpropagation algorithms. ” For a variety of reasons, backpropagation isn’t compatible with the brain’s anatomy and physiology, particularly in the cortex”. So, they are looking for other techniques to accomplish the same.

Where have we seen pyramidal neurons before?

There is this paper that I wrote two posts about a while ago that states: “This perspective makes one quite specific prediction: cortical processing that does not include L5p neurons will be unconscious. More generally, the present perspective suggests that L5p neurons have a central role in the mechanisms underlying consciousness”.

There is also the Susan Pockett electromagnetic theory of consciousness with this diagram of pyramidal neurons in the layers of the brain.

Her theory: “The hypothesis proposed here is that one necessary (albeit clearly not sufficient) characteristic of conscious as opposed to non-conscious EM fields or patterns of charge is a spatial structure something like that shown… The essence of the proposal is that in the radial direction (perpendicular to the surface of the cortex) conscious fields will have a surface layer of negative charge above two deeper layers of positive charge, separated by a distinct neutral layer.”

Is the missing part of the explanation feedback generated by the brain electromagnetic field?

There is an entire post I wrote on learning and consciousness that argues, in part, for  Stephen Grossberg’s Adaptive Resonance Theory. I submitted a comment on the Quanta article:

Baars, Grossberg, and others have pointed out the role of consciousness in any sort of complex learning. Grossberg almost equates learning and consciousness and his description matches what this article is discussing.

“The processes whereby our brains continue to learn about a changing world in a stable fashion throughout life are proposed to lead to conscious experiences. These processes include the learning of top-down expectations, the matching of these expectations against bottom-up data, the focusing of attention upon the expected clusters of information, and the development of resonant states between bottom-up and top-down processes as they reach an attentive consensus between what is expected and what is there in the outside world. It is suggested that all conscious states in the brain are resonant states and that these resonant states trigger learning of sensory and cognitive representations.“

So the missing piece seems to be consciousness itself or exactly how it works.

To my surprise, Stephen Grossberg himself responded.

Thanks for noting that the authors seem not to be aware that many of their aspirations have been realized years ago through the work that I, with many gifted collaborators, have carried out during the past 40 years. See sites.bu.edu/steveg for downloadable lectures and articles that illustrate this progress. One problem with their models is that their foundational hypotheses are incompatible with thousands of known experimental facts about how our brains make our minds. A second problem is that the authors’ models do not explain any of these facts. In science, the theories that warrant attention are the ones that successfully explain and predict the most facts in a principled and testable way. The theories that I have been lucky enough to develop over the past 40 years past that test. Stephen Grossberg

Grossberg thinks backpropagation isn’t required as an explanation. He believes his Adaptive Resonance Theory solves all of the problems that backpropagation is trying to solve without entering into any problems that backpropagation creates. In one of his presentations he writes: “These efforts to overcome catastrophic forgetting created additional conceptual and computational problems I view them as adding EPICYCLES to ameliorate a fundamental flaw in the model reminiscent of adding epicycles to correct problems in the
Ptolemaic model of the solar system. The Copernican model that we now accept did not require epicycles”!