the power of sequential thinking

A short while ago I was mentioning to another computing academic at a meeting the curious fact that the computational power of the complete internet is now roughly similar to that of a single human brain [[see article here]]. While this little factoid is deliberatly provocative, I did not expect the strength of the response.

“that’s impossible” he said.

“why” I asked, “I’m not saying they are similar, just that there is the same computational potential”

“Computers are sequential” he said, “brains are associative”.

Further attempts to reason, likening it to other forms of simulation or emulation, simply met with the same flat response, a complete unwillingness to entertain the concept.

Partly this is to do with the feeling that this somehow diminishes us as people, what for me was a form of play with numbers, for him was perhaps an assault on his integrity as a human. I guess as a Christian I’m used to the idea that the importance of a person is not that we are clever or anything else, but that we are loved and chosen. So, I guess, for me this is less of an insult to my idea of being who I am.

This aside it is interesting that the reason given was about the mode of computation: “computers are sequential” vs. the massively parallel associativity of the human brain.

Of course if the computational substrate is all the PCs connected to the Intenet then this is hardly purely sequential and in fact one of the reasons that you could not ‘run’ a brain simulation on the Internet is that communication is too slow. Distributed computation over 100s of millions of PCs on the internet could not synchronise in the way that long-range synapses do within our brains.

Amongst other things it is suggested that our sense of consciousness is connected with the single track of synchronised activity enabled by the tight interconnections and rapid feedback loops within our brains1. In contrast, individual computers connected to the onternet compute far faster than they can communicate, there could be not single thread of attention switching at the rate that our minds can.

If the internet were to think it would be schizophrenic.

Sequence is also imprtant in other ways. As the man said, our brians are associative. When considering spreading activation mechanisms for intelligent internet interfaces, one of the problems is that associative stuff gets ‘mixed up’. If London has a high level of activation, why is that? In a designed computational framework it is possible to consider mutiple ‘flavours’ of activations spreading through a network of concepts, but our brains do not do this, so how do they mange to separate things.

Now to some extent they don’t – we get an overall feel for things, not seeing the world as little pieces. However, it is also important to be able to more or less accurately ascribe feelings and associations to things. Consider one of those FBI training ranges were bank terrorists and hostages pop out from behind windows or doors. Your aim is to shoot the terrorists and save the hostages. But, if you see a robber holding a hostage how do you manage to separate the ‘bad and kill’ feelings and properly ascribe them only to the terrorist and not the hostage.

The answer may well be due exactly to the switching of attention. Even with both terrorist and hostage are next to each other, as mental attention shifts momentarily to one and then the other, the mental associations also shift. Rodney Cotterill in Enchanted Looms describes two levels of attention switich2. One near conscious and taking around 500ms and one connected with more low-level visual attention (sometimes called a visual searchlight) at 20-50ms. It is probably the slower timescales that allow fuller webs of association to build and decay, but maybe there are other intermediate timescales of attention switching as well.

If this is right then the rapid sequential shifts of attention could be essential for maintaining the individual identity of percepts and concepts.

If we look at concepts on their own, another story of sequence unfolds.

There is a bit of a joke among neuroscientists about grandmother cells. This is the idea that there is a single neuron that in someway encodes or represents your grandmother3

Looking at this purely from a computing science perspective, even if there were not neurological reasons for looking for more distrubuted representations, there are computational ones. If concepts were stored in small local assemblies of neurons (not single ones to allow some redundancy and robsutness) and even a reasonably large part of our brains were dedicated to concept memory, then there just seems too few ‘concept-slots’.

If we used 100 neurons per concept and 10% of the brain for concept memory, we would only have space for around 10 million concepts. A quick scan through the dictionary suggests I have a reconition vocabuary of arounf 35,000 words, so that means I’d have less than 300 other concepts per dictinary word one. Taking into account memories of various kinds, it justs seems a little small. If we take into account the interconnections then we have plenty of potential long-term storage capacity (1/2 petabyte or so), but not if we try to use indiviudal groups of neorons to represent things. Gradmother cells are simpy an inefficient use of neurons!

Now there is also plenty of neurological evidence for more distributed storage. Walter Freeman describes how he and his team lovingly chopped the tops off rabbits’ skulls, embeded electrodes into their olfactory bulbs and then gently nursed them back to health4. The rabbits were then presented with different smells and each smell produced a distinctive pattern of neuron firings, but these patterns exteded across the bulb, not localised to a few neurons.

If neurons had ‘continuous’ levels of activation it would be possible to represent things like “1/2 think it is a dog 1/2 think it is a fox”, simply as an overlay of the activation of each. However, if this were the case, and one could have in mind any blend of concepts, then an assembly of N neurons would still only be able to encode up to N concepts as the concepts patterns would form a set of basis vectors for the N-dimensional vector space of possible activation levels (a bit of standard linear algebra).

In fact, neurons tend to behave non-linearly and in many areas there are patterns of inhibition as well as mutual excitement and disinhibition, leading to winner-takes-all effects. If this is true of the places where we represent concepts for short term memory, conscious attention, etc., then this means instead of representations that ‘add up’, we have each pattern potentially completely different, similar to the way binary numers are encoded in computer memory: 1010 is not a combination of 1000 and 0010 but completely different.

In principle this kind of representation allows 2^N (two to the power of N) rather than N different concepts using the same N neurons … In reality, almost certainly representations are less ‘precise’ allowing some levels of similarity in representations etc., so the real story will be more complex, but the basic principle holds that combinations of thresholding and winner-takes-all allow more distinct concepts than would be possible if combinations of concepts can occur more freely.

However, notice again that higher capacity to deal with more concepts is potentially bought at the cost of being able to think of less things ‘at once’ – and the side effect is that we have to serialise.

Returning back to the “computers are sequential, brains are associative” argument, whilst not denying the incredible parallel associativity of human memory, actually there seems as much to wonder about in the mechanisms that the brain ‘uses’ for sequentiality and the gains it gets because of this.

  1. see Gerald Edelman, Wider then the Sky, Yale University Press, 2004, ISBN 0-300-10229-1[back]
  2. Rodney Cotterill, Enchanted Looms: Conscious Networks in Brains and Computers, Cambridge University Press, 1998, ISBN 0-521-62435-5. See p. 244 for 500ms switching and pp. 261 and 265 for 20-50ms spotlight/searchlight of attention[back]
  3. Although the grandmother cell this is generally derided as oversimplisitic, there is evidence that there is more neuron specialisation then previously thought [[see Mind Hacks: evidence for ‘Grandmother Cells’]]. Also it is easier to encode relationships if there are single patches than configuratiin sof neurons, so perhaps we have both mechanisms at work.[back]
  4. Walter J. Freeman, How Brains Make Up Their Minds, Phoenix, 1999, ISBN 0-75381-068-9. See p. 95 onwards for rabbit olfactory bulb experiments.[back]

Dennett’s Sweet Dreams – consciousness and the Turing test

I read Dennett’s Sweet Dreams a few months ago. Although I am also interested in dreams this book is about consciousness … indeed subtitled “Philosophical Obstacles to a Science of Consciousness”

The book is largely about one message – that a scientific study of consiousness can only take into account third party accessible knowledge about first part experience. In other words I can only base a scientific study on what I can tell of other people’s consciousness from their actions, words and any available brain scanning etc.

Dennett has a meticulous rhetoric, but I found two broad facets of his argument weak, one more about rheteric and one substance.

First somewhat disingenuously he does not say that a scientific study of consciousness would yield a complete account of consciouness, but effectively the implication is there. That is he does not say that consciouness is no more than its phenomenial effects … but implies it.

Second, being a philosopher he focuses on incontrovertible evidence, whereas as scientists and humans often reasonable evidence is sufficient.

The first point is obvious and yet easily underestimated. A ‘scientific’ study of literature could formulate many known literary rules (aliteration, rhyme, etc.) and may even find new ones, and indeed poets in particular are happy to perform such analyses. However, we do not expect such rules to be a complete account of literture.

The second point is more substantive, but does interact with the first.
Dennett takes issue with philosophers who posit some form of non-sentient zombie (often called ‘Mary’) who/which nonetheless behaves otherwise exactly like a human including things that might appear to be conscious. They then say “but of course Mary is not conscious”. Dennett objects to the ‘of course’, which is really a statement about prior beliefs/assumptions (although Dennett, of course, frequently does the same with his beliefs!).

Dennett posits a Robo-Mary which is entirely mechanical/electronic and yet emulates perfectly the brain circuitry of a person and so can work out how the person would react and then reacts similarly. From the outside and by all her (emulated) subjective reactions she appears to be conscious. She would pass any ‘Turing Test’ for consciousness and yet many, perhaps most, would say she is not. The implication (from the first weakness) is that we are no more conscius than she (it?).

Actually I don’t object to the idea that such a creature may indeed be conscious, but I’d need more evidence than I would for a human, not because Robo-Mary is a machine, but becasue she is designed to appear conscious.

Robo-Mary is in fact a Robo-Mata-Hari, a spy, a robot in human clothing.

A good enough actor may convince you he is feeling happy, sad, or in love, and you may not be able to tell the differece between the act and the real thing, but that does not mean happiness, saddness and love are no more than their appearance.

As a philosopher, you cannot have incontrovertible evidence that a person’s emotions are real, not just a facade. However, as a human it would be unreasonable to therefore dismiss all expressions of emotion.

Some (well many) years ago, I worked with people at York who creating one of the first ADA compilers. There was a validation suite of programs that had to compile and run correctly for the compiler to get an official stamp from the ADA standards agency. I used to wonder about writing a program that recognised each of the tests cases and simply spat out the right code for each one. Any other program given to the program would simply print an error message and stop. The program would pass the test suite and could get the stamp as being a validated compiler, and yet would be completely useless. It would be a cheat ADA compiler.

Imagine if I sold such a cheat compiler. Any judge would regard it as fraud – whilst it passed the test, it is clearly not an ADA compiler. The test is there to validate things that are designed to be ADA compilers, not things designed to pass the test. So, the cheat ADA compiler is not adequately validated by the test, just becase it is designed to pass it.

Robo-Mary is designed to pass the consciousness test … indeed any consciousness test. We perhaps could never incontrovertibly tell whether Robo-Mary was conscious or simply acting conscious. However, when faced with another human being, an ordinary Mary, who is not designed specifically to appear conscious, it is reasonable to assume that she experiences similar things to me when she describes her experience in similar terms. I can never incontrovertibly tell that Mary is conscious, but it is reasonable to believe so. And it is equally reasonable to base a scientific study on such defeasible observations.

Turning back to Robo-Mary; convincing machine cosciousness would not come from machines designed to appear conscious, but more ‘by accident’. Perhaps one day my intelligent automated vacuum cleaner will say to me “Alan, have you ever watched those dust motes in the sunlight”.