# Incompleteness in the natural world

Gödel Incompleteness Theorem (Photo credit: janoma.cl)

A post inspired by Godel, Escher, Bach, Complexity: A Guided Tour, an article in this week’s New Scientist about the clash between general relativity and quantum mechanics and personal humiliation.

The everyday incompleteness: This is the personal humiliation bit. For the first time ever I went on a “Parkrun” today – the 5km Finsbury Park run, but I dropped out after 2.5km 2km – at the top of a hill and about 250 metres from my front door – I simply thought this is meant to be a leisure activity and I am not enjoying it one little bit. I can offer some excuses – it was really the first time ever I had run outdoors and so it was a bit silly to try a semi-competitive environment for that, I had not warmed up properly and so the first 500 metres were about simply getting breathing and limbs in co-ordination - mais qui s’excuse, s’accuse.

But the sense of incompleteness I want to write about here is not that everyday incompleteness, but a more fundamental one – our inability to fully describe the universe, or rather, a necessary fuzziness in our description.

Let’s begin with three great mathematical or scientific discoveries:

The diagonalisation method and the “incompleteness” of the real numbers: In 1891 Georg Cantor published one of the most beautiful, important and accessible arguments in number theory – through his diagonalisation argument, that proved that the infinity of the real numbers was qualitatively different from and greater than the infinity of the counting numbers.

The infinity of the counting numbers is just what it sounds like – start at one and keep going and you go on infinitely. This is the smallest infinity – called aleph null ($\aleph_0$).

Real numbers include the irrationals – those which cannot be expressed as fractions of counting numbers (Pythagoras shocked himself by discovering that $\sqrt 2$ was such a number). So the reals are all the numbers along a counting line – every single infinitesimal point along that line.

Few would disagree that there are, say, an infinite number of points between 0 and 1 on such a line. But Cantor showed that the number was uncountably infinite – i.e., we cannot just start counting from the first point and keep going. Here’s a brief proof…

Imagine we start to list all the points between 0 and 1 (in binary) – and we number each point, so…

1 is 0.00000000…..
2 is 0.100000000…..
3 is 0.010000000……
4 is 0.0010000000….
n is 0.{n – 2 0s}1{000……}

You can see this can go on for an infinitely countable number of times….

and so on. Now we decide to ‘flip’ the o or 1 at the index number, so we get:

1 is 0.1000000….
2 is 0.1100000….
3 is 0.0110000….
4 is 0.00110000….

And so on. But although we have already used up all the counting numbers we are now generating new numbers which we have not been able to count – this means we have more than $\aleph_0$ numbers in the reals, surely? But you argue, let’s just interleave these new numbers into our list like so….

1 is 0.0000000….
2 is 0.1000000…..
3 is 0.0100000….
4 is 0.1100000….
5 is 0.0010000….
6 is 0.0110000….

And so on. This is just another countably infinite set you argue. But, Cantor responds, do the ‘diagonalisation’ trick again and you get…

1 is 0.100000…..
2 is 0.110000….
3 is 0.0110000….
4 is 0.1101000…
5 is 0.00101000…
6 is 0.0110010….

And again we have new numbers, busting the countability of the set. And the point is this: no matter how many times you add the new numbers produced by diagonalisation into your counting list, diagonalisation will produce numbers you have not yet accounted for. From set theory you can show that while the counting numbers are of order (analogous to size) $\aleph_0$, the reals are of order $2^{\aleph_0}$, a far far bigger number – literally an uncountably bigger number.

Gödel’s Incompleteness Theorems: These are not amenable to a blog post length demonstration, but amount to this – we can state mathematical statements we know to be true but we cannot design a complete proof system that incorporates them – or we can state mathematical truths but we cannot build a self-contained system that proves they are true. The analogy with diagonalisation is that we know how to write out any real number between 0 and 1, but we cannot design a system (such as a computer program) that will write them all out – we have to keep ‘breaking’ the system by diagonalising it to find the missing numbers our rules will not generate for us. Gödel’s demonstration of this in 1931 was profoundly shocking to mathematicians as it appeared to many of them to completely undermine the very purpose of maths.

Turing’s Halting Problem: Very closely related to both Gödel’s incompleteness theorems and Cantor’s diagonalisation proof is Alan Turing’s formulation of the ‘halting problem’. Turing proposed a basic model of a computer – what we now refer to as a Turing machine – as an infinite paper tape and a reader (of the tape) and writer (to the tape). The tape’s contents can be interpreted as instructions to move, to write to the tape or to change the machine’s internal state (and that state can determine how the instructions are interpreted).

Now such a machine can easily be made of go into an infinite loop e.g.,:

• The machine begins in the ‘start’ state and reads the tape.  If it reads a 0 or 1 it moves to the right and changes its state to ‘even’.
• If the machine is in the state ‘even’ it reads the tape. If it reads a 0 or 1 it moves to the left and changes its state to ‘start’

You can see that if the tape is marked with two 0s or two 1s or any combination of 0 or 1 in the first two places the machine will loop for ever.

The halting problem is this – can we design a Turing machine that will tell us if a given machine and its instructions will fall into an infinite loop? Turing proved  we cannot without having to discuss any particular methodology … here’s my attempt to recreate his proof:

We can model any other Turing machine though a set of instructions on the tape, so if we have machine $T$ we can have have it model machine $M$ with instructions $I$: i.e., $T(M, I)$

Let us say $T$ can tell whether $M$ will halt or loop forever with instructions $I$ – we don’t need to understand how it does it, just suppose that it does. So if $(M, I)$ will halt $T$ writes ‘yes’, otherwise it writes ‘no’.

Now let us design another machine $T^\prime$ that takes $T(M,I)$ its input but here $T^\prime$ loops forever if $T$ writes ‘yes’ and halts if $T$ writes ‘no’.

Then we have:

$M(I)$ halts or loops – $T(M, I)$ halts – $T^\prime$ loops forever.

But what if we feed $T^\prime$ the input of $T^\prime(T(M, I)$?

$M(I)$ halts or loops – $T(M, I)$ halts – $T^\prime(T(M,I))$ loops forever – $T^\prime(T^\prime(T(M,I)))$ – ??

Because if the second $T^\prime(T^\prime(T(M,I)))$ halted then that would imply that the first had halted – but it is meant to loop forever, and so on…

As with Gödel we have reached a contradiction and so we cannot go further and must conclude that we cannot build a Turing machine (computer) that can solve the halting problem.

Quantum mechanics: The classic, Copenhagen, formulation of quantum mechanics states that the uncertainty of the theory collapses when we observe the world, but the “quantum worlds” theory suggests that actually the various outcomes do take place and we are just experiencing one of them at any given time. The experimental backup for the many worlds theory comes from quantum ‘double-slit’ experiments which suggest particles leave traces of their multiple states in every ‘world’.

What intrigues me: What if our limiting theories – the halting problem, Gödel’s incompleteness theorem, the uncountable infinite, were actually the equivalents of the Copenhagen formulation and, in fact, maths was also a “many world” domain where the incompleteness of the theories was actually the deeper reality – in other words the Turing machine can both loop forever and halt? This is probably, almost certainly, a very naïve analogy between the different theories but, lying in the bath and contemplating my humiliation via incompleteness this morning, it struck me as worth exploring at least.

# A (partial) answer to my Goedelian conundrum?

Douglas Hofstadter in Bologna, Italy (Photo credit: Wikipedia)

Last week I puzzled over what seemed to me to be the hand waiving dismissal, by both Alan Turing and Douglas Hofstadter of what I saw as the problem of humans being able to write true statements that the formal systems employed by computers could not determine – the problem thrown up by Goedel’s Incompleteness Theorems.

Well, Douglas Hofstadter has now come to his own (partial) rescue as I continue to read on through Godel, Escher, Bach – as he describes Tarski’s Theorem, which essentially states that humans cannot determine all such statements either (unless we posit the Church-Turing thesis is wrong, of course and there is some inner human computational magic we have yet to describe).

I am now going to quickly run through Hofstadter’s exposition – it might not mean too much to those of you not familiar with GEB, but if so and if you are interested in computation (and genetics and music) and you want to improve your mind this summer you could always think about buying the book. I don’t promise it’s an easy read, the style can vary from the nerdy to the deeply frustrating, but it is still a rewarding read.

So here goes:

We imagine we have a procedure that can determine the truth of a number theoretical statement, i.e.:

$TRUE\{a\}$ can tell us whether the number theoretical statement with Goedel number a is true.

So now we posit, $T$ with Goedel number $t$:

$\exists a:<\sim TRUE\{a\} \wedge ARITHMOQUINE\{a^{\prime\prime},a\}>$

Now, of you have read GEB you will know that to “arithmoquine” a number theoretical statement is to replace the free variable – in this case $a^{\prime\prime}$, with the Goedel number for the statement itself…

$\exists a:<\sim TRUE\{a\} \wedge ARITHMOQUINE\{SSS \ldots SSS0,a\}>$

Which we can state as “the arithmoquinification of $t$ is the Goedel number of a false statement”.

But the arithmoquinification of $t$ is $T$‘s own number, so this statement is the equivalent of saying “this statement is false”: just another version of the famous Epimenides Paradox, but one that is decidedly not hand waving in form: it’s about natural numbers.

The outcome is that $TRUE\{a\}$ cannot exist without our whole idea of natural numbers collapsing and we are forced to conclude there is no formal way of deciding what is a true statement of theoretical number theory using only theoretical number theory – and so humans are no better off than computers in this regard: we use concepts from without the formal theory to establish truth and we could surely program our computers to do the same. Turing’s “imitation game” conception of intelligent machines thus survives.

At least I think so!

# Cambridge University and computer science

The University of Cambridge Computer Laboratory in West Cambridge, south of the Madingley Road. (Photo credit: Wikipedia)

Cambridge University has a stellar reputation for Computer Science in the UK.

The Computer Laboratory can trace its history back over more than 75 years (to a time when ‘computers’ where humans making calculations), while the wider University can claim Alan Turing for one of its own. And Sinclair Research, ARM, the Cambridge Ring – the list of companies and technical innovations associated with the University is a long one: they even had what was possibly the world’s first webcam.

But, according to today’s Guardian, they might need to work a bit harder with their undergraduates – the Guardian’s 2014 University Guide rates Cambridge as the best University in Britain overall but slots it in only at 8th in computer science and conspicuously gives it the worst rating (1/10) for “value added” – namely the improvement from entry to degree for students.

Now, possibly this is because it is the toughest computer science course in the country to get a place in – the average student needs more than 3 A* grades at A level (and 3 As at AS) to get a place, compared to Imperial, the next place down where 3 A*s would probably set you right – but there has to be more to it than that. It is even harder to get into biosciences at Cambridge and yet they are rated 8/10 in the value added score.

Don’t get me wrong – I am sure Cambridge is fantastic at teaching computer science, but it is also given a lot of money on the basis that it is an elite institution and so it seems reasonable to ask for an explanation (from the Guardian too of course!)

(Incidentally, it seems that Oxford teaches so few undergraduates computer science it cannot be rated at all.)

# Turing, Hofstadter, Bach – with some Cherla thrown in

The imitation game, as described by Alan Turing in “Computing Machinery and Intelligence”. Player C, through a series of written questions, attempts to determine which of the two players is a man, and which of the two is the woman. Player A – the man – tires to trick player C into making the wrong decision, while player B tries to help player C. Turing uses this game as the basis for his test for intelligence. (Photo credit: Wikipedia)

A few weeks ago I attended the morning (I had to go back to work in the afternoon) of the BCS doctoral consortium in Covent Garden in London – watching various PhD students present their work to audience of peers.

The presentation which most interested me was that of Srikanth Cherla who is researching connectionist models for music analysis and prediction and to use generative models to produce short passages of music that are in a similar style to the music passages that his systems learn.

It’s not a field that I have any expertise in or indeed much knowledge of, though in essence (I hope I get this right): a specialised form of neural network is used to analyse musical passages (Bach’s chorale works were highlighted) and from there it is possible to get the computer to play some passages it has composed based on the style it has learnt.

Srikanth emphasised that it was not a case of applying a rigid rule that guessed or picked the next note – there is a semi-random/stochastic element that can be attributed to certain musical patterns in the works of the great composers and capturing that is important.

And the music he played at the end – while plainly not matching Bach, did certainly sound like Bach.

As I am (still! It’s a big book) reading Godel, Escher, Bach: An Eternal Golden Braid by Douglas Hofstadter my thoughts were immediately turned to the idea of artificial intelligence and then on to Alan Turing’s idea of the “Imitation Game” and the concept that anything that looks like it is thinking should, in fact, be thought of as thinking.

Today, prior to writing this blog I read through Turing’s October 1950 paper “Computing Machinery and Intelligence“, from which we get the idea of a “Turing Test” (though obviously he doesn’t call it that).

The paper begins:

I propose to consider the question, ‘Can machines think?’

And goes on to discuss ways in which it might be possible “by the end of the century” to have machines which could fool a remote observer, able only to read typed answers to questions, that a digital computer was in fact a person.

The paper is not, for Turing at least, in a completely different field to “On computable numbers”: Turing’s essential point is anything a human computer can do, a digital computer can do, and he goes on to explicitly call humans machines.

The idea that great works of art, such as the “next” set of Bach chorales,  might in the future be composed by computer no doubt horrifies many readers, as it plainly did in Turing’s day too – as he deals specifically with what he calls “the theological objection” – an extreme objection based on the idea that “God gives an immortal soul to every man and woman, but not to any other animal or machine”:

I am unable to accept any part of this… I am not very impressed with theological arguments whatever they may be used to support

But in any case, from within the theological paradigm dismisses it as a human imposition on what is meant to be an unlimited Godly power:

It appears to me the argument quoted above implies a serious restriction on the omnipotence of the Almighty

…before going on to swat aside Biblical literalism as an argument by citing how it was used against Galileo (maybe there are still fundamentalists out there who believe in the literal truth of Psalm 104 and an unmoving Earth but if so they keep quiet about it).

Then he deals with the argument that machines could not appear human because they have no consciousness by essentially asking what is consciousness anyway – and how can we prove others have it and then goes on to deal with “various disabilities” – such as computers being unable to appreciate the taste of strawberries with cream:

The inability to enjoy strawberries and cream may have struck the reader as frivilous. Possibly a machine might be made to enjoy this delicious dish, but any attempt to make one would be idiotic. What is important about this disability is that it contributes to some of the other diabilities e.g. to the difficulty of the same kind of friendliness occurring between man and machine as between white man and white man, or between black man and black man.

This passage is worth quoting as it both suggests that Turing is far from the 100% progressive superhero later admirers are tempted to paint him as – he beat the Nazis and was persecuted as a gay man and therefore can do no wrong: in fact he was a man of his times with all that implies – as well as because I find it less than fully satisfying an answer.

In context I think the point he is seeking to make is that we could make a machine that liked “eating” strawberries and could be friends with its fellows (so long as they had the same skin colour, don’t lets get too radical!) but why would we bother… but it is not totally clear.

Similarly he, like Hofstadter, deals with the so-called Goedalisation argument less than satisfactorily: this states that we, humans, can state true statements about numbers that machines cannot determine (i.e. we know they are true but the machine cannot decide if they are true or false). Hence we could, in the imitation game, pose a Goedel Number type puzzle that the computer could never answer.

Actually, of course, the computer could guess, as humans often do! But the more general point – that humans can do something machines cannot and so we are not truly Turing Machines seems unanswered to me by both Turing and Hofstadter’s argument: that we can also find questions humans cannot determine if we make them complex enough.

Perhaps an expert would care to comment?

Update: Following some feedback from Srikanth I have edited the passages referring to his work slightly – haven’t changed the sense I think but just made it a bit clearer. I also updated the Psalm number – as I had misread Turing’s reference to line 5 of the Psalm for the Psalm itself

# How I discovered the fundamental theorem of arithmetic by chance

Cover via Amazon

Actually, of course, I rediscovered it.

I have been attempting to read, for the third time Douglas Hofstadter‘s celebrated Godel, Escher, Bach: I bought a copy in Washington DC in 2009 and loved it (though didn’t get very far before I put it down for some reason) but I have always struggled to get deeply into it: it’s plainly a great book – though having read more on Turing these days I am not sure I’d subscribe to what appears to be the core thesis – but it’s not an easy read.

(Though Hofstadter’s preface did inspire me three years ago to read the brilliantly compact Godel’s Proof- buy, steal or borrow a copy if you are at all interested in maths.)

This time round I am feeling more confident I’ll make progress, it just seems a bit easier to grasp – possibly because I am feeling more confident about maths these days (not that there is any higher maths in it – but grasping some of the more complex stuff does seem to help with all of it.)

So I got the part known as the “MU puzzle” – can you go from the axiom MI to the theorem MU following the rules of the formal system he sets out. I thought – oh, yes, easy, just find a power of two that is also a multiple of three – and in my head I start going 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096 – a list I have been familiar with for thirty three years or so. But none of them are multiples of three.

Now, presumably anyone who knows anything about prime numbers has by now said “of course they are not, haven’t you heard of the fundamental theorem of arithmetic?” - well it seems the answer to that is “err, no”.

Or, if I had, I never really grasped its meaning. Each number cn be factorised into a unique set of primes (if a prime then just itself, of course). That precludes any power of two having as a factor any other prime number (of which three is of course one). It’s really epically important in terms of understanding numbers and I am slightly shocked and puzzled I have never really understood it before now.

# Stephen Fry is wrong about Alan Turing

The case for Turing is, of course, a strong one. Not just because of the computer you are reading this blog on, but because, probably even more importantly, Turing’s concept was at the heart of the British “Station X” codebreaking effort in the Second World War, shortening the war by years and perhaps was even essential to national survival. Defeating Hitler – something that it is hard to believe could have been done without Britain even if others paid a higher price in money and lives to finally deliver the triumph – remains the great British gift to humanity.

But it is utterly wrong.

Because the Universal Machine is not an invention or innovation at all. It is a discovery – of a mathematical idea – and it has existed (at least) since the beginning of our universe and will still exist long after we (humanity) are all gone.

Admittedly my formulation of mathematics – essentially Platonic – is not uncontroversial and I have strong “political” motivations for making it (of I will be writing more shortly) – seeing mathematical ideas as discoveries rather than inventions is core to keeping patent lawyers’ hands off them (in this jurisdiction at least).

But why treat maths as any different from physics? We are surely not about to grant Peter Higgs (lovely man that he is) a patent over mass, even though we are on the verge of confirming his theoretical paper of 1962 was essentially correct in its formulation of the “Higgs field” that gives particles mass.

Update: Stephen Fry is of course describing the Universal Machine as an “innovation” and not an “invention” as I originally descried it at the top of this article (changed now). The argument doesn’t change though – I don’t think it is an innovation, because it has always existed. It is a great discovery, arguably the greatest, but that’s a different matter.

# Unreal Tournament at the forefront of AI research (really)

I am not much of a computer games player, but I do have a fondness for Unreal Tournament- a network shoot-em-up game at which I have always been hopeless if

Human brain – midsagittal cut (Photo credit: Wikipedia)

enthusiastic (though I’ve not played for a few years now).

So I was pleasantly surprised to read that Unreal is now, according to the New Scientist, at the forefront of artificial intelligence research, (subscribers only at present).

Next week Unreal bots will battle human players at the IEEE Conference of Computational Intelligence and Games in Grenada, Spain and if a bot can convince human players it is real then its developers could win $7000. In past years the bots have only won a maximum of$2000 – the money that goes to the best bot that is not convincing as a human.

This year, though, hopes seem high that one bot – ‘Neurobot’ – has a real crack at the \$7000 prize (it came second to ICE-CIG amongst the bots last year but Neurobot’s developers, from Imperial College in London, are hoping that improvements they have made put it in poll.)

The interesting thing is that Neurobot is the algorithm/concept being used – the bot doesn’t try to use computational power to fully absorb the scene and act on every piece of information, but instead discriminates using the principles of “global workplace theory” (GWT) which states that the human brain only pushes a small number of things into the forefront of thought – the “global workplace”.

Neurobot models the brain’s GWT with about 20,000 simulated neurons as opposed to the estimated 120 billion in the human brain.

Neurobot’s prospects for success might then suggest that the barrier to  successful AI has not really been the inability of computers to match the computational power of the human brain, but the failure, thus far at least, for human AI researchers to model how the brain works. In other words – we are not really as clever as we like to think (a thought which dominated much of the latter work of Alan Turing – as much discussed in Alan Turing: The Enigma (which I am still listening to – though I have got down to the final three hours of thirty).

# SCOOPING THE LOOP SNOOPER

## A proof that the Halting Problem is undecidable

Geoffrey K. Pullum
(School of Philosophy, Psychology and Language Sciences, University of Edinburgh)

In October 2000, after a refereeing delay of nearly a year, an earlier and incorrect version of this poetic proof was published in Mathematics Magazine (73, no. 4, 319–320). But it had an error. I am very grateful to Philip Wadler (Informatics, University of Edinburgh) and Larry Moss (Mathematics, Indiana University) for helping with the development of this corrected version, which is now free of bugs (trust me; you can check it). Thanks also to the late Dr. Seuss for the style, and of course to the pioneering work of Alan Turing (and Martin Davis’s nice simplified presentation) for the content. Copyright © 2008 by Geoffrey K. Pullum. Permission is hereby granted to reproduce or distribute this work for non-commercial, educational purposes relating to the teaching of computer science, mathematics, or logic, provided this attribution is included.

# From The Selfish Gene to Alan Turing: the Enigma

As I spend a lot of time in the gym (honestly) I have decided to use that a little more productively and listen to decent non-fiction audio books – and I have just finished listening to Richard Dawkins’s The Selfish Gene.

Cover of Alan Turing: The Enigma

I quite enjoyed the book (or the recording) in the end – certainly added to my knowledge of genetics (not that that would be tough) and made a convincing case for its central argument: that genes the fundamental replicators of the biological world and it is their ‘drive’ to selfish self-preservation that shapes so much of our world. In fact my biggest criticism of the book is the way it anthropomorphises genes – a weakness reflected in the title itself. Genes are not selfish really – they are molecules that, statistically, interact with other molecules in a way that ensures their molecular formation survives. The book’s unwillingness to discuss anything but the most minimal amount of maths means Dawkins refuses to discuss that perspective in anything but the briefest of terms.

I also found his explanation of intelligence unconvincing, but this is a tricky and exceptionally difficult subject and I don’t have an original idea to counter Dawkins with.

And, of course, there is no discuss of the chemical processes that allow these proteins to shape their ‘machines’ or their phenotypes generally.

But, yes, it’s worth reading or listening to.

Next up? Andrew Hodges‘s Alan Turing: The Enigma. This, at 30 hours, is almost twice as long as Dawkins’s book, so suspect will be listening to this into August (unless I go mad for the gym) – will be interesting to see how, if at all, the maths are dealt with.

# Leonardo DiCaprio to play Alan Turing?

I hadn’t realised this before (update: err, actually I did – I wrote about this in October 2011 too, oops) but apparently Leonardo DiCaprio is to play Alan Turing in “The Imitation Game“, a biopic.

Leonardo DiCaprio at the Body of Lies film premiere in London (Photo credit: Wikipedia)

It’s fantastic news, not just because the idea of the film itself is appealing but because DiCaprio is surely one of the greatest film actors of this or any other generation. I sense that he’s never quite had the recognition he deserves because his first big role, in Titantic, was rather soft and slushy, but he is so good that even when he is in an otherwise mediocre film – J Edgar is a recent example – he lifts it to a higher plane.