The very fabric of our physical world, space itself, could be a purely mathematical object in the sense that its only intrinsic properties are mathem… - Max Tegmark

My guess is that we’ll one day understand consciousness as yet another phase of matter. I’d expect there to be many types of consciousness just as there are many types of liquids, but in both cases, they share certain characteristic traits that we can aim to understand.

Suppose that humanity flourishes thanks to the enslaved-god AI. Would this be ethical? If the AI has subjective conscious experiences, then would it feel that “life is suffering,” as Buddha put it, and it was doomed to a frustrating eternity of obeying the whims of inferior intellects? After all, the AI “boxing” we explored in the previous chapter could also be called “imprisonment in solitary confinement.” Nick Bostrom terms it mind crime to make a conscious AI suffer.4 The “White Christmas” episode of the Black Mirror TV series gives a great example. Indeed, the TV series Westworld features humans torturing and murdering AIs without moral qualms even when they inhabit human-like bodies.

Life 1.0”: life where both the hardware and software are evolved rather than designed. You and I, on the other hand, are examples of “Life 2.0”: life whose hardware is evolved, but whose software is largely designed. By your software, I mean all the algorithms and knowledge that you use to process the information from your senses and decide what to do — everything from the ability to recognize your friends when you see them to your ability to walk, read, write, calculate, sing and tell jokes.