Whatever reality is, it’s not what you see.

What you see is just an adaptive fiction.

The following is a conversation with Donald Hoffman,

professor of cognitive sciences at UC Irvine,

focusing his research on evolutionary psychology,

visual perception, and consciousness.

He’s the author of over 120 scientific papers

on these topics and his most recent book

titled The Case Against Reality,

Why Evolution Hid the Truth from Our Eyes.

I think some of the most interesting ideas in this world,

like those of Donald Hoffman’s,

attempt to shake the foundation

of our understanding of reality,

and thus they take a long time to internalize deeply.

So proceed with caution.

Questioning the fabric of reality

can lead you to either madness or to truth.

And the funny thing is, you won’t know which is which.

This is the Lex Friedman podcast.

To support it, please check out our sponsors

in the description.

And now, dear friends, here’s Donald Hoffman.

In your book, The Case Against Reality,

Why Evolution Hid the Truth from Our Eyes,

you make the bold claim that the world we see

with our eyes is not real.

It’s not even an abstraction of objective reality.

It is completely detached from objective reality.

Can you explain this idea?

Right, so this is a theorem

from evolution by natural selection.

So the technical question that I and my team asked was,

what is the probability that natural selection

would shape sensory systems

to see true properties of objective reality?

And to our surprise,

we found that the answer is precisely zero,

except for one kind of structure

that we can go into if you want to.

But for any generic structure

that you might think the world might have,

a total order, a topology, metric,

the probability is precisely zero

that natural selection would shape any sensory system

of any organism to see any aspect of objective reality.

So in that sense, what we’re seeing

is what we need to see

to stay alive long enough to reproduce.

So in other words, we’re seeing what we need

to guide adaptive behavior, full stop.

So the evolutionary process,

the process that took us from the origin of life on Earth

to the humans that we are today,

that process does not maximize for truth,

it maximizes for fitness, as you say, fitness beats truth.

And fitness does not have to be connected to truth,

is the claim.

And that’s where you have an approach

towards zero of probability

that we have evolved human cognition,

human consciousness, whatever it is,

the magic that makes our mind work,

evolved not for its ability to see the truth of reality,

but its ability to survive in the environment.

That’s exactly right.

So most of us intuitively think that surely

the way that evolution will make our senses more fit

is to make them tell us more truths,

or at least the truths we need to know

about objective reality, the truths we need in our niche.

That’s the standard view, and it was the view I took.

I mean, that’s sort of what we’re taught

or just even assume.

It was just sort of like the intelligent assumption

that we would all make.

But we don’t have to just wave our hands.

Evolution of a natural selection

is a mathematically precise theory.

John Maynard Smith in the 70s

created evolutionary game theory.

And we have evolutionary graph theory

and even genetic algorithms that we can use to study this.

And so we don’t have to wave our hands.

It’s a matter of theorem and proof and or simulation

before you get the theorems and proofs.

And a couple of graduate students of mine,

Chester Mark and Brian Marion,

did some wonderful simulations that tipped me off

that there was something going on here.

And then I went to a mathematician, Chetan Prakash,

and Manish Singh, and some other friends of mine,

Chris Fields.

But Chetan was the real mathematician behind all this.

And he’s proved several theorems

that uniformly indicate that with one exception,

which has to do with probability measures,

there’s no, the probability is zero.

The reason there’s an exception for probability measures,

so called sigma algebras or sigma additive classes,

is that for any scientific theory,

there is the assumption that needs to be made

that whatever structure,

whatever probabilistic structure the world may have

is not unrelated to the probabilistic structure

of our perceptions.

If they were completely unrelated,

then no science would be possible.

So this is technically the map from reality to our senses

has to be a so called measurable map,

has to preserve sigma algebras.

But that means it could be infinite to one,

and it could collapse all sorts of event information.

But other than that, there’s no requirement

in standard evolutionary theory

for fitness payoff functions, for example,

to preserve any specific structures of objective reality.

So you can ask the technical question.

This is one of the avenues we took.

If you look at all the fitness payoffs

from whatever world structure you might want to imagine.

So a world with say a total order on it.

So it’s got end states and they’re totally ordered.

And then you can have a set of maps from that world

into a set of payoffs, say from zero to a thousand

or whatever you want your payoffs to be.

And you can just literally count all the payoff functions

and just do the combinatorics and count them.

And then you can ask the precise question,

how many of those payoff functions preserve the total order?

If that’s what you’re looking for,

or how many preserve the topology?

And you just count them and divide.

So the number that are homomorphisms

versus the total number, and then take the limit

as the number of states in the world

and the number of payoff values goes very large.

And when you do that, you get zero every time.

Okay, there’s a million things to ask here.

But first of all, just in case people

are not familiar with your work,

let’s sort of linger on the big bold statement here,

which is the thing we see with our eyes

is not some kind of limited window into reality.

It is completely detached from reality,

likely completely detached from reality.

You’re saying 100% likely.

Okay, so none of this is real in the way we think is real.

In the way we have this intuition,

there’s like this table is some kind of abstraction,

but underneath it all, there’s atoms.

And there’s an entire century of physics

that describes the functioning of those atoms

and the quirks that make them up.

There’s many Nobel Prizes about particles and fields

and all that kind of stuff that slowly builds up

to something that’s perceivable to us,

both with our eyes, with our different senses as this table.

Then there’s also ideas of chemistry

that over layers of abstraction, from DNA to embryos,

the cells that make the human body.

So all of that is not real.

It’s a real experience,

and it’s a real adaptive set of perceptions.

So it’s an adaptive set of perceptions, full stop.

We want to think that the perceptions are real.

So their perceptions are real as perceptions, right?

We are having our perceptions,

but we’ve assumed that there’s a pretty tight relationship

between our perceptions and reality.

If I look up and see the moon,

then there is something that exists in space and time

that matches what I perceive.

And all I’m saying is that if you take evolution

by natural selection seriously, then that is precluded.

That our perceptions are there.

They’re there to guide adaptive behavior, full stop.

They’re not there to show you the truth.

In fact, the way I think about it is

they’re there to hide the truth

because the truth is too complicated.

It’s just like if you’re trying to use your laptop

to write an email, right?

What you’re doing is toggling voltages in the computer,

but good luck trying to do it that way.

The reason why we have a user interface

is because we don’t want to know that quote unquote truth,

the diodes and resistors and all that terrible hardware.

If you had to know all that truth,

your friends wouldn’t hear from you.

So what evolution gave us was perceptions

that guide adaptive behavior.

And part of that process, it turns out,

means hiding the truth and giving you eye candy.

So what’s the difference between hiding the truth

and forming abstractions,

layers upon layers of abstractions

over low level voltages and transistors

and chips and programming languages

from assembly to Python that then leads you

to be able to have an interface like Chrome

where you open up another set of JavaScript and HTML

programming languages that lead you

to have a graphical user interface

and which you can then send your friends an email.

Is that completely detached from the zeros and ones

that are firing away inside the computer?

It’s not.

Of course, when I talk about the user interface

on your desktop, there’s this whole sophisticated

backstory to it, right?

That the hardware and the software

that’s allowing that to happen.

Evolution doesn’t tell us the backstory, right?

So the theory of evolution is not going to be adequate

to tell you what is that backstory.

It’s gonna say that whatever reality is,

and that’s the interesting thing,

it says whatever reality is, you don’t see it.

You see a user interface,

but it doesn’t tell you what that user interface is,

how it’s built, right?

Now, we can try to look at certain aspects

of the interface, but already we’re gonna look at that

and go, okay, before I would look at neurons

and I was assuming that I was seeing something

that was at least partially true.

And now I’m realizing that it could be like looking

at the pixels on my desktop or icons on my desktop

and good luck going from that to the data structures

and then the voltages and I mean, good luck.

There’s just no way.

So what’s interesting about this is that

our scientific theories are precise enough

and rigorous enough to tell us certain limits,

but, and even limits of the theories themselves,

but they’re not going to tell us what the next move is

and that’s where scientific creativity comes in.

So the stuff that I’m saying here, for example,

is not alien to physicists.

The physicists are saying precisely the same thing

that space time is doomed.

We’ve assumed that space time is fundamental.

We’ve assumed that for several centuries

and it’s been very useful.

So all the things that you were mentioning,

the particles and all the work that’s been done,

that’s all been done in space time,

but now physicists are saying space time is doomed.

There’s no such thing as space time fundamentally

in the laws of physics.

And that comes actually out of gravity

together with quantum field theory,

which just comes right out of it.

It’s a theorem of those two theories put together,

but it doesn’t tell you what’s behind it.

So the physicists know that their best theories,

Einstein’s gravity and quantum field theory put together

entail that space time cannot be fundamental

and therefore particles in space time cannot be fundamental.

They’re just irreducible representations

of the symmetries of space time.

That’s what they are.

So we have, so space time, so we put the two together.

We put together what the physicists are discovering

and we can talk about how they do that.

And then we, the new discoveries

from evolution of a natural selection.

Both of these discoveries are really in the last 20 years.

And what both are saying is space time

has had a good ride.

It’s been very useful.

Reductionism has been useful, but it’s over.

And it’s time for us to go beyond.

When you say space time is doomed,

is it the space, is it the time,

is it the very hard coded specification of four dimensions?

Or are you specifically referring

to the kind of perceptual domain

that humans operate in, which is space time?

You think like there’s a 3D, like our world

is three dimensional and time progresses forward.

Therefore, three dimensions plus one, 4D.

What exactly do you mean by space time?

And what do you mean by space time is doomed?

Great, great.

So this is, by the way, not my quote.

This is from, for example, Nima Arkanihaim Ed

at the Institute for Advanced Study at Princeton.

Ed Witten, also there.

David Gross, Nobel Prize winner.

So this is not just something the cognitive scientists,

this is what the physicists are saying.

Yeah, the physicists, they’re space time skeptics.

Well, yeah, they’re saying that,

and I can say exactly why they think it’s doomed.

But what they’re saying is that,

because your question was what aspect of space time,

what are we talking about here?

It’s both space and time.

They’re union into space time as an Einstein’s theory.

That’s doomed.

And they’re basically saying that even quantum theory,

this is with Nima Arkanihaim Ed, especially.

So Hilbert spaces will not be fundamental either.

So that the notion of Hilbert space,

which is really critical to quantum field theory,

quantum information theory,

that’s not going to figure

in the fundamental new laws of physics.

So what they’re looking for

is some new mathematical structures beyond space time,

beyond Einstein’s four dimensional space time

or super symmetric version,

geometric algebra signature two comma four kind of.

There are different ways that you can represent it,

but they’re finding new structures.

And then by the way, they’re succeeding now.

They’re finding, they found something

called the amplituhedron.

This is Nima and his colleagues,

the cosmological polytope.

So there are these like polytopes,

these polyhedra in multi dimensions,

generalizations of simplices that are coding for,

for example, the scattering amplitudes of processes

in the Large Hadron Collider and other colliders.

So they’re finding that if they let go of space time,

completely, they’re finding new ways

of computing these scattering amplitudes

that turn literally billions of terms into one term.

When you do it in space and time,

because it’s the wrong framework,

it’s just a user interface from,

that’s not from the evolutionary point of view,

it’s just user interface.

It’s not a deep insight into the nature of reality.

So it’s missing deep symmetry

is something called a dual conformal symmetry,

which turns out to be true of the scattering data,

but you can’t see it in space time.

And it’s making the computations way too complicated

because you’re trying to compute all the loops

in the Feynman diagrams and all the Feynman integrals.

So see the Feynman approach to the scattering amplitudes

is trying to enforce two critical properties of space time,

locality and unitarity.

And so by, when you enforce those,

you get all these loops and multiple,

different levels of loops.

And for each of those,

you have to add new terms to your computation.

But when you do it outside of space time,

you don’t have the notion of unitarity.

You don’t have the notion of locality.

You have something deeper

and it’s capturing some symmetries

that are actually true of the data.

And, but then when you look at the geometry

of the facets of these polytopes,

then certain of them will code for unitarity and locality.

So it actually comes out of the structure

of these deep polytopes.

So what we’re finding is there’s this whole new world.

Now beyond space time that is making explicit symmetries

that are true of the data

that cannot be seen in space time.

And that is turning the computations

from billions of terms to one or two or a handful of terms.

So we’re getting insights into symmetries

and all of a sudden the math is becoming simple

because we’re not doing something silly.

We’re not adding up all these loops in space time.

We’re doing something far deeper.

But they don’t know what this world is about.

Also, they’re in an interesting position

where we know that space time is doomed.

And I should probably tell you why it’s doomed,

what they’re saying about why it’s doomed.

But they need a flashlight to look beyond space time.

What flashlight are we gonna use

to look into the dark beyond space time?

Because Einstein’s theory and quantum theory

can’t tell us what’s beyond them.

All they can do is tell us that when you put us together,

space time is doomed at 10 to the minus 33 centimeters,

10 to the minus 43 seconds.

Beyond that, space time doesn’t even make sense.

It just has no operational definition.

So, but it doesn’t tell you what’s beyond.

And so they’re just looking for deep structures

like guessing is really fun.

So these really brilliant guys, generic brilliant men

and women who are doing this work, physicists,

are making guesses about these structures,

informed guesses, because they’re trying to ask,

well, okay, what deeper structure could give us

the stuff that we’re seeing in space time,

but without certain commitments

that we have to make in space time, like locality.

So they make these brilliant guesses.

And of course, most of the time you’re gonna be wrong,

but once you get one or two that start to pay off

and then you get some lucky breaks.

So they got a lucky break back in 1986.

Couple of mathematicians named Park and Taylor

took the scattering amplitude for two gluons coming in

at high energy and four gluons going out at low energy.

So that kind of scattering thing.

So it’s like apparently for people who are into this,

that’s sort of something that happens so often

you need to be able to find it and get rid of those

cause you already know about that and you need to.

So you needed to compute them.

It was billions of terms and they couldn’t do it

even though for the supercomputers couldn’t do that

for the many billions or millions of times per second

they needed to do it.

So the experimentals begged the theorists,

please, you got it.

And so Park and Taylor took the billions of terms,

hundreds of pages and miraculously turned it into nine.

And then a little bit later,

they guessed one term expression

that turned out to be equivalent.

So billions of terms reduced to one term,

that so called famous Park Taylor formula, 1986.

And that was like, okay, where did that come from?

This is a pointer into a deep realm, beyond space and time,

but no one, I mean, what can you do with it?

And they thought maybe it was a one off,

but then other formulas started coming up.

And then eventually Neymar, Connie, Hamid and his team

found this thing called the amplituhedron,

which really sort of captures the whole,

a big part of the whole ball of wax.

I’m sure they would say, no, there’s plenty more to do.

So I won’t say they did it all by any means.

They’re looking at the cosmological polytope as well.

So what’s remarkable to me is that two pillars

of modern science, quantum field theory with gravity

on the one hand and evolution by natural selection

on the other, just in the last 20 years

have very clearly said space time has had a good run.

Reductionism has been a fantastic methodology.

So we had a great ontology of space time,

a great methodology of reductionism.

Now it’s time for a new trick.

But now you need to go deeper and show,

but by the way, this doesn’t mean we throw away

everything we’ve done, not by a long shot.

Every new idea that we come up with beyond space time

must project precisely into space time.

And it better give us back everything that we know

and love in space time or generalizations,

or it’s not gonna be taken seriously and it shouldn’t be.

So we have a strong constraint on whatever we’re going to do

beyond space time, it needs to project into space time.

And whatever this deeper theory is,

it may not itself have evolution by natural selection.

This may not be part of this deeper realm.

But when we take whatever that thing is beyond space time

and project it into space time,

it has to look like evolution by natural selection

or it’s wrong.

So that’s a strong constraint on this work.

So even the evolution by natural selection

and quantum field theory could be interfaces

into something that doesn’t look anything like,

like you mentioned.

I mean, it’s interesting to think that evolution

might be a very crappy interface

into something much deeper.

That’s right.

They’re both telling us that the framework that you’ve had

can only go so far and it has to stop.

And there’s something beyond.

And the very framework that is space and time itself.

Now, of course, evolution by natural selection

is not telling us about like Einstein’s relativistic

space time.

So that was another question you asked a little bit earlier.

It’s telling us more about our perceptual space and time,

which we have used as the basis for creating

first Newtonian space versus time

as a mathematical extension of our perceptions.

And then Einstein then took that and extended it even further.

So the relationship between what evolution is telling us

and what the physicists are telling us is that

in some sense, the Newton and Einstein space time

are formulated as sort of rigorous extensions

of our perceptual space,

making it mathematically rigorous

and laying out the symmetries that they find there.

So that’s sort of the relationship between them.

So it’s the perceptual space time

that evolution is telling us

is just a user interface effectively.

And then the physicists are finding

that even the mathematical extension of that

into the Einsteinian formulation has to be as well,

not the final story, there’s something deeper.

So let me ask you about reductionism and interfaces

as we march forward from Newtonian physics

to quantum mechanics.

These are all, in your view, interfaces.

Are we getting closer to objective reality?

How do we know if these interfaces in the process of science,

the reason we like those interfaces

is because they’re predictive of some aspects,

strongly predictive about some aspects of our reality.

Is that completely deviating

from our understanding of that reality

or is it helping us get closer and closer and closer?

Well, of course, one critical constraint

on all of our theories

is that they are empirically tested

and pass the experiments that we have for them.

So no one’s arguing against experiments being important

and wanting to test all of our current theories

and any new theories on that.

So that’s all there.

But we have good reason to believe

that science will never get a theory of everything.

Everything, everything.

Everything, everything, right.

A final theory of everything, right.

I think that my own take is, for what it’s worth,

is that Gödel’s incompleteness theorem

sort of points us in that direction,

that even with mathematics,

any finite axiomatization that’s sophisticated enough

to be able to do arithmetic,

it’s easy to show that there’ll be statements that are true,

that can’t be proven,

can’t be deduced from within that framework.

And if you add the new statements to your axioms,

then there’ll be always new statements that are true,

but can’t be proven with a new axiom system.

And the best scientific theories in physics, for example,

and also now evolution, are mathematical.

So our theories are gonna be,

they’re gonna have their own assumptions

and they’ll be mathematically precise.

And there’ll be theories, perhaps,

of everything except those assumptions,

because the assumptions are,

we say, please grant me these assumptions.

If you grant me these assumptions,

then I can explain this other stuff.

So you have the assumptions that are like miracles,

as far as the theory is concerned.

They’re not explained.

They’re the starting points for explanation.

And then you have the mathematical structure

of the theory itself, which will have the Gödel limits.

And so my take is that reality,

reality, whatever it is, is always going to transcend

any conceptual theory that we didn’t come up with.

There’s always gonna be mystery at the edges.

Right.

Contradictions and all that kind of stuff.

Okay.

And truths.

So there’s this idea that is brought up

in the financial space of settlement of transactions.

It’s often talked about in cryptocurrency, especially.

So you could do, you know, money, cash,

is not connected to anything.

It used to be connected to gold, to physical reality,

but then you can use money to exchange,

to exchange value, to transact.

So when it was on the gold standard,

the money would represent some stable component of reality.

Isn’t it more effective to avoid things like hyperinflation

if we generalize that idea?

Isn’t it better to connect your,

whatever we humans are doing

in the social interaction space with each other,

isn’t it better from an evolutionary perspective

to connect it to some degree to reality

so that the transactions are settled

with something that’s universal,

as opposed to us constantly operating

in something that’s a complete illusion?

Isn’t it easy to hyperinflate that?

Like where you really deviate very, very far away

from the underlying reality,

or do you not never get in trouble for this?

Can you just completely drift far, far away

from the underlying reality and never get in trouble?

That’s a great question, on the financial side,

there’s two levels at least

that we could take your question.

One is strictly like evolutionary psychology

of financial systems, and that’s pretty interesting.

And there the decentralized idea,

the DeFi kind of idea in cryptocurrencies

may make good sense

from just an evolutionary psychology point of view.

Having human nature being what it is,

putting a lot of faith in a few central controllers

depends a lot on the veracity of those

and trustworthiness of those few central controllers.

And we have ample evidence time and again

that that’s often betrayed.

So it makes good evolutionary sense, I would say,

to have a decentralized,

I mean, democracy is a step in that direction, right?

We don’t have a monarch now telling us what to do,

we decentralize things, right?

Because if the monarch,

if you have Marcus Aurelius as your emperor, you’re great.

If you have Nero, it’s not so great.

And so we don’t want that.

So democracy is a step in that direction,

but I think the DeFi thing is an even bigger step

and is going to even make the democratization even greater.

So that’s one level of it.

Also, the fact that power corrupts

and absolute power corrupts absolutely

is also a consequence of evolution.

That’s also a feature, I think, right?

You can argue from the long span of living organisms,

it’s nice for power to corrupt for you to,

so mad men and women throughout history

might be useful to teach us a lesson about ourselves.

We can learn from our negative example, right?

Exactly.

Right, right, right.

Power does corrupt and I think that you can think about that

again from an evolutionary point of view.

But I think that your question was a little deeper

when that was, does the evolutionary interface idea

sort of unhinge science from some kind of important test

for the theories, right?

We don’t want, it doesn’t mean that anything goes

in scientific theory, but there’s no,

if we don’t see the truth,

is there no way to tether our theories and test them?

And I think there’s no problem there.

We can only test things in terms of what we can measure

with our senses in space and time.

So we’re going to have to continue to do experiments

and, but we’re going to re,

we’re going to understand a little bit differently

what those experiments are.

We had thought that when we see a pointer

on some machine in an experiment,

that the machine exists, the pointer exists

and the values exist even when no one is looking at them

and that they’re an objective truth.

And our best theories are telling us no,

the pointers are just pointers

and that’s what you have to rely on

for making your judgments.

But even the pointers themselves

are not the objective reality.

So, and I think Gödel is telling us that,

not that anything goes, but as you develop

new axiom systems, you will find out what goes

within that axiom system

and what testable predictions you can make.

So I don’t think we’re untethered.

We continue to do experiments.

What I think we won’t have that we want

is a conceptual understanding

that gives us a theory of everything

that’s final and complete.

I think that this is, to put it another way,

this is job security for scientists.

Our job will never be done.

It’s job security for neuroscience.

Because before we thought that when we looked in the brain,

we saw neurons and neural networks

and action potentials and synapses and so forth.

And that was it, that was the reality.

Now we have to reverse engineer that.

We have to say, what is beyond space time?

What is going on?

What is a dynamical system beyond space time?

That when we project it into Einstein’s space time,

gives us things that look like neurons

and neural networks and synapses.

So we have to reverse engineer it.

So there’s gonna be lots more work for neuroscience.

It’s gonna be far more complicated

and difficult and challenging.

But that’s wonderful, that’s what we need to do.

We thought neurons exist when they are perceived

and they don’t.

In the same way that if I show you,

when I say they don’t exist,

I should be very, very concrete.

If I draw on a piece of paper,

a little sketch of something that is called the Necker cube,

it’s just a little line drawing of a cube, right?

It’s not a flat piece of paper.

If I execute it well, and I show it to you,

you’ll see a 3D cube and you’ll see it flip.

Sometimes you’ll see one face in front,

sometimes you’ll see the other face in front.

But if I ask you, which face is in front

when you don’t look?

The answer is, well, neither face is in front

because there’s no cube.

There’s just a flat piece of paper.

So when you look at the piece of paper,

you perceptually create the cube.

And when you look at it,

then you fix one face to be in front and one face to be.

So that’s what I mean when I say it doesn’t exist.

Space time itself is like the cube.

It’s a data structure that your sensory systems construct,

whatever your sensory systems mean now,

because we now have to not even take that for granted.

But there are perceptions that you construct on the fly

and they’re data structures in a computer science sense,

and you garbage collect them when you don’t need them.

So you create them and garbage collect them.

But is it possible that it’s mapped well

in some concrete, predictable way to objective reality?

The sheet of paper, this two dimensional space,

or we can talk about space time,

maps in some way that we maybe don’t yet understand,

but we’ll one day understand what that mapping is,

but it maps reliably.

It is tethered in that way.

Well, yes.

And so the new theories that the physicists are finding

beyond space time have that kind of tethering.

So they show precisely how you start with an epileptic hedron

and how you project this high dimensional structure

into the four dimensions of space time.

So there’s a precise procedure that relates the two.

And they’re doing the same thing

with the cosmological polytopes.

So they’re the ones that are making the most concrete

and fun advances going beyond space time.

And they’re tethering it, right?

They say this is precisely the mathematical projection

from this deeper structure into space time.

One thing I’ll say about, as a non physicist,

what I find interesting is that they’re finding just geometry,

but there’s no notion of dynamics.

Right now, they’re just finding

these static geometric structures, which is impressive.

So I’m not putting them down.

This is what they’re doing is unbelievably complicated

and brilliant and adventurous, it’s all those things.

And beautiful from a human aesthetic perspective

because geometry is beautiful.

It’s absolutely.

And they’re finding symmetries that are true of the data

that can’t be seen in space time.

But I’m looking for a theory beyond space time

that’s a dynamical theory.

I would love to find, and we can talk about that

at some point, a theory of consciousness

in which the dynamics of consciousness itself

will give rise to the geometry

that the physicists are finding beyond space time.

If we can do that,

then we’d have a completely different way

of looking at how consciousness is related

to what we call the brain or the physical world

more generally, right?

Right now, all of my brilliant colleagues,

well, 99% of them are trying to,

they’re assuming space time is fundamental.

They’re assuming that particles are fundamental,

quarks, gluons, leptons, and so forth.

Elements, atoms, and so forth are fundamental

and that therefore neurons and brains

are part of objective reality.

And that somehow when you get matter

that’s complicated enough,

it will somehow generate conscious experiences

by its functional properties.

Or if you’re panpsychist, maybe you,

in addition to the physical properties of particles,

you add your consciousness property as well.

And then you combine these physical and conscious properties

to get more complicated ones.

But they’re all doing it within space time.

All of the work that’s being done on consciousness

and its relationship to the brain

is all assumed something that our best theories

are telling us is doomed, space time.

Why does that particular assumption bother you the most?

So you bring up space time.

I mean, that’s just one useful interface

we’ve used for a long time.

Surely there’s other interfaces.

Is space time just one of the big ones

that you, to build up people’s intuition

about the fact that they do assume a lot of things strongly?

Or is it in fact the fundamental flaw

in the way we see the world?

Well, everything else that we think we know

are things in space time.

Sure.

And so when you say space time is doomed,

this is a shot to the heart of the whole framework,

the whole conceptual framework that we’ve had in science.

Not to the scientific method,

but to the fundamental ontology

and also the fundamental methodology,

the ontology of space time and its contents,

and the methodology of reductionism,

which is that as we go to smaller scales in space time,

we will find more and more fundamental laws.

And that’s been very useful for space and time for centuries,

reductionism for centuries.

But now we realize that that’s over.

Reductionism is in fact dead, as is space time.

What exactly is reductionism?

What is the process of reductionism

that is different than some of the physicists

that you mentioned that are trying to think,

trying to let go of the assumption of space time?

Looking beyond, isn’t that still trying to come up

with a simple model that explains this whole thing?

Isn’t it still reducing?

It’s a wonderful question,

because it really helps to clarify two different notions,

which is scientific explanation on the one hand,

and a particular kind of scientific explanation on the other,

which is the reductionist.

So the reductionist explanation is saying,

I will start with things that are smaller in space time

and therefore more fundamental,

where the laws are more fundamental.

So we go to just smaller and smaller scales.

Whereas in science more generally,

we just say like when Einstein

did the special theory of relativity,

he’s saying, let me have a couple of postulates.

I will assume that the speed of light is universal

for all observers in uniform motion,

and that the laws of physics,

so if you’re for uniform motion are,

that’s not a reductionist.

Those are saying, grant me these assumptions.

I can build this entire concept of space time out of it.

It’s not a reductionist thing.

You’re not going to smaller and smaller scales of space.

You’re coming up with these deep, deep principles.

Same thing with his theory of gravity, right?

It’s the falling elevator idea, right?

So this is not a reductionist kind of thing.

It’s something different.

So simplification is a bigger thing than just reductionism.

Reductionism has been a particularly useful

kind of scientific explanation,

for example, in thermodynamics, right?

Where the notion that we have of heat,

some macroscopic thing like temperature and heat,

it turns out that Neil Boltzmann and others discovered,

well, hey, if we go to smaller and smaller scales,

we find these things called molecules or atoms.

And if we think of them as bouncing around

and having some kind of energy,

then what we call heat really can be reduced to that.

And so that’s a particularly useful kind of reduction,

is a useful kind of scientific explanation

that works within a range of scales within space time.

But we know now precisely where that has to stop.

At 10 to the minus 33 centimeters

and 10 to the minus 43 seconds.

And I would be impressed

if it was 10 to the minus 33 trillion centimeters.

I’m not terribly impressed at 10 to the minus 33 centimeters.

I don’t even know how to comprehend

either of those numbers, frankly.

Just a small aside,

because I am a computer science person,

I also find cellular automata beautiful.

And so you have somebody like Stephen Wolfram,

who recently has been very excitedly exploring

a proposal for a data structure

that could be the numbers that would make you

a little bit happier in terms of scale,

because they’re very, very, very, very tiny.

So do you like this space of exploration

of really thinking, letting go of space time,

letting go of everything and trying to think

what kind of data structures

could be underneath this whole mess?

That’s right.

So if they’re thinking about these as outside of space time,

then that’s what we have to do.

That’s what our best theories are telling us.

You now have to think outside of space time.

Now, of course, I should back up and say,

we know that Einstein surpassed Newton, right?

But that doesn’t mean that there’s not good work

to do on Newton.

There’s all sorts of Newtonian physics

that takes us to the moon and so forth,

and there’s lots of good problems

that we want to solve with Newtonian physics.

The same thing will be true of space time.

It’s not like we’re gonna stop using space time.

We’ll continue to do all sorts of good work there.

But for those scientists who are really looking

to go deeper, to actually find the next,

just like what Einstein did to Newton,

what are we gonna do to Einstein?

How do we get beyond Einstein and quantum theory

to something deeper?

Then we have to actually let go.

And if we’re gonna do like this automata kind of approach,

it’s critical that it’s not automata in space time,

it’s automata prior to space time,

from which we’re going to show how space time emerges.

If you’re doing automata within space time,

well, that might be a fun model,

but it’s not the radical new step that we need.

Yeah, so the space time emerges from that whatever system.

Like you’re saying, it’s a dynamical system.

Do we even have an understanding what dynamical means

when we go beyond?

When you start to think about dynamics,

it could mean a lot of things.

Even causality could mean a lot of things

if we realize that everything’s an interface.

Like how much do we really know is an interesting question.

Because you brought up neurons,

I gotta ask you yet another tangent.

There’s a paper I remember a while ago looking at

called Could a Neuroscientist Understand a Microprocessor?

And I just enjoyed that thought experiment

that they provided, which is they basically,

it’s a couple of neuroscientists,

Eric Jonas and Conrad Cording,

who use the tools of neuroscience

to analyze a microprocessor, so a computer chip.

Now, if we lesion it here, what happens and so forth,

and if you go and lesion in a computer,

it’s very, very clear that lesion experiments on computers

are not gonna give you a lot of insight into how it works.

And also the measurement devices and the kind of sort of,

just using the basic approaches of neuroscience,

collecting the data, trying to intuit

about the underlying function of it.

And that helps you understand that

our scientific exploration of concepts,

depending on the field,

are maybe in the very, very early stages.

I wouldn’t say it leaves us astray,

perhaps it does sometimes,

but it’s not anywhere close to some fundamental mechanism

that actually makes a thing work.

I don’t know if you can sort of comment on that

in terms of using neuroscience

to understand the human mind and neurons.

Are we really far away potentially

from understanding in the way we understand

the transistors enough to be able to build a computer?

So one thing about understanding

is you can understand for fun.

The other one is to understand so you could build things.

And that’s when you really have to understand.

Exactly.

In fact, what got me into the field at MIT

was work by David Marr on this very topic.

So David Marr was a professor at MIT,

but he’d done his PhD in neuroscience,

studying just the architectures of the brain.

But he realized that his work, it was on the cerebellum.

He realized that his work, as rigorous as it was,

left him unsatisfied

because he didn’t know what the cerebellum was for

and why it had that architecture.

And so he went to MIT and he was in the AI lab there.

And he said, he had this three level approach

that really grabbed my attention.

So when I was an undergrad at UCLA,

I read one of his papers in a class and said,

who is this guy?

Because he said, you have to have a computational theory.

What is being computed and why?

An algorithm, how is it being computed?

What are the precise algorithms?

And then the hardware,

how does it get instantiated in the hardware?

And so to really do neuroscience, he argued,

we needed to have understanding at all those levels.

And that really got me.

I loved the neuroscience, but I realized this guy was saying,

if you can’t build it, you don’t understand it effectively.

And so that’s why I went to MIT.

And I had the pleasure of working with David

until he died just a year and a half later.

So there’s been that idea that with neuroscience,

we have to have, in some sense, a top down model

of what’s being computed and why

that we would then go after.

And the same thing with the, you know,

trying to reverse engineer a computing system

like your laptop.

We really need to understand

what the user interface is about

and what are keys on the keyboard for and so forth.

You need to know why to really understand

all the circuitry and what it’s for.

Now, we don’t, evolution of a natural selection

does not tell us the deeper question that we’re asking,

the answer to the deeper question, which is why?

What’s this deeper reality and what’s it up to and why?

All it tells us is that whatever reality is,

it’s not what you see.

What you see is just an adaptive fiction.

So just to linger on this fascinating, bold question

that shakes you out of your dream state.

Does this fiction still help you in building intuitions

as literary fiction does about reality?

The reason we read literary fiction

is it helps us build intuitions and understanding

in indirect ways sneak up to the difficult questions

of human nature, great fiction.

Same with this observed reality.

Does this interface that we get, this fictional interface,

help us build intuition about deeper truths

of how this whole mess works?

Well, I think that each theory that we propose

will give its own answer to that question, right?

So when the physicists are proposing these structures

like the amplituhedron and cosmological polytope,

associahedron and so forth beyond space time,

we can then ask your question for those specific structures

and say, how much information, for example,

does evolution by natural selection

and the kinds of sensory systems that we have right now

give us about this deeper reality?

And why did we evolve this way?

We can try to answer that question from within the deep.

So there’s not gonna be a general answer.

I think what we’ll have to do is posit

these new deeper theories

and then try to answer your question

within the framework of those deeper theories,

knowing full well that there’ll be an even deeper theory.

So is this paralyzing though?

Because how do we know we’re not completely adrift

out to sea, lost forever from,

so like that our theories are completely lost.

So if it’s all,

if we can never truly deeply introspect to the bottom,

if it’s always just turtles on top of turtles infinitely,

isn’t that paralyzing for the scientific mind?

Well, it’s interesting that you say introspect

to the bottom.

Because there is one,

again, this isn’t the same spirit of what I said before,

which is it depends on what answer you give

to what’s beyond space time,

what answer we would give to your question, right?

So, but one answer that is interesting to explore

is something that spiritual traditions have said

for thousands of years, but haven’t said precisely.

So we can’t take it seriously in science

until it’s made precise,

but we might be able to make it precise.

And that is that they’ve also said something like

space and time aren’t fundamental,

they’re Maya, they’re illusion.

And, but that if you look inside, if you introspect

and let go of all of your particular perceptions,

you will come to something that’s beyond conceptual thought.

And that is, they claim,

being in contact with the deep ground of being

that transcends any particular conceptual understanding.

If that is correct, and I’m not saying it’s correct,

but, and I’m not saying it’s not correct,

I’m just saying, if that’s correct,

then it would be the case that as scientists,

because we also are in touch with this ground of being,

we would then not be able

to conceptually understand ourselves all the way,

but we could know ourselves just by being ourselves.

And so we would, there would be a sense

in which there is a fundamental grounding

to the whole enterprise,

because we’re not separate from the enterprise.

This is the opposite of the impersonal third person science.

This would make science go personal all the way down.

And, but nevertheless, scientific,

because the scientific method would still be

what we would use all the way down

for the conceptual understanding.

Unfortunately, I still don’t know

if you went all the way down.

It’s possible that this kind of whatever consciousness is

and we’ll talk about it,

is getting the cliche statement of be yourself.

It is somehow digging at a deeper truth of reality,

but you still don’t know when you get to the bottom.

A lot of people, they’ll take psychedelic drugs

and they’ll say, well, that takes my mind to certain places

where it feels like that is revealing

some deeper truth of reality,

but you still, it could be interfaces on top of interfaces.

That’s, in your view of this, you really don’t know.

I mean, it’s Gato’s incompleteness

is that you really don’t know.

My own view on it, for what it’s worth,

because I don’t know the right answer,

but my own view on it right now is that it’s never ending.

I think that there will never,

that this is great, as I said before,

great job security for science.

And that we, if this is true,

and if consciousness is somehow important

or fundamental in the universe,

this may be an important fundamental fact

about consciousness itself,

that it’s a never ending exploration

that’s going on in some sense.

Well, that’s interesting.

Push back on the job security.

Okay.

So maybe as we understand this kind of idea

deeper and deeper,

we understand that the pursuit is not a fruitful one.

Then maybe we need to,

maybe that’s why we don’t see aliens everywhere,

is you get smarter and smarter and smarter,

you realize that exploration is,

there’s other fun ways to spend your time than exploring.

You could be sort of living maximally

in some way that’s not exploration.

There’s all kinds of video games you can construct

and put yourself inside of them

that don’t involve you going outside of the game world.

It’s a feeling, from my human perspective,

what seems to be fun is challenging yourself

and overcoming those challenges.

So you can constantly artificially generate challenges

for yourself, like Sisyphus and his boulder,

just, and that’s it.

So the scientific method

that’s always reaching out to the stars,

that’s always trying to figure out

the puzzle on the bottom puzzle,

we’re always trying to get to the bottom turtle.

Maybe if we can build more and more the intuition

that that’s infinite pursuit,

we agree to start deviating from that pursuit

and start enjoying the here and now

versus the looking out into the unknown always.

Maybe that’s looking out into the unknown

as a early activity for a species that’s evolved.

I’m just sort of saying, pushing back,

as you probably got a lot of scientists excited

in terms of job security,

I could envision where it’s not job security,

where scientists become more and more useless.

Maybe they’re like the holders of the ancient wisdom

that allows us to study our own history,

but not much more than that, just to push back.

That’s good pushback.

I’ll put one in there for the scientists again,

but sure, but then I’ll take the other side too.

So when Faraday did all of his experiments

with magnets and electricity and so forth,

he came up with all this wonderful empirical data

and James Clerk Maxwell looked at it

and wrote down a few equations,

which we can now write down in a single equation,

the Maxwell equation if we use geometric algebra,

just one equation.

That opened up unbelievable technologies.

People are zooming and talking to each other

around the world, the whole electronics industry.

There was something that transformed our lives

in a very positive way.

With the theories beyond space time,

here’s one potential, right now,

most of the galaxies that we see, we can see them,

but we know that we could never get to them

no matter how fast we traveled.

They’re going away from us at the speed of light or beyond.

So we can’t ever get to them.

So there’s all this beautiful real estate

that’s just smiling and waving at us

and we can never get to it.

Yeah.

But that’s if we go through space time.

But if we recognize that space time

is just a data structure, it’s not fundamental.

We’re not little things inside space time.

Space time was a little data structure in our perceptions.

It’s just the other way around.

Once we understand that,

and we get equations for the stuff that’s beyond space time,

maybe we won’t have to go through space time.

Maybe we can go around it.

Maybe I can go to Proxima Centauri

and not go through space.

I can just go right there directly.

It’s a data structure.

We can start to play with it.

So I think that for what it’s worth,

my take would be that the endless sequence of theories

that we could contemplate building

will lead to an endless sequence of new remarkable insights

into the potentialities, the possibilities

that would seem miraculous to us.

And that we will be motivated to continue the exploration

partly just for the technological innovations

that come out.

But the other thing that you mentioned though,

what about just being?

What if we decide instead of all this doing and exploring,

what about being?

My guess is that the best scientists will do both

and that the act of being will be a place

where they get many of their ideas

and that they then pull into the conceptual realm.

And I think many of the best scientists,

like Einstein comes to mind, right?

Where these guys say, look,

I didn’t come up with these ideas by a conceptual analysis.

I was thinking in vague images

and it was just something nonconceptual.

And then it took me a long, long time

to pull it out into concepts

and then longer to put it into math.

But the real insights didn’t come from data.

The real insights didn’t come from just slavishly

playing with equations.

They came from a deeper place.

And so there may be this going back and forth

between the complete nonconceptual

where there’s essentially no end to the wisdom

and then conceptual systems

where there’s the girdle limits that we have to that.

And that may be, if consciousness is important

and fundamental, that may be what consciousness,

at least part of what consciousness is about

is this discovering itself, discovering its possibilities,

so to speak, and we can talk about what that might mean,

by going from the nonconceptual to the conceptual

and back and forth.

So you get better and better and better at being.

Right.

Let me ask you just to linger on the evolutionary,

because you mentioned evolutionary game theory

and that’s really where you,

the perspective from which you come

to form the case against reality.

At which point in our evolutionary history

do we start to deviate the most from reality?

Is it way before life even originated on Earth?

Is it in the early development from bacteria and so on?

Or is it when some inklings of what we think of

as intelligence or maybe even complex consciousness

started to emerge?

So where did this deviation,

just like with the interfaces in a computer,

you start with transistors and then you have assembly

and then you have C, C++, then you have Python,

then you have GUIs, all that kind,

you have layers upon layers.

When did we start to deviate?

Well, David Marr, again, my advisor at MIT,

in his book, Vision,

suggested that the more primitive sensory systems

were less realistic, less theoretical,

but that by the time you got to something

as complicated as the humans,

we were actually estimating the true shapes

and distances to objects and so forth.

So his point of view, and I think it was probably,

it’s not an uncommon view among my colleagues

that, yeah, the sensory systems of lower creatures

may just not be complicated enough

to give them much, much truth.

But as you get to 86 million neurons,

you can now compute the truth,

or at least the parts of the truth that we need.

When I look at evolutionary game theory,

one of my graduate students, Justin Mark,

did some simulations using genetic algorithms.

So there he was just exploring,

we start off with random organisms,

random sensory genetics and random actions.

And the first generation was unbelievably,

it was a foraging situation.

They were foraging for resources.

Most of them stayed in one place,

didn’t do anything important.

But we could then just look at how the genes evolved.

And what we found was,

what he found was that basically you never even saw

the truth organisms even come on the stage.

If they came up, they were gone in one generation,

they just weren’t.

So they came and went even just in one generation.

They just are not good enough.

The ones that were just tracking,

their senses just were tracking the fitness payoffs

were far more fit than the truth seekers.

So an answer at one level,

I want to give an answer at a deeper level,

but just with evolutionary game theory,

because my attitude as a scientist is,

I don’t believe any of our theories.

I take them very, very seriously.

I study them, I look at their implications,

but none of them are the gospel.

They’re just the latest ideas that we have.

And so the reason I study evolutionary game theory

is because that’s the best tool we have right now

in this area.

There is nothing else that competes.

And so as a scientist, it’s my responsibility

to take the best tools and see what they mean.

And the same thing the physicists are doing.

They’re taking the best tools

and looking at what they entail.

But I think that science now has enough experience

to realize that we should not believe our theories

in the sense that we’ve now arrived.

In 1890, a lot of physicists thought we’d arrived.

They were discouraging bright young students

from going into physics, because it was all done.

And that’s precisely the wrong attitude forever.

It’s the wrong attitude forever.

The attitude we should have is a century from now,

they’ll be looking at us and laughing

at what we didn’t know.

And we just have to assume that that’s going to be the case.

Just know that everything that we think

is so brilliant right now, our final theory.

A century from now, they’ll look at us

like we look at the physicists of 1890 and go,

how could they have been so dumb?

So I don’t want to make that mistake.

So I’m not doctrinaire about any

of our current scientific theories.

I am doctrinaire about this.

We should use the best tools we have right now.

That’s what we’ve got.

And with humility.

Well, so let me ask you about game theory.

I love game theory, evolutionary game theory.

But I’m always suspicious of it, like economics.

When you construct models,

it’s too easy to construct things that oversimplify

just because we, our human brains,

enjoy the simplification of constructing a few variables

that somehow represent organisms or represent people

and running a simulation that then allows you

to build up intuition and then it feels really good

because you can get some really deep

and surprising intuitions.

But how do you know your models aren’t,

the assumptions underlying your models

aren’t some fundamentally flawed?

And because of that,

your conclusions are fundamentally flawed.

So I guess my question is what are the limits

in your use of game theory, evolution game theory,

your experience with it?

What are the limits of game theory?

So I’ve gotten some pushback from professional colleagues

and friends who have tried to rerun simulations

and try to, the idea that we don’t see the truth

is not comfortable and so many of my colleagues

are very interested in trying to show that we’re wrong.

And so the idea would be to say that somehow

we did something, as you’re suggesting,

maybe something special that wasn’t completely general.

We got some little special part of the whole search space

in evolutionary game theory in which this happens to be true

but more generally organisms would evolve

to see the truth.

So the best pushback we’ve gotten is from a team at Yale.

And they suggested that if you use

thousands of payoff functions,

so we in our simulations, we just use a couple,

one or two, because it was our first simulations, right?

So that would be a limit.

We had one or two payoff functions,

we showed the result of those,

at least for the genetic algorithms.

And they said, if you have 20,000 of them,

then we can find these conditions in which

truth seeing organisms would be the ones

that evolved and survived.

And so we looked at their simulations

and it certainly is the case that you can find

special cases in which truth can evolve.

So when I say it’s probability zero,

it doesn’t mean it can’t happen.

It can happen, in fact, it could happen infinitely often.

It’s just probability zero.

So probability zero things can happen infinitely often.

When you say probability is zero, you mean probability

close to zero.

To be very, very precise.

So for example, if I have a unit square on the plane

and I use a measure on a probability measure

in which the area of a region is this probability.

Then if I draw a curve in that unit square,

it has measure precisely zero,

precisely not approximately, precisely zero.

And yet it has infinitely many points.

So there’s an object that for that probability measure

has probability zero, and yet there’s

infinitely many points in it.

So that’s what I mean when I say that things

that are probability zero can happen

infinitely often in principle.

Yeah, but infinity, as far as, and I look outside often,

I walk around and I look at people.

I have never seen infinity in real life.

That’s an interesting issue.

I’ve been looking, I’ve been looking.

I don’t notice it, infinitely small or the infinitely big.

And so the tools of mathematics,

you could sort of apply the same kind of criticism

that it is a very convenient interface into our reality.

That’s a big debate in mathematics,

the intuitionists versus the ones who take,

for example, the real numbers as real.

And that’s a fun discussion.

Nicholas Giesen, a physicist,

has really interesting work recently

on how if you go with intuitionist mathematics,

you could effectively quantize Newton,

and you find that the Newtonian theory

and quantum theory aren’t that different

once you go with it.

It’s funny.

It’s really quite interesting.

So the issue he raises is a very, very deep one,

and one that I think we should take quite seriously,

which is how should we think about the reality

of the contours hierarchy?

Aleph one, aleph two, and all these different infinities

versus just a more algorithmic approach, right?

So where everything’s computable,

in some sense, everything’s finite,

as big as you want, but nevertheless finite.

So yeah, that ultimately boils down to

whether the world is discrete or continuous

in some general sense.

And again, we can’t really know,

but there’s just a mind breaking thought,

just common sense reasoning,

that something can happen,

and as yet, probability of it happening is 0%.

That doesn’t compute for common sense computer.

Right.

This is where you have to be a sharp mathematician

to really, and I’m not.

Sharp is one word.

What I’m saying is common sense computer is,

I mean that in a very kind of,

in a positive sense,

because we’ve been talking about perception systems

and interfaces, if we are to reason about the world,

we have to use the best interfaces we got.

And I’m not exactly sure that game theory

is the best interface we got for this.

Oh, right.

In application of mathematics, tricks and tools

in mathematics, the game theory is the best we got

when we are thinking about the nature of reality

and fitness functions and evolution, period.

Right.

Well, that’s a fair rejoinder,

and I think that that was the tool that we used.

And if someone says, here’s a better mathematical tool

and here’s why, this mathematical tool

better captures the essence of Darwin’s idea,

John Maynard Smith didn’t quite get it

with evolutionary game theory.

There’s this thing.

Now there are tools like evolutionary graph theory,

which generalize evolutionary game theory,

and then there’s quantum game theory.

So you can use quantum tools like entanglement,

for example, as a resource in games

that change the very nature of the solutions,

the optimal solutions of the game theory.

Well, the work from Yale is really interesting.

It’s a really interesting challenge of these ideas

where, okay, if you have a very large number

of fitness functions, or let’s say you have

a nearly infinite number of fitness functions

or a growing number of fitness functions,

what kind of interesting things start to emerging

if you are to be an organism?

If to be an organism that adapts means

having to deal with an ensemble of fitness functions.

Right, and so we’ve actually redone some of our own work

based on theirs, and this is the back and forth

that we expect in science, right?

And what we found was that in their simulations,

they were assuming that you couldn’t carve the world

up into objects, and so we said,

well, let’s relax that assumption.

Allow organisms to create data structures

that we might call objects,

and an object would be you take,

you would do hierarchical clustering

of your fitness payoff functions,

the ones that have similar shapes.

If you have 20,000 of them, maybe these 50

are all very, very similar,

so I can take all the perception, action, fitness stuff

and make that into a data structure,

and we’ll call that a unit or an object.

And as soon as we did that,

then all of their results went away.

It turned out they were the special case

and that the organisms that were allowed

to only see, that were shaped to see only fitness payoffs

were the ones that were.

So the idea is that objects then,

what are objects from an evolutionary point of view?

This bottle, we thought that when I saw a bottle,

it was because I was seeing a true object

that existed whether or not it was perceived.

Evolutionary theories suggest a different interpretation.

I’m seeing a data structure that is encoding

a convenient way of looking at various fitness payoffs.

I can use this for drinking.

I could use it as a weapon, not a very good one.

I could be somewhere with head with it.

If my goal is mating, this is pointless.

So I’m seeing for, what I’m coding here

is all sorts of actions and the payoffs that I could get.

When I pick up an apple,

now I’m getting a different set of actions and payoffs.

When I pick up a rock, I’m getting, so for every object,

what I’m getting is a different set of payoff functions

and with various actions.

And so once you allow that,

then what you find is once again that truth goes extinct

and the organisms that just get an interface

are the ones that win.

But the question, just sneaking up on, this is fascinating.

From where do fitness functions originate?

What gives birth to the fitness functions?

So if there’s a giant black box

that just keeps giving you fitness functions,

what are we trying to optimize?

You said that water has different uses than an apple.

So there’s these objects.

What are we trying to optimize?

And why is not reality a really good generator

of fitness functions?

So each theory makes its own assumptions and says,

grant me this, then I’ll explain that.

So evolutionary game theory says,

grant me fitness payoffs, right?

And grant me strategies with payoffs.

And I can write down the matrix

for this strategy interacts with that strategy.

These are the payoffs that come up.

If you grant me that,

then I can start to explain a lot of things.

Now you can ask for a deeper question like,

okay, how does physics evolve biology

and where do these fitness payoffs come from, right?

Now that’s a completely different enterprise.

And of course, evolutionary game theory then

would be not the right tool for that.

It would have to be a deeper tool

that shows where evolutionary game theory comes from.

My own take is that there’s gonna be a problem

in doing that because space time isn’t fundamental.

It’s just a user interface.

And that the distinction that we make

between living and nonliving

is not a fundamental distinction.

It’s an artifact of the limits of our interface, right?

So this is a new wrinkle and this is an important wrinkle.

It’s so nice to take space and time as fundamental

because if something looks like it’s inanimate,

it’s inanimate and we can just say it’s not living.

Now it’s much more complicated.

Certain things are obviously living.

I’m talking with you, I’m obviously interacting

with something that’s alive and conscious.

I think we’ve let go of the word obviously

in this conversation.

I think nothing is obvious.

Nothing’s obvious, that’s right.

But when we get down to like an ant,

it’s obviously living, but I’ll say it appears to be living.

But when we get down to a virus, now people wonder

and when we get down to protons,

people say it’s not living.

And my attitude is look, I have a user interface.

Interface is there to hide certain aspects of reality

and others to, it’s an uneven representation,

put it that way.

Certain things just get completely hidden.

Dark matter and dark energy are most of the energy

and matter that’s out there.

Our interface just plain flat out hides them.

The only way we get some hint is because

gravitational things are going wrong within our,

so most things are outside of our interface.

The distinction between living and nonliving

is not fundamental.

It’s an artifact of our interface.

So if we really, really want to understand

where evolution comes from,

to answer the question, the deep question you asked,

I think the right way we’re gonna have to do that

is to come up with a deeper theory than space time

in which there may not be the notion of time

and show that whatever this dynamics of that deeper theory

is, by the way, I’ll talk about how you could have dynamics

without time, but the dynamics of this deeper theory,

when we project it into, in certain ways,

then we do get space time and we get what appears

to be evolution by natural selection.

So I would love to see evolution by natural selection,

nature, red and tooth and claw, people fighting,

animals fighting for resources and the whole bit,

come out of a deeper theory in which perhaps

it’s all cooperation, there’s no limited resources

and so forth, but as a result of projection,

you get space and time, and as a result of projection,

you get nature, red and tooth and claw,

the appearance of it, but it’s all an artifact

of the interface.

I like this idea that the line between living

and nonliving is very important

because that’s the thing that would emerge

before you have evolution, the idea of death.

So that seems to be an important component

of natural selection, and if that emerged,

because that’s also asking the question,

I guess, that I ask, where do fitness functions come from?

That’s like asking the old meaning of life question, right?

It’s the why, why, why?

And one of the big underlying whys,

okay, you can start with evolution on Earth,

but without living, without life and death,

without the line between the living and the dead,

you don’t have evolution.

So what if underneath it, there’s no such thing

as the living and the dead?

There’s no, like this concept of an organism, period.

There’s a living organism that’s defined

by a volume in space time that somehow interacts,

that over time maintains its integrity somehow.

It has some kind of history, it has a wall of some kind.

The outside world, the environment,

and then inside, there’s an organism.

So you’re defining an organism,

and also you define that organism

by the fact that it can move, and it can become alive,

which you kind of think of as moving,

combined with the fact that it’s keeping itself

separate from the environment,

so you can point out that thing is living,

and then it can also die.

That seems to be all very powerful components of space time

that enable you to have something

like natural selection and evolution.

Well, and there’s a lot of interesting work,

some of it by collaborators of Carl Friston and others,

where they have Bayes net kind of stuff

that they build on the notion of a Markov blanket.

So you have some states within this network

that are inside the blanket, then you have the blanket,

and then the states outside the blanket.

And the states inside this Markov blanket

are conditionally independent of the states

outside the blanket conditioned on the blanket.

And what they’re looking at is that the dynamics inside

of the states inside the Markov blanket

seem to be trying to estimate properties of the outside

and react to them in a way.

So it seems like you’re doing probabilistic inferences

in ways that might be able to keep you alive.

So there’s interesting work going on in that direction.

But what I’m saying is something slightly different,

and that is, like, when I look at you,

all I see is skin, hair, and eyes, right?

That’s all I see.

But I know that there’s a deeper reality.

I believe that there’s a much deeper reality.

There’s the whole world of your experiences,

your thoughts, your hopes, your dreams.

In some sense, the face that I see

is just a symbol that I create, right?

And as soon as I look away, I delete that symbol.

But I don’t delete you.

I don’t delete the conscious experience,

the whole world of your…

So I’m only deleting an interface symbol.

But that interface symbol is a portal, so to speak.

Not a perfect portal, but a genuine portal

into your beliefs, into your conscious experiences.

That’s why we can have a conversation.

Your consciousness is genuinely affecting mine,

and mine is genuinely affecting yours,

through these icons, which I create on the fly.

I mean, I create your face.

When I look, I delete it.

I don’t create you, your consciousness.

That’s there all the time, but I do…

So now, when I look at a cat,

I’m creating something that I still call living,

and I still think is conscious.

When I look at an ant, I create something

that I still would call living, but maybe not conscious.

When I look at something I call a virus,

now I’m not even sure I would call it living.

And when I look at a proton, I would say,

I don’t even think it’s not alive at all.

It could be that I’m nevertheless interacting

with something that’s just as conscious as you.

I’m not saying the proton is conscious.

The face that I’m creating when I look at you,

that face is not conscious.

That face is a data structure in me.

That face is an experience.

It’s not an experiencer.

Similarly, a proton is something that I create

when I look or do a collision

in the Large Hadron Collider or something like that.

But what is behind the entity in space time?

So I’ve got this space time interface,

and I’ve just got this entity that I call a proton.

What is the reality behind it?

Well, the physicists are finding these big, big structures.

The amplitude hadron, the sociahedron,

cause what’s behind those?

Could be consciousness, what I’m playing with.

In which case, when I’m interacting with a proton,

I could be interacting with consciousness.

Again, to be very, very clear,

because it’s easy to misunderstand,

I’m not saying a proton is conscious.

Just like I’m not saying your face is conscious.

Your face is a symbol I create and then delete as I look.

So your face is not conscious,

but I know that that face in my interface,

the Lex Friedman face that I create,

is an interface symbol that’s a genuine portal

into your consciousness.

The portal is less clear for a cat,

even less clear for an ant.

And by the time we get down to a proton,

the portal is not clear at all.

But that doesn’t mean I’m not interacting

with consciousness, it just means my interface gave up.

And there’s some deeper reality that we have to go after.

So your question really forces out a big part

of this whole approach that I’m talking about.

So it’s this portal and consciousness.

I wonder why you can’t,

your portal is not as good to a cat,

to a cat’s consciousness than it is to a human.

Does it have to do with the fact that you’re human

and just similar organisms, organisms of similar complexity

are able to create portals better to each other?

Or is it just as you get more and more complex,

you get better and better portals?

Well, let me answer one aspect of it

that I’m more confident about,

then I’ll speculate on that.

Why is it that the portal is so bad with protons?

Well, and elementary particles more generally.

So quarks, leptons and gluons and so forth.

Well, the reason for that is because those are just

symmetries of space time.

More technically, they’re irreducible representations

of the Poincare group of space time.

So they’re just literally representations

of the data structure of space time that we’re using.

So that’s why they’re not very much insightful.

They’re just almost entirely tied

to the data structure itself.

There’s not much,

they’re telling you only something about the data structure,

not behind the data structure.

It’s only when we get to higher levels

that we’re starting to, in some sense,

build portals to what’s behind space time.

Sure.

Yeah, so there’s more and more complexity built

on top of the interface of space time with the cat.

So you can actually build a portal, right?

Yeah.

Yeah, right.

Yeah, this interface of face and hair and so on, skin.

There’s some syncing going on between humans though,

where we synced, like you’re getting

a pretty good representation of the ideas in my head

and starting to get a foggy view of my memories in my head.

Even though this is the first time we’re talking,

you start to project your own memories.

You start to solve like a giant hierarchy of puzzles

about a human, because we’re all,

there’s a lot of similarities, a lot of it rhymes.

So you start to make a lot of inferences

and you build up this model of a person.

You have a pretty sophisticated model

what’s going on underneath.

Again, I just, I wonder if it’s possible

to construct these models about each other

and nevertheless be very distant from an underlying reality.

There’s a lot of work on this.

So there’s some interesting work called signaling games

where they look at how people can coordinate

and come to communicate.

There’s some interesting work that was done

by some colleagues and friends of mine,

Louis Narens, Natalia Komarova, and Kimberly Jamieson,

where they were looking at evolving color words.

So you have a circle of colors, the color circle,

and they wanted to see if they could get people to cooperate

and how they carved the color circle up into a circle.

Two units of words.

And so they had a game theoretic kind of thing

that they’d had people do.

And what they found was that when they included,

so most people are trichromats,

you have three kinds of cone photoreceptors,

but there are some, a lot of men,

7% of men are dichromats.

They might be missing the red cone photoreceptor.

They found that the dichromats had an outsized influence

on the final ways that the whole space of colors

was carved up and labels attached.

You needed to be able to include the dichromats

in the conversation.

And so they had a bigger influence

on how you made the boundaries of the language.

And I thought that was a really interesting kind of insight

that there’s going to be, again, a game,

perhaps a game where evolutionary or genetic algorithm

kind of thing that goes on in terms of learning

to communicate in ways that are useful.

And so, yeah, you can use game theory to actually explore

that are signaling games.

There’s a lot of brilliant work on that.

I’m not doing it, but there’s work out there.

So if it’s okay, let us tackle once more

and perhaps several more times

after the big topic of consciousness.

Okay, this very beautiful, powerful things

that perhaps is the thing that makes us human, what is it?

What’s the role of consciousness in,

let’s say even just the thing we’ve been talking about,

which is the formation of this interface, any kind of ways

you want to kind of start talking about it.

Well, let me say first what most of my colleagues say.

99% are, again, assuming that space time is fundamental,

particles and space time, matter is fundamental,

and most are reductionist.

And so the standard approach to consciousness

is to figure out what complicated systems of matter

with the right functional properties

could possibly lead to the emergence of consciousness.

That’s the general idea, right?

So maybe you have to have neurons,

maybe only if you have neurons, but that might not be enough.

They have to certain kinds of complexity

in their organization and their dynamics,

certain kind of network abilities, for example.

So there are those who say, for example,

that consciousness arises from orchestrated collapse

of quantum states of microtubules and neurons, certainly.

So this is Hamroff and Penrose, that’s kind of.

So you start with something physical,

a property of quantum states of neurons,

of microtubules and neurons,

and you say that somehow an orchestrated collapse

of those is consciousness or conscious experiences.

Or integrated information theory.

Again, you start with something physical,

and if it has the right kind of functional properties,

it’s something they call phi,

with the right kind of integrated information,

then you have consciousness.

Or you can be a panpsychist, Philip Goff, for example,

where you might say, well,

in addition to the particles and space and time,

those particles are not just matter,

they also could have, say, a unit of consciousness.

And so, but once again, you’re taking space and time

and particles as fundamental,

and you’re adding a new property to them,

say, consciousness, and then you have to talk about how

when a proton and an electron get together

to form hydrogen, then how those consciousnesses

merge to or interact to create the consciousness

of hydrogen and so forth.

There’s attention schema theory,

which again, this is how neural network processes

representing to the network itself,

its attentional processes, that could be consciousness.

There’s global workspace theory,

and neuronal global workspace theory.

So there’s many, many theories of this type.

What’s common to all of them is they assume

that space time is fundamental.

They assume that physical processes

and space time is fundamental.

Panpsychism adds consciousness as an additional thing,

it’s almost dualist in that regard.

And my attitude is our best science is telling us

that space time is not fundamental.

So why is that important here?

Well, for centuries, deep thinkers thought of earth, air,

fire, and water as the fundamental elements.

It was a reductionist kind of idea.

Nothing was more elemental than those,

and you could sort of build everything up from those.

When we got the periodic table of elements,

we realized that, of course,

we want to study earth, air, fire, and water.

There’s combustion science for fire.

There’s sciences for all these other things,

water and so forth.

So we’re gonna do science with these things,

but fundamental, no, no.

If you’re looking for something fundamental,

those are the wrong building blocks.

Earth has many, many different kinds of elements

that project into the one thing that we call earth.

If you don’t understand that there’s silicon,

that there’s iron,

that there’s all these different kinds of things

that project into what we call earth,

you’re hopelessly lost.

You’re not fundamental, you’re not gonna get there.

And then after the periodic table,

then we came up with quarks, leptons, and gluons,

the particles of the standard model of physics.

And so we actually now know

that if you really want to get fundamental,

the periodic table isn’t it.

It’s good for chemistry,

and it’s just wonderful for chemistry,

but if you’re trying to go deep fundamental,

what is the fundamental science?

That’s not it.

You’re gonna have to go to quarks, leptons,

and gluons and so forth.

Well, now we’ve discovered space time itself is doomed.

Quarks, leptons, and gluons

are just irreducible representations

of the symmetries of space time.

So the whole framework

on which consciousness research is being based right now

is doomed.

And for me, these are my friends and colleagues

that are doing this, they’re brilliant.

They’re absolutely, they’re brilliant.

I, my feeling is I’m so sad

that they’re stuck with this old framework

because if they weren’t stuck with earth, air, fire,

and water, you could actually make progress.

So it doesn’t matter how smart you are.

If you start with earth, air, fire, and water,

you’re not gonna get anywhere, right?

Can I actually just,

because the word doomed is so interesting,

let me give you some options, multiple choice quiz.

Is space time, we could say is reality

the way we perceive it doomed,

wrong or fake?

Because doomed just means it could still be right

and we’re now ready to go deeper.

It would be that.

So it’s not wrong, it’s not a complete deviation

from a journey toward the truth.

Right, it’s like earth, air, fire, and water is not wrong.

There is earth, air, fire, and water.

That’s a useful framework, but it’s not fundamental.

Right, well, there’s also wrong,

which is they used to believe, as I recently learned,

that George Washington was the president,

the first president of the United States,

was bled to death for something

that could have been easily treated

because it was believed that you can get,

actually, I need to look into this further,

but I guess you get toxins out or demons out.

I don’t know what you’re getting out

with the bleeding of a person.

So that ended up being wrong,

but widely believed as a medical tool.

So it’s also possible that our assumption of space time

is not just doomed, but is wrong.

Well, if we believe that it’s fundamental, that’s wrong.

But if we believe it’s a useful tool, that’s right.

But bleeding somebody to death

was believed to be a useful tool.

And that was wrong.

It wasn’t just not fundamental.

It was very, I’m sure there’s cases

in which bleeding somebody would work,

but it would be a very tiny, tiny, tiny percentage of cases.

So it could be that it’s wrong,

like it’s a side road that’s ultimately leading

to a dead end as opposed to a truck stop or something

that you can get off of.

My feeling is not the dead end kind of thing.

I think that what the physicists are finding

is that there are these structures beyond space time,

but they project back into space time.

And so space time, when they say space time is doomed,

they’re explicit.

They’re saying it’s doomed in the sense

that we thought it was fundamental.

It’s not fundamental.

It’s a useful, absolutely useful and brilliant data structure,

but there are deeper data structures

like cosmological polytope and space time is not fundamental.

What is doomed in the sense that it’s wrong

is reductionism.

Which is saying space time is fundamental, essentially.

Right, right.

The idea that somehow being smaller in space and time

or space time is a fundamental nature of reality,

that’s just wrong.

It turned out to be a useful heuristic

for thermodynamics and so forth.

And in several other places,

reductionism has been very useful,

but that’s in some sense an artifact

of how we use our interface.

Yeah, so you’re saying size doesn’t matter.

Okay, this is very important for me to write down.

Ultimately. Ultimately, right.

It’s useful for theories like thermodynamics

and also for understanding brain networks

in terms of individual neurons and neurons

in terms of chemical systems inside cells.

That’s all very, very useful,

but the idea that we’re getting

to the more fundamental nature of reality, no.

When you get all the way down in that direction,

you get down to the quarks and gluons,

what you realize is what you’ve gotten down to

is not fundamental reality,

just the irreducible representations of a data structure.

That’s all you’ve gotten down to.

So you’re always stuck inside the data structure.

So you seem to be getting closer and closer.

I went from neural networks to neurons,

neurons to chemistry, chemistry to particles,

particles to quarks and gluons.

I’m getting closer and closer to the real.

No, I’m getting closer and closer to the actual structure

of the data structure of space and time,

the irreducible representations.

That’s what you’re getting closer to,

not to a deeper understanding of what’s beyond space time.

We’ll also refer, we’ll return again

to this question of dynamics

because you keep saying that space time is doomed,

but mostly focusing on the space part of that.

It’s very interesting to see why time gets the bad cred too

because how do you have dynamics without time

is the thing I’d love to talk to you a little bit about.

But let us return your brilliant whirlwind overview

of the different theories of consciousness

that are out there.

What is consciousness if outside of space time?

If we think that we want to have a model of consciousness,

we as scientists then have to say,

what do we want to write down?

What kind of mathematical modeling

are we gonna write down, right?

And if you think about it, there’s lots of things

that you might want to write down about consciousness.

For all the complicated subject.

So most of my colleagues are saying,

let’s start with matter or neurons

and see what properties of matter

could create consciousness.

But I’m saying that that whole thing is out.

Space time is doomed, that whole thing is out.

We need to look at consciousness qua consciousness.

In other words, not as something that arises

in space and time, but perhaps as something

that creates space and time as a data structure.

So what do we want?

And here again, there’s no hard and fast rule,

but what you as a scientist have to do

is to pick what you think are the minimal assumptions

that are gonna allow you to boot up a comprehensive theory.

That is the trick.

So what do I want?

So what I chose to do was to have three things.

I said that there are conscious experiences.

Feeling of headache, the smell of garlic,

experiencing the color red.

There are, those are conscious,

so that’s the primitive of the theory.

And the reason I want few primitives, why?

Because those are the miracles of the theory, right?

The primitives, the assumptions of the theory

are the things you’re not going to explain.

Those are the things you assume.

And those experiences, you particularly mean

there’s a subjectiveness to them.

That’s the thing when people refer

to the hard problem of consciousness,

is it feels like something to look at the color red, okay.

Exactly right, it feels like something to have a headache

or to feel upset to your stomach.

It feels like something.

And so I’m going to grant that in this theory,

there are experiences and they’re fundamental in some sense.

So conscious experience.

So they’re not derived from physics.

They’re not functional properties of particles.

They are sui generis, they exist.

Just like we assume space time exists.

I’m now saying space time is just a data structure.

It doesn’t exist independent of conscious experiences.

Sorry to interrupt once again,

but should we be focusing in your thinking on humans alone?

Or is there something about in relation

to other kinds of organisms that have

a sufficiently high level of complexity?

Or even, or is there some kind of generalization

of the panpsychist idea that all consciousness permeates,

all matter?

Outside of the usual definition

of what matter is inside space time.

So it’s beyond human consciousness.

Human consciousness, from my point of view,

would be one of a countless variety of consciousnesses.

And even within human consciousness,

there’s countless variety of consciousnesses within us.

I mean, you have your left and right hemisphere.

And apparently if you split the corpus callosum,

the personality of the left hemisphere

and the religious beliefs of the left hemisphere

can be very different from the right hemisphere.

And their conscious experiences can be disjoint.

One could have one conscious experience.

They can play 20 questions.

The left hemisphere can have an idea in its mind

and the right hemisphere has to guess.

And it might not get it.

So even within you,

there is more than just one consciousness.

It’s lots of consciousnesses.

So the general theory of consciousness that I’m after

is not just human consciousness.

It’s going to be just consciousness.

And I presume human consciousness is a tiny drop

in the bucket of the infinite variety of consciousnesses.

That said, I should clarify that the black hole

of consciousness is the home cat.

I’m pretty sure cats lack, is the embodiment of evil

and lack all capacity for consciousness or compassion.

So I just want to lay that on the table.

That’s the theory I’m working on.

I don’t have any good evidence, but it’s just an intuition.

It’s just a shout out.

Sorry to distract.

So that’s the first assumption.

The first assumption, that’s right.

The second assumption is that

these experiences have consequences.

So I’m going to say that conscious experiences

can trigger other conscious experiences somehow.

So really in some sense, there’s two basic assumptions.

There’s some kind of causality.

Is there a chain of causality?

Does this relate to dynamics?

I’ll say there’s a probabilistic relationship.

So I’m trying to be as nonspecific to begin with

and see where it leads me.

So what I can write down are probability spaces.

So a probability space, which contains

the conscious experiences that this consciousness can have.

So I call this a conscious agent, this technical thing.

Annika Harris and I’ve talked about this

and she rightly cautions me that people will think

that I’m bringing in a notion of a self or agency

and so forth when I say conscious agent.

So I just want to say that I use the term conscious agent

merely as a technical term.

There is no notion of self in my fundamental definition

of a conscious agent.

There are only experiences and probabilistic relationships

of how they trigger other experiences.

So the agent is the generator of the conscious experience?

The agent is a mathematical structure

that includes a probability measure,

the probability space of a possible conscious experiences

and a Markovian kernel, which describes how

if this agent has certain conscious experiences,

how that will affect the experiences

of other conscious agents, including itself.

But you don’t think of that as a self?

No, there is no notion of a self here.

There’s no notion of really of an agent.

But is there a locality?

Is there an organism?

There’s no space.

There’s no.

So this is, these are conscious units, conscious entities.

But they’re distinct in some way

because they have to interact.

Well, so here’s the interesting thing.

When we write down the mathematics,

when you have two of these conscious agents interacting,

the pair satisfy a definition of a conscious agent.

So they are a single conscious agent.

So there is one conscious agent.

But it has a nice analytic decomposition

into as many conscious agents as you wish.

So that’s a nice interface.

It’s a very useful scientific interface.

It’s a scale free or if you like a fractal like approach

to it in which we can use the same unit of analysis

at all scales in studying consciousness.

But if I want to talk about,

so there’s no notion of learning, memory, problem solving,

intelligence, self, agency.

So none of that is fundamental.

So, and the reason I did that was

because I want to assume as little as possible.

Everything I assume is a miracle in the theory.

It’s not something you explain, it’s something you assume.

So I have to build networks of conscious agents.

If I want to have a notion of a self,

I have to build a self.

I have to build learning, memory, problem solving,

intelligence and planning, all these different things.

I have to build networks of conscious agents to do that.

It’s a trivial theorem that networks of conscious agents

are computationally universal, that’s trivial.

So anything that we can do with neural networks

or automata, you can do with networks of conscious agents.

That’s trivial.

But you can also do more.

The events in the probability space need not be computable.

So the Markovian dynamics is not restricted

to computable functions

because the very events themselves need not be computable.

So this can capture any computable theory.

Anything we can do with neural networks,

we can do with conscious agent networks.

But it leaves open the door for the possibility

of noncomputable interactions between conscious agents.

So if we want a theory of memory, we have to build it.

And there’s lots of different ways you could build.

We’ve actually got a paper,

Chris Fields took the lead on this.

And we have a paper called Conscious Agent Networks

where Chris takes the lead and shows how to use

these networks of conscious agents to build memory

and to build primitive kinds of learning.

But can you provide some intuition

of what conscious networks,

networks of conscious agents helps you?

First of all, what that looks like.

And I don’t just mean mathematically.

Of course, maybe that might help build up intuition.

But how that helps us potentially solve

the hard problem of consciousness.

Or is that baked in, that that exists?

Can you solve the hard problem of consciousness,

why it tastes delicious when you eat a delicious ice cream

with networks of conscious agents?

Or is that taken as an assumption?

So the standard way the hard problem is thought of

is we’re assuming space and time and particles

or neurons, for example.

These are just physical things that have no consciousness.

And we have to explain how the conscious experience

of the taste of chocolate could emerge from those.

So the typical hard problem of consciousness

is that problem, right?

How do you boot up the taste of chocolate,

the experience of the taste of chocolate from neurons, say,

or the right kind of artificial intelligence circuitry?

How do you boot that up?

That’s typically what the hard problem of consciousness

means to researchers.

Notice that I’m changing the problem.

I’m not trying to boot up conscious experiences

from the dynamics of neurons or silicon

or something like that.

I’m saying that that’s the wrong problem.

My hard problem would go in the other direction.

If I start with conscious experiences,

how do I build up space and time?

How do I build up what I call the physical world?

How do I build up what we call brains?

Because I’m saying consciousness

is not something that brains do.

Brains are something that consciousness makes up.

It’s among the experience,

it’s an ephemeral experience in consciousness.

I look inside, so to be very, very clear,

right now, I have no neurons.

If you looked, you would see neurons.

That’s a data structure that you would create on the fly,

and it’s a very useful one.

As soon as you look away,

you garbage collect that data structure,

just like that Necker cube that I was talking about

on the piece of paper.

When you look, you see a 3D cube you created on the fly.

As soon as you look away, that’s gone.

When you say you, you mean a human being scientist.

Right now, that’s right.

More generally, it’ll be conscious agents,

because as you pointed out,

am I asking for a theory of consciousness

only about humans?

No, it’s consciousness,

which human consciousness is just a tiny sliver.

But you are saying that there is,

that’s a useful data structure.

How many other data structures are there?

That’s why I said you human.

If there’s another Earth,

if there’s another alien civilization

and doing these kinds of investigations,

would they come up with similar data structures?

Probably not.

What is the space of data structures,

I guess is what I’m asking.

My guess is that if consciousness is fundamental,

consciousness is all there is,

then the only thing that mathematical structure

can be about is possibilities of consciousness.

And that suggests to me

that there could be an infinite variety of consciousnesses,

and a vanishingly small fraction of them

use space time data structures

and the kinds of structures that we use.

There’s an infinite variety of data structures.

Now, this is very similar

to something that Max Tegmark has said,

but I want to distinguish it.

He has this level four multiverse idea.

He thinks that mathematics is fundamental.

And so that’s the fundamental reality.

And since there’s an infinite variety of,

endless variety of mathematical structures,

there’s an infinite variety of multiverses in his view.

I’m saying something similar in spirit,

but importantly different.

There’s an infinite variety

of mathematical structures, absolutely.

But mathematics isn’t the fundamental reality

in this framework.

Consciousness is,

and mathematics is to consciousness

like bones are to an organism.

You need the bones.

So mathematics is not divorced from consciousness,

but it’s not the entirety of consciousness by any means.

And so there’s an infinite variety of consciousnesses

and signaling games that consciousnesses could interact via.

And therefore worlds, your common worlds,

data structures that they can use to communicate.

So space and time is just one of an infinite variety.

And so I think that what we’ll find is that

as we go outside of our little space time bubble,

we will encounter utterly alien forms

of conscious experience that we may not be able

to really comprehend in the following sense.

If I ask you to imagine a color

that you’ve never seen before,

does anything happen?

Nothing happens.

And that’s just one color.

I’m asking for just a color.

We actually know, by the way,

that apparently there are women called tetraphams

who have four color receptors, not just three.

And Kimberly Jameson and others who’ve studied these women

have good evidence that they apparently have

a new dimension of color experience

that the rest of us don’t have.

So these women are apparently living in a world of color

that you and I can’t even concretely imagine.

No man can imagine them.

Yeah.

And yet they’re real color experiences.

And so in that sense, I’m saying,

now take that little baby step,

oh, there are women who have color experiences

that I could never have.

Well, that’s shocking.

Now take that infinite.

There are consciousnesses where every aspect

of their experiences is like that new color.

It’s something utterly alien to you.

You have nothing like that.

And yet these are all possible varieties

of conscious experience.

And when you say there’s a lot of consciousnesses,

as a singular consciousness,

basically the set of possible experiences you can have

in that subjective way,

as opposed to the underlying mechanism.

Because you say that, you know,

having a extra color receptor,

ability to have new experiences

that somehow a different consciousness,

is there a way to see that as all the same consciousness,

the subjectivity itself?

Right.

Because when we have two of these conscious agents

interacting, the mathematics,

they actually satisfy the definition of a conscious agent.

So in fact, they are a single conscious agent.

So in fact, one way to think about what I’m saying,

I’m postulating with my colleagues,

Chaiton and Chris and others,

Robert Pretner and so forth.

There is one big conscious agent, infinitely complicated.

But fortunately, we can, for analytic purposes,

break it down all the way to,

in some sense, the simplest conscious agent,

which has one conscious experience, one.

This one agent can experience red 35, and that’s it.

That’s what it experiences.

You can get all the way down to that.

So you think it’s possible that consciousness,

whatever that is,

is much more, is fundamental,

or at least much more in the direction of the fundamental

than is space time as we perceive it?

That’s the proposal.

And therefore, what I have to do,

in terms of the hard problem of consciousness,

is to show how dynamical systems of conscious agents

could lead to what we call space and time

and neurons and brain activity.

In other words, we have to show how you get space time

and physical objects entirely from a theory

of conscious agents outside of space time,

with the dynamics outside of space time.

So that’s, and I can tell you how we plan to do that,

but that’s the idea.

Okay, the magic of it, that chocolate is delicious.

So there’s a mathematical kind of thing

that we could say here, how it can emerge

within this system of networks of conscious agents,

but is there going to be at the end of the proof

why chocolate is so delicious?

Or no?

I guess I’m going to ask different kinds of dumb questions

to try to sneak up.

Oh, well, that’s the right question, and when I say

that I took conscious experiences as fundamental,

what that means is, in the current version of my theory,

I’m not explaining conscious experiences

where they came from.

That’s the miracle, that’s one of the miracles.

So I have two miracles in my theory.

There are conscious experiences, like the taste of chocolate,

and that there’s a probabilistic relationship.

When certain conscious experiences occur,

others are more likely to occur.

Those are the two miracles that are possible.

Is it possible to get beyond that

and somehow start to chip away

at the miracleness of that miracle,

that chocolate is delicious?

I hope so.

I’ve got my hands full with what I’m doing right now,

but I can just say at top level how I would think about that.

That would get at this

consciousness without form.

This is really tough, because it’s consciousness without form

versus the various forms that consciousness takes

for the experiences that it has.

Right, right.

So when I write down a probability space

for these conscious experiences, I say,

here’s a probability space

for the possible conscious experiences, right?

It’s just like when I write down a probability space

for an experiment.

Like I’m gonna flip a coin twice, right?

And I want to look at the probabilities of various outcomes.

So I have to write down a probability space.

There could be heads, heads, heads, tails,

tails, heads, tails, tails.

So any class of probability you’re told,

write down your probability space.

If you don’t write down your probability space,

you can’t get started.

So here’s my probability space for consciousness.

How do I want to interpret that structure?

The structure is just sitting there.

There’s gonna be a dynamics that happens on it, right?

Experiences appear and then they disappear,

just like heads appears and disappears.

So one way to think about that fundamental

probability space is that corresponds

to consciousness without any content.

The infinite consciousness that transcends

any particular content.

Well, do you think of that as a mechanism,

as a thing, like the rules that govern the dynamics

of the thing outside of space time?

Isn’t that, if you think consciousness is fundamental,

isn’t that essentially getting like,

it is solving the hard problem,

which is like from where does this thing pop up,

which is the mechanism of the thing popping up,

whatever the consciousness is,

the different kinds and so on, that mechanism.

And also, the question I want to ask is how tricky

do you think it is to solve that problem?

You’ve solved a lot of difficult problems

throughout the history of humanity.

There’s probably more problems to solve left

than we’ve solved by like an infinity.

But along that long journey of intelligent species,

when will we solve this consciousness one?

Which is one way to measure the difficulty of the problem.

So I’ll give two answers.

There’s one problem I think we can solve,

but we haven’t solved yet.

And that is the reverse

of what my colleagues call the hard problem.

The problem of how do you start with conscious experiences

in the way that I’ve just described them and the dynamics

and build up space and time and brains,

that I think is a tough technical problem,

but it’s in principle solvable.

So I think we can solve that.

So we would solve the hard problem,

not by showing how brains create consciousness,

but how networks of conscious agents

create what we call the symbols that we call brains.

So that I think.

But does that allow you to, so that’s interesting.

That’s an interesting idea.

Consciousness creates the brain,

not the brain creates consciousness.

But does that allow you to build the thing?

My guess is that it will enable unbelievable technologies.

Once, and I’ll tell you why.

I think it plugs into the work

that the physicists are doing.

So this theory of consciousness will be even deeper

than the structures that the physicists are finding,

like the amplituhedron.

But the other answer to your question is less positive.

As I said earlier, I think that there is no such thing

as a theory of everything.

So that I think that the theory that my team is working on,

this conscious agent theory, is just a 1.0 theory.

We’re using probability spaces and Markovian curls.

I can easily see people now saying,

well, we can do better if we go to category theory.

And we can get a deeper, perhaps more interesting.

And then someone will say,

well, now I’ll go to topoi theory.

So I imagine that there’ll be conscious agents,

five, 10, 3 trillion, 0.0, but I think it will never end.

I think ultimately this question

that we sort of put our fingers on of,

how does the formless give birth to form,

to the wonderful taste of chocolate?

I think that we will always go deeper and deeper,

but we will never solve that.

That in some sense, that will be a primitive.

I hope I’m wrong.

Maybe it’s just the limits of my current imagination.

So I’ll just say my imagination right now

doesn’t peer that deep.

By the way, I’m saying this,

I don’t want to discourage some brilliant 20 year old

who then later on proves me dead wrong.

I hope to be proven dead wrong.

Just like you said, essentially from now,

everything we’re saying now, everything you’re saying,

all your theories will be laughing stock.

They will respect the puzzle solving abilities

and how much we were able to do with so little.

But outside of that, you will all be just,

the silliness will be entertainment for a teenager.

Especially the silliness when we thought

that we were so smart and we knew it all.

So it would be interesting to explore your ideas

by contrasting, you mentioned Annika, Annika Harris,

you mentioned Philip Goff.

So outside of, if you’re not allowed to say

the fundamental disagreement is the fact

that space time is fundamental.

What are interesting distinctions

between ideas of consciousness

between you and Annika, for example?

You guys have, you’ve been on a podcast together,

I’m sure in private you guys

have some incredible conversations.

So where are some interesting sticking points,

some interesting disagreements,

let’s say with Annika first.

Maybe there’ll be a few other people.

Well, Annika and I just had a conversation this morning

where we were talking about our ideas

and what we discovered really in our conversation

was that we’re pretty much on the same page.

It was really just about consciousness.

Our ideas about consciousness

are pretty much on the same page.

She rightly has cautioned me to,

when I talk about conscious agents,

to point out that the notion of agency

is not fundamental in my theory.

The notion of self is not fundamental

and that’s absolutely true.

I can use this network of conscious agents,

I now use as a technical term,

conscious agent is a technical term

for that probability space with the Markovian dynamics.

I can use that to build models of a self

and to build models of agency,

but they’re not fundamental.

So she has really been very helpful

in helping me to be a little bit clear about these ideas

and not say things that are misleading.

This is the interesting thing about language, actually,

is that language, quite obviously,

is an interface to truth.

It’s so fascinating that individual words

can have so much ambiguity

and the specific choices of a word

within a particular sentence,

within the context of a sentence,

can have such a difference in meaning.

It’s quite fascinating,

especially when you’re talking about topics

like consciousness, because it’s a very loaded term.

It means a lot of things to a lot of people

and the entire concept is shrouded in mystery.

So a combination of the fact that it’s a loaded term

and that there’s a lot of mystery,

people can just interpret it in all kinds of ways.

And so you have to be both precise

and help them avoid getting stuck

on some kind of side road of miscommunication,

lost in translation because you used the wrong word.

That’s interesting.

I mean, because for a lot of people,

consciousness is ultimately connected to a self.

I mean, that’s our experience of consciousness

is very, it’s connected to this ego.

I mean, I just, I mean, what else could it possibly be?

I can’t even, how do you begin to comprehend,

to visualize, to conceptualize a consciousness

that’s not connected to like this particular organism?

I have a way of thinking about this whole problem now

that comes out of this framework that’s different.

So we can imagine a dynamics of consciousness,

not in space and time, just abstractly.

It could be cooperative for all we know.

It could be very friendly, I don’t know.

And you can set up a dynamics, a Markovian dynamics

that is so called stationary.

And that’s a technical term,

which means that the entropy effectively is not increasing.

There is some entropy, but it’s constant.

So there’s no increasing entropy.

And in that sense, the dynamics is timeless.

There is no entropic time, but it’s a trivial theorem,

three line proof that if you have a stationary

Markovian dynamics, any projection that you make

of that dynamics by conditional probability.

And if you want, I can state a little bit more,

even more mathematically precisely

for some readers or listeners.

But if any projection you take by conditional probability,

the induced image of that Markov chain

will have increasing entropy.

You will have entropic time.

So I’ll be very, very precise.

I have a Markov chain X1, X2 through Xn

where Xn, n goes to infinity, right?

The entropy H, capital H of Xn is equal to the entropy H

of Xn minus one for all n.

So the entropy is the same.

But it’s a theorem that H of Xn,

say given X sub one is greater than or equal to

H of Xn minus one given X1.

Sure, where does the greater come from?

Because with the theorem, the three line proof,

H of Xn given X1 is greater than or equal to H of Xn

given X1 and X2 because conditioning reduces.

But then H of Xn minus one given X1, X2

is equal to H of Xn given X2,

Xn minus one given X2 by the Markov property.

And then because it’s stationary, it’s equal to H of X.

I have to write it down.

Anyway, there’s a three line proof.

Sure, but the assumption of stationarity,

we’re using a lot of terms that people won’t understand,

doesn’t matter.

So there’s some kind of, some Markovian dynamics

is basically trying to model some kind of system

with some probabilities and there’s agents

and they interact in some kind of way

and you can say something about that system

as it evolves stationarity.

So a stationary system is one that has certain properties

in terms of entropy, very well.

But we don’t know if it’s stationary or not.

We don’t know what the properties.

Right.

So you have to kind of take assumptions

and see, okay, well, what does the system behave like

under these different properties?

The more constraints, the more assumptions you take,

the more interesting, powerful things you can say,

but sometimes they’re limiting.

That said, we’re talking about consciousness here.

Right.

How does that, you said cooperative, okay, competitive.

It just, I like chocolate.

I’m sitting here, I have a brain, I’m wearing a suit.

It sure as hell feels like I’m myself.

Right.

Now, what, am I tuning in?

Am I plugging into something?

Am I a projection, a simple, trivial projection

into space time from some much larger organism

that I can’t possibly comprehend?

How the hell, you’re saying some,

you’re building up mathematical intuitions, fine, great.

But I’m just, I’m having an existential crisis here

and I’m gonna die soon.

Well, I’ll die pretty quickly.

So I wanna figure out why chocolate’s so delicious.

So help me out here.

So let’s just keep sneaking up to this.

Right, so the whole technical thing was to say this.

Even if the dynamics of consciousness is stationary

so that there is no entropic time,

any projection of it, any view of it

will have the artifact of entropic time.

That’s a limited resource.

Limited resources, so that the fundamental dynamics

may have no limits, limited resources whatsoever.

Any projection will have certainly time

as a limited resource

and probably lots of other limited resources.

Hence, we could get competition and evolution

and nature, red and tooth and claw

as an artifact of a deeper system

in which those aren’t fundamental.

And in fact, I take it as something

that this theory must do at some point

is to show how networks of conscious agents,

even if they’re not resource limited,

give rise to evolution by natural selection

via a projection.

Yeah, but you’re saying,

I’m trying to understand how the limited resources

that give rise to,

so first the thing gives rise to time,

that gives rise to limited resource,

that gives rise to evolution by natural selection,

how that has to do with the fact that chocolate’s delicious?

Well, it’s not gonna do that directly.

It’s gonna get to this notion of self.

Oh, it’s gonna give you?

The notion of self.

Oh, the evolution gives you the notion of self.

And also of a self separate from other selves.

So the idea would be that.

It’s competition, it has life and death,

all those kinds of things.

That’s right.

So it won’t, I don’t think,

as I said, I don’t think that I can tell you

how the formless gives rise

to the experience of chocolate.

Right now, my current theory says

that’s one of the miracles I’m assuming.

Yeah.

So my theory can’t do it.

And the reason my theory can’t do it

is because Hoffman’s brain can’t do it right now.

But the notion of self, yes.

The notion of self can be an artifact

of the projection of it.

So there’s one conscious agent.

Because anytime conscious agents interact,

they form a new conscious agent.

So there’s one conscious agent.

Any projection of that one conscious agent

gives rise to time,

even if there wasn’t any time in that one conscious agent.

And it gives rise, I want to,

now I haven’t proven this.

So this is, so now this is me guessing

where the theory is going to go.

I haven’t done this.

There’s no paper on this yet.

So now I’m speculating.

My guess is I’ll be able to show,

or my brighter colleagues working with me

will be able to show

that we will get evolution of a natural selection,

the notion of individual selves,

individual physical objects and so forth

coming out as a projection of this thing.

And that the self, this then will be really interesting

in terms of how it starts to interact

with certain spiritual traditions, right?

Where they will say that there is a notion of self

that needs to be let go,

which is this finite self

that’s competing with other selves

to get more money and prestige and so forth.

That self in some sense has to die.

But there’s a deeper self,

which is the timeless being

that precludes, not precludes,

but precedes any particular conscious experiences,

the ground of all experience.

That there’s that notion of a deep capital self.

But our little capital, lowercase s selves

could be artifacts of projection.

And it may be that what consciousness is doing

in this framework is, right?

It’s projected itself down into a self

that calls itself dawn

and a self that calls itself lax.

And through conversations like this,

it’s trying to find out about itself

and eventually transcend the limits

of the dawn and lax little icons that it’s using

and that little projection of itself.

Through this conversation,

somehow it’s learning about itself.

So that thing dressed me up today

in order to understand itself.

And in some sense, you and I are not separate

from that thing and we’re not separate from each other.

Yeah, well, I have to question the fashion choices

on my end then.

All right, so you mentioned you agree

in terms of consciousness on a lot of things with Anika.

Is there somebody, friend or friendly foe

that you disagree with in some nuanced, interesting way

or some major way about consciousness,

about these topics of reality that you return to?

Often, it’s like Christopher Hitchens

with Rabbi David Wolpe have had interesting conversations

through years that added to the complexity

and the beauty of their friendship.

Is there somebody like that that over the years

has been a source of disagreement with you

that’s strengthened your ideas?

Hmm, my ideas have been really shaped by several things.

One is the physicalist framework

that my scientific colleagues, almost to a person,

have adopted and that I adopted too.

And the reason I walked away from it was

because it became clear that we couldn’t start

with unconscious ingredients and boot up consciousness.

Can you define physicalist in contrast to reductionist?

So a physicalist, I would say as someone

who takes space time and the objects within space time

as ontologically fundamental.

Right, and then reductionist is saying

the smaller, the more fundamental.

That’s a methodological thing.

That’s saying within space time,

as you go to smaller and smaller scales in space,

you get deeper and deeper laws, more and more fundamental

laws and the reduction of temperature

to particle movement was an example of that.

But I think that the reason that worked

was almost an artifact of the nature of our interface.

That was for a long time and your colleagues,

including yourself, were physicalists

and now you broke away.

Broke away because I think you can’t start

with unconscious ingredients and boot up consciousness.

And so even with Roger Penrose

where there’s like a gray area.

Right, and here’s the challenge I would put

to all of my friends and colleagues

who give one specific conscious experience

that you can boot up, right?

So if you think that it’s integrated information

and I’ve asked this of Giulio Tononi a couple times

back in the 90s and then just a couple years ago.

I asked Giulio, okay, so great, integrated information.

So we’re all interested in explaining

some specific conscious experiences.

So what is, you know, pick one, the taste of chocolate.

What is the integrated information precise structure

that we need for chocolate and why does that structure

have to be for chocolate and why is it

that it could not possibly be vanilla?

Is there any, I asked him, is there any one specific

conscious experience that you can account for?

Because notice, they’ve set themselves the task

of booting up conscious experiences from physical systems.

That’s the task they’ve set themselves.

But that doesn’t mean they’re,

I understand your intuition,

but that doesn’t mean they’re wrong

just because they can’t find a way to boot it up yet.

That’s right.

No, that doesn’t mean that they’re wrong.

It just means that they haven’t done it.

I think it’s principled.

The reason is principled,

but I’m happy that they’re exploring it.

But the fact is, the remarkable fact is

there’s not one theory.

So integrated information theory,

orchestrated collapse of microtubules,

global workspace theory.

These are all theories of consciousness.

There’s not a single theory that can give you

a specific conscious experience that they say,

here is the physical dynamics or the physical structure

that must be the taste of chocolate

or whatever one they want.

So you’re saying it’s impossible.

They’re saying it’s just hard.

Yeah.

My attitude is, okay, no one said

you had to start with neurons or physical systems

and boot up consciousness.

You guys are just taking that.

You chose that problem.

So since you chose that problem,

how much progress have you made?

Well, when you’ve not been able to come up

with a single specific conscious experience

and you’ve had these brilliant people

working on it for decades now,

that’s not really good progress.

Let me ask you to play devil’s advocate.

Can you try to steel man, steel man meaning

argue the best possible case for reality?

The opposite of your book title.

So, or maybe just stick into consciousness.

Can you take the physicalist view?

Can you steel man the physicalist view

for a brief moment playing devil’s advocate too?

Or steel man the person you used to be?

Right, right. She’s a physicalist.

What’s a good, like saying that you might be wrong

right now, what would be a convincing argument for that?

Well, I think the argument I would give

that I believed was, look,

when you have very simple physical systems,

like a piece of dirt,

there’s not much evidence of life or consciousness.

It’s only when you get really complicated physical systems

like that have brains and really,

the more complicated the brains,

the more it looks like there’s consciousness

and the more complicated that consciousness is.

Surely that means that simple physical systems

don’t create much consciousness or if maybe not any,

or maybe panpsychists,

they create the most elementary kinds

of simple conscious experiences,

but you need more complicated physical systems to boot up,

to create more complicated consciousnesses.

I think that’s the intuition

that drives most of my colleagues.

And you’re saying that this concept of complexity

is ill defined when you ground it to space time.

Oh, I think it’s well defined

within the framework of space time, right?

No, it’s ill defined relative to what you need

to actually understand consciousness

because you’re grounding complexity in just in space time.

Oh, got you, right, right.

Yeah, what I’m saying is if it were true

that space time was fundamental,

then I would have to agree

that if there is such a thing as consciousness,

given the data that we’ve got,

that complex brains have consciousness and dirt doesn’t,

that somehow it’s the complexity of the dynamics

or organization, the function of the physical system

that somehow is creating the consciousness.

So under those assumptions, yes,

but when the physicists themselves are telling us

that space time is not fundamental, then I can understand.

See, then the whole picture starts to come into focus.

Why, my colleagues are brilliant, right?

These are really smart people.

I mean, Francis Crick worked on this

for the last 20 years of his life.

These are not stupid people.

These are brilliant, brilliant people.

The fact that we’ve come up

with not a single specific conscious experience

that we can explain and no hope.

There’s no one that says, oh, I’m really close.

I’ll have it for you in a year.

No, there’s just like, there’s this fundamental gap.

So much so that Steve Pinker in one of his writings says,

look, he likes the global workspace theory,

but he says the last dollop of the theory

in which there’s something it’s like to,

he says, we may have to just stipulate that as a brute fact.

Pinker is brilliant, right?

He understands the state of play

on this problem of the hard problem of consciousness,

starting with physicalist assumptions

and then trying to put up consciousness.

So you’ve set yourself the problem.

I’m starting with physical stuff

that’s not conscious.

I’m trying to get the taste of chocolate out

as maybe some kind of function of the dynamics of that.

We’ve not been able to do that.

And so Pinker is saying, we may have to punt.

We may have to just stipulate that last bit.

He calls it the last dollop.

And just stipulate it as a bare fact of nature

that there is something it’s like.

Well, from my point of view as the physical,

the whole point, the whole promise of the physicalist

was we wouldn’t have to stipulate.

I was gonna start with the physical stuff

and explain where the consciousness came from.

If I’m going to stipulate consciousness,

why don’t I just stipulate consciousness

and not stipulate all the physical stuff too?

So I’m stipulating less.

I’m saying, okay, I agree.

Which is the panpsychist perspective.

Well, it’s actually what I call

the conscious realist perspective.

Consciousness.

Panpsychists are effectively dualists, right?

They’re saying there’s physical stuff

that really is fundamental and then consciousness stuff.

So I would go with Pinker and say, look,

let’s just stipulate the consciousness stuff,

but I’m not gonna stipulate the physical stuff.

I’m gonna actually now show how to boot up

the physical stuff from just the consciousness stuff.

So I’ll stipulate less.

Is it possible, so if you stipulate less,

is it possible for our limited brains to visualize reality

as we delve deeper and deeper and deeper?

Is it possible to visualize somehow?

With the tools of math, with the tools of computers,

with the tools of our mind, are we hopelessly lost?

You said there’s ways to intuit what’s true

using mathematics and probability

and sort of a Markovian dynamics, all that kind of stuff,

but that’s not visualizing.

That’s a kind of building intuition.

But is it possible to visualize

in the way we visualize so nicely in space time

in four dimensions, in three dimensions, sorry.

Well, we really are looking through a two dimensional screen

until it’s what we intuit to be a three dimensional world

and also inferring dynamic stuff, making it 4D.

Anyway, is it possible to visualize some pretty pictures

that give us a deeper sense of the truth of reality?

I think that we will incrementally be able to do that.

I think that, for example, the picture that we have

of electrons and photons interacting and scattering,

it may have not been possible

until Faraday did all of his experiments

and then Maxwell wrote down his equations.

And we were then sort of forced by his equations

to think in a new way.

And then when Planck in 1900,

desperate to try to solve the problem

of black body radiation,

what they call the ultraviolet catastrophe

where Newton was predicting infinite energies

where there weren’t infinite energies

in black body radiation.

And he in desperation proposed packets of energy.

Then once you’ve done that,

and then you have an Einstein come along five years later

and show how that explains the photoelectric effect.

And then eventually in 1926, you get quantum theory.

And then you get this whole new way of thinking

that was, from the Newtonian point of view,

completely contradictory and counterintuitive, certainly.

And maybe if Giesen is right, not contradictory.

Maybe if you use intuitionist math, they’re not contradictory,

but still, certainly you wouldn’t have gone there.

And so here’s a case where the experiments

and then a desperate mathematical move,

sort of we use those as a flashlight into the deep fog.

And so that science may be the flashlight into the deep fog.

I wonder if it’s still possible to visualize

in the, like we talk about consciousness

from a self perspective experience it.

Hold that idea in our mind,

the way you can experience things directly.

We’ve evolved to experience things in this 3D world.

And that’s a very rich experience.

When you’re thinking mathematically,

you still in the end of the day have to project it down

to a low dimensional space to make conclusions.

Your conclusions will be a number or a line

or a plot or a visual.

So I wonder like how we can really touch some deep truth

in a subjective way, like experience it,

really feel the beauty of it, you know,

in the way that humans feel beauty.

Right, are we screwed?

I don’t think we’re screwed.

I think that we get little hints of it

from psychedelic drugs and so forth.

We get hints that there are certain interventions

that we can take on our interface.

I apply this chemical,

which is just some element of my interface

to this other, to a brain I ingested.

And all of a sudden I seem like I’ve opened new portals

into conscious experiences.

Well, that’s very, very suggestive.

That’s like the black body radiation doing something

that we didn’t expect, right?

It doesn’t go to infinity

when we thought it was gonna go to infinity

and we’re forced to propose these quanta.

So once we have a theory of conscious agents

and this projection to space,

I should say, I should sketch what I think

that projection is.

But then I think we can then start

to ask specific questions.

When you’re taking DMT or you’re taking LSD

or something like that,

now that we have this deep model

that we’ve reverse engineered space and time

and physical particles,

we’ve pulled them back to this theory of conscious agents.

Now we can ask ourselves in this idealized future,

what are we doing to conscious agents

when we apply five MEO DMT?

What are we doing?

Are we opening a new portal, right?

So when I say that, I mean,

I have a portal into consciousness

that I call my body of Lex Friedman that I’m creating.

And it’s a genuine portal, not perfect,

but it’s a genuine portal.

I’m definitely communicating with your consciousness.

And we know that we have one technology

for building new portals.

We know one technology and that is having kids.

Having kids is how we build new portals into consciousness.

It takes a long time.

Can you elaborate that?

Oh, oh, oh, you mean like?

Your son and your daughter didn’t exist.

That was a portal.

You’re having contact with consciousness

that you never would have had before,

but now you’ve got a son or a daughter.

You went through this physical process,

they were born, then there was all the training.

But is that portal yours?

So when you have kids, are you creating new portals

that are completely distinct from the portals

that you’ve created with other consciousness?

Like can you elaborate on that?

To which degree are the consciousness of your kids

a part of you?

Well, so every person that I see,

that symbol that I see, the body that I see,

is a portal potentially for me to interact

with a consciousness.

And each consciousness has a unique character.

We call it a personality and so forth.

So with each new kid that’s born,

we come in contact with a personality

that we’ve never seen before.

And a version of consciousness

that we’ve never seen before.

At a deeper level, as I said,

the theory says there’s one agent.

So this is a different projection of that one agent.

But so that’s what I mean by a portal

is within my own interface, my own projection,

can I see other projections of that one consciousness?

So can I get portals in that sense?

And I think we will get a theory of that,

that we will get a theory of portals

and then we can ask how the psychedelics are acting.

Are they actually creating new portals or not?

If they’re not, we should nevertheless then understand

how we could create a new portal, right?

Maybe we have to just study what happens

when we have kids.

We know that that technology creates new portals.

So we have to reverse engineer that and then say,

okay, could we somehow create new portals de novo?

With something like brain computer interfaces, for example.

Yeah, well, maybe just a chemical or something.

It’s probably more complicated than a chemical.

That’s why I think that the psychedelics may,

because they might be affecting this portal

in certain ways that it turns it around and opens up.

In other words, maybe once we understand

what this thing is a portal, your body is a portal,

and understand all of those complexities,

maybe we’ll realize that that portal can be shifted

to different parts of the deeper consciousness

and can be used to create new portals.

And give new windows on it.

And so in that way, maybe yes,

psychedelics could open up new portals

in the sense that they’re taking something

that’s already a complex portal

and just tweaking it a bit.

Well, but creating is a very powerful difference

between morphing.

Right, right, tweaking versus creating, I agree.

But maybe it gives you intuition

to at least the full space of the kinds of things

that this particular system is capable of.

I mean, the idea of the consciousness creates brains.

I mean, that breaks my brain because,

I guess I’m still a physicalist in that sense

because it’s just much easier to intuit the world.

It’s practical to think there’s a neural network

and what are the different ways

fascinating capabilities can emerge

from this neural network.

I agree, it’s easier.

And so you start to,

and then present to yourself the problem of,

okay, well, how does consciousness arise?

How does intelligence arise?

How does emotion arise?

How does memory arise?

How do we filter within the system

all the incoming sensory information

we’re able to allocate attention

in different interesting ways?

How do all those mechanisms arise?

To say that there’s other fundamental things

we don’t understand outside of space time

that are actually core to how this whole thing works

is a bit paralyzing because it’s like,

oh, we’re not 10% done, we’re like 0.001% done.

It’s the immediate feeling.

Certainly understand that.

My attitude about it is,

if you look at the young physicists

who are searching for these structures beyond space time,

like amplitude and so forth,

they’re having a ball.

Space time, that’s what the old folks did.

That’s what the older generation did.

We’re doing something that really is fun and new

and they’re having a blast

and they’re finding all these new structures.

So I think that we’re going to

succeed in getting a new deeper theory.

I can just say what I’m hoping with the theory

that I’m working on, I’m hoping to show

that I could have this timeless dynamics of consciousness,

no entropic time.

I take a projection and I show how this timeless dynamics

looks like the Big Bang

and the entire evolution of space time.

In other words, I see how my whole space time interface.

So not just the projection

doesn’t just look like space time,

you can explain the whole from the origin of the universe.

That’s what we have to do

and that’s what the physicists understand.

When they go beyond space time to the amplitude heat

and the cosmological polytope,

they ultimately know that they have to get back

the Big Bang story and the whole evolution,

that whole story where there were no living things.

There was just a point and then the explosion

and then just particles at high energy

and then eventually the cooling down

and the differentiation and finally matter condenses

and then life and then consciousness.

That whole story has to come out of something

that’s deeper and without time.

And that’s what we’re up to.

So the whole story that we’ve been telling ourselves

about Big Bang and how brains evolved in consciousness

will come out of a much deeper theory.

And yeah, for someone like me, it’s a lot.

But for the younger generation, this is like, oh wow,

all the low cherries aren’t picked.

This is really good stuff.

This is really new fundamental stuff that we can do.

So I can’t wait to read the papers of the younger generation

and I wanna see them.

Kids these days with their non space time assumptions.

It’s just interesting looking at the philosophical tradition

of this difficult ideas you struggle with.

If you look like somebody like Emmanuel Kant,

what are some interesting agreements and disagreements

you have with a guy about the nature of reality?

So there’s a lot in agreement, right?

So Kant was an idealist, transcendental idealist

and he basically had the idea

that we don’t see nature as it is.

We impose a structure on nature.

And so in some sense, I’m saying something similar.

I’m saying that, by the way,

I don’t call myself an idealist.

I call myself a conscious realist

because idealism has a long history.

A lot of different ideas come under idealism

and there’s a lot of debates and so forth.

It tends to be identified with, in many cases,

anti science and anti realism.

And I don’t want either connection with my ideas

and so I just called mine conscious realism

with an emphasis on realism and not anti realism.

But one place where I would, of course,

disagree with Kant was that he thought

that Euclidean space time was a priori, right?

We just know that that’s false.

So he went too far on that.

But in general, the idea that we don’t start with space time,

that space and time is in some sense

the forms of our perceptions.

Yes, absolutely.

And I would say that there’s a lot in common

with Berkeley in that regard.

There’s a lot of ingenious arguments in Berkeley.

Leibniz in his monodology understood very clearly

that the hard problem was not solvable.

He posed the hard problem and basically dismissed it.

He just said, you can’t do this.

And so if he came here and saw where we are,

he said, look, guys, I told you this 300 years ago.

And he had his monodology.

He was trying to do something like,

it’s different from what I’m doing,

but he had these things that were not in space and time,

these monads.

He was trying to build something.

I’m trying to build a theory of conscious agents.

My guess is that if he came here,

I could just, if he saw what I was doing,

he would say, he would understand it

and immediately take off with it

and go places that I couldn’t.

He would have no problem with this.

Right, there would be overlap of the spirit

of the ideas would be totally overlapping.

But his genius would then just run with it

far faster than I could.

I love the humility here.

So let me ask you about sort of practical implications

of your ideas to our world, our complicated world.

When you look at the big questions of humanity,

of hate, war, what else is there?

Evil, maybe there’s the positive aspects of that,

of meaning, of love.

What is the fact that reality is an illusion perceived?

What is the conscious realism when applied to daily life?

What kind of impact does it have?

A lot, and it’s sort of scary.

We all know that life is ephemeral

and spiritual traditions have said wake up to the fact

that anything that you do here is going to disappear.

But it’s even more ephemeral than perhaps we’ve thought.

I see this bottle because I create it right now.

As soon as I look away,

that data structure has been garbage collected.

That bottle, I have to recreate it every time I look.

So I spend all my money and I buy this fancy car.

That car, I have to keep recreating it

every time I look at it.

It’s that ephemeral.

So all the things that we invest ourselves in,

we fight over, we kill each other over,

we have wars over, these are all,

it’s like people in a virtual reality simulation, right?

And there’s this Porsche and we all see the Porsche.

Well, that Porsche exists when I look at it.

I turn my headset and I look at it.

And then if Joe turns his headset the right way,

he’ll see his Porsche.

It’s not even the same Porsche that I see.

He’s creating his own Porsche.

So these things are exceedingly ephemeral.

And now just imagine saying that that’s my Porsche.

Well, you can agree to say that it’s your Porsche,

but really the Porsche only exists as long as you look.

So this all of a sudden,

what the spiritual traditions have been saying

for a long, long time,

this gets cashed out in mathematically precise science.

It’s saying ephemeral, yes.

In fact, it lasts for a few milliseconds,

a few hundred milliseconds while you look at it.

And then it’s gone.

So the whole idea, why are we fighting?

Why do we hate?

We fight over possessions

because we think that we’re small little objects

inside this preexisting space time.

We assume that that mansion and that car

exists independent of us.

And that somehow we, these little things

can have our sense of self and importance

enhanced by having that special car

or that special house or that special person.

When in fact, it’s just the opposite.

You create that mansion every time you look.

That’s, you’re something far deeper than that mansion.

You’re the entity which can create that mansion on the fly.

And there’s nothing to the mansion

except what you create in this moment.

So all of a sudden, when you take this point of view,

it has all sorts of implications

for how we interact with each other,

how we treat each other.

And again, a lot of things

that spiritual traditions have said, it’s a mixed bag.

Spiritual traditions are a mixed bag.

So let me just be right up front about that.

I’m not promoting any particular,

but they do have some insights.

Yeah, they have wisdom.

They have certain wisdom.

They have, I can point to nonsense.

I won’t go into it,

but I can also point to lots of nonsense.

So the issue is to then to look for the key insights.

And I think they have a lot of insights

about the ephemeral nature of objects in space and time

and not being attached to them, including our own bodies.

And reversing that I’m not this little thing,

a little consciousness trapped in the body.

And the consciousness itself is only a product of the body.

So when the body dies, the consciousness disappears.

It turns completely around.

The consciousness is fundamental.

The body, my hand exists right now

because I’m looking at it.

My hand is gone.

I have no hand.

I have no brain.

I have no heart.

If you looked, you’ll see a heart.

Whatever I am is this really complicated thing

in consciousness.

That’s what I am.

All the stuff that I thought I was

is something that I create on the fly and delete.

So this is completely radical restructuring

of how we think about possessions, about identity,

about survival of death and so forth.

This is completely transformative.

But the nice thing is that this whole approach

of conscious agents, unlike the spiritual traditions,

which have said in some cases similar things,

they’ve said it imprecisely.

This is mathematics.

We can actually now begin to state precisely,

here’s the mathematical model of consciousness,

conscious agents, here’s how it maps onto space time,

which I should sketch really briefly.

And here’s why things are ephemeral

and here’s why you shouldn’t be worried

about the ephemeral nature of things

because you’re not a little tiny entity

inside space and time, quite the opposite.

You’re the author of space and time.

The I and the am and the I am

is all kind of emerging through this whole process

of evolution and so on that’s just surface waves

and there’s a much deeper ocean

that we’re trying to figure out here.

So how does, you said some of the stuff

you’re thinking about maps to space time,

how does it map to space time?

So just a very, very high level and I’ll keep it brief.

The structures that the physicists are finding,

like the amplituhedron, it turns out

they’re just static structure, they’re polytopes.

But they, remarkably, most of the information in them

is contained in permutation matrices.

So it’s a matrix, like an end by end matrix

that just has zeros and ones.

That contains almost all of the information

and you can, they have these plebic graphs

and so forth that they use to boot up the scattering.

You can compute those scattering amplitudes

almost entirely from these permutation matrices.

So that’s just, now from my point of view,

I have this conscious agent dynamics.

It turns out that the stationary dynamics

that I was talking about,

where the entropy isn’t increasing,

all the stationary dynamics are sketched out

by permutation matrices.

So there’s so called Burkhoff polytope.

All the vertices of this polytope,

all the points are permutation matrices.

All the internal points are Markovian kernels

that have the uniform measure as a stationary measure.

Now I need to intuit a little better

what the heck you’re talking about.

So basically, there’s some complicated thing

going on with the network of conscious agents

and that’s mappable to this,

you’re saying a two dimensional matrix

that scattering has to do with what?

With our perception, like that’s like photon stuff?

I mean, I don’t know if it’s useful

to sort of dig into detail.

I’ll do just the high level thing.

Yes.

So the high level is the long term behavior

of the conscious agent dynamics.

So that’s the projection of just looking

at the long term behavior.

I’m hoping we’ll give rise to the amplituhedron.

The amplituhedron then gives rise to space time.

So then I can just use their link

to go all the way from consciousness

through its asymptotics to,

through the amplituhedron into space time

and get the map all the way into our interface.

And that’s why you mentioned the permutation matrix

because it gives you a nice thing to try to generate.

That’s right, it’s the connection with the amplituhedron.

The permutation matrices are the core of the amplituhedron

and it turns out they’re the core

of the asymptotic description of the conscious agents.

So not to sort of bring up the idea of a creator,

but I like, first of all, I like video games

and you mentioned this kind of simulation idea.

First of all, do you think of it as an interesting idea,

this thought experiment that will live in a simulation?

And in general, do you think we’ll live in a simulation?

So the Nick Bostrom’s idea about the simulation

is typically couched in a physicalist framework.

Yes.

So there is the bottom level,

there’s some programmer in a physical space time

and they have a computer that they’ve programmed

really cleverly where they’ve created conscious entities.

So you have the hard problem of consciousness, right?

The standard hard problem.

How could a computer simulation create a conscious,

which isn’t explained by that simulation theory.

But then the idea is that the next level,

the entities that are created in the first level simulation

then can write their own simulations

and you get this nesting.

So the idea that this is a simulation is fine,

but the idea that it starts with a physicalist base,

I think, isn’t fine.

Well, there’s different properties here.

The partial rendering, and to me that’s the interesting idea

is not whether the entirety of the universe is simulated,

but how efficiently can you create interfaces

that are convincing to all other entities

that can appreciate such interfaces?

How little does it take?

Because you said like partial rendering

or like temporal, ephemeral rendering of stuff.

Only render the tree falling in the forest

when there’s somebody there to see it.

It’s interesting to think,

how can you do that super efficiently

without having to render everything?

And that to me is one perspective on the simulation,

just like it is with video games,

where a video game doesn’t have to render

every single thing.

It’s just the thing that the observer is looking at.

Right, there is actually, that’s a very nice question.

And there’s whole groups of researchers

that are actually studying in virtual reality,

what is the sort of minimal requirements on the system?

How does it have to operate

to give you an immersion experience,

to give you the feeling that you have a body,

to get you to take it real?

And there’s actually a lot of really good work

on that right now.

And it turns out it doesn’t take that much.

You do need to get the perception action loop tight

and you have to give them the perceptions

that they’re expecting if you want them to.

But if you can lead them along,

if you give them perceptions

that are close to what they’re expecting,

you can then maybe move their reality around a bit.

Yeah, it’s a tricky engineering problem,

especially when you’re trying to create a product

that costs little, but that’s,

it feels like an engineering problem,

not a deeply scientific problem.

Or meaning, obviously it’s a scientific problem,

but as a scientific problem,

it’s not that difficult to trick us descendants of apes.

But here’s a case for just us, you know, our own,

if this is a virtual reality

that we’re experiencing right now.

So here’s something you can try for yourself.

If you just close your eyes

and look at your experience in front of you,

be aware of your experience in front of you,

what you experience is just like a modeled dark gray,

where there’s all sort of, there’s some dynamics to it,

but it’s just dark gray.

But now I ask you, instead of having your attention forward,

put your attention backward.

What is it like behind you with your eyes closed?

And there, it’s like nothing.

It’s real.

So what is going on here?

What am I experiencing back there?

Right?

Well, it’s, I don’t know if it’s nothing.

It’s like, I guess it’s the absence of,

it’s not even like darkness or something.

It’s not even darkness.

There’s no qualia to it.

And yet there is a sense of being.

And that’s the interesting thing.

There’s a sense of being back.

So I close my, when I put my attention forward,

I have the qualia of a gray model thing.

But when I put my attention backward,

there’s no qualia at all, but there is a sense of being.

Yeah.

I personally, now you haven’t been to that side of the room.

I have been to that side of the room.

So for me, memories, I start playing the engine

of memory replay, which is like,

I take myself back in time and think about that place

where I was hanging out in that part.

That’s where I see what I’m behind.

So that’s an interesting quirk of humans too,

we’re able to, we’re collecting these experiences

and we can replay them in interesting ways

whenever we feel like it.

And it’s almost like being there,

but not really, but almost.

That’s right.

And yet we can go our entire lives in this.

You’re talking about the minimal thing for VR.

We can go our entire lives and not realize

that all of my life, it’s been like nothing behind me.

Yeah, right.

We’re not even aware that all of our lives,

if you just pay attention to us behind me,

we’re like, oh, holy smoke, it’s totally scary.

I mean, it’s like nothing.

There’s no qualia there at all.

How did I not notice that my entire life?

We’re so immersed in the simulation, we buy it so much.

Yeah, I mean, you could see this with children, right?

Though with persistence, you could do the peekaboo game.

You can hide from them and appear and they’re fully tricked.

And in the same way, we’re fully tricked.

There’s nothing behind us and we assume there is.

And that’s really interesting.

These theories are pretty heavy.

You as a human being, as a mortal human being,

how has these theories been to you personally?

Like, are there good days and bad days

when you wake up and look in the mirror

and the fact that you can’t see anything behind you?

The fact that it’s rendered,

like, is there interesting quirks?

Nietzsche with his, if you gaze long into the abyss,

the abyss gazes into you.

How has these theories, these ideas,

changed you as a person?

It’s been very, very difficult.

And this stuff is not just abstract theory building

because it’s about us.

Sometimes I realize that there’s this big division in me.

My mind is doing all this science

and coming up with these conclusions

and the rest of me is not integrating.

I’m just like, I don’t believe it.

I just don’t believe this.

I mean, it seems, so as I start to take it seriously,

I get scared myself.

It’s like, but it’s very much,

then I read these spiritual traditions

and realize they’re saying very, very similar things.

Like, there’s a lot of convergence.

So for me, I have,

the first time I thought it might be possible

that we’re not seeing the truth was in 1986.

It was from some mathematics we were doing.

And when that hit me, it hit me like a ton of bricks.

I had to sit down.

It was, it really, it was scary.

It was really a shock to the system.

And then to realize that everything

that has been important to me,

like, you know, getting a house,

getting a car, getting a reputation and so forth.

Well, that car is just like the car I see

in the virtual reality.

It’s just there when you perceive it and it’s not there.

So the whole question of, you know,

what am I doing and why?

What’s worthwhile doing in life?

Clearly, getting a big house and getting a big car.

I mean, we all knew that we were gonna die.

So we tend not to know that.

We tend to hide it, especially when we’re young.

Before age 30, we don’t believe we’re gonna die.

But we factually maybe know that you kind of

are supposed to, yeah.

But they’ll figure something out and we’ll be the generation

that is the first one that doesn’t have to die.

That’s the kind of thing.

But when you really face the fact that you’re going to die,

and then when I start to look at it from this point of view

that, well, this thing was an interface to begin with.

So what I’m really, is what I’m really gonna be doing,

just taking off a headset.

So I’ve been playing in a virtual reality game all day

and I got lost in the game and I was fighting over a Porsche.

And I shot some guys up and I punctured their tires

and I got the Porsche.

Now I take the headset off and what was that for?

Nothing, it was just, it was a data structure

and the data structure is gone.

So all of the wars, the fighting and the reputations

and all this stuff, it’s just a headset.

So my theory says that intellectually,

my mind, my emotions rebel all over the place.

It’s like I, you know, and so I have to meditate.

I meditate a lot.

What percent of the day would you say you spend

as a physicalist sort of living life,

pretending your car matters, your reputation matters?

Like how much, what’s that Tom Waits song,

I like my town with a little drop of poison.

How much poison do you allow yourself to have?

I think my default mode is physicalist.

I think that that’s just the default.

When I’m not being conscious, consciously attentive.

Then intellectually consciously attentive,

because if you’re just, you’re still,

if you’re tasting coffee and not thinking

or drinking or just taking in the sunset,

you’re not being intellectual,

but you’re still experiencing it.

So it’s when you turn on the introspective machine,

that’s when you can start.

And turn off the thinker,

when I actually just start looking without thinking.

So that’s when I feel like I,

all of a sudden I’m starting to see through.

Sort of like, okay, part of the addiction to the interface

is all the stories I’m telling about it.

It’s really important for me to get that,

really important to do that.

So I’m telling all these stories and so I’m all wrapped up.

Almost all of the mind stuff that’s going on in my head

is about attachment to the interface.

And so what I found is that the,

essentially the only way to really detach

from the interface is to literally let go

of thoughts altogether.

And then all of a sudden, even my identity,

my whole history, my name, my education,

all this stuff is almost irrelevant

because it’s just now here is the present moment.

And this is the reality right now.

And all of that other stuff is an interface story.

But this conscious experience right now,

this is the only reality as far as I can tell.

The rest of it’s a story.

But that is, again, not my default.

That is, I have to make a really conscious choice

to say, okay, I know intellectually

this is all an interface.

I’m gonna take the headset off and so forth.

And then immediately sink back into the game

and just be out there playing the game and get lost in it.

So I’m always lost in the game

unless I literally consciously choose to stop thinking.

Isn’t it terrifying to acknowledge

that, to look beyond the game?

Isn’t it?

It scares the hell out of me.

It really is scary because I’m so attached.

I’m attached to this body.

I’m attached to the interface.

Are you ever worried about breaking your brain a bit?

Meaning like, it’s, I mean, some of these ideas,

some of these ideas, when you think about reality,

even with like Einstein, just realizing,

you said interface, just realizing that light,

that there’s a speed of light

and you can’t go faster than the speed of light

and what kind of things black holes can do with light,

even that can mess with your head.

Yes.

But that’s still space time.

That’s a big mess, but it’s still just space time.

It’s still a property of our interface.

That’s right.

But it’s still like, even Einstein realized

that this particular thing,

some of the stories we tell ourselves

is constructing interfaces

that are oversimplifying the way things work

because it’s nice.

The stories are nice.

Stories are nice.

I mean, just like video games, they’re nice.

Right, but Einstein was a realist, right?

He was a famous realist in the sense

that he was very explicit in a 1935 paper

with Podolsky and Rosen, the EPR paper, right?

They said, if without in any way disturbing a system,

I can predict with probability one,

the outcome of a measurement,

then there exists in reality that element, right?

That value that, and we now know from quantum theory

that that’s false.

Einstein’s idea of local realism is strictly speaking false.

Yeah.

And so we can predict, we can set up,

in quantum theory, you can set up,

and there’s a paper by Chris Fuchs, quantum Bayesianism,

where he scouts this out.

It was done by other people,

but he gives a good presentation of this,

where they have a sequence of something

like nine different quantum measurements that you can make.

And you can predict with probability one

what a particular outcome will be,

but you can actually prove that it’s impossible

that the value existed before you made the measurement.

So you know with probability one what you’re gonna get,

but you also know with certainty

that that value was not there

until you made the measurement.

So we know from quantum theory

that the act of observation is an act of fact creation.

And that is built into what I’m saying

with this theory of consciousness.

If consciousness is fundamental,

space time itself is an act of fact creation.

It’s an interface that we create, consciousness creates,

plus all the objects in it.

So local realism is not true.

Quantum theory has established that.

Also noncontextual realism is not true.

And that fits in perfectly with this idea

that consciousness is fundamental.

These things are, these exist as data structures

when we create them.

As Chris Fuchs says, the act of observation

is an act of fact creation.

But I must say on a personal level,

I’m having to spend,

I spend a couple hours a day

just sitting in meditation on this

and facing the rebellion in me

that goes to the core,

it feels like it goes to the core of my being,

rebellion against these ideas.

So here it’s very, very interesting

for me to look at this because,

so here I’m a scientist and I’m a person.

The science is really clear.

Local realism is false.

Noncontextual realism is false.

Space time is doomed.

It’s very, very clear.

It couldn’t be clearer.

And my emotions rebel left and right.

When I sit there and say, okay,

I am not something in space and time.

Something inside of me says, you’re crazy.

Of course you are.

And I’m completely attached to it.

I’m completely attached to all this stuff.

I’m attached to my body.

I’m attached to the headset.

I’m attached to my car.

I’m attached to people.

I’m attached to all of it.

And yet I know as an absolute fact,

I’m gonna walk away from all of it.

I’m gonna die.

In fact, I almost died last year.

COVID almost killed me.

I sent a goodbye text to my wife.

So I thought I was done.

You really did.

I sent her a goodbye.

I was in the emergency room and it had attacked my heart

and it had been at 190 beats per minute for 36 hours.

I couldn’t last much longer.

I knew I couldn’t, they couldn’t stop it.

So that was it.

So I texted her goodbye from the emergency room.

I love you, goodbye kind of thing.

Yeah, right.

Yeah, that was it.

So, so.

Were you afraid?

God, it scares the hell out of you, right?

But there was, you’re just feeling so bad anyway

that sort of you’re scared, but you’re just feeling so bad

that in some sense you just want it to stop anyway.

So I’ve been there and faced it just a year ago.

How did that change you, by the way?

Having this intellectual reality that’s so challenging

that you meditate on, it’s just an interface.

And one of the hardest things to come to terms with

is that that means that it’s gonna end.

How did that change you having come so close

to the reality of it?

It’s not just an intellectual reality,

it’s a reality of death.

It’s forced, I’ve meditated for 20 years now.

And I would say averaging three or four hours a day.

But it’s put a new urgency,

but urgency is not the right word

because it’s riveted my attention, I’ll put it that way.

It’s really riveted my attention and I’ve really paid,

I spent a lot more time looking up

what spiritual traditions say.

I don’t, by the way, again, not taking it with the,

take it all with a grain of salt.

But on the other hand, I think it’s stupid for me

to ignore it.

So I try to listen to the best ideas

and to sort out nonsense from,

and we all have to do it for ourselves, right?

It’s not easy.

So what makes sense?

And I have the advantage of some science

so I can look at what science says

and try to compare with spiritual tradition.

I try to sort it out for myself.

But then I also look and realize

that there’s another aspect to me,

which is this whole emotional aspect.

The, I seem to be wired up

as evolutionary psychology says I’m wired up, right?

All these defensive mechanisms, you know,

I’m inclined to lie if I need to.

I’m inclined to be angry, to protect myself,

to have an in group and an out group,

to try to make my reputation as big as possible,

to try to demean the out group.

There’s all these things

that evolutionary psychology is spot on.

It’s really bright about the human condition.

And yet I think evolution, as I said, evolutionary theory

is a projection of a deeper theory

where there may be no competition.

So how, so I’m in this very interesting position

where I feel like, okay,

according to my own theory, I’m consciousness.

And maybe this is what it means

for consciousness to wake up.

It’s not easy.

It’s almost like I have,

I feel like I have real skin in the game.

It really is scary.

I really was scared when I was about to die.

It really was hard to say goodbye to my wife.

It really, it really pained.

And to then look at that and then look at the fact

that I’m gonna walk away from this anyway

and it’s just an interface.

How do I, so it’s trying to put all this stuff together

and really grok it, so to speak,

not just intellectually, but grok it at an emotional level.

Yeah, what are you afraid of,

you silly evolved organism

that’s gotten way too attached to the interface?

What are you really afraid of?

That’s right.

Is there a…

Very personal, you know, it’s very, very personal.

Yeah.

I mean, speaking of the text,

what do you think is this whole love thing?

What’s the role of love in our human condition?

This interface thing we have,

is this somehow interweaved,

interconnected with consciousness?

This attachment we have to other humans

and this deep, like some quality to it

that seems very interesting, peculiar.

Well, there are two levels I would think about that.

There’s love in the sexual sense

and there’s love in a deeper sense.

And in the sexual sense,

we can give an evolutionary account of that and so forth.

And I think that’s pretty clear to people.

In this deeper sense, right?

So of course, I love my wife in a sexual sense,

but there is a deeper sense as well.

When I was saying goodbye to her,

there was a much deeper love that was really at play there.

That’s one place where I think

that the mixed bag from spiritual traditions

has something right.

When they say, love your neighbor as yourself,

that in some sense, love is fundamental.

I think that they’re onto something,

something very, very deep and profound.

And every once in a while,

I can get a personal glimpse of that,

especially when I’m in the space with no thought.

When I can really let go of thoughts,

I get little glimpses of a love

in the sense that I’m not separate.

It’s a love in the sense that I’m not different from that.

If you and I are separate, then I can fight you.

But if you and I are the same, if there’s a union there.

The togetherness of it, yeah.

What, who’s God?

All those gods, the stories that have been told

throughout history, you said through the spiritual traditions.

What do you think that is?

Is that us trying to find that common thing at the core?

Well, in many traditions, not all.

The one I was raised in, so my dad was a Protestant minister.

We tend to think of God as a being.

But I think that that’s not right.

I think the closest way to think about God is being, period.

Not a being, but being, the very ground of being itself is God.

I think that’s the deep, and from my point of view,

that’s the ground of consciousness.

So the ground of conscious being is what we might call God.

But the word God has always been,

for example, you don’t believe the same God as my God,

so I’m gonna fight you, or we’ll have wars over,

because the being, the specific being that you call God

is different from the being that I call God,

and so we fight.

Whereas if it’s not a being, but just being,

and you and I share a being,

then you and I are not separate,

and there’s no reason to fight.

We’re both part of that one being,

and loving you is loving myself,

because we’re all part of that one being.

The spiritual traditions that point to that,

I think are pointing in a very interesting direction,

and that does seem to match with the mathematics

of the conscious agent stuff

that I’ve been working on as well,

that it really fits with that, although that wasn’t my goal.

Is there, you mentioned,

you mentioned that the young physicists that you talk to,

or whose work you follow, have quite a lot of fun

breaking with the traditions of the past,

the assumptions of the past.

What advice would you give to young people today,

in high school, in college, not just physicists,

but in general, how to have a career they can be proud of,

how they can have a life they can be proud of,

how to make their way in the world,

from the lessons, from the wins and the losses

in your own life, what little insights could you pull out?

I would say the universe is a lot more interesting

than you might expect, and you are a lot more special

and interesting than you might expect.

You might think that you’re just a little, tiny,

irrelevant, 100 pound, 200 pound person

in a vast billions of light years across space,

and that’s not the case.

You are, in some sense, the being that’s creating

that space all the time, every time you look.

So, waking up to who you really are,

outside of space and time,

as the author of space and time,

as the author of everything that you see.

The author of space and time, sorry.

You’re the author of space and time,

and I’m the author of space and time,

and space and time is just one little data structure.

Many other consciousnesses are creating

other data structures, they’re authors

of various other things.

So, realizing, and then realizing that,

I had this feeling growing up, going to college,

reading all these textbooks, oh man, it’s all been done.

If I’d just been there 50 years ago,

I could have discovered this stuff,

but it’s all in the textbooks now.

Well, believe me, the textbooks are gonna look silly

in 50 years, and it’s your chance

to write the new textbook.

So, of course, study the current textbooks.

You have to understand them.

There’s no way to progress until you understand

what’s been done, but then,

the only limit is your imagination, frankly.

That’s the only limit.

The greatest books, the greatest textbooks

ever written on Earth are yet to be written.

Exactly.

What do you think is the meaning of this whole thing?

What’s the meaning of life from your limited interface?

Can you figure it all out, like why?

So, you said the universe is kind of trying to figure

itself out through us.

Why?

Yeah, that’s the closest I’ve come.

So, I’ll give you, so I will say that I don’t know,

but here’s my guess, right?

That’s a good first sentence.

That’s a good starting point.

And maybe that’s gonna be a profound part

of the final answer is to start with the I don’t know.

It’s quite possible that that’s really important

to start with the I don’t know.

My guess is that if consciousness is fundamental

and if Gödel’s incompleteness theorem holds here,

and there’s infinite variety of structures

for consciousness to some sense explore,

that maybe that’s what it’s about.

This is something that Annika and I talked about a little bit

and she doesn’t like this way of talking about it.

And so I’m gonna have to talk with her some more

about this way of talking.

But right now I’ll just put it this way

and I’ll have to talk with her more

and see if I can say it more clearly.

But the way I’m talking about it now is that

there’s a sense in which there’s being

and then there’s the experiences or forms

that come out of being.

That’s one deep, deep mystery.

And the question that you asked, what is it all about?

Somehow it’s related to that.

Why does being, why doesn’t it just stay without any forms?

Why do we have experiences?

Why not just have, when you close your eyes

and pay attention to what’s behind you, there’s nothing.

But there’s being.

So why don’t we just stop there?

Why didn’t we just stop there?

Why did we create all tables and chairs

and the sun and moon and people?

All this really complicated stuff, why?

And all I can guess right now,

and I’ll probably kick myself in a couple of years

and say that was dumb, but all I can guess right now

is that somehow consciousness wakes up to itself

by knowing what it’s not.

So here I am, I’m not this body.

And I sort of saw that, it was sort of in my face

when I sent a text goodbye.

But then as soon as I’m better, it’s sort of like,

okay, I sort of don’t wanna go there, right?

I, okay, so I just, so I am my body.

I go back to the standard thing, I am my body

and then I want to get that car.

And even though I was just about to die a year ago,

so that comes rushing back.

So consciousness immerses itself fully

into a particular headset.

Gets lost in it and then slowly wakes up.

Just so it can escape and that is the waking up,

but it needs to have a negative.

It needs to know what it’s not.

It needs to know what you are.

You have to say, oh, I’m not that, I’m not that.

That wasn’t important, that wasn’t important.

That’s really powerful.

Don, let me just say that because I’ve been

a long term fan of yours and we’re supposed

to have a conversation during this very difficult moment

in your life, let me just say you’re a truly special person

and I for one, I know there’s a lot of others that agree.

I’m glad that you’re still here with us on this earth

if for a short time.

So whatever, whatever the universe,

whatever plan it has for you that brought you close

to death to maybe enlighten you some kind of way,

I think it has an interesting plan for you.

You’re one of the truly special humans

and it’s a huge honor that you would sit

and talk with me today.

Thank you so much.

Thank you very much, Lex.

I really appreciate that, thank you.

Thanks for listening to this conversation

with Donald Hoffman.

To support this podcast, please check out our sponsors

in the description and now, let me leave you

with some words from Albert Einstein,

relevant to the ideas discussed in this conversation.

Time and space are modes by which we think

and not conditions in which we live.

Thank you for listening and hope to see you next time.