How are these neurons on this
multi-elerode array playing pong?
>> What the free energy principle says at a
sort of at a gross oversimplification is
that a system will try to predict its
environment by minimizing what it calls
its free energy. But you can also think
of as a form of information entropy or
the amount of disorganization. There's
really only two ways to try to get
better. Either better prediction or
better control. We asked a question well
what if then we made better prediction
impossible. What if upon the cells doing
a set of activity patterns that meant
let's say pong they missed the ball we
gave them something that couldn't be
predicted random white noise. Would the
cells then actually change their
behavior to get better over time at
hitting the ball cuz that's the only way
they could predict their environment. If
you miss you don't know what's going to
happen. And what we found was actually
indeed they did end up remarkably
adjusting their activity patterns very
rapidly to try and create a more
predictable environment and actually
control it which meant from our
perspective not the cell's perspective
but our perspective the cells look like
they were playing the game pong.
Brett Kagan is the chief scientific
officer at Cortical Labs, the Australian
startup fusing living human neurons with
silicon chips. His team published the
famous dish brain paper in Neuron,
showing neurons in a dish could learn to
play pong in just under five minutes,
providing some of the strongest
experimental evidence for Carl Fristen's
free energy principle. Their team went
viral just a few weeks ago as the same
dish brain setup played the video game
Doom. How far can neurons in a dish
really go? Brett thinks quite far.
Subscribe to find out if 200,000 neurons
on a chip ever wondered why the hell
they're playing a video game from the
90s.
>> A lot of people find it a little weird
to be using brain cells as a source of
intelligence. We actually think it's the
most natural thing in the world. And
perhaps technically it is legitimately
the most natural thing in the world to
explore intelligence with. When we're
looking at machine learning, the
question is, is it possible for silicon
to show general intelligence? We don't
know, right? We do not know if that's
possible yet. Current attempts, despite
trillions of investment, have not done
it, but we know for biology it is
possible. The question is, how do you
get it? Do you imagine a world where
we're building just giant buildings of
neurons that are basically just
computation hubs that people are APIing
into to do computation? And these things
are, you know, many orders of magnitude
larger than any individual human brain.
Well, look, I I think there's
Brett Kagan. Everyone saw the headlines.
Neurons in a dish are playing Doom. It
went crazy viral. And I'm super excited
to get into all of the fascinating
nitty-gritty details of that experiment.
But I want to understand and I want the
listener to understand what's the most
important context that led up to that
experiment. What were the stepping stone
discoveries that led up to that? Where
does that story begin in your mind?
That's a great question. Where's where's
the story begin? Because uh at the end
of the day, while a lot of people find
it a little weird to be using brain
cells as a source of intelligence, we
actually think it's the most natural
thing in the world. And perhaps
technically, it is legitimately the most
natural thing in the world to explore
intelligence with. To me, it's sort of a
weird thing that we've taken silicon,
pounded it out, tried to infuse that
with something that could resemble
intelligence. And so, I suppose you
could say it began with evolution. Uh
but to bring us up just a little bit to
more recent history, we began with a
question back in 2019
where we wanted to know could we get
cells in a dish to do anything at all
that we might want. We initially started
with a simpler game. We started with
pong and we managed to get that work. We
published some work on that which did
attract some attention back in 2022
and then from there the last few years
we've been pretty quiet and people have
wanted to know why. What have we been
doing? When's the next thing? Is there
anything else happening? And it's
because behind the scenes what we've
been doing is actually building
foundations.
And this foundation is a device called
the CL1. And essentially it's the
world's first commercially available
device intended for neural biomputation
to explore information processing and
the basic fundamental intelligence that
cells can exhibit these neural cells can
exhibit. And we launched that last year
and then this year we launched cloud
access. So now anyone across the world
can actually log on and interact through
the API that we've built, this
programming interface we've built that
we've simplified and interact with these
cells.
And of course with that, one of the
things we did was actually provide a
hackathon to students giving them access
and seeing could they actually build and
get one of the most classic computer
games do to actually show some learning
effects. And so this wasn't a rigorous
scientific experiment. It was actually
just seeing what can people who had no
experience in the area do with the tools
we built. And one of them, Sean Cole,
actually managed to do a very workable
version of sales playing Doom. Still
lots of work to do, but that's the sort
of origin story behind it and it's
caught a lot of people's imagination.
>> Incredible. Really, really well
explained. How would you summarize the
ultimate moonshot goal of Cortical Labs
to be? What are you really trying to
achieve in the long term?
So long-term we really want to be able
to provide people a new tool in the
intelligence toolbox. At the moment
obviously there's a huge amount of
attention and media and social media
hype around the LLMs and other AI
implementations. I mean shoe companies
are adding AI to their name and sending
their stock skyrocketing. There's a lot
of hype around that. But at the end of
the day it's all the same tool more or
less or a spectrum of them, right? It's
all using classic volume and
architecture, GPU, CPU, and trying to
run it through essentially variations of
the artificial neural network. And we we
asked the question really early on, why
would we want to model in silicon or in
code what we can harness in reality? And
we don't see that as a way to replace it
necessarily, but we do see it as a way
to augment it and provide another tool.
So just as you use your CPU and your GPU
and current silicon computing, what if
you add biology and the benefits biology
has into the way we process and handle
information? And so we aim to be the
company that provides the platform, the
methods, and the techniques to do that,
but also to allow other people to build
their own methods, techniques, and
applications upon it.
>> So cool. So, do you, this is probably
jumping ahead a bit, but do you imagine
a world, you know, we're building all of
these servers at the moment as fast as
we can build them. Do you imagine a
world where we're building just giant
buildings of neurons that are basically
just computation hubs that people are
aping into to do computation and these
things are, you know, many orders of
magnitude larger than any individual
human brain?
>> Well, look, I I think there's there's
two big questions there. one will we be
building server racks of biological
computation we already are we already
have our first small scale admittedly uh
biomp computing data center here in
Melbourne and there are groups setting
up there's a recent news article in
Singapore where we're partnered with a
data center provider day one and they're
looking to set up a thousand there in
the future and so this is happening now
the second part of your question
actually is is perhaps more nuanced
because you ask will we have cells or
systems many times larger than the human
brain. And I think that is a technical
question we don't have the answer to
yet. Our personal focus is not actually
to build human brain in a dish, right?
Because largely there's already a lot of
human brains in the world and people are
able to use them already to do pretty
cool things. We don't want to replace
people. But we are interested in the
material that the brains are made out of
these neural cells. And we want to know
the question of what if we actually
build these systems from the material
circuits of neurons to achieve things
that silicon can't achieve, but maybe
also to achieve things human brains
can't achieve. And so we're not aiming
to replace human brains or build these
brains in a dish. We're able to actually
create something that's totally new, but
with a shared starting point. And I
think this addresses that we work a lot
on the ethics and there are a lot of
ethical concerns about what if you
create something in a in a dish that
could experience or suffer and we work a
lot on that and it gets pretty deep.
But actually that's not even our core
goal.
There's a lot of different approaches to
go forward. We don't have the answer yet
but we are actively working with many
researchers around the world to try and
figure out what is the best way to build
these systems and the most ethical and
responsible way to build these systems.
>> Very cool. If you want to truly
understand the contents of your own
inner conscious experience, like your
mental imagery, your own inner speech,
or your emotions, then click the link in
the description and take my 7-day inner
experience challenge. Throughout the
challenge, you will learn about the five
frequent phenomena that make up our
conscious reality and engage in the
actual process used in research to
discover your own completely unique
breakdown. How often do you really inner
speak? What kind of mental imagery do
you have? Do you see in vivid pictures,
in three dimensions, in two dimensions,
black or white? Maybe you're part of the
5% of people among us that have no
mental imagery at all. Where do you feel
and process your emotions? We all have
fascinatingly different answers to these
questions. And by day seven, you'll
finally be able to confidently answer
these questions for yourself.
It is completely free to do and you will
get the most comprehensive breakdown of
your actual conscious reality using only
research proven methods. I am genuinely
so excited about this. Just click the
link in the description. I hope to see
you inside the community. Well, can you
lay out in as simple terms as possible
what you mean by these systems? So, I
think a lot of people when they hear
neuromorphic computing, biological
computing, they're not entirely sure
what these words mean. So how exactly
are is biology and computing being fused
together in what's being done already,
what you say you're already building and
what you hope to build in the future.
What does that system really look like?
So essentially, I mean, when you use
words like fuse together, it kind of
gives you some interesting vibes. But in
practice, we're able to grow cells,
especially neural cells, they are
evolved to connect. they're evolved to
network together. And so when we place
these cells on these what we call
microelerode arrays or meas which is
sort of a a bed of small electrodes that
allow us to measure the electrical
impulses of these cells as they're
active and deliver small electrical
pulses to the cells. They actually
naturally do this. There is nothing
particularly it's synthetic in that we
bring it together manually. But it's not
wholly artificial. It's leveraging the
natural properties of these systems. And
what we do then is grow these cells on
these MEA and then we have to code
information and decode information and
that's where electricity comes in.
Electricity is this shared language
between silicon and between the biology
and so we can actually uh pattern the
types of electrical pulses we deliver to
convey information and then we can
decode the patented electrical responses
from these cells to understand how
they're responding. Now, there's still a
lot of work to do here, but we've been
making some really interesting strides
forward in figuring out the best way to
do this part.
>> Really cool. You're a great
communicator, man. Um, really, really
clear. Let's, so let's let's
contextualize this within the Pong
system because I think you laid it out
really nicely there. Maybe people have
some idea what this looks like. How are
these neurons on this multi-elerode
array playing Pong?
So the the old pawn work that we did was
actually not the simplest but one of the
simplest approaches we can took we could
have taken. When we started out we had a
lot of options. There's a huge amount of
ways to go about this. And so we did
what I thought at the time was the
simplest idea and said let's do the
simplest ones first. And so we took the
inspiration from how biology can decode
the world which is effectively uh what's
called topographic. So in our for our
eyes for example if we see something in
the left of our visual field there'll
actually be a little map topographic map
they call it in our visual uh uh visual
cortices that actually will decode it
and have a onetoone mapping. And then we
took the rate and there the closest
analogy for the whole system is actually
a rodent barrel whisker cell it's
called. Uh and these barrel cells have a
map of where the whisker is on the
animal's face. And the more you tweak
the whisker, the faster it fires. And so
that's what we did. So we had this map
that was initially inspired by the
visual system, but then we added rate to
it. So the closer the ball got to the
end, the uh faster the rate would fire.
So in the end it ended up kind of like
what a how a rodent's whisker would
work. And then what we did was we said
to the cells, we're going to put some
information into one area. We're going
to read information out of a couple of
counterbalanced regions to make sure
there was no bias. We don't want to be
the case where we sort of stimulated in
one area and the cells just learned to
do something really simple. They had to
have a bit more control over their
system than that. And we wanted to make
sure there was no bias. And then we read
out of this counterbalanced regions. And
what we found was that if the cells
would actually understand for the given
value of cells in a dish, right? This is
not understand like we would understand.
They would understand like the most
basic primitive systems you could
imagine. Uh the what their task was in
other words move paddle to hit ball.
They would actually change their
patterns of activity to do that.
So you're encoding the particular
properties of the game Pong, which is
all you control in that game is actually
the paddle, right? And then the reward
function is, you know, you're rewarding
the neurons, which we'll get into in a
bit when the ball hits the paddle at the
right angle. and then you are providing
some sort of negative reinforcement or
negative feedback to the neurons when
the ball misses the paddle or the paddle
is far away from the ball. Are those
sort of the two central things that you
are trying to encode and then stimulate
in those neural populations?
>> Yeah. So concretely there's two aspects.
We encode the position of the ball
relative to the paddle. So if the paddle
is aligned to the ball, it gets one sort
of stimulation, left, another, right,
another, we had in that case eight
points where we would use to stimulate
and then we would also give it a
different type of stimulation whether it
successfully hit the ball or missed it.
So the setup itself was actually quite
simple in that regard.
And where are you choosing to stimulate?
So you have all of these plated neurons
that you grew from stem cells, right?
Where do you choose to stimulate them?
What neurons in that population do you
choose to stimulate? And how do you make
that decision?
>> Well, in those initial cases, it was
pretty arbitrary because we were growing
these what's called a mono layer, which
is a single layer, although they do tend
to have some more structure in there,
but it's sort of undirected,
uncontrolled of cells. And so we looked
at it and initially when we started out
we used to set up a specific input
output relationship for every culture.
And that was incredibly time consuming
with our old with the old hardware and
software that we were doing. It would
take literally hours to map it. And
eventually we asked the question look if
cells neural cells are meant to be
plastic and meant to be able to
reorganize their activity. Why are we
the ones sitting here trying to shape
the input and alkal relationship instead
of letting the cells shape how they
respond? And so we then just created a
very fixed pattern. We had a little area
at the top that we called the sensory
region. These are all cortical cells,
but we called it a sensory region
because we put sensation, these
electrical pulses in. And we had these
counterbalanced areas we call motor
regions because they moved the paddle.
Again, all the same cell type and we'd
read activity out of. And so that was
the same for all the cultures back then.
Uh with the new systems we built the
CL1, it's very very easy to change these
mappings and you can do it rapidly even
in real time pretty much if you want um
or close to it. And so we have a lot
more options now than we did back then.
We also have a bigger team now than we
had back then. Uh and so it's opened up
a lot more possibilities. But the very
fixed kind of simple and boring approach
is what we tried before. The remarkable
thing though was indeed that for many
cultures uh the majority even as our
result showed they were able to
reorganize their activity. There were
caveats to that. If you had the culture
was too uneven or they were just dead
all on one side that broke down but we
represented all of this data in the
paper. Um but yeah so there there was a
lot there.
>> Yeah. So how so how specific do these
starting conditions need to be or is it
really the beauty of these biological
systems is that provided the right
inputs they will self-organize into a
functional larger system. Does that sort
of always happen or are there loads of
situations where your starting situation
wasn't right enough? the sensory neurons
weren't set up in the right way
connected to the motor neurons and then
it just didn't learn the game. Explain
sort of how robust the system is in
response in with that initial starting
conditions and the inputs from the game
itself.
>> Yeah, very big question there and one
we've been doing a lot of work on. Uh in
fact a a very recent paper that's not
out now but will be out by the time this
episode airs for sure uh has spent a lot
of time looking at this question how
does the structure and the function
relate what setup leads to the best
performance
>> and so as you'll be able to see in this
work we actually have the case where
we've done all sorts of things like
Morse code classification emnest which
is a type of handwritten digit task that
people use in machine learning often and
we find really strongly that the
structure that you set these cell
cultures up in have a very strong effect
in how that they're actually able to
perform certain tasks. But it varies and
some tasks benefit from some structure
more than others. So there's a lot of
complexity here and a lot of nuance. But
that also means there's a lot of control
in how we build these systems. And this
goes all the way back to that point I
made before is we're not necessarily
trying to build a human brain in a
ditch. We're trying to figure out how do
we build, how do we bioengineer,
the best type of cell culture, the best
material to achieve the goals we want to
achieve. And I'll give you like an
example is that bees are far simpler
than a human. Uh and yet bees are
remarkable in what they do and in many
things they do humans could not achieve.
And so again, why we don't need to make
more humans. There's plenty of humans
out there who are looking for work to
do. Let's do that. Let's figure out how
to do something totally different.
>> I find this so unbelievably cool. So,
how much how much inspiration are you
able to take from what we know about the
architecture of the human brain and the
type of problem and task that that part
of the brain is equipped to solve? How
well are you able to replicate that
system in your monollayer multi-elerode
array? And how informative is that? Can
you, you know, take a particular brain
region? it's we know it's good at doing
this task like language like Brock's
area for example something like that
does that map onto the type of problems
that your monollayer neuron system is
then able to is it better equipped to to
solve
>> yeah I should clarify as well that we
don't just do mono layers that's what we
did in the initial pong work
>> but we we do all range these things
called organoids which are a little more
spherical and they're meant to be sort
of a mini organ uh but they're far
simpler than braids it's not quite right
to call them a mini brain because they
don't have that complexity or
connections or the diversity of cell
types, but they do have a lot more
complexity at least geometrically
than a uh a monollayer. And then we also
do these bio-engineered modular units
which kind of move further from
physiology but still have some really
unique properties. Uh in terms of how
much do we take inspiration from the
human brain specifically?
Quite a lot. I mean the team uh my
background is in neuroscience and we
have several other neuroscientists on
the team and so it would certainly
natural I guess pun intended uh to take
inspiration from the the thing itself
right uh and so if we want to look at
things like let's say the hippoc campus
which famously has these very powerful
place coding subs would be silly not to
take what we know about that when we go
and develop hippocample cells which
we've got and done and we've got a paper
out on
at the moment. Um, but there's a big
difference from taking inspiration
from something and trying to copy it
exactly. And so we don't throw away what
evolution and physiology has done. That
would be insane.
But we also aren't going to try to
recreate it, which I think has a similar
degree, recreate it exactly for those
purposes, at least anytime soon. Uh,
until we had some sort of evidence or
justification to do so.
uh because that would also similarly be
incredibly technically difficult.
>> Yeah. Very cool. So talk about the
process if you have a new task or
problem that you want the um neural
system to solve say emnest or you know
any any of the other tasks. What would
be the process by which you start to
think about or try to start model the
type of architecture that might be
optimized for that system as a starting
place? like how do you think about that
process?
>> There's
thinking what I can talk about right
now. There'll be some exciting stuff
coming out soon, but I can't talk about
it right now. Sorry.
>> You can talk about it right now, Brett.
can I can
>> what what I what I'd say is uh
cautiously that there's
there's a lot of different approaches
and actually there's far more that we
don't even know that we don't know still
about this space and so a lot of our
work needs to be from ground principles
exploratory we sometimes take
inspiration from machine learning which
is kind of closing the cycle because a
lot of machine learning was inspired by
neuroscience so now we sometimes take
inspiration from that. Um for example
like on like reservoir computing
approaches and other things that people
might have heard of. Um there's a whole
range of things. Uh sometimes we do take
inspiration from again the physiological
basis. So we might say well actually be
brains have this type of connectivity.
So we want to try and achieve this type
of connectivity. But a lot of the time
it just comes down to good oldfashioned
science. You try something it works or
normally it doesn't work. you try
something else, you write it down, you
move to the next thing. Where where
we've sort of reached an unlock is not
necessarily that we are so much more
clever than any other lab out there. Uh
and there are many labs out there who
are doing good work that also we can be
informed by. Of course that may to me it
goes without saying but I should say it
actually there is a lot of amazing work
being done and we collaborate with a lot
of amazing scientists all around the
world. uh but where we have a particular
advantage is that we have as I said
server racks of these units. So we can
actually iterate and run through
thousands of hours or more a week
just to try and find the answer to a
question that previously would have
taken labs years to do. Take for example
this Doom game, right? Again that was
built by someone who's a non-expert in
the field in about a week or so.
It took us something like 18 months to
really get a pond environment working to
a similar degree of control I would say.
Uh and that's because we had to start
all the way from the ground floor. Don't
get me wrong, that was a huge
engineering challenge. And um Andy
Kitchen, who did the the substantive of
that work, uh did an amazing job in
building it, but he started with with
very little, right? It all had to be
built from the ground up. That's so
slow. When you've got the foundations
that were laid, you can move much
quicker.
It's so cool. What maybe give a few
other examples of tasks that this that
you have been able to get good very good
or good outputs from like what other
things can these systems do right now?
>> Yeah. So, we've we've been looking at a
few things. Um, some of the ones I can
talk about are things like this
handwritten digit classification. Uh,
seeing if cells can identify the 0 to 9
digits. uh which has been really
exciting. Morse code has been fun. Um
you know can
>> explain that elaborate on elaborate on
exactly teaching at Morse code what that
means exactly you're having a
conversation with this thing.
>> Yeah the re well the way we started out
initially was we kind of did what we we
refer to sometimes internally as a wine
maze test. Y maze is a really classic
neuroscience test. If you put the
rodent, normally people have built
bigger ones for I think dogs and monkeys
and other things, but normally it's
rodents and it's literally just a
Y-shaped maze and the the rodent has to
choose to go left or right. And we would
sort of drop in a stimulation either on
the left or the right and we just want
to see would the cells respond very
differently to this and would they
respond in a way that would kind of
align. So we were kind of doing this
spatial classification
and we wanted to move on to a rate based
classification and so we were thinking
well how's the best way to do that and I
realized well rate there is already a
rate based classification it's called
Morse code so why invent our own rate
based classification when we could just
say to people we're going to try Morse
code because that already exists and
it's essentially the same thing and what
we found is actually cells are really
sensitive to rate coding so we found
that Morse code is a very strong effect
cells are pretty good at it. Um, so
that's why I went to MOS code. So we put
in sort of different frequencies, you
know, saying that would represent an S
or an O. Uh, can cells can cells decode
SOS? These sorts of things that we could
we would find sort of interesting to
explore the dynamics and this was the
question, right? wasn't can cells learn
Morse code which is a cute question but
the real question is how does the
structure and function dynamics the cell
types the uh way we provide feedback how
does that shape how the cells actually
decode this information can we get them
to improve so simple task but with a way
to ask some really powerful questions so
when you say decoding there if you're
you're the input is SOS encoded in Morse
code and what does it give you on the
other end like what is the output?
>> So for that particular task we were
basically mapping particular types of
responses from the cells and then trying
to improve the consistency in which the
cells would display a certain type of
response. So it's we would post hawk say
all righty this response means you've
identified you've seen an essence this
response means that you've se seen again
seen is used um metaphorically here as
much as anything but perceived may be a
word but again not like how we perceive
again in this very rudimentary
physics-based analogy it's hard to talk
and this is one thing that pops up is
it's very hard to talk about this
>> this because the the word remains like
You know, bacteria to some extent
perceives and so do cells in a dish. But
if I say perceives, you're going to
anthropomorphize it. And this is this is
sometimes where people get a bit upset
with us, but but ultimately like the
most basic form of a thing is still the
thing.
>> I definitely won't get upset. I've had
many podcast a big thread on this
podcast has been biology is cognition
the whole way down. You know, the work
of Michael Lean that this
>> Yeah, exactly. He like he's really
informed a lot of these podcast
episodes. So you could basically use any
human term for a neuron in a dish. And I
am totally cool with that because I I
believe all these things are spectrums
and continuums now and you have to tell
me why those neurons aren't seeing. I I
think that it's actually a a kind of
totally legitimate way of speaking about
it. And maybe we can get into those
differences later, but yeah, I I I think
this is a completely fine way of talking
about these systems.
>> Mhm.
So maybe it's worth talking about the
reward function because this is sort of
an important aspect of this I think
because like what is the dog treat
analogy here you know what is the you're
such a good boy hand them the treat
feedback signal happens how are you
doing that for neurons on a plate
>> yeah and initially we used the words
reward and punishment and we were using
that my my before neuroscience my
background was psychology
uh in the strict behavioral sense, a
reward is something that increases the
probability that a behavior will arise
and a punishment is something that
decreases the probability of the
behavior arising. It has nothing to do
with it being pleasant or painful or
anything like that. It's simply the
probability of a behavior and cells
having electrical activity is again one
of the most rudimentary behaviors but it
is a type of behavior
um at the most basic level. Uh however
again many people misunderstood that. So
we now use the words and I think these
words are more intuitive to understand
anyway. They're probably better. We just
didn't think about it initially. Um
we use the words now reinforcing in that
it's going to keep a set of patterns
happening or plasticity inducing in that
it's going to incentivize the cells to
change that pattern and how it relates.
So what we used initially to be
plasticity inducing was based on this
theory uh that's been worked on by many
many people but it was originally
developed by professor KL Fristen over
at University College London and it was
called the free energy principle and
what the free energy principle says at a
sort of at a gross oversimplification as
it is a very overarching theory uh and a
broad branching theory at that is that a
system will try to predict its
environment by minimizing what it calls
its free energy. But you can also think
of as a form of information entropy or
the amount of disorganization.
And what that basically says is that we
are and there's this part of it that of
course active inference. We are going to
aim to predict our environment and align
those predictions with what we
experience. So if I reach for my cup, I
don't really in any sort of direct sense
see the cup except in a gestalt sense.
My brain interprets signals it receives
from my eyes that the photons are
bouncing on. And I'm going to make a
model and say it's likely this
distribution of possibilities that there
is indeed a cup there based on what
sensory information I've seen. And if I
was to engage in a particular action
like reaching for it, I would
successfully pick it up. And if that
happens, my expectation
aligns with the future sensation of the
world. It's a it gets a little
complicated, but that's the core of it
is can you predict the world? And if
not, how do you then in future align it?
There's only two ways. One is to get
better at reaching for the cup. If I
reach for the cup, I knock it over. That
surprises me. It doesn't align with what
I expected as an outcome. And so then I
have to either get better at knowing
that if I reach for the cup in that way
I will knock it over or I have to get
better at picking up the cup and
controlling the world. These are sort of
the two options to align.
>> Yeah. Well explained hard concept to
explain. I've spoke to I've spoke to
Fristen and that's probably one of the
best explanations I've gotten because
it's a it's a damn hard concept to
really explain simply. So I so I I bow
to you on that one. Good. Nice one. Nice
job.
>> Well, he he he's the he's the uh the the
godfather of it of the theory. Um, so he
has a lot of nuance that I'm sure he
would um with very good grace because
he's an absolute absolute gentleman
scientist.
>> Yes, 100%
>> would probably say well actually Brett
but but at a high oversimplification I I
believe that this is this would be
accurate. Um uh the thing is though if
we get down to this conclusion right
that there's really only two ways to try
to get better either better prediction
or better control. We asked a question
well what if then we made better
prediction impossible. What if upon the
cells doing a set of activity patterns
that meant let's say pong they missed
the ball we gave them something that
couldn't be predicted random white
noise. Would the cells and it was a
question. It was a question. Uh would
the cells then actually
uh change their behavior to get better
over time at hitting the ball? Because
that's the only way they could predict
their environment. If you miss you don't
know what's going to happen. It's kind
of the equivalent of uh if I'm in a room
and I do something wrong and let's say
you like you turn the lights off like
over time I'll just be like oh I did it
wrong and it's going to be dark for a
bit that's okay I know that will happen
but if every time like one time you
turned the lights off one time you
played loud music the other time you
dropped me through a trap door like I'm
not know I don't know what's going to
happen the only thing I could do is be
like all right I better hit that ball
because I just don't know what's going
to happen next u now this is
anthromorphizing it but You could
imagine that this is true of that level
of information physics as well. And so
that's what we did for the cells. And
what we found was actually indeed they
did end up remarkably adjusting their
activity patterns very rapidly to try
and create a more predictable
environment and actually control it,
which meant from our perspective, not
the cell's perspective, but our
perspective, the cells look like they
were playing the game pong.
So when when the paddle hit the ball at
the absolute perfect angle, you would
give it the most patterned, predictable,
lovely, gorgeous stimulation that it
likes, quote unquote. And then
>> it was it was it was a bit rougher than
that. It was any any hit would get
something predictable. Any miss would
get something unpredictable. Again, we
took the simplest approach possible.
>> Gotcha. So I was trying to figure out
then is there like a distribution of
okay that hit that exact ball hitting
the paddle just 1 cm over is slightly
worse. So then the pattern is slightly
slightly more jumbled, slightly more
noisy and then you kind of have a
>> it's not exactly
>> no we we did discuss this and as I said
we started with the simplest approach
and uh essentially like as we described
in the paper there were two simpler
approaches and the one that worked quite
well. Uh and what we found is as we
improved the sort of complexity we could
start to see more and more measurable
effect. Now, we could have continued to
go down that road, try to refine and get
better and better at Pong, except we're
not set up to be a pong playing company.
I mean, Pong's a cute example, but it's
not in of itself what's going to really
matter in the world.
>> That should be the big goal.
>> Once we've had the effect company,
>> we're not a pong playing company,
although we do do sit.
>> Um,
we realized very quickly like, yeah,
this is a nice effect. We can see it.
Cells in a dish can indeed do some
things. And we realized, but you need
the foundation to do it not just for us,
but for the world. Because
if we're the only ones with this
technology and with access to this
technology, we we will make progress,
but it'll be a fraction of the progress
if we can bring on a community of people
around the world who want to explore
these capabilities with us.
>> Yeah. Yeah. I want to build something.
Now, after this podcast, I'm going to
try and get access to this and see what
I can do because that sounds fun.
>> You can sign up at Cortical Cloud and uh
if you look at Corticle Labs, we have
Corticle Cloud and you can actually sign
up and join. There's a bit of a wait
list, but you can sign up and get
access.
>> Cool. I I originally came across
Dishbrain while preparing for my episode
with Carl Fristen in the first place.
This was maybe eight months ago or so
now. It was the first time I came across
Cortical Labs and your work and it just
blew me away. I was like, "Wow, that's
that seems like really convincing
evidence that the free energy principle
is happening not just on the level of
the organism but at all of the smaller
levels of organization as well. As you
go down the biological hierarchy, at all
levels of self-organization, it seems
like this principle is playing out." And
this seemed like really strong evidence
for that where I hadn't necessarily seen
strong experimental evidence elsewhere
of this actual model prediction
minimization. Like it's a great
framework. It sounds nice, but I'd never
seen an experiment that was like, "Wow,
that's that seems like really good
evidence that that's actually what's
happening." Do you think that that that
result is one of the strongest pieces of
evidence for that that's what's
happening at the lower levels of
biological self-organization?
I think lower levels is an interesting
term and the answer is I I do think that
it is evidence that there is drivers
information based drivers in this and
and bear in mind like uh professor
Fristen and I uh we've worked together
um before you know on these papers he
was he was a senior author in this paper
um but we do have slight difference of
opinions on these I I believe that it is
an formational driver. Uh but I do
believe there are others out there and
we've been working on exploring some of
those drivers over time and we'll have
some work out fairly soon putting real
numbers and math behind that. But I I
believe that there are multiple systems
running. Uh the free energy principle in
different iterations has covered the
idea that there are interacting drivers
and but generally propose that this is
the principal driver which I don't find
as convincing as the idea that it's one
piece of a much broader puzzle.
>> Right?
>> So ultimately though the the nice thing
about the way we're doing it is we can
test it. Um, and a lot of the concerns
some people have with the free energy
principle is that as a broad, it is a
principle. It's not a theory. There's a
reason it's not called the free energy
theory. It's a principle and it can't
really be falsified in of itself. You
can test different applications of it.
And that's one thing we did and we found
support for it. But it as a principle
itself, it's very hard to falsify. And
so we're very keen on looking at
particular applications that can be
falsified or or supported and figuring
out how we put together a much larger
picture on how these systems work.
>> Yes. Very cool. You keep teasing me with
these upcoming results. Brett, you keep
you keep dangling these carrots in front
of me just out of my reach. Um, no,
that's it's a really good point. It's an
interesting piece of nuance because the
free energy principle really does make a
very all-encompassing claim that this is
theformational
driver for systems of all sizes, shapes,
and forms. And I've never heard the the
the push back on that that it seems to
be important and we have a very we
reasons to be like it it could be part
of the equation but I haven't
necessarily heard that you know there
could be many other or a few other or we
just don't know how many
otherformational drivers are forming or
driving this self-organization.
Can you drop any other that you think
might be important here and why would
you believe that?
at a at a high level, one thing that
I'll talk a little about because we'll
have something out relatively soon on it
is the idea of complexity
uh at a high level. And so complexity is
I think a very interesting feature and
we we find many complex things innately
attractive to us as biological creatures
and it's not just us. animals do it,
insects even to some extent um do this.
And so, but it's hard to explain why we
might find beauty attractive. You could
argue beauty is predictable because it
follows some sort of symmetry, but we
don't typically, at least I personally
don't typically find a plain circle or a
square particularly beautiful.
Um there still needs to be a level of
complexity. And so there's this balance
in my mind between
complexity
uh and the structure the information
structure behind that and
predictability, right? And then
personally I believe these are different
biological drivers that interact in some
very interesting ways that actually lead
to far more complex behavior than single
driver alone.
>> Very interesting. I'm really interested
in this at the moment because I've just
had a few conversations with
mathematicians that are modeling the
shape of DMT experiences. So I think
>> I think what a lot of what you just said
there resonates with as people take
escalating doses of DMT, what they see
is this greater complexification and
symmetrical nature of their environment.
you know, things go from three-fold
symmetries to four-fold, five-fold,
sixfold, sevenfold. And it just as these
symmetries and levels of complexity
expand, you know, people
note experientially, phenomenologically
that this is a more intense experience
that it's more all-encompassing.
Do you think that there's some
connection there? Is that is that a a
link that doesn't relate to your work or
or does that sound related to what you
just said?
I think that there's a relation there.
Um, of course, bear in mind like there's
there's the work of cortical labs where
we really are looking to provide a
platform to people. But as a scientist,
I have a particular interest in uh I
guessformational drivers of intelligence
foundationally, what I like to sort of
refer to as like the information
physics. Uh so my my I believe that
there probably is a link there and I
think though there's plenty of examples
and uh in particular I was inspired uh I
have a little 15-month-old now but I was
inspired because I've been working this
idea of complexity for a while and I saw
her as her eyes began to develop around
the first few months and she just loved
fractals
and I thought why why like why would she
love fractals
>> more than the square on the circle?
You'd show the circle, she'd look at it,
look away. Where's the mill? You'd show
her some sort of fractal pattern.
Ah, she would not look away. She loved
the fractal pattern. So, this sort of
led to me saying, I can't explain this
with the tools that I currently have.
And I've been working on these other
approaches for a while, but I've kind of
been playing around this fuzzy edge of
an idea. And it was upon seeing this
that I really went, "Huh, there's
actually something we can do with this."
And again with apologies for sort of
dropping hints. We have been doing some
explorations there and and I think some
of the data is really interesting and I
think it really adds a new perspective
on how we go forward with these things.
You would be a really fun guest to get
on with a few of these people I've been
speaking to. Are you familiar with the
work of Andre Gomez Emerson? He's the
research director at the Qualia Research
Institute and he's really putting
mathematical formalizations that I don't
remotely understand. And I want to be
clear about that behind qualia, behind
experience, behind consciousness. And it
just seems and it's it's very fractal
fractal in nature. And there just seems
to be a lot of connections there um with
what you're speaking about. And he's
done a lot of the really rigorous
computation and he's brilliant guy. I
think you guys will have a fun
conversation. Well, I think I think it
is interesting and it does relate back I
think also to these questions on ethics
that we're talking about because
figuring out what
experience if any these cells in a dish
could have and I will flag very clearly
there is no evidence that these cells in
a dish can have an experience in any way
we would remotely associate with that
word but there are many people who are
still very very worried I'm like without
any evidence for it and we I work with a
lot of these people who do have these
concerns and I don't judge them for
those concerns but it is good to sort of
you can be concerned about the future
but like grounded in the present
evidence and the people I work with do
that and that's great. Uh but one of the
things we're doing to try and address
the risk that could exist one day is to
develop metrics like you're talking
about. So we recently have a paper uh
out at the moment just as a pre-print
under review currently but we expect it
to be out soon uh looking at agency.
Agency is a big topical word at the
moment with all the LLMs.
>> Yes.
>> And we actually developed this
mathematical framework. It doesn't prove
something has agency that can exclude
things that don't based on how they
process and transform information they
receive.
>> Very good.
>> And so it's quite a powerful tactic and
you could imagine extending that to
things like consciousness or whatever
term. I mean terms are very badly
defined as we said before but you could
extend this and start to put these uh
borders around things where you can say
look we don't understand exactly what
consciousness is and maybe we never will
but at the very least we can say if you
don't have this
we don't even need to worry about for
example if you can't transform the
information you receive in a meaningful
way you almost certainly do not have
agency because all you're doing is being
a response like a reflex
or lesser extent. There's a few
varieties of those, but
>> it's a way to sort of say, hey, look, is
a thermostat agent? It changes when it
feels cold.
>> A thermostat isn't an agent. But it's
very hard formally to say why that is.
>> So, we're putting these boundaries
around it and we hope to use that in our
systems to also sort of identify
anything that requires further look. So,
people say, well, how do you know it's
not conscious? we can say well we've
done this analysis and it actually does
not show the same metrics that a human
does or we find out like in some other
cases that metric is actually not
related to consciousness at all um
there's some work we did there at
criticality and we're able to really
demonstrate that this is necessary but
not at all sufficient
>> I find this so fascinating I've had a
bunch of conversations again around
biological agency at different levels of
self-organization lost a guests on that
are really confident that individual
cells display information. And I guess
the one one piece of that that you
dropped there is that, you know, can the
system transform a signal from the
environment to make it meaningful to
itself is one of these, you know,
hallmarkers of agency. I've spoken to
Philip Ball about about this who wrote a
fantastic book called How Life Works.
really looking at agency being the one
of the most foundational pillars to
think about biology. And he makes the
argument, you know, very hard to prove.
Again, we don't have a framework, but it
really seems like what individual cells
are doing is display a level of
problemsolving agency. They explore
their environments. They make meaningful
use of ambiguous information. So, I'm
curious for you to dive into this. And I
asked him this question. You're like,
what? Because complex behavior can look
like agency, right? And it's very hard
to disentangle them. How do you
>> like how do you disentangle what's like
truly an agent and what is doing
something that looks complex like you
know bacteria and like chemotaxis as an
example, right? They come they will
follow chemical gradients that kind of
seems like agency, but then you could
also see how that's not agency. And then
the exact boundaries are always blurry.
Boundaries don't really exist. But like
talk about how we could develop that
framework because I think this has huge
implications for biology.
>> For sure. Yeah, I fully agree with you.
Uh I should specify the work we've
recently done doesn't actually allow you
to prove something as an agent, but it
does allow you to prove what isn't an
agent. And that said, that has that's
helped on its own.
>> That's a lot of progress. That's more
than a lot of guests have been able to
say. So yeah.
>> Yeah, it it at least that's our
proposal. I mean, we we think it's
pretty well supported. has received
quite a bit of support from people who
have seen it so far. But of course, you
know, science science progresses one
step at a time. Uh so the short the
short example of this in between agency
is we basically propose this three-
layered frameworks which we call
information processing orders and
essentially the first order of
information processing is effectively
just a reflex. you you pass some sort of
threshold and then there's an action but
the actual information that's been
transmitted it's it it remains basically
the same that it is. It doesn't cause
any complex change. All you have to do
is reach a threshold and it passes. the
second level that we have the second
order complexity
uh actually has some sort of uh
rudimentary
um transformation. So you might think of
this as like uh you send something in
and then it might uh double or triple or
it may change in some small way but it's
not actually creating any sort of
meaningful qualitative compounded change
over time. Right? So it might have
actually quite complex cycles like a
memor memor memory stuff in
neuromorphics
uh can actually have some fairly complex
dynamics but it doesn't actually
physically change uh or meaningfully
change the way the information is in a
dynamic way. It doesn't have the real
case of learning in any complex
biological sense. This third order
though does add this feature of actually
qualitative memory over time where the
change that occurs can actually then
change the information yet again and so
it can compound over time and basically
observing this compound over time
suggests that the system itself changes
how it treats information that it will
then receive over time. And I think
that's the core of being an agent in the
environment. being able to have not just
a response to information, but for lack
of a better word, agency over how you
respond in that situation.
And so the ability of these systems to
monitor and change their change over
time is really powerful, I think, as a
defining feature. If you can't have
that, we would argue you can't have
agency. And it's kind of the simple
straightforward question uh approach but
it's not one that people had adopted
before and really gotten down to saying
what's the most fundamental need in
terms of how you deal with information
and it's that ability to change how you
change over time.
>> So time frame sounds like the key
dimension there in terms of moving down
the order of agency. the the first order
of information processing is basically
responding very immediately um to a to
something in the environment that
doesn't really persist in as you say a
dynamic way through time. It might
change its state once, but it's not
going to change its state again and
again and again based on some sort of
signal at one time. And then as you go
second order, third order, it's it's
basically sort of about the time frame
that a system can maintain some sort of
dynamic change based on something it
responded to in the environment. Again,
that's even simpler way that even
lacking and more nuance the way you
described it. But is that maybe sort of
the the one-dimensional simple way to
think about it that it's really about
time frame? Yeah, I don't think it's
it's not just about time frame. It's how
the systems and there's many layers of
system, right? Like think about like a
synapse versus cell versus a network of
cells. So there's many systems here,
right? And you might have a case where
you have like the synapse synapse
probably doesn't have agency, right?
Does a cell have agency? I don't know.
We'd have to test it. Probably not, but
we'd have to test. What about the
network? like at some level you might
see something emerge but it's not at all
levels. I'd say the core thing is not
just time. It's actually how that change
to the incoming information occurs over
time.
>> And if it can adapt in a specific way
over time and actually transform that
information in a specific or in any way
really over time that then in itself
changes. That is the most fundamental
basis for agency. And again it doesn't
prove that just because your system does
that suddenly it has agency. There's a
whole lot of other work that says, you
know, you need capacity, intentionality.
People have looked at like
counterfactualities.
There's a lot there that still needs to
come into place. But at the very least,
it can resolve questions like is a
thermostat an agent because it feels
cold.
>> And I think like that at least this is
actually how the how the uh this all
started because a friend of mine would
uh often argue sort of from a panist
perspective. ah everything's
everything's got some agency and I'd say
no it doesn't or how prove it so fine
here's here's here's an answer that I
think addresses it um yeah sometimes
sometimes uh science comes from sort of
uh very deep profound Prometheian
desires to seize fire and other times
from a from a argument with a friend
>> a wage over a few beers yeah I relate as
an as an Irishman I definitely relate to
that But I'm kind of I'm kind of
interested to hear that you would lean
towards a cell not having agency. Of
course,
>> being a good scientist, we can't say one
way or the other, but I'd c like again
depends on definitions, but based on
what you described there, I'm like tick
tick tick. Like a cell I think a cell
absolutely displays that.
>> Yeah, it may. But the nice thing is we
don't have to query it, right? like like
with that little uncertainty, we can
just set out and test it and we'll get
an answer that will say yes or no. Um,
and that's the nice thing about what we
put forward again is that this isn't
just a sort of, you know, castle in the
sky idea. It's one that we can
rigorously test and answer and say, how
does that fit in the framework? Uh I'm s
you know one of my frustrations I'm not
in academia anymore but one of my
frustrations
uh with a lot of the work that's coming
out of the scientific community is uh a
preference to be vague instead of raw
>> right
>> and I think that this is you know and I
was actually amused to find um Freeman
Dyson famous for Dyson spheres and not
the vacuum company I think they took his
name but Freeman Dyson uh who was
obviously a famous scientist proposed
uh maybe you should edit that out. I
don't know if they took his name or if
they've got another dice and I might get
sued for that retracted.
Um but he actually said the same thing
all the way back in his day as well is
that like this the biggest detriment to
science is that there's an unwillingness
to be prepared to put down your ideas
and be wrong. And I don't think there's
anything wrong with that. I I'm wrong
all the time. I say, "Hey, I think the
cells will do this." We test that. Cells
don't do that. Okay, we know we know
more than we did. You know, we've just
had we tried a new cell type recently
introduced into our cultures. It seems
like they've all collapsed. Not great
for us, but we didn't think that would
happen. We've learned something.
>> Yeah,
>> that's powerful. It's excellent to be
wrong. Scientists should be rewarded for
being wrong and writing it down because
the next time you might get it right,
but if you always just refuse uh only
want to be vague, okay, well done.
you're not wrong, but have you really
helped us progress? Unfortunately, the
incentives just aren't there anymore in
the this leads to a whole another
discussion, but the incentives in most
scientific academic environments are
just aren't there to be prepared to take
the big swing and potentially be wrong.
And so, you get more and more iterative
work building upon something that
already exists rather than people trying
to create something that has never been
before. Yes, I was really going to say
that that that's not a dig at the
scientists, but that is a symptom of the
infrastructure and incentives that we
have. If you have a series of two or
three, you know, two or three
>> hypotheses that you were wrong on, that
fourth grant proposal seriously becomes
a lot harder to fund. But in the private
sector, when you have a much larger
mission and more funding, you're able to
take bigger swings. you're not relying
on, you know, the cycle of grant reports
that really are relying on the last one.
Of course, you have to also make
successful do successful things and do
meaningful things and have, you know, do
things that are valuable, but you just,
you know, you get a little bit more or a
lot more leeway to take big swings, be
wrong. Um, yeah,
>> but it should be the other way around. I
mean our public funded science and I
know yeah again pretty much all the
scientists I talked to would love the
opportunity to take those big swings.
These are uh for the most part creative
inspiring individuals who just want to
go out there and try and do something
but they have to do the grant and uh in
Australia grants are successful at the
moment 7% of the time it's got a less
than 10%.
>> Wow.
>> And so people just can't take that big
risk
>> and it's such a pity. I mean we again we
can go really off track with this
because I feel quite passionate about it
and I see so much wasted potential from
amazing people who are just stuck in an
environment that they don't have any
other option but
>> yes
>> we could do so much more across the
world if only we could just fund fund
science a little more and a little more
ambitiously
>> is incredibly yeah I'm incredibly
grateful to be uh have Cortical Labs and
have the funders and investors we have
who literally look at the world and you
know don't just see it the way it is but
ask how could it be and are prepared to
back us in our vision to try and create
something. Uh I'm a little relieved that
we keep finding nice results though.
>> Yes.
>> Because we just don't know a lot of the
time like they are just questioned. We
never knew if the free energy principle
would work. We tried to set up the
experiment in such a way that if we
found evidence against it, it would have
been an interesting story as well. But
um it obviously wouldn't have been as
interesting a story to find scientists
showing this doesn't work.
But every step of the way there's
questions and uncertainty.
>> Yeah, I completely I completely mirror
your passion for this subject and I've
talked to a bunch of frustrated
scientists about exactly these concerns.
But I'm really curious on how your
stance on agency might have shifted over
the last six, seven years working at
Cortical Labs. Did you have sort of an
idea of what agency looked like that has
shifted over that time? Because you are
watching neurons in a dish do these
incredible things, things that I think,
you know, maybe it would have been
impossible to think about 100 years ago
this being this being a reality. Does it
change the way you think about neural
systems? Does it change the way you
think about system again down the
biological hierarchy? Does it has it
shifted your view about this at all?
I
not not not not personally, but I mean,
you know, we were prepared to put our
careers and lives work into the idea
that cells indeed would have these
ability to actually reorganize and show
these amazing properties. So, we I mean,
if we really thought it was impossible,
we probably wouldn't have started. Um,
but I will say like people have had
these ideas for a long time. I mean the
first work I could find about these sort
of brains in like you know described in
rather interesting ways but was 1929
I think um Bernell an author by Bernell
who's a science historian who wrote a
pretty wild book called the world the
flesh and the devil um I'd recommend
anyone who has a spare two hours it's
only 80 pages or something long go read
it um Grenell was a staunch communist uh
as well which sort of reflects a bit on
some of his views of what society should
be. Uh take take that as you will when
reading it. Uh but he he proposes as
well. He sort of came down and said,
"Hey, look, the most important and
interesting things about human beings,
we're pretty bad animals in most
respects except for our brains. Uh our
brains are remarkable." And so the
inevitable conclusion is that our brains
are the things that should go on and
survive and we should propagate those.
Um now we're not quite taking the same
approach as I said but the idea has been
thought on for a long time and there's
been work also going back to
probably 1998 was the first work really
looking at responses of cells in the
dish. Um there might be something
earlier if so I apologize to those
researchers. Um there was some really
foundational work uh that had some
interesting early results out of a lab
from led by Steve Potter over in um uh
the states back in the early 2000s.
There's been other work done throughout
the world since looking at interesting
things. Where we came in was really uh
looking at hard closed loop real time so
that receiver response from the cells
and immediately update their game world
and combining it with these neuro
computational theories and doing it at a
level of scale and rigor that
unfortunately people previously couldn't
do. And we're fortunate with the timing
then because it means that where we are
now in the world or especially where we
were back in 2019 uh there were these
resources available.
So again sometimes
uh you know luck always has a little bit
to do with it and the ability to you
know this is called the giant shoulder
uh so of course standing on the
shoulders of giants right people who
have gone and laid out so much of the
ground work. I think one of the more
interesting things about our early pong
work was the ability for us to do uh
human neurons from IPS-C's induced
preotent stem cells. We never developed
IPS-C's the developer of that rightly
so. So won a Nobel Prize. So it's this
important piece of work, but we're able
to use those cells and we never came up
with the first differentiations of those
stem cells, cortical cells, but we could
use that work. And so we we sort of were
able to stands on the shoulders of these
giants to build something that hadn't
really been built in that way before and
take it to the sort of one or two extra
steps that previous works had not yet
been able to do.
>> Yes. Love it. Love when guests bring up
the giant shoulder as well. Fantastic.
really really appreciate it. How so how
how sophisticated a game can these
systems play do you think? Have you have
you got a chess bot out there? I mean
like I think that I would be I would
feel very inadequate if I lost a game of
chess to 800 neurons in a dish. But but
can it play more sophisticated games
where the mapping like this is really
high level strategy, lots of playing
down different routes and like the game
to model the game, you just need like a
really complex data structure, right?
And like you know l and like AI systems
will destroy any human chess player now
will destroy any go player. I don't know
if there is actually a game that exists
that AI can't beat humans in, but I'm
like how do you think that maps to your
kind of systems? Can you could you get a
really good AI or a really good cortical
chess system?
>> There there are games there are games
but and this is interesting things the
and there's this Morx paradox. Have you
heard of this?
>> I I I've heard it mentioned but please
refresh.
>> So essentially this states that uh
things that computers find easy humans
find hard. Things humans find easy
computers find hard. So, if I was to do
the square root of like 10,25,
you know, 0892 in my head, like I'm I
can't do that in my head. I'm going to
get bored. Um, it'll take me a long time
to do with a pen and paper. Uh, but a
calculator, very simple, can do it
immediately, right? Because it's a
deterministic system. But likewise, no
AI system, no physical AI system can
actually go into an unknown, unmapped
environment and navigate it and make a
cup of tea. If you put a robot in my
house and you say, "Go make a cup of
tea, it's not doing that without
shattering most of my cups, if ever,
right? It requires but you you could do
it, right? You learn other people's
houses. You would guess, you would
figure it out."
>> So, it's actually, you know, something
like a chess. Why is chess such an
interesting thing? Because it's really
hard for humans to do it perfectly. So,
we have competitions. Who's better?
That's a really hard thing. But
computers are quite good at it. uh games
that computers find hard. For example,
like first person shooters with
incomplete information. If you give the
perfect game world, you know, to the
computer, uh it can be pretty good,
right? But you don't get a perfect
information environment when you play a
first person shooter, right? Especially
not against other humans. Uh so the
these are the sorts of things. So we're
not aiming to build a system that
is going to compete with what computers
are good at. We're aiming to build a
system to augment what computers are bad
at. And so it's things with real time
changes, with fuzzy information, with uh
dynamic real time or limited
information, things that biology has to
learn to be very good at. When we
evolved and we saw a tiger or lion in
the grass, we weren't able to sample
10,000 times. Is that a lion? Is a lion
going to eat me? We had to make that
decision, right? Oh, well, Fred got
eaten by the lion last time, so let's
get out of here. Like, that's that's how
we have to evolve to learn. And so,
biology still has these properties. The
question isn't, and I think this is the
beautiful thing about it. The question
is not, is it possible for brain cells
to show intelligence?
When we're looking at machine learning,
the question is, is it possible for
silicon to show general intelligence? We
don't know, right? We do not know if
that's possible yet. Current attempts
despite trillions of investment have not
done it, but we know for biology it is
possible. The question is how do you get
it and that that is a powerful question
I think to begin this work and it's
really what led us to begin this work in
the first place.
>> Yeah, it's really well said and I
appreciate the I appreciate what you
mean but I'm still curious if you think
you could do it. So say Cortical Labs,
they're pivoting tomorrow, right? your
entire job is to now make a chess
cortical system. Um, and Magnus
Carlson's coming in. He's helping you as
much as you want. Like, you have
infinite money. Like, do you think that
the best cortical chess system could
beat the best chess AI system? I think
that's a really interesting question,
even if that's not your mission.
>> Probably not. Probably not. Right.
That's kind of what I was getting at
before cuz the current machine learning
AI systems, whatever you want to call
them, uh can play the game essentially
perfectly,
right?
And so humans can't do it. Uh so
probably the answer is no. Um
we we'd be looking to solve different
problems. Now there are hybrid
approaches that could get quite
interesting. um and we are interested in
those but in that case it's not a purely
biological system. So do I think there's
huge capabilities in what people call
heterogeneous compute? Yes. I think
heterogeneous computer or hybrid
computing is incredibly interesting.
But uh a pure biological system, no
humans can't do it. I I don't think
we're going to be building anything
that's better at these sorts of things
uh in anytime soon on their own. But
could we augment an AI system to give it
something that's even better? Very
possibly. Right.
>> Yes.
>> Um and cortical labs of course looks at
these heterogeneous systems as well um a
little more in our research arm. So we
we do take a lot of different approaches
there and I think the future of
computing is headed that way.
heterogeneous compute, not just GPU and
CPU, but your biological unit, your
quantum unit, your neuromorphic chip,
figuring out how to use the various
tools in our toolkit the best way
possible, right tool for the right job?
Yes. Yeah. I guess I was curious, you
know, do neurons in a dish, do they
exhibit the same sort of scaling
properties as LLMs, where you just keep
getting this better performance the more
you cram in? Like if we did build
trillions upon trillions of neural
neuron systems, you know, how much
better can those systems be? Do do you
think they or do you know do they
exhibit a sort of similar scaling law to
LLM? Is it just different when it's
biological intelligence? Do you have any
thoughts on that?
>> Different is probably the right word. Uh
if you consider like consider bees again
like I like bees. I go back to bees a
lot. Consider bees. Uh bees are pretty
small, 800ish, 200 to 800,000 cells
depending the bee you're talking about.
Probably there's more variability than
that, but it's roughly that range. Uh
yet they navigate their environment
pretty well. They can find things from a
good good long distance for relative
especially relative to the bee. Uh they
can do pretty complicated stuff. They
build. It's remarkable what bees do. But
then consider an elephant, right?
Elephants have much larger braids than
humans. Does the elephant navigate its
environment that much better than the
bee given that it has so many magnitudes
more cells? No. So, the scaling isn't
the same. And if it was, we would see
humans with bigger brains be that much
smarter. That's just uh not the case,
right?
>> Yes.
>> So, so I think like the answer to that
is like it's going to be different. Um,
a lot of it depends on the structure.
We've actually moved to smaller cultures
than what we did back in the early
prototype days. We used to use about
800,000 to a million cells. Now we've
gone back down from about 200,000 as our
standard setup. And we get better
results, more controllable results
because it's better specified.
>> Interesting.
>> So I think there is a story there about
growth, but I don't think it's nearly as
straightforward as bigger is better,
>> right? Like I was very interesting. And
I was curious, you know, do you get a
better Doom player if you build if you
just like linearly as you as you add
more neurons? It's just it's just not
that it's not that simple. It's not how
>> certainly
>> it's certainly not the case. The
complexity and structure matters a lot.
Uh so unfortunately not as not as
straightforward a story as that. If only
only it was the case cuz growing bigger
cells is not the hardest [ __ ] you know,
we throw away most of the cells we ever
make because we don't have a, you know,
have things to put them on. Um, if all
we had to do was grow a 100red billion
cells and suddenly we would have a
device that was so much better, my job
would be very easy. Um, unfortunately
uh unfortunately there's a lot more
complexity uh there, but it's that very
complexity which makes this such a
beautiful and promising approach.
>> Yes. So, how do you think about the
future of these heter heterogeneous
machines? Will there be neurons in my
Mac 7 or or do you know what I mean?
Like what like where will we see this in
the world? How exactly will the
biological systems interact with the AI
that's being built at the moment? Um how
do you think about that in the future?
>> Yeah, look, you know what they say, uh
predictions are hard, especially when
they're about the future. Um the
do I think that we'll have biological
neurons in like sort of consumer laptops
anytime soon? Probably not. I I don't
but cloud use for certain tasks and
applications. Uh a lot especially with
the use of new um you know sort of
frontier models and all that today a lot
of processing you might use your laptop
to do isn't actually done on your
laptop. Um, a lot of the like for my
laptop, most of my stuff is stored in
the cloud. I bring it down into my
laptop when I need it, right? A lot of
the processing for some of the more most
complicated things we do is done off my
laptop. So, might that be a use case for
your computers relatively soon? Yeah. I
mean, if you logged on to the cortical
cloud, you could do it today, albeit,
you know, we're just at the start of the
journey, but you could do at least some
of it uh in a very very basic
rudimentary experimental sense right
now. So I think that's possible. Um but
I think there are other types of devices
other than laptops you know edge devices
for example where some of the properties
of neural systems like the very low
power very high sample efficiency in
other words doesn't need much data to
learn becomes very promising.
>> Interesting. Can you give an example of
that?
>> So I think things like edge edge
robotics are particularly promising. Uh
we've often used robots as an example
because again you have a ground truth
that we are essentially neat robots uh
walking around the world doing all
right. You know we seldom crash into
things sometimes but seldom. Uh and so
we know kind of like yeah controlling a
entity in the real world is a thing
biological cells can do quite well. So
we think robots is a super interesting
example and physical AI is receiving a
lot of attention at the moment. Uh but
the truth is this there's a reason these
results haven't really bridged into the
dynamic worlds just yet on their own.
So those are sort of key that that's
like one key example.
>> Amazing. Brett, you're so cool. This
work is amazing. You're such a good
communicator. Um I've loved this
conversation. I think this has been so
much fun. I think we are able to really
lay a good foundation for how this
works, where it could go, what are the
applications, what's happening at the
moment. I'm personally incredibly
excited to see all of these little
carrots that you've been dangling in
front of me the whole time. All of these
little papers and results that you're
that are going to come out soon. I'm
super excited to to read them, to hear
about them, to to see the headlines, to
see how all the headlines have
sensationalized the work and and created
all of this crazy stuff that you didn't
actually do. I am excited to have you
back on when all this work can be talked
about properly. Brett, thank you so much
for your work, for everything. And uh
yeah, I really enjoyed that. I thought
that was so much fun. It
>> was a pleasure. Pleasure chatting with
you. Thanks for having me on the show.
>> The Giant Shoulder mission is to explore
radical ideas in biology, neuroscience,
and consciousness and elevate those
stories to the highest possible level
while keeping them accessible to
everyone. If this interests you and you
want to support independent science,
then please consider subscribing to the
clips channel. Check out our 26
neuroscience book. Can download it for
free.