Name: Joe Rogan Experience #2156 - Jeremie & Edouard Harris
Uploaded: 2024-05-25T17:00:31.000Z
Description: Watch the stand-up comedy video 'Joe Rogan Experience #2156 - Jeremie & Edouard Harris' from PowerfulJRE on Comedy Buff. Published 2024-05-25. Transcript available.

Watch the stand-up comedy video 'Joe Rogan Experience #2156 - Jeremie & Edouard Harris' from PowerfulJRE on Comedy Buff. Published 2024-05-25. Transcript available.

Joe Rogan Experience #2156 - Jeremie & Edouard Harris

Joe Rogan podcast, check it out.
The Joe Rogan experience.
Train by day, Joe Rogan podcast by night, all day.
What's happening?
Oh, you know, not too much.
Just another typical week in AI.
Just the beginning of the end of time.
That's all happening right now.
Just for the sake of the listeners, please just give us your names and tell us
what you do.
So I'm Jeremy Harris.
I'm the CEO and co-founder of this company, Gladstone AI, that we co-founded.
So we're essentially a national security and AI company.
We can get into the backstory a little bit later, but that's the high level.
Yeah, and I'm Ed Harris.
I'm actually, I'm his co-founder and brother and the CTO of the company.
Keep this, like, pull this up like a fist from your face.
There you go.
Perfect.
So how long have you guys been involved in the whole AI space?
For a while in different ways.
So we actually, we started off as physicists.
Like, that was our background.
And in, like, around 2017, we started to go into AI startups.
So we founded a startup, took it through Y Combinator, this, like, Silicon
Valley, you
know, accelerator program.
At the time, actually, Sam Altman, who's now the CEO of OpenAI, was the
president of Y Combinator.
So he, like, opened up our batch at YC with this big speech.
And we got some, you know, some conversations in with him over the course of
the batch.
Then in 2020, so this thing happened that we could talk about.
Essentially, this was like the moment that there's like a before and after in
the world
of AI, before and after 2020.
And it launched this revolution that brought us to ChatGPT.
Essentially, there was an insight that OpenAI had and doubled down on that you
can draw a
straight line to ChatGPT, GPT-4, Google Gemini, everything that makes AI
everything it is today
started then.
And when it happened, we kind of went, well, Ed gave me a call, this, like, panicked
phone
call.
He's like, dude, I don't think we can keep working, like, business as usual in
a regular
company anymore.
Yeah.
Yeah.
So there was this AI model called GPT-3.
So, like, everyone has, you know, maybe played with GPT-4.
That's like ChatGPT.
GPT-3 was the generation before that.
And it was the first time that you had an AI model that could get, that could
actually,
let's say, do stuff like write news articles that the average person, like in a
paragraph
of a news article, could not tell the difference between it wrote this news
article and a real
person wrote this news article.
So that was an inflection.
And that was, you know, significant in itself.
But what was most significant was that it represented a point along this line,
this, like, scaling
trend for AI, where the signs were that you didn't have to be clever.
You didn't have to come up with necessarily a revolutionary new algorithm or be
smart about
it.
You just had to take what works and make it way, way, way bigger.
And the significance of that is you increase the amount of computing cycles you
put against
something.
You increase the amount of data.
All of that is an engineering problem, and you can solve it with money.
So you've got, you can scale up the system, use it to make money and put that
money right
back into scaling up the system some more money in IQ points come out.
Jeez.
That was kind of the 2020 moment.
Like that's, and that's what we said in 2020.
Exactly.
I spent about two hours trying to argue him out of it.
I was like, no, no, no, like we can keep working at our company because we're
having fun.
Like we like founding companies.
And yeah, he just like wrestled me to the ground.
And we're like, shit, we got to do something about this.
We, you know, we reached out to like a family friend who, you know, he was non-technical,
but he had some some connections in government in DOD.
And we're like, dude, the way this is set up right now, you can really start
drawing straight
lines and extrapolating and saying, you know what, the government is going to
give a shit
about this in not very long, two years, four years, we're not sure.
But the, the knowledge about what's going on here is so siloed in the frontier
labs.
Like our friends are, you know, all over the frontier labs, the open AIs, the
Google Deep
Minds, all that stuff.
The shit they were saying to us that was like mundane reality, like water
cooler conversation.
When you then went to talk to people in policy and even like pretty senior
people in government
not tracking the story remotely.
In fact, you're hearing almost the diametric opposite.
This is like overlearning the lessons of the AI winters that came before when
it's pretty
clear, like we're on a very, at least interesting trajectory, let's say that
should, should
change the way we're thinking about the technology.
What, what was your fear?
Like what, what was it that hit you that made you go, we have to stop doing
this?
So it's basically any, you know, anyone can draw a straight line, right?
On a graph.
The key is looking ahead and actually at that point, three years out, three
years out, four
years out and asking, like you're asking, what does this mean for the world?
What does it mean?
What does the world have to look like if we're at this point?
And we're already seeing the first kind of wave of risk sets just begin to
materialize.
And that's kind of the weaponization risk sets.
So you think about stuff like large scale psychological manipulation of social
media, actually really
easy to do now.
You train a model on just a whole bunch of tweets.
You can actually direct it to push a narrative like, you know, maybe China
should own Taiwan
or, you know, whatever, something like that.
And you actually, you can, you can train it to adjust the discourse and have
increasing
levels of effectiveness to that.
Just as you increase the general capability surface of these systems, we don't
know how
to predict what exactly comes out of them at each level of scale, but it's just
general increasing
power.
And then the, the, the kind of next beat of risk after that.
So we're scaling these systems.
We're on track to scale systems that are at human level, like generally as
smart, however
you define that as a person or greater.
And OpenAI and the other labs are saying, yeah, it might be two years away,
three years away,
four years away, like insanely close at the same time, and we can go into the
details of
this, but we actually don't understand how to reliably control these systems.
We don't understand how to get these systems to do what it is we want.
We can kind of like poke them and prod them and get them to kind of adjust, but
you've seen,
and we can go over these examples.
We've seen example after example of, you know, Bing Sydney yelling at users,
Google showing
a 17th century British scientists that are racially diverse, all that kind of
stuff.
We, we don't really understand how to like aim it or align it or steer it.
And so then you can ask yourself, well, we're on track to get here.
We are not on track to control these systems effectively.
How bad is that?
And the risk is if you have a system that is significantly smarter than humans
or human
organization, that we basically get disempowered in various ways relative to
that system.
And we can go into some details on that too.
Now, when, when a system does something like what Gemini did, like what says,
show us Nazi
soldiers and it shows you Asian women and like, what is for, what's the
mechanism?
Like how does that happen?
So it's maybe worth, yeah, taking a step back and looking at like how these
systems actually
work, you know, because that's going to give us a bit of a frame too for
figuring out when
we see weird shit happen, how weird is that shit?
Is that shit just explainable by just the basic mechanics of, you know, what
you would expect
to happen based on the way we're training these things or is there's something
new and fundamentally
different happening?
So, um, we're talking about this idea of scaling these AI systems, right?
What does that actually mean?
Well, you imagine the AI model, which is kind of like you think of it as like
the artificial
brain here that actually does the thinking.
That model contains, it's kind of like a human brain, it's got these things
called neurons.
We, in the human brain, call them biological neurons in the context of AI, it's
artificial
neurons, but it doesn't really matter.
They're the cells that do the thinking for the machine.
And the realization of AI scaling is that you can basically take this model,
increase the
number of artificial neurons it contains, um, and at the same time, increase
the amount
of computing power that you're putting into kind of like wiring the connections
between
those neurons.
That's the training process.
Pause here right there.
Yeah.
How does it, how does the neuron think?
Yeah.
So, okay.
So, so let's, let's get a little bit more concrete then.
So in your brain, right, we have these neurons, they're all connected to each
other with different
connections.
And when you go out into the world and you learn new skill, what really happens
is you
try out that skill, you succeed or fail.
And based on your succeeding or failing, the connections between neurons that
are associated
with doing that task well, get stronger.
The connections that are associated with doing it badly, get weaker.
And over time through this like glorified process really of trial and error,
eventually
you're going to hone in and really in a very real sense, everything you know
about the
world gets implicitly encoded in the strengths of the connections between all
those neurons.
If I can X-ray your brain and get all the connection strengths of all the
neurons, I have everything
Joe Rogan has learned about the world.
That's like basically the, a good sketch, let's say, of what's going on here.
So now we apply that to AI, right?
That's, that's the next step.
And here really it's the same story.
We have these massive systems, artificial neurons connected to each other.
The strength of those connections is secretly what encodes all the knowledge.
So if I can, if I can steal all of those connections, those weights, as they're
sometimes
called, I've stolen the model, I've stolen the artificial brain, I can use it
to do whatever
the model could do initially.
That is kind of the artifact of central interest here.
And so if you can, so if you can build the system right now, you got so many
moving parts.
Like if you look at GPT-4, it has people think around a trillion of these
connections, and
that's a trillion little pieces that all have to be jiggered together to work
together coherently.
And you need computers to go through and like tweak those numbers.
So massive amounts of computing power.
The bigger you make that model, the more computing power you're going to need
to kind of tune it
in.
And now you have this relationship between the size of your model, the amount
of computing
power you're going to use to train it.
And if you can increase those things at the same time, what Ed was saying is
your IQ points
basically drop out.
Very roughly speaking, that was what people realized in 2020.
And the effect that had, right, was now all of a sudden the entire AI industry
is looking
at this equation.
Everybody knows the secret sauce.
I make it bigger.
I make more IQ points.
I can get more money.
So Google's looking at this.
Microsoft, OpenAI, Amazon, everybody's looking at the same equation.
You have the makings for a crazy race.
Like right now today, sorry, Microsoft is engaged in the single biggest
infrastructure
in human history.
Build out.
The biggest infrastructure build out.
Build out.
$50 billion a year, right?
So on the scale of the Apollo moon landings, just in building out data centers
to house
the compute infrastructure, because they are betting that these systems are
going to get
them to something like human level AI pretty damn soon.
So I was reading some story about, I think it was Google, that's saying that
they're going
to have multiple nuclear reacto