Thread 16834075

https://www.youtube.com/watch?v=21EYKqUsPfg [Embed]

Indian podcaster Dwarkesh Patel debates the 'father of reinforcement learning' Prof. Richard Sutton on whether LLMs are a dead-end.
Sutton believes so, while Dwarkesh Patel proclaims LLMs are the foundation on which experiential learning can happen, citing recent LLM models' success at high-school math problems. Patel claims humans all learn by imitation too.

The following exchange summarizes the debate:
>Patel: But there are phases of learning where there's the programming in your biology early on, you're not that useful. And then kind of why you exist is to understand the world and learn how to interact with it. It seems like a training phase [in LLMs].
>Sutton: There's NOTHING where you have training of what you should do; there's nothing. You see things that happen, you're not told what to do. Don't be difficult, I mean, this is obvious.
>Patel: I-I mean you're literally taught what to do, this is where the word training comes from, from humans.
>Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing up

What's /sci/'s verdict? Who won?
Is Dwarkesh Patel right in saying humans are literally taught what to do the moment they're born? Is all learning just imitation?

>>16834075 (OP)
>Indian
Stopped reading.

>>16834075 (OP)
>>Patel: I-I mean you're literally taught what to do, this is where the word training comes from, from humans.
this guy is unbelievably retarded. The core component of intelligence is not imitation. It's exactly as Sutton puts it: you're not told what to do, intelligence is about observing the environment and exhibiting novel, goal-oriented behavior based on it.

>>16834075 (OP)
It's both, isn't it? A child with someone helping them is usually going to advance a lot faster than if nobody helped them. And some things, like a child learning English to be able to interact with other humans, can't really be done by just trying things out on their own, especially if they're alone

>>16834101
yeah however the question is what makes an animal intelligent and most would agree that mindlessly imitating things isn't it but for that to not be mindless in the first place you need some kind of intelligence. Not to mention, human brains are about a million times quicker and a billion times less energy-consuming than LLMs, and LLMs can't learn on-the-job, so for Patel to say LLMs are the foundation on which experiential learning can happen is just wrong. Clearly, the architecture of animal brains is much more intricate and effective and should be studied to make any kind of progress towards AGI

I think I take a young optimist Indian over a super grumpy super old CSler

>>16834075 (OP)
>We don't think about training when we think of an infant growing up
Has this man ever raised a child? Obviously not. He either left it to a woman or he's a childless gay.

>>16834075 (OP)
>[Embed] [Embed]
retard

>>16834138
I watched this video a couple of weeks ago, and IIRC, his point isn't that you "train" the kid, it's that the kid is able to learn by himself and, most importantly, to act on what he learned. you don't train the kid, he can learn himself, (sorta kinda) reflect on it and use the learning and knowledge as new tools.

>>16834138
>stay-at-home chud to train the children
Now this is a novel approach

I found this guy through his interview with Richard Rhodes a couple years ago and he seemed to play the "good interviewer" pretty well, which consists in my opinion of:
>get a good guest
>ask reasonably well-researched questions occasionally to keep them talking
>otherwise shut the fuck up
When he (or any interviewer) tries to make it a real-time debate I usually find it ends up just being a mess. That said I haven't seen this specific interview, maybe it's good.

>>16834075 (OP)
i dont think this poo understands that LLMs are just recycled logic machines

it cant invent anything

the government can make it happen through some ways but they wont release it to the public

ideation LLMs are not going to come out lmao

>these models do seem to have very robust world models
>SEEM
jeet could have stopped there.

LLMs have dazzled retards so much they are like a reverse Turing test

>>16834075 (OP)
I'm just glad he was finally explicit in saying all the llm stuff has nothing to do with what he meant in the Bitter Lesson - none of it is le bitter lesson-pilled. 2+ years of hearing that. It's a short essay and not hard to understand at all. Techdrones don't read so good.

>>16834075 (OP)
Stupid debate. Old guy says new thing not as good as old thing. Young guy says new thing good, because new. Why not just get old ham radio enthusiast to debate a Gen Z on the benefits of smart phones. That might actually be better.

>>16834075 (OP)
notice how the white person is the inventor and knows how ML works at the core RL specifically and the poo loo gets excited over mundane bullshit thinking its some revolutionary piece of tech lmao

its a google search api on steroids thats it

too bad my 2019 search engine was controlled

>>16834075 (OP)
>>Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing up
>Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing up


theres an element of innate learning to humans that a markov model or token bin distributed model cant replicate i guess we need RL generative activation functions or some bullshit agi pure math invention for activation functions otherwise we're stuck at this lmao

>>16834814
>Old guy says new thing not as good as old thing.
Old guy is an expert in the field. Young guy is a tard. Hope this helps.

>>16834138
>>16834814
>>16834132
t. p zombie

>>16834083
>The core component of intelligence is not imitation. It's exactly as Sutton puts it: you're not told what to do, intelligence is about observing the environment and exhibiting novel, goal-oriented behavior based on it.
You're correct, obviously. But Indians don't see that, the average Indian really does just imitate what everyone around them does. That 'jeet thinks this is intelligence because that's exactly how the 'jeet mond works.

>>16834075 (OP)
Dwarkesh Patel is subhuman, he cannot comprehend the idea of figuring things out for himself

>>16834814
you're never getting anywhere with that ignorant attitude.

It's amazing someone as stupid as Dwarkesh is surrounded by intellectuals such as Paine and Sutton.

>>16834075 (OP)
>Is all learning just imitation?
A lot of it instinctual and hard-coded into our DNA most likely. You never learn breathe by imitation, for example.

>>16834075 (OP)
>>16835294
https://www.youtube.com/watch?v=u3HBJVjpXuw

>>16835591
not interested in hearing any more of this retard's musings. There also was no "misunderstanding" here, Dwarkesh's position just isn't based in reality and fell apart on the slightest pushback by Sutton

>named dwarf pajeet
>iq as high as a midget
Who wrote this shit?

learning isn't just imitation
learning is when you change your reaction to future stimuli based on past stimuli that's it

for you to imitate irl you have to have some kind of planning and abstract thinking and whatever its obvious animals don't do that and neither do babies

some stuff is built in through genetics and everything else is either learned reaction to stimuli or some shit to do with planning and abstract thinking and whatever which requires a model of the world learned through aforementioned stimuli anyways

>>16834075 (OP)
i swear these tech people like patel have got to be llms themselves

>>16834105
Mindless imitation is how we create a mind, then superstructure in the mind is formed after lot of mindless imitation. A baby's mind isn't doesnt have full consciousness, they're similar to animal as they have not evolved an inner mind.

>>16834075 (OP)
For once I'm with the jeet on this one. Learning is training. We have a reward system in our brain that is analogous to ML scoring functions.

>>16834786
The government is retarded and can only do a fraction of what the private sector can. It can absolutely invent things just like you can. Inventing things is just combining things that already exist. It has the same primitives that you have, and can combine them in new ways if given appropriate stimuli, just like you or any other human.

>>16834956
That’s how hindpoofags are they lack actual intelligence like us aryans

>>16836608
kek nice bait

>>16834075 (OP)
>The child tries things and sees what happens. We don't think about training when we think of an infant growing up
what. kids are basically forced to guzzle down trivia at gunpoint(or the parents go to jail)

>>16834075 (OP)
Sutton is right LLM's are a dead end however this is old news because literally all competent AI companies haven't been using pure LLM's for a while now.

I think Deepseek started off the branching away from the mentality that larger LLM=smarter ai by giving their R1 model the ability to use some of its tokens to think rather than having to regurgitate the answer right away kind of like giving someone a paper to do long division on rather than telling them to blurt out the answer. LLM's are old news now basically every AI use a bunch of different AI agents some that have access to a wolfram alpha like calculator, others that can edit images to better read text or make out objects or more effectively look up stuff on the web. These agents put together make up what we call AI in modern times. A lot of them are still AI i think people have forgotten that AI=/=LLM and could just be image recognition software.

>>16836095
>some stuff is built in through genetics and everything else is either learned reaction to stimuli or some shit to do with planning and abstract thinking and whatever which requires a model of the world learned through aforementioned stimuli anyways
they're called instincts

>Patel claims humans all learn by imitation too.

We do. We’re parrots. However we don’t mimic each other well enough, and this is resulting in competency crises.

Because for some reason nobody wants to trust each other anymore and zoomers would rather trust TikTok (the absolute WORST thing to parrot) than their daddy.

>>16834075 (OP)
I can suggest an equation

>>16834075 (OP)
but that's intelligence, not learning

>>16838633
>>16834083

poo poo patel
can never be right
>>16838625
kek

>science has a problem
>philosophy has to solve it
it's always funny when this happens

>>16834814
You’re right this guy should pivot to mkbhd videos lmao
>>16835319
Stfu nigga attitude is irrelevant to innate mind innovative ideas

>>16834828
The guy on the right is just an interviewer that hosts talks with experts in their fields. Look up his videos with Sarah Paine. He just looks into whatever field the expert is in and prepares some questions into their work.

e=mc2+ai