Yes. You can watch the wasteful iterations in a locally run AI.
It will take time to “tell” itself things that only a sleepy toddler would need to be told. And it may have to pause to “remind” itself of those things, later, if it does not “forget”.
It is really interesting the way brute-forcing through a space with a giant pile of associations can achieve some things that look like intelligence (to folks who fail the Turing test.)
Edit: Adding various air-quotes to emphasize that these things don’t think, per comment below.
Modern online LLMs also expose the thinking process. A few years ago, they didn’t have that feature at all, so the first answer appeared to be the only answer. As far as I can tell, it really was the only answer. The obvious problem with that is the lack of processing. A human could come up with a few stupid responses, but would quickly filter out the worst ones and only say the one that seemed most reasonable. An old LLM didn’t seem do that at all. It would just spit out the first version and let the user deal with the stupidity.
Modern LLMs also spit out stupid idiot BS, but they have an internal processing step where the answer is refined. In some cases, it takes multiple steps to process the answer. If you want, you can also track the internal “train of thought” to see how the initial wrong answer got kicked out and replaced by something semi-useful.
Regardless, I don’t see that a brute forcing it. It looks more like the internal thinking process a human would go through before opening their mouth.
That’s not how it works at all. You are talking about “thinking” models, which are just models prompting themselves.
LLMs are fancy next word predictors. It’s just returning what mathematically is the next likely word given their training set (really token since it doesn’t actually understand words).
I’m not sure what part of my "explain like I’m five " version you’re taking issue with enough to correct me. You may want to clarify if you feel I’ve misinformed.
To my read, you just restated what I was trying to make more accessible.
Edit: I added some air quotes to help clarify that the terms I used are for illustration only.
Yes. You can watch the wasteful iterations in a locally run AI.
It will take time to “tell” itself things that only a sleepy toddler would need to be told. And it may have to pause to “remind” itself of those things, later, if it does not “forget”.
It is really interesting the way brute-forcing through a space with a giant pile of associations can achieve some things that look like intelligence (to folks who fail the Turing test.)
Edit: Adding various air-quotes to emphasize that these things don’t think, per comment below.
Modern online LLMs also expose the thinking process. A few years ago, they didn’t have that feature at all, so the first answer appeared to be the only answer. As far as I can tell, it really was the only answer. The obvious problem with that is the lack of processing. A human could come up with a few stupid responses, but would quickly filter out the worst ones and only say the one that seemed most reasonable. An old LLM didn’t seem do that at all. It would just spit out the first version and let the user deal with the stupidity.
Modern LLMs also spit out stupid idiot BS, but they have an internal processing step where the answer is refined. In some cases, it takes multiple steps to process the answer. If you want, you can also track the internal “train of thought” to see how the initial wrong answer got kicked out and replaced by something semi-useful.
Regardless, I don’t see that a brute forcing it. It looks more like the internal thinking process a human would go through before opening their mouth.
That’s not how it works at all. You are talking about “thinking” models, which are just models prompting themselves.
LLMs are fancy next word predictors. It’s just returning what mathematically is the next likely word given their training set (really token since it doesn’t actually understand words).
I’m not sure what part of my "explain like I’m five " version you’re taking issue with enough to correct me. You may want to clarify if you feel I’ve misinformed.
To my read, you just restated what I was trying to make more accessible.
Edit: I added some air quotes to help clarify that the terms I used are for illustration only.