I don’t believe there’s any requirement to learn during a particular part of the lifecycle? And if there was, the short term memory of a context window fulfils that
You need to be more specific. If your point is that you literally cannot backprop while doing a forward pass, then sure. But you can certainly do inference then backprop on the outcome…
I fail to see how that is substantially different from a human doing something then reflecting and learning.
You can critique LLMs and transformers generally, but to say it’s somehow a problem with backprop is a bit of a stretch.
My read is that the user is saying that creating a world view contains trial and error. Online learning if you will. With the current setup this learning is very much not like this. While that makes a human to machine analogy I am also not too convinced by this reasoning. Just allow for a larger time lag and then the inference and learning is at the same scale.
Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
How do you know?
Backpropagation, the algorithm behind current machine learning systems, precludes an active world-model; it can’t learn during inference.
I don’t believe there’s any requirement to learn during a particular part of the lifecycle? And if there was, the short term memory of a context window fulfils that
I thought learning was an important part of intelligence.
Huh, didn’t know there was a word for it, thanks.
You need to be more specific. If your point is that you literally cannot backprop while doing a forward pass, then sure. But you can certainly do inference then backprop on the outcome…
I fail to see how that is substantially different from a human doing something then reflecting and learning.
You can critique LLMs and transformers generally, but to say it’s somehow a problem with backprop is a bit of a stretch.
My read is that the user is saying that creating a world view contains trial and error. Online learning if you will. With the current setup this learning is very much not like this. While that makes a human to machine analogy I am also not too convinced by this reasoning. Just allow for a larger time lag and then the inference and learning is at the same scale.
Yes but that’s related to LLMs specifically, not backpropagation. There’s plenty of ML paradigms that use backprop and have continual learning setups.
Backprop is the process of how the weights are updated.
Have you used LLMs? They have about as much personality as a pigeon.
They’re just very neurodivergent.
Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
Wikipedia Chinese room. It’s pretty airtight at the moment.
Chinese room is not at all airtight. We don’t know what it means to have emotions so we can’t prove a Chinese room can’t have emotions.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
This is not what the Chinese room is about.