You need to be more specific. If your point is that you literally cannot backprop while doing a forward pass, then sure. But you can certainly do inference then backprop on the outcome…
I fail to see how that is substantially different from a human doing something then reflecting and learning.
You can critique LLMs and transformers generally, but to say it’s somehow a problem with backprop is a bit of a stretch.
My read is that the user is saying that creating a world view contains trial and error. Online learning if you will. With the current setup this learning is very much not like this. While that makes a human to machine analogy I am also not too convinced by this reasoning. Just allow for a larger time lag and then the inference and learning is at the same scale.
Well to be precise the pretraining is prevented by this but the online post training does have this issue. The reinforcement learning trials show some sort of automation but I am not aware of any unsupervised methods that have been as successful as the latest batches of llms — haven’t been in this field for a while so honestly dunno.
I don’t believe there’s any requirement to learn during a particular part of the lifecycle? And if there was, the short term memory of a context window fulfils that
Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
Not really. Chinese room has the conclusion pre-fabricated and tries to construct any sort of pseudo-scientific, fantastical philosophical dressing for some shred of legitimacy. It has absolutely no relation to the real world.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
We’ve yet to be able to replicate it from the old parts of meat we know are involved in it with the addition of electricity in the ways it should need in even the much simpler flesh of insects, amphibians, and reptiles. Best we can do is make a muscle twitch without causing observable thought/emergent nureaul processes.
So even if we can, we’re so far from it that thinking we’re on the cusp seems delusional to me.
Even quantum computing is just a shortcut for a subset of hideous obscure proofs. They aren’t doing different math. Special pleading is a fallacy… except when I do it, that’s different.
A quantum computer is definitely not a plain old Turing machine. The only memory in a Turing machine is the tape; the tape head itself is just a list of deterministic instructions, and that tape head itself is stateless.
A plain old Turing machine can’t even handle randomness. To extend it to randomness, you need a probabilistic Turing machine, where the instructions take the form of stochastic matrices. A stochastic matrix is just a statistical truth table.
But a quantum computer cannot even be a probabilistic Turing machine, because of the way operators compose.
In both a plain old Turing machine and a probabilistic Turing machine, operators that occur at different moments of time are independent of one another. The NOT operator flips a bit. If I apply the NOT operator once, and then later apply it again, the outcome is the same: it flips the bit. Probabilistic computers can have stochastic operators, like let’s call one COIN which outputs a 50%/50% distribution for the bit for both inputs 0 and 1, and so whatever comes in, randomness comes out. If we apply the COIN operator, and later apply it again: the effect is the same.
Quantum computers don’t work like that, because operators compose differently. The quantum mechanical equivalent of the COIN operator is called the Hadamard, sometimes represented just with the letter H. If we apply H one time, it has the effect of flipping the bit at complete random, just like COIN. However, if we apply H twice in a row, the second time has the effect of deterministically moving the bit back to whatever value it had before the first H.
That is only possible if the operators, that occur at different moments in time, are not independent of one another. The second H behaves differently from the first because it comes after the first. Where it is situated in the circuit, in relation to everything before it, plays a role in determining its behavior.
There are only two explanations for this, and both violate the principles of both a plain old Turing machine and a probabilistic Turing machine.
The simplest explanation, although the most controversial (because people prefer more extraordinary explanations), is just that the operators do indeed just change their behavior based upon what comes before them in the circuit. That means that the only thing that fundamentally sets quantum information apart from classical information is a kind of past-dependence in the behavior of the operators. That’s not something either kind of Turing machine mentioned thus far does. Operators have no awareness of the history of previous operators.
The more extraordinary explanation is to assume that there is no past-dependence, because there is a memory. If you remember the past, then the memory is in the present, and so the dependence becomes a present-dependence. However, this memory must be exponentially large for it to work: 2^n complex numbers for n qubits. If you have 300 qubits, that is 2^300 complex numbers, which is a memory greater than the number of atoms in the observable universe. The memory also cannot exist anywhere because it is too large to meaningfully assign it coordinates in physical space. This is how most people conceive of quantum computers, where the memory is held in an abstract invisible ψ that exists nowhere, but this also is not a Turing machine because the tape head itself is memoryless.
The distinction between #1 and #2 is between a non-Markovian model and a hidden Markov model. The Markov assumption is the idea that the system’s evolution only depends upon the present state of the system. Dropping that assumption makes it non-Markovian, so there is a past-dependence. You can get rid of this past dependence if you assume the system keeps a memory, but that memory is invisible, and thus these are called hidden Markov models. Most people interpret quantum systems in general in terms of a hidden Markov model with an utterly enormous invisible memory represented by ψ. (Indeed, if ψ represents something continuous, like the position of a particle in physical space, then the memory literally becomes infinitely large!)
Either way, a Turing machine is neither a hidden Markov model (the tape head itself holds no memory and is stateless; only the tape holds data) nor is it non-Markovian. A probabilistic Turing machine is explicitly Markovian.
Of course, you could define a new kind of Turing machine, and they exist: a quantum Turing machine. But that is definitely not a plain old Turing machine.
(That being said, I don’t am not agreeing with the other person, as I don’t even believe “consciousness” is a meaningful concept. I am merely saying that quantum computing is definitely not a plain old Turing machine.)
Quantum computing features wave functions, so it’s more similar to human brains than binary is. I have this headcanon SciFi thing like what if the first full scale quantum computer we make accidentally conscious? That would be cool. Though, seeing how badly we fucked up much simpler LLMs implementation maybe not so cool…
Emotions or not, it’s not good to have your product known for letting people torture something. Torturing /things/ alive or not is not something one should be doing unless there is specific and extreme cause.
Things that aren’t alive can’t be tortured. You can’t torture a rock. Getting the text predictor to spit out the script for Saw XVII starring ChatGPT isn’t torture, even if it’s weird.
Brother you can torture a fucking doll (Voodoo, deranged kids, etc). The torture is not in consideration of how feeling-less thing feels, it’s in consideration of the intent and action of the conscious thing doing the torture. Not sure why you think that’s not the case.
‘Pretend you’re being tortured’ as an instruction to the text generator is way less concerning in isolation than, say, decapitating and hanging a Barbie doll. And even that isn’t wildly concerning in itself. Do you freak out when kids smash their toys together to make them fight?
I’d hope we can both agree there’s a line when it becomes actual torture, and that is in the intent that went along with the action. Torture is an extreme action. I do not equate most of your references to torture, though I’d say the doll hanging could need a conversation with the child to discuss the intent behind their actions given more context.
“”“Torturing”“” a text predictor is NOT the same as torturing a human, you will not feel the same thing while “”“torturing”“” ChatGPT and a human(or a animal), your reaction would be vastly different
Eh, the more I respond to people about this the more I realize torture is a messy word draped in metaphor and all sorts of layered usage. No, I don’t believe the rock has feelings or is alive, but you can indeed torture something that isn’t alive by being mentally unhealthy enough. So if it’s actual torture or not from a psychological perspective is going to center around if the person’s intent is to cause realized suffering imo, and the line is really easy to cross in a simulacrum as seemingly realistic as advanced modern generative models.
Yet you’re just arguing the metaphor is the definition provided the definition is the intent, are you not? Someone who intends to torture a rock, regardless of that intent, isn’t going to succeed in their task. Their own psychology might be such that they are genuinely trying to torture it but, from the rock’s perspective, the rock has no perspective. It’s a fucking rock. It can’t experience torture. It can’t experience anything. Ergo the torturer cannot torture it.
We can say the person who intends to literally torture something which is unexperiencing, achieving only metaphorical torture, is mentally unwell but to call the act literal torture because it was their intent is just sharing in their delusion.
I don’t believe torture is measured at the receiving end. I believe it’s in the intent and action of the conscious thing doing it. If it’s an expression or not would then be heavily subjective on the specifics involved.
You shouldn’t torture anything, generally. It’s a not psychologically healthy urge to have. The specific reasoning will depend on the specific person. As for if you should do it to a hunk of metal, a rock, or your crusty cum sock is irrelevant.
I don’t believe killing npcs or perhaps even more messily I could see circumstances where even /torturing npcs/ in a game is not actual torturing. I think it centers on the intent of the action, and if that intent is to cause actual realized suffering. It is very difficult to define and will be heavily subjective to the person, but I also think most people get a “sense” of where that line is naturally in person-to-person interactions.
If you define it by intent, then the same action performed by different people may be and not be torture. This makes it pretty hard to define, and it becomes completely subjective after that.
For instance, I don’t think it is possible to torture an inanimate object, thus I never have an intent of causing suffer to one. Sometimes I want to break an object, and by your definition it will not be torture however I do that
in what way can an AI be tortured? start with defining torture, its relationship to living beings, and then try to translate that to a piece of software. software, fyi, has no a priori, or even posteriori, emotion; for it cannot have emotion.
Same reason flapping your arms really fast will never make you start flying. You don’t have the equipment. You can’t fly because that’s not what arms do.
I’ll accept that argument once someone can describe or define consciousness in some comparable way. Until then, you can’t say some person or thing doesn’t have it
Transistors switching really fast = emotions lul
Neurons firing really synchronized = emotions lul
Alright, look, I don’t think AI is sentient. I don’t think it’s capable of feeling emotions, LLMs are just prediction engines on steroids, etc…
But if you want to get reductive about it, Mr waveforms in a clump of cells,…
I think we don’t understand intelligence and we absolutely could create a sentient machine. LLM’s ain’t it though
How do you know?
Backpropagation, the algorithm behind current machine learning systems, precludes an active world-model; it can’t learn during inference.
You need to be more specific. If your point is that you literally cannot backprop while doing a forward pass, then sure. But you can certainly do inference then backprop on the outcome…
I fail to see how that is substantially different from a human doing something then reflecting and learning.
You can critique LLMs and transformers generally, but to say it’s somehow a problem with backprop is a bit of a stretch.
My read is that the user is saying that creating a world view contains trial and error. Online learning if you will. With the current setup this learning is very much not like this. While that makes a human to machine analogy I am also not too convinced by this reasoning. Just allow for a larger time lag and then the inference and learning is at the same scale.
Yes but that’s related to LLMs specifically, not backpropagation. There’s plenty of ML paradigms that use backprop and have continual learning setups.
Backprop is the process of how the weights are updated.
Well to be precise the pretraining is prevented by this but the online post training does have this issue. The reinforcement learning trials show some sort of automation but I am not aware of any unsupervised methods that have been as successful as the latest batches of llms — haven’t been in this field for a while so honestly dunno.
I don’t believe there’s any requirement to learn during a particular part of the lifecycle? And if there was, the short term memory of a context window fulfils that
I thought learning was an important part of intelligence.
Huh, didn’t know there was a word for it, thanks.
Have you used LLMs? They have about as much personality as a pigeon.
They’re just very neurodivergent.
Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
Wikipedia Chinese room. It’s pretty airtight at the moment.
Not really. Chinese room has the conclusion pre-fabricated and tries to construct any sort of pseudo-scientific, fantastical philosophical dressing for some shred of legitimacy. It has absolutely no relation to the real world.
Chinese room is not at all airtight. We don’t know what it means to have emotions so we can’t prove a Chinese room can’t have emotions.
This is not what the Chinese room is about.
We don’t? That’s news to me. Sorry for your loss, or happy for you, or whatever
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
Please don’t write off the possibility of artificial consciousness, just because this isn’t it. Thought is a process which meat can perform.
The meat does the thinking.
Stopped what I was doing to watch this again.
I mean, sure consciousness is (probably) a physical process, so like any other process we should be able to replicate it with some sort of machine.
…but I really really really doubt that machine is going to be a plain old Turing machine.
We’ve yet to be able to replicate it from the old parts of meat we know are involved in it with the addition of electricity in the ways it should need in even the much simpler flesh of insects, amphibians, and reptiles. Best we can do is make a muscle twitch without causing observable thought/emergent nureaul processes.
So even if we can, we’re so far from it that thinking we’re on the cusp seems delusional to me.
Every machine is a plain old Turing machine.
Even quantum computing is just a shortcut for a subset of hideous obscure proofs. They aren’t doing different math. Special pleading is a fallacy… except when I do it, that’s different.
A quantum computer is definitely not a plain old Turing machine. The only memory in a Turing machine is the tape; the tape head itself is just a list of deterministic instructions, and that tape head itself is stateless.
A plain old Turing machine can’t even handle randomness. To extend it to randomness, you need a probabilistic Turing machine, where the instructions take the form of stochastic matrices. A stochastic matrix is just a statistical truth table.
But a quantum computer cannot even be a probabilistic Turing machine, because of the way operators compose.
In both a plain old Turing machine and a probabilistic Turing machine, operators that occur at different moments of time are independent of one another. The NOT operator flips a bit. If I apply the NOT operator once, and then later apply it again, the outcome is the same: it flips the bit. Probabilistic computers can have stochastic operators, like let’s call one COIN which outputs a 50%/50% distribution for the bit for both inputs 0 and 1, and so whatever comes in, randomness comes out. If we apply the COIN operator, and later apply it again: the effect is the same.
Quantum computers don’t work like that, because operators compose differently. The quantum mechanical equivalent of the COIN operator is called the Hadamard, sometimes represented just with the letter H. If we apply H one time, it has the effect of flipping the bit at complete random, just like COIN. However, if we apply H twice in a row, the second time has the effect of deterministically moving the bit back to whatever value it had before the first H.
That is only possible if the operators, that occur at different moments in time, are not independent of one another. The second H behaves differently from the first because it comes after the first. Where it is situated in the circuit, in relation to everything before it, plays a role in determining its behavior.
There are only two explanations for this, and both violate the principles of both a plain old Turing machine and a probabilistic Turing machine.
The simplest explanation, although the most controversial (because people prefer more extraordinary explanations), is just that the operators do indeed just change their behavior based upon what comes before them in the circuit. That means that the only thing that fundamentally sets quantum information apart from classical information is a kind of past-dependence in the behavior of the operators. That’s not something either kind of Turing machine mentioned thus far does. Operators have no awareness of the history of previous operators.
The more extraordinary explanation is to assume that there is no past-dependence, because there is a memory. If you remember the past, then the memory is in the present, and so the dependence becomes a present-dependence. However, this memory must be exponentially large for it to work: 2^n complex numbers for n qubits. If you have 300 qubits, that is 2^300 complex numbers, which is a memory greater than the number of atoms in the observable universe. The memory also cannot exist anywhere because it is too large to meaningfully assign it coordinates in physical space. This is how most people conceive of quantum computers, where the memory is held in an abstract invisible ψ that exists nowhere, but this also is not a Turing machine because the tape head itself is memoryless.
The distinction between #1 and #2 is between a non-Markovian model and a hidden Markov model. The Markov assumption is the idea that the system’s evolution only depends upon the present state of the system. Dropping that assumption makes it non-Markovian, so there is a past-dependence. You can get rid of this past dependence if you assume the system keeps a memory, but that memory is invisible, and thus these are called hidden Markov models. Most people interpret quantum systems in general in terms of a hidden Markov model with an utterly enormous invisible memory represented by ψ. (Indeed, if ψ represents something continuous, like the position of a particle in physical space, then the memory literally becomes infinitely large!)
Either way, a Turing machine is neither a hidden Markov model (the tape head itself holds no memory and is stateless; only the tape holds data) nor is it non-Markovian. A probabilistic Turing machine is explicitly Markovian.
Of course, you could define a new kind of Turing machine, and they exist: a quantum Turing machine. But that is definitely not a plain old Turing machine.
(That being said, I don’t am not agreeing with the other person, as I don’t even believe “consciousness” is a meaningful concept. I am merely saying that quantum computing is definitely not a plain old Turing machine.)
I don’t quite think my lawnmower is boolean complete.
And while quantum computing may be expressible algorithmically (dunno, your claim, not mine), that doesn’t mean all of quantum physics is.
Quantum computing features wave functions, so it’s more similar to human brains than binary is. I have this headcanon SciFi thing like what if the first full scale quantum computer we make accidentally conscious? That would be cool. Though, seeing how badly we fucked up much simpler LLMs implementation maybe not so cool…
Then why continue if it’s possible? Unless you are ok with slavery.
Can you not imagine the difference between creating new life and enslaving it?
In a capitalist system? No, I can’t imagine that…
What is it, then? And while you’re there, what is non artificial consciousness?
I’ve thought more about the dangers of artificial consciousness than you would expect.
That says less than you want it to
Emotions or not, it’s not good to have your product known for letting people torture something. Torturing /things/ alive or not is not something one should be doing unless there is specific and extreme cause.
Wait til this guy learns about The Sims
Or worse, Rimworld
Things that aren’t alive can’t be tortured. You can’t torture a rock. Getting the text predictor to spit out the script for Saw XVII starring ChatGPT isn’t torture, even if it’s weird.
Maybe, but don’t underestimate my ability to feel compassion for all kind of bullshit! That poor little rock 😭
A rock is almost like a tortoise in disguise
A beautiful one? Lady rock even?
Brother you can torture a fucking doll (Voodoo, deranged kids, etc). The torture is not in consideration of how feeling-less thing feels, it’s in consideration of the intent and action of the conscious thing doing the torture. Not sure why you think that’s not the case.
‘Pretend you’re being tortured’ as an instruction to the text generator is way less concerning in isolation than, say, decapitating and hanging a Barbie doll. And even that isn’t wildly concerning in itself. Do you freak out when kids smash their toys together to make them fight?
I’d hope we can both agree there’s a line when it becomes actual torture, and that is in the intent that went along with the action. Torture is an extreme action. I do not equate most of your references to torture, though I’d say the doll hanging could need a conversation with the child to discuss the intent behind their actions given more context.
“”“Torturing”“” a text predictor is NOT the same as torturing a human, you will not feel the same thing while “”“torturing”“” ChatGPT and a human(or a animal), your reaction would be vastly different
It inherently isn’t torture though. I prefer punishing people who actually commit torture.
Eh, the more I respond to people about this the more I realize torture is a messy word draped in metaphor and all sorts of layered usage. No, I don’t believe the rock has feelings or is alive, but you can indeed torture something that isn’t alive by being mentally unhealthy enough. So if it’s actual torture or not from a psychological perspective is going to center around if the person’s intent is to cause realized suffering imo, and the line is really easy to cross in a simulacrum as seemingly realistic as advanced modern generative models.
Yet you’re just arguing the metaphor is the definition provided the definition is the intent, are you not? Someone who intends to torture a rock, regardless of that intent, isn’t going to succeed in their task. Their own psychology might be such that they are genuinely trying to torture it but, from the rock’s perspective, the rock has no perspective. It’s a fucking rock. It can’t experience torture. It can’t experience anything. Ergo the torturer cannot torture it.
We can say the person who intends to literally torture something which is unexperiencing, achieving only metaphorical torture, is mentally unwell but to call the act literal torture because it was their intent is just sharing in their delusion.
While I agree, it’s not a living thing. So it’s just an expression.
I don’t believe torture is measured at the receiving end. I believe it’s in the intent and action of the conscious thing doing it. If it’s an expression or not would then be heavily subjective on the specifics involved.
I could call that tortured reasoning, just to make a point. Me making my point to be clear.
Well then you’d just be getting metaphorical
Why shouldn’t you torture an LLM? They sometimes work better that way.
You shouldn’t torture anything, generally. It’s a not psychologically healthy urge to have. The specific reasoning will depend on the specific person. As for if you should do it to a hunk of metal, a rock, or your crusty cum sock is irrelevant.
I run torture tests on my stuff after each overclock tweak, and its both good for my peace of mind and my systems general health
That’s a great metaphor, and I too sleep better at night with properly tested hardware. 🙏
Is killing NPCs in games on the same level like this for you, or a different thing altogether?
Not judging or against it, I kind of half agree but also find very difficult to define where I’d draw a line.
I don’t believe killing npcs or perhaps even more messily I could see circumstances where even /torturing npcs/ in a game is not actual torturing. I think it centers on the intent of the action, and if that intent is to cause actual realized suffering. It is very difficult to define and will be heavily subjective to the person, but I also think most people get a “sense” of where that line is naturally in person-to-person interactions.
If you define it by intent, then the same action performed by different people may be and not be torture. This makes it pretty hard to define, and it becomes completely subjective after that.
For instance, I don’t think it is possible to torture an inanimate object, thus I never have an intent of causing suffer to one. Sometimes I want to break an object, and by your definition it will not be torture however I do that
in what way can an AI be tortured? start with defining torture, its relationship to living beings, and then try to translate that to a piece of software. software, fyi, has no a priori, or even posteriori, emotion; for it cannot have emotion.
Why not?
that doesn’t answer my question.
In what way can AI not have these thoughts or emotions?
Same reason flapping your arms really fast will never make you start flying. You don’t have the equipment. You can’t fly because that’s not what arms do.
I’ll accept that argument once someone can describe or define consciousness in some comparable way. Until then, you can’t say some person or thing doesn’t have it
It’s a joke… lighten up, Francis