Fascinating and well-written! Addresses so many questions I have about language in such a short space. Interesting how it agrees more with Cormac McCarthy in "The Kekulé Problem" than it does with Chomsky's theory of language - beyond the idea 'Human vs. ape communication is apples and oranges', it doesn't go along with much of the latter's thesis at all.
I'm not sure whether it was right at the time, but it's certainly not right now to say that reinforcement learning is always or even mainly achieved via self-play. The reinforcement learning that forms the second main phase of training an LLM chatbot ("post-training"), turning it from a pure next-token predictor into a turn-taking, "helpful, honest, harmless assistant" persona is done by human feedback, with no self-play at all, and I'm pretty sure the chain-of-thought reinforcement learning in the newer reasoning models is similar. I'd be surprised if this is entirely a very new development, and that reinforcement learning was all done by self-play before this. Either way, it definitely isn't now.
This isn't a central point at all and I thoroughly enjoyed this excerpt, but I thought this was a confusion worth noting.
But at a certain point, this argument switches from what we know to speculation. The key indicator of this is when the author stops using statements like "the more joint attention . . . The more vocabulary . . ." & " at this age human infants develop . . ." And starts using words like "instincts" & "[M]ay have" & "[T]hey naturally start . . ." We don't actually know why or how infants go from learning labels to using grammar to form sentences. Probably they learn grammar use from their parents but where exactly does grammar come from originally? It is not hardwired. It does not obviously develop from labeling objects b/c many animals have sounds that denote specific objects such as predators. Different labels for different predators, for example, tell the group which direction danger lies. But it does not lead to grammar or syntax. And what is the origin of questioning? That also does not follow — the author simply speculates that questioning is hardwired. But how? Hardwired questioning poses the same unknown as hardwired language itself.
So this discussion makes sense as far as it goes, but does not elucidate or even indicate, the remaining huge mystery of how humans developed complicated language.
While this is an insightful piece about the way in which humans apparently learn to coordinate sounds (speech), feelings (Braille), hand signs (sign language), and coloured squiggles (writing) with sensory images (colours, odours, flavours, feelings, and sounds), there remains an unbridgeable gap between this process (governed entirely by physical laws), and the emergence of ideas of number, form, and meaning.
The reason why it is unbridgeable is that you cannot get meaning from meaninglessness.
At birth, only meaningless colours are Seen, meaningless sounds are Heard, and meaningless feelings are Felt, as well as meaningless odours-Smelled and meaningless flavours-Tasted.
Look at a plain swatch of colour. The colour itself is meaningless. No pattern of meaningless colours can manifest meaning.
It's plain that meaning is not 'in' any 'pattern'. Otherwise, it would be impossible for one word to have many meanings, or many words to have the same meaning, or no meaning at all when viewed from the perspective of an illiterate person.
At no point in the exchange between baby and parent is there any process by which an idea can form. There is simply a non-stop flow of electro-chemical impulses within the brain and moving molecules and lightwaves outside the brain, all governed by physical laws.
What is an 'idea'?
It's simply 'that which is Known'.
That ideas are Known requires no theory or belief. It is 'self-evident': two (number) cubed (form) dice (meaning).
It's plain also that no idea can exist apart from the capacity to Know it. Which is to say Knowing-ideas in an inseparable 'non-dual' whole.
Knowing-ideas is also self-evidently a facet of Awareness.
Ideas have no odour, or flavour. They cannot be Seen or Heard or Touched. Objectively, they appear as if they are not. They can only be Known subjectively.
That Knowing is 'immaterial', is also self-evident. Plainly, Knowing has no substance or form. You cannot See or Hear or Feel or Taste of Smell 'Knowing'.
What Knowing-ideas 'is', is impossible to Know.
Take the number '4'. The numerical meaning is not in the 'numeral' (4) or the word (four, spoken or written), nor in any group of apparent objects. There are infinite ways in which the number may be represented, none of which are the number-Known. It is the number-Known that gives numerical meaning to any apparent symbol or set of apparent objects. As such, it is impossible to learn the number (or any idea) from experience, as nothing in experience is the idea-Known.
It's plain when looking that no 'things' are perceived until the sensory image is associated with an idea. Say when looking at the trunk of a tree, a camouflaged moth may not be Seen, even though the colours are apparent. Or take the case of the well-known 'illusion' where a line drawing may be Seen as a young woman, or an old woman, depending on the ideas associated with the image.
It's also self-evident that no idea can ever change, for if it did, it would no longer be the idea that it is.
It's plain too, that no idea can stand on its own. Each takes its number form and meaning by reference to other ideas that ultimately take their number form and meaning from the set of all possible 'things, properties, processes, events, and relations'.
Try to define any idea without bringing in other ideas. It cannot be done.
Knowing-all-ideas (ever-unchanged) is a facet of Consciousness.
Of course, there is no Awareness of Knowing-all-ideas (ever-unchanged).
But notice: 'Gorilla'
Until this word appeared in Awareness , there was no Awareness of Knowing or not-Knowing the meaning associated with the word. Awareness of any idea arises only as it is paired with a sensory image. How this occurs remains an eternal mystery.
All observations and theories about the apparent world appear in and to Awareness. Without Awareness there is no experience, no perceived world... no 'thing' at all.
If you'd like to explore this mystery further, I recommend a new book - 'The Door to NowHere: A Portal into the Timeless Reality of Non-dual Consciousness', available on Amazon.
The majority of humans are still limited to object-oriented language, on par with the apes, and are almost entirely bereft of the capacity to convey abstraction. They mimic word-patterns just like they mimic gestures, to get the reactions they want, and they do not mean anything else by the words they use.
Is it possible that the innate drive or instinct for language and indeed other higher functions are not in fact based in the brain at all. If they are usually based in Broca and Wernickes and post language acquisition damage causes lingual failure BUT the ability can move plasticly to other areas in the brain in pre-lingual children then it seems that there is something *outside* the brain which expects new areas to develop in order to hold the curriculum.
More simply - where does the impetus for language acquisition come from and why?
When I see a phrase like 600 million years ago, I basically switch off. I am not saying it isn’t an area that people can’t be interested in, but it just doesn’t appeal to me.
Fascinating and well-written! Addresses so many questions I have about language in such a short space. Interesting how it agrees more with Cormac McCarthy in "The Kekulé Problem" than it does with Chomsky's theory of language - beyond the idea 'Human vs. ape communication is apples and oranges', it doesn't go along with much of the latter's thesis at all.
I'm not sure whether it was right at the time, but it's certainly not right now to say that reinforcement learning is always or even mainly achieved via self-play. The reinforcement learning that forms the second main phase of training an LLM chatbot ("post-training"), turning it from a pure next-token predictor into a turn-taking, "helpful, honest, harmless assistant" persona is done by human feedback, with no self-play at all, and I'm pretty sure the chain-of-thought reinforcement learning in the newer reasoning models is similar. I'd be surprised if this is entirely a very new development, and that reinforcement learning was all done by self-play before this. Either way, it definitely isn't now.
This isn't a central point at all and I thoroughly enjoyed this excerpt, but I thought this was a confusion worth noting.
But at a certain point, this argument switches from what we know to speculation. The key indicator of this is when the author stops using statements like "the more joint attention . . . The more vocabulary . . ." & " at this age human infants develop . . ." And starts using words like "instincts" & "[M]ay have" & "[T]hey naturally start . . ." We don't actually know why or how infants go from learning labels to using grammar to form sentences. Probably they learn grammar use from their parents but where exactly does grammar come from originally? It is not hardwired. It does not obviously develop from labeling objects b/c many animals have sounds that denote specific objects such as predators. Different labels for different predators, for example, tell the group which direction danger lies. But it does not lead to grammar or syntax. And what is the origin of questioning? That also does not follow — the author simply speculates that questioning is hardwired. But how? Hardwired questioning poses the same unknown as hardwired language itself.
So this discussion makes sense as far as it goes, but does not elucidate or even indicate, the remaining huge mystery of how humans developed complicated language.
Don’t miss Ray Birdwhistell and Charles F. Hockett… Or wait for my new book: “One World, Four Words.”
While this is an insightful piece about the way in which humans apparently learn to coordinate sounds (speech), feelings (Braille), hand signs (sign language), and coloured squiggles (writing) with sensory images (colours, odours, flavours, feelings, and sounds), there remains an unbridgeable gap between this process (governed entirely by physical laws), and the emergence of ideas of number, form, and meaning.
The reason why it is unbridgeable is that you cannot get meaning from meaninglessness.
At birth, only meaningless colours are Seen, meaningless sounds are Heard, and meaningless feelings are Felt, as well as meaningless odours-Smelled and meaningless flavours-Tasted.
Look at a plain swatch of colour. The colour itself is meaningless. No pattern of meaningless colours can manifest meaning.
It's plain that meaning is not 'in' any 'pattern'. Otherwise, it would be impossible for one word to have many meanings, or many words to have the same meaning, or no meaning at all when viewed from the perspective of an illiterate person.
At no point in the exchange between baby and parent is there any process by which an idea can form. There is simply a non-stop flow of electro-chemical impulses within the brain and moving molecules and lightwaves outside the brain, all governed by physical laws.
What is an 'idea'?
It's simply 'that which is Known'.
That ideas are Known requires no theory or belief. It is 'self-evident': two (number) cubed (form) dice (meaning).
It's plain also that no idea can exist apart from the capacity to Know it. Which is to say Knowing-ideas in an inseparable 'non-dual' whole.
Knowing-ideas is also self-evidently a facet of Awareness.
Ideas have no odour, or flavour. They cannot be Seen or Heard or Touched. Objectively, they appear as if they are not. They can only be Known subjectively.
That Knowing is 'immaterial', is also self-evident. Plainly, Knowing has no substance or form. You cannot See or Hear or Feel or Taste of Smell 'Knowing'.
What Knowing-ideas 'is', is impossible to Know.
Take the number '4'. The numerical meaning is not in the 'numeral' (4) or the word (four, spoken or written), nor in any group of apparent objects. There are infinite ways in which the number may be represented, none of which are the number-Known. It is the number-Known that gives numerical meaning to any apparent symbol or set of apparent objects. As such, it is impossible to learn the number (or any idea) from experience, as nothing in experience is the idea-Known.
It's plain when looking that no 'things' are perceived until the sensory image is associated with an idea. Say when looking at the trunk of a tree, a camouflaged moth may not be Seen, even though the colours are apparent. Or take the case of the well-known 'illusion' where a line drawing may be Seen as a young woman, or an old woman, depending on the ideas associated with the image.
It's also self-evident that no idea can ever change, for if it did, it would no longer be the idea that it is.
It's plain too, that no idea can stand on its own. Each takes its number form and meaning by reference to other ideas that ultimately take their number form and meaning from the set of all possible 'things, properties, processes, events, and relations'.
Try to define any idea without bringing in other ideas. It cannot be done.
Knowing-all-ideas (ever-unchanged) is a facet of Consciousness.
Of course, there is no Awareness of Knowing-all-ideas (ever-unchanged).
But notice: 'Gorilla'
Until this word appeared in Awareness , there was no Awareness of Knowing or not-Knowing the meaning associated with the word. Awareness of any idea arises only as it is paired with a sensory image. How this occurs remains an eternal mystery.
All observations and theories about the apparent world appear in and to Awareness. Without Awareness there is no experience, no perceived world... no 'thing' at all.
If you'd like to explore this mystery further, I recommend a new book - 'The Door to NowHere: A Portal into the Timeless Reality of Non-dual Consciousness', available on Amazon.
Mind blowing. Loved it
There is a kind of “chicken and egg” problem. If learning a language requires a teacher, then who was the first teacher and how did she learn?
The majority of humans are still limited to object-oriented language, on par with the apes, and are almost entirely bereft of the capacity to convey abstraction. They mimic word-patterns just like they mimic gestures, to get the reactions they want, and they do not mean anything else by the words they use.
Is it possible that the innate drive or instinct for language and indeed other higher functions are not in fact based in the brain at all. If they are usually based in Broca and Wernickes and post language acquisition damage causes lingual failure BUT the ability can move plasticly to other areas in the brain in pre-lingual children then it seems that there is something *outside* the brain which expects new areas to develop in order to hold the curriculum.
More simply - where does the impetus for language acquisition come from and why?
When I see a phrase like 600 million years ago, I basically switch off. I am not saying it isn’t an area that people can’t be interested in, but it just doesn’t appeal to me.
Good to know
Yes, because I am normally very engaged in the things you post about.