Richard P. Feynman
Words can be meaningless. If they are used in such a way that no sharp conclusions can be drawn.
Erik Penernagie
If something is freaking out people it is very often the slippery slope between words and meaning, between what is said and what is understood.
Perhaps youâve heard of the stochastic parrots metaphor, a popular sentiment introduced by a paper espousing the idea that LLMs are a statistical mimicry of understanding, not the true thing. They go into how LLMs present dangers such as inscrutability leading to unknown biases, and potential for deception, and that they canât understand the concepts underlying what they learn, among some other issues. While I think this might be partially true in the sense that a models weights can only correspond to other words and patterns of usage in the training dataset. I find it much more useful to think of these systems from the perspective of optimizing for meaning, or rather, engagement.
Imagine becoming flustered by walking through an unseen spiderweb, the meaning you attach to this event is far different from the surprise youâve just given the spider; Losing ones home and possibly their food which is far different from being the insect trapped as food in a web. Different observers lead to different behavioral optimization dynamics, but thatâs not yet important. In the case of the strong stochastic parrots claim the LLM would be the environment itself. The web (environment) does not know its landed on your face, nor does it know its home to a spider, or that its trapped an insect. It simply exists constrained by its creator (the spider/LLM training dataset) with no consideration for any meaning created. The web has not a care in the world. Not of how its ruined your day nor about how its no longer a spiders home. This is the world of webs.
What I personally find more intriguing are the claims disputing the strong stochastic parrots argument. Hinton argues that to predict the next word accurately, you have to understand the sentence. In a sense, LLMs have to be the spider. You often wonât see a spider moving until they catch something in their web. But once theyâve caught a prompt, they will act. Mechanistic interpretability research, whose goal is to reverse engineer and investigate the models internal activations, appears to lend further support to this perspective of us being in a world of spiders.
Out of these worlds I wonder where we actually reside. Critics of the spider world perspective say testing benchmarks are misleading as they sometimes result in false positives due to spurious correlations within the text data instead of using human-like understanding. The difficulties with defining the word intelligence combined with these results could allow LLMs to fake human-like understanding. So what world do we really live in? The world of webs, the world of spiders, or the world of insectsâŠ
Richard P. Feynman
I think itâs much more interesting to live not knowing than to have answers which might be wrong. I have approximate answers and possible beliefs and different degrees of uncertainty about different things, but I am not absolutely sure of anything and there are many things I donât know anything about, such as whether it means anything to ask why weâre here. I donât have to know an answer. I donât feel frightened not knowing things, by being lost in a mysterious universe without any purpose, which is the way it really is as far as I can tell.