Why we must be careful about how we speak of large language models

For decades, we’ve customized our devices and apps with verbs like “thinks,” “sees,” and “believes.” And in most cases, such anthropological interpretations are harmless.

But we’re entering an age where we need to be careful how we talk about software. Artificial Intelligence (AI) and, in particular, Large Language Models (LLMs)It has made impressive progress in mimicking human behavior at a fundamentally different level from the human mind.

Murray Shanahan, professor of cognitive robotics and research scientist at Imperial College London, warns that it would be a big mistake to use the same instincts we have with each other without mirroring them to artificial intelligence systems. deep mindIn a new paper titled, “We are talking about large language models.” To make the most of the remarkable capabilities that AI systems possess, we need to know how they work and avoid assuming capabilities that they do not have.

read more: OpenAI CEO acknowledges ChatGPT risks What now? | AI beat

Humans vs LLMs

“It’s amazing how human-like LLM-based systems can be, and they’re improving so quickly. After talking with them for a while, it’s Everything is very easy Start thinking of them as companies with minds like ours,” Shanahan told VentureBeat. “But they’re actually an alien form of intelligence, and we don’t fully understand them yet. So we have to be careful when incorporating them into human affairs.

Human use of language is an aspect of collective behavior. We acquire language by interacting with our community and the world we share with them.

“As a child, your parents and caregivers provided running commentary in natural language as they pointed to things, put things in your hands or picked them up, moved things in your view, played together, and so on,” Shanahan said. “LLMs are trained in a very different way, not living in our world.”

LLMs are mathematical models that represent the statistical distribution of tokens (tokens can be words, parts of words, letters, or punctuation marks) in a corpus of human-generated text. They produce text in response to a question or question, but not in the same way a human does.

Shanahan simplifies the interaction with the LL.M.: “Here’s a piece of text. Tell me how this piece can be continued. According to your model of the statistics of human language, what words are likely to come next?”

Trained on enough examples, LLM can produce correct answers at an impressive rate. Nevertheless, the difference between humans and LLMs is very important. For humans, different parts of language may have different relationships to truth. We can tell the difference between fact and fiction, like Neil Armstrong’s trip to the moon and Frodo Baggins’ return to the Shire. For an LLM that produces statistically probable terms, these differences are invisible.

“That’s one reason why it’s good for users to be reminded of something over and over again

LLMs actually do,” Shanahan writes. And this reminder reminds developers that “philosophically, words like ‘belief,’ ‘knowledge,’ ‘understanding,’ ‘self,’ or ‘feeling’ are used to describe the capabilities of LLMs. Helps avoid misuse of rich words.

Blurred barriers

When we’re talking about phones, calculators, cars, etc., there’s usually no harm in using anthropomorphic language (eg, “I don’t realize my watch keeps daylight savings”). We know these words are convenient shorthand for complex processes. However, in the case of LLMs, “such is their power, things can get a little blurry,” warns Shanahan.

For example, there is a large research organization Instant engineering tricks LLMs can improve their performance in complex tasks. In some cases, adding a simple sentence such as “Let’s think step by step” can improve the LLM’s ability to complete reasoning and planning tasks. Such results may “increase the desire to view [LLMs] It has human-like characteristics, Shanahan warns.

But again, we need to keep in mind the differences between reasoning in humans and meta-reasoning in LLMs. For example, if we ask a friend, “Which country is south of Rwanda?” And they said, “I think it’s Burundi,” and we know that they understand our perspective, our background knowledge, and our interests. At the same time, they know our ability and means to check their answer like looking at a map or googling or asking others.

However, when the same question is asked of an LLM, that fertile environment is not there. In many cases, some context is rendered by adding bits to the background, such as building it into a script-like structure that the AI ​​exposed during training. This makes the LLM more likely to produce the correct answer. But the AI ​​doesn’t “know” about Rwanda, Burundi, or their connection to each other.

“Knowing that the word ‘Burundi’ is likely to follow the words ‘country south of Rwanda’ is not the same as knowing that Burundi is south of Rwanda,” writes Shanahan.

Careful application of LLMs in real-world applications

While LLMs continue to improve, we as developers need to be careful about how we build applications on top of them. As users, we need to be careful about how we think about our interactions with them. Our frame of mind about LLMs and AI in general will have a major impact on the safety and robustness of their applications.

The expansion of LLMs may require a change in the way familiar psychological terms such as “beliefs” and “thoughts” are used, or the introduction of new terms, Shanahan said.

“This new type of artefact may require an extensive period of interaction and living with it before learning how to speak effectively about it,” writes Shanahan. “In the meantime, we must try to resist the siren call of anthropomorphism.”