From James Somers, "The Case That A.I. Is Thinking: ChatGPT does not have an inner life. Yet it seems to know what it’s talking about," The New Yorker: ...Was ChatGPT mindlessly stringing words together, or did it understand the problem? The answer could teach us something important about understanding itself. “Neuroscientists have to confront this humbling truth,” Doris Tsao, a neuroscience professor at the University of California, Berkeley, told me. “The advances in machine learning have taught us more about the essence of intelligence than anything that neuroscience has discovered in the past hundred years.”
Tsao is best known for decoding how macaque monkeys perceive faces. Her team learned to predict which neurons would fire when a monkey saw a specific face; even more strikingly, given a pattern of neurons firing, Tsao’s team could render the face. Their work built on research into how faces are represented inside A.I. models. These days, her favorite question to ask people is “What is the deepest insight you have gained from ChatGPT?” “My own answer,” she said, “is that I think it radically demystifies thinking.”
...People in A.I. were skeptical that neural networks were sophisticated enough for real-world tasks, but, as the networks got bigger, they began to solve previously unsolvable problems. People would devote entire dissertations to developing techniques for distinguishing handwritten digits or for recognizing faces in images; then a deep-learning algorithm would digest the underlying data, discover the subtleties of the problem, and make those projects seem obsolete. Deep learning soon conquered speech recognition, translation, image captioning, board games, and even the problem of predicting how proteins will fold.
...Today’s leading A.I. models are trained on a large portion of the internet, using a technique called next-token prediction. A model learns by making guesses about what it will read next, then comparing those guesses to whatever actually appears. Wrong guesses inspire changes in the connection strength between the neurons; this is gradient descent. Eventually, the model becomes so good at predicting text that it appears to know things and make sense. So that is something to think about. A group of people sought the secret of how the brain works. As their model grew toward a brain-like size, it started doing things that were thought to require brain-like intelligence. Is it possible that they found what they were looking for?
...Jonathan Cohen, a cognitive neuroscientist at Princeton, emphasized the limitations of A.I., but argued that, in some cases, L.L.M.s seem to mirror one of the largest and most important parts of the human brain. “To a first approximation, your neocortex is your deep-learning mechanism,” Cohen said. Humans have a much larger neocortex than other animals, relative to body size, and the species with the largest neocortices—elephants, dolphins, gorillas, chimpanzees, dogs—are among the most intelligent.
...I do not believe that ChatGPT has an inner life, and yet it seems to know what it’s talking about. Understanding—having a grasp of what’s going on—is an underappreciated kind of thinking, because it’s mostly unconscious. Douglas Hofstadter, a professor of cognitive science and comparative literature at Indiana University, likes to say that cognition is recognition... Hofstadter was one of the original A.I. deflationists, and my own skepticism was rooted in his. He wrote that most A.I. research had little to do with real thinking, and when I was in college, in the two-thousands, I agreed with him. There were exceptions. He found the U.C.S.D. group interesting. And he admired the work of a lesser-known Finnish American cognitive scientist, Pentti Kanerva, who noticed some unusual properties in the mathematics of high-dimensional spaces...
Hofstadter realized that Kanerva was describing something like a “seeing as” machine. “Pentti Kanerva’s memory model was a revelation for me,” he wrote in a foreword to Kanerva’s book. “It was the very first piece of research I had ever run across that made me feel I could glimpse the distant goal of understanding how the brain works as a whole.” Every kind of thinking—whether Joycean, Proustian, or logical—depends on the relevant thing coming to mind at the right time. It’s how we figure out what situation we’re in...
...L.L.M.s appear to have a “seeing as” machine at their core. They represent each word with a series of numbers denoting its coördinates—its vector—in a high-dimensional space. In GPT-4, a word vector has thousands of dimensions, which describe its shades of similarity to and difference from every other word. During training, a large language model tweaks a word’s coördinates whenever it makes a prediction error; words that appear in texts together are nudged closer in space. This produces an incredibly dense representation of usages and meanings, in which analogy becomes a matter of geometry. In a classic example, if you take the word vector for “Paris,” subtract “France,” and then add “Italy,” the nearest other vector will be “Rome.” L.L.M.s can “vectorize” an image by encoding what’s in it, its mood, even the expressions on people’s faces, with enough detail to redraw it in a particular style or to write a paragraph about it.
When [my friend] Max asked ChatGPT to help him out with [a] sprinkler at the park, the model wasn’t just spewing text. The photograph of the plumbing was compressed, along with Max’s prompt, into a vector that captured its most important features. That vector served as an address for calling up nearby words and concepts. Those ideas, in turn, called up others as the model built up a sense of the situation. It composed its response with those ideas “in mind.” ...
Dwan Ev ceremoniously soldered the final connection with gold. The eyes of a dozen television cameras watched him and the subether bore throughout the universe a dozen pictures of what he was doing.
He straightened and nodded to Dwar Reyn, then moved to a position beside the switch that would complete the contact when he threw it. The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies.
Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said, "Now, Dwar Ev."
Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.
Dwar Ev stepped back and drew a deep breath. "The honor of asking the first question is yours, Dwar Reyn."
"Thank you," said Dwar Reyn. "It shall be a question which no single cybernetics machine has been able to answer."
He turned to face the machine. "Is there a God?"
The mighty voice answered without hesitation, without the clicking of a single relay.
"Yes, now there is a God."
Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.
A bolt of lightning from the cloudless sky struck him down and fused the switch shut.
