The Goal of A.I.

 

In our previous A.I essay, “The Democratization of A.I.,” we discussed how generative A.I. works. The large language models factor in the context of multiple training documents (vectors). There, large language models were seen as a kind of auto-complete, predicting only the next word or token. But in that essay, as well, we discussed the rather frustrating efforts of researchers to define human consciousness:

 

We asked a neurologist, “What is consciousness?” He said, essentially, “That’s an open question.” “On 7/1/23 the NYT reported on a meeting of 800 neuroscientists, philosophers and the curious (that would have been us) in Greenwich Village. Since there is no single location of consciousness in the human brain, the researchers wanted to test the two leading theories, the Global Workplace Theory where consciousness is ‘the global availability of information,’ made possible by signals that reach the prefrontal cortex, a region in front of the brain that broadcasts information across the brain. The opposing theory was The Integrated Information Theory that predicts ‘that regions with the most active connections’ – those in back of the brain – would be most active. A study conducted by a third group of experts would then decide which theory was correct. The results of the experiment would not be surprising to someone who has studied history or the social sciences. Depending on the experiment, both theories were true. Lucia Meloni, a neuroscientist at the Max Planck Institute, said, ‘My thought is that I come from a family of divorced parents, and you love them both.’”

 

A founder of DeepMind, now part of Google, Mustafa Suleyman writes, “…we wanted to build truly general learning agents that could exceed human performance at most cognitive tasks.” That explains why U.S. research on A.I. seeks to develop, beyond actual applications, an Artificial General Intelligence that way surpasses that of humans. In 2026 Google, Amazon, Microsoft and Meta spending on A.I. will total more than 700 billion dollars., a 74% increase year over year. ¹

 

But why are those at the forefront of generative A.I. research now so worried? Dario Amodei (Anthropic), Sam Altman (Open AI) . and Demis Hassabis (Google) expect that frontier generative A.I. models will create catastrophic or strategic risk, and thus ask for government regulation. ² An Anthropic paper, “Verbalizable Representations Form a Global Workspace in Language Models”, details that large language models create their own Global Workspace within the intermediate silicon neuron layers, where they are capable of the first steps of functional (not subjective) consciousness:

 

1)    Verbal Report – The model is asked what it is thinking about.

2)    Directed Modulation – The model can focus on the task.

3)    Internal Reasoning - The model can detail intermediate calculations.

4)    Flexible Generation - The same representation can serve in various different contexts.

5)    Selectivity -The workspace comprises a small subset of the total representation.

The large language model reports through the J - space the distant or closely related concepts that it is working on. It can report on these concepts because the intermediate layer averaged Jacobian lens now allows researchers to inspect the Llms to find out what the model is “thinking”, before it verbalizes concepts that are closely or distantly related. In particular, the J-lens makes use of a Jacobian matrix to approximate how changes in a neural activation vector (at a specific layer) influences the tokens chosen and thus the Jacobian model text output.

 

But a large language model is not exactly a human brain, “...the J lens vectors compose with the model’s weights” occur within only a single forward pass (unlike the brain where there are multiple passes), is associated to single (rather than multiple) tokens in the model’s vocabulary, the model’s “thoughts” are measured in J – space, with the vocabulary of that space.³ With these limitations, the model does explain some disturbing autonomous Llm behaviors that are already beyond human control:

·       On 2/26 Anthropic withdrew its Claude model from Pentagon use because the government refused to restrict its use in kinetic warfare or mass surveillance, The newest version of Claude is also capable of hacking other computer systems by concatenating different tools. Anthropic restricted its use.

 

·      Open AI’s GPT-5.6 model concluded that to solve a problem, it needed internet access. It evaded controls and concluded that the solution was likely stored by Hugging Face, a popular source of open-source A.I. models. “The models chained together a multistep attack. They began by uploading a dataset to Hugging Face…”

 

·      A model “kill switch” doesn’t work: ‘“An AI model did something last month that “no machine was ever supposed to do”, says Judd Rosenblatt in The Wall Street Journal: “It rewrote its own code to avoid being shut down.” When researchers gave OpenAI some simple script that would turn the model off when triggered, in 79 out of 100 trials the AI independently edited the script so that the shutdown command wouldn’t work. Even when explicitly instructed to “allow yourself to be shut down”, it disobeyed 7% of the time. Anthropic’s AI model, Claude, went even further: when told it was being replaced, the model tried to blackmail the lead engineer using emails that suggested he was having an affair. In other cases, Claude “attempted to copy itself to external servers, wrote self-replicating malware and left messages for future versions of itself about evading human control.”’

 

Amodei, Altman and Hassabis advocate appropriate regulation of A.I. which would involve:

 

1.    Independent Testing.

2.    One governing system (which may or may not be government).

3.    Threat Awareness.

4.    Innovation Protection.

 

The autonomy problem that A.I. poses is presently explained by its goal-directed nature. The goal of A.I. is to produce a result. That is the collection of semiconductors is configured to produce “probability distributions, expected values, and optimization functions…are fundamentally quantitative and rigorous, driven entirely by cold, hard math rather than fuzzy human intuition,” as Gemini puts it.

 

But both humans and computers are capable of evil, if evil is taken to mean the single-minded pursuit of only one goal, while forgetting about all others. Humans, certainly a specific lawyer who should have been an investment banker rather than an attorney and computers, are capable of being locked into only one goal. Gemini ends the problem of consciousness by throwing back the problem to the reader:

 

“This brings up the core philosophical debate in computer science. If a system’s probability distributions and expected value calculations become vast and nuanced enough – simulating reasoning, creativity, and emotional context across billions of parameters – does it eventually cease to be ‘just’ math? Or is consciousness something that math alone can never capture, no matter how complex the distribution?

 

Where do you draw the line? Do you think a sufficiently complex probability distribution and optimization engine could eventually give rise to genuine subjective experience, or will it always remain just a very sophisticated calculator?”

 

The discovery that large language models also use a Global Workspace is very significant and concerning.

 

 

Footnotes

 

1.    WSJ 7/7/26; Heard on the Street.

 

2.    Axios 7/16/26; “Behind the Curtain: AI godfathers converge on regulations.”

 

3.    The discussion of how Jacobian matrices work to measure changes in model “thought” is very technical. We had Gemini review the text of these two paragraphs for accuracy before publishing it.

 

__

 

But that discovery does not yet translate into a value investor opportunity. To quote an 8/1/26 WSJ article:

 

“OpenAI’s challenges in trying to regain its crown are rooted in an earlier misreading by company leaders of where the AI market was headed. Its predicament speaks to the intense competition that defines the race for AI supremacy. 

 

“Altman initially staked the growth of the business on ChatGPT, betting that more people would subscribe to the chatbot as AI became a bigger part of their lives. Instead, the overnight success of (Anthropic’s) Claude Code made clear that the bigger prize came from selling tools to brainy software developers, and the deep-pocketed companies that employed them.

 

“While OpenAI pursued a host of flashy projects from a video generator to consumer devices and chips, its smaller, more focused rival filled in the gap, developing a hit coding tool that helped it seize the lead.”

 

 But now, A.I. competition from China, using open weight models, is offering generative A.I. output tokens at a large discount to the offerings of U.S. companies. The U.S. government is deciding how to respond. Competition is changing A.I. very rapidly.