Footnotes
1.
Large amounts of corporate time and energy involve the training of 500 billion to
1 trillion neuron networks. Your question is treated as a separate test set.
Training a large network involves going through the massive libraries on the
internet, for instance Wikipedia or the NYT (who will be
compensated). The MIT Technology Review (2023) reports that training
Open AI’s GPT-3 cost $4MM (consider now GPT-4). “Training an AI model is
energy-consuming; it can expend more electricity than 100 typical U.S. homes
over an entire year….These models are some of the most
compute-heavy models…Open AI is reportedly spending $40MM a month to process
the queries entered by users...More efficient smaller models may be a promising
alternative.”
In
computer science, a major issue isn’t only getting algorithms to work; it’s
designing the most efficient ones. As we have illustrated, generative AI relies
heavily on matrix multiplication, of what in practice is very large matrices.
2.
The word, “context” is now probably now the “C” word. In December,
2023 three college presidents were called before the House Committee on
Education. They were asked whether it was appropriate for political groups on
campus to advocate genocide. The correct answer would have been, “Murder is
always wrong.” Their answers were - it depends upon, “context.” Two out of
three were subsequently forced to resign. Colleges should question; but they
should also teach at least a few moral values of the society, but not
necessarily a long list of them.
Like
language, liberal societies are complex. Effective AI, as we see, honors that
complexity in varying contexts. If the U.S. had thought that way, it
would have avoided the foreign policy catastrophes of Vietnam and Iraq.
3.
Mustafa Suleyman; “The Coming Wave”; Crown Publishers; New York, N.Y.; 2023; p.
51.
4.
Since we are talking about emerging phenomena and complexity, we might as well
talk about the financial markets. We have shown that the financial markets can
be divided into idiosyncratic sub-markets. For this reason, a general model of
the financial market, say the Capital Asset Pricing Model, will be less
effective than an acceptable value calculation of bonds or stocks - as value
investors know. In additional to the vagaries of markets; the present value
model of stocks and bonds; there is also the Federal Reserve to finally restore
logic to overall markets, aligning risk with return – by taking away the punch
bowl.
5.
In probability theory, the central
limit theorem (CLT) states that, under appropriate conditions, the distribution of a normalized version of the
sample mean converges to a standard normal distribution. This holds even if the original variables themselves are
not normally distributed.
Wikipedia
This
theorem justifies the entire field of econometrics and the complicated algebra
of economics. It is, unfortunately, usually wrong. Economic data series, say
inflation, commodity prices or the stock market, have very large variances. The
central limit theorem assumes a rather placid economy with moderate variances,
capable of optimization. Econometrics cannot answer a question that we asked
years ago, “What is the disutility of a Saddam Hussein?” Now, what about Hamas
or a Putin?
That
having been said, econometrics is good to know, because it expresses the
scientific method and its assumptions very clearly. That also helped us
understand generative AI and might also indicate its limitations. This IBM
article, “What is model drift?”
essentially says the same. add: Simple AI is probably most applicable in
those cases where variances are moderate and controlled.
Unlike
models in physics or chemistry, AI social models change rapidly. Like CAPM,
they always have to be monitored for changes in the input variables. For a CEO,
the cost of model expertise has to be weighed against the benefits of
operational improvement. One simple way to analyze the effects of change is to
use model perturbation, removing or changing an input variable. But that
approach can be expensive because the model has to be rerun each time. A second
way, that Stanford’s (Shrikumar, Greenside and Kundaje, 2019) explore, is to backpropagate the network
output to every feature of the input.
The
field of generative AI did not just appear out of thin air. The
Austro-Hungarian philosopher, Ludwig Wittgenstein (1889-1951) became deeply
interested in how (as Gemini summarizes), “…language creates meaning and how we
connect words to the world. He questioned traditional philosophical theories
that saw meaning as residing in objects or mental representations, instead
focusing on language use and context.” * His work resulted in computational
linguistics. As a practical example, foreign languages used to be taught
according to tables and grammatical rules; it is now taught from how people actually use it, from context. Similarly, answers
are now generated by AI, word for word, from a context defined by the (Q,K,V)
matrices. There is, within this, another possibly substantial problem; how do
you affix characteristics or authorship to a specific object or document, as this
world/society also requires? There is a difference between the real world and a
virtual one.
We
note that field is fast changing, and progress will occur. But humans are
necessary to judge what is important, when, and where. The view that AI can
have substantial upsides and downsides is likely correct. An important positive
factor is that AI can scale, opening up new markets for
a certain perception. An important negative factor is its possible effect on
employment.
*Microsoft
Word can now translate, maybe too easily, the above passage excerpt into
French:
Le domaine de l’IA générative n’est pas apparu de nulle part. Le philosophe austro-hongrois Ludwig Wittgenstein (1889-1951) s’est profondément intéressé à la façon dont (comme le résume Gemini), « ... Le langage crée du sens et la façon dont nous relions les mots au monde. Il a remis en question les théories philosophiques traditionnelles qui considéraient que le sens résidait dans des objets ou des représentations mentales, en se concentrant plutôt sur l’utilisation du langage et le contexte.
