Here Comes the God: All the Secrets of Wittgenstein, Anthropic and OpenAI

In the summer of 1918, Ludwig Wittgenstein, an artillery officer in the Austro-Hungarian Army, finished a short book of less than 30,000 words in the lulls of artillery fire on the Italian front. The manuscript was carried in his backpack as he moved positions; in November of that year, he was captured by Italian troops in South Tyrol and imprisoned in the Cassino POW camp. Thanks to the mediation of John Maynard Keynes, the British economist who would later save the world from the Great Depression, he was allowed to correspond in the camp, send the manuscript to Russell, and leave the famous declaration: All these problems have, in all essential respects, been solved.

This book is Tractatus Logico-Philosophicus. Russell, who wrote the introduction for it, said as early as before the war that this student was "the most perfect example of genius I have ever met". And the tone in Wittgenstein's own preface still feels strikingly sharp when read a hundred years later:

I therefore believe that I have finally solved these problems in all essential respects. If I am not mistaken, the second value of this book lies in the fact that it shows how little has been accomplished when these problems are solved. — Vienna, 1918

He truly practiced what he preached. After finishing this book, he gave up one of the largest private inheritances in Europe at that time, went to a mountain village in Austria to work as an elementary school teacher for six years, then worked as a gardener in a monastery, and later designed a house for his sister that is still regarded as a modernist classic to this day. Philosophy has come to an end, and a decent person should go do something else.

Ten years later, he changed his mind. In January 1929, Wittgenstein returned to Cambridge. Keynes, who was on the same train as him, wrote a note to his wife Lydia that same day:

Well, God has arrived. I met him on the 5:15 train. He plans to stay in Cambridge for a long time. I think the exhaustion will be devastating, but I cannot let him talk to me for more than two or three hours a day.

Cambridge fully understood what it had welcomed. Keynes was too lazy to use the word "genius", he simply wrote: God has arrived.

The title of this article is taken from this note. Because a hundred years later, another group of people are taking this joke literally. Except this time, "God" does not take the 5:15 train, it evolves itself in the computing cluster.

More than 100 years after the publication of the Tractatus Logico-Philosophicus in San Francisco, two laboratories update their flagship models every few months, declaring the same belief: General artificial intelligence is an engineering problem, and it is a problem that is about to be solved. One of them prints its ultimate goal on every prospectus, and the other incorporates the word "anthropos" (meaning human) into its name.

The core theme to be explained in this article is: All the secrets of Anthropic and OpenAI, these two companies, have long been written in a short philosophical book published in 1921. This is not alarmist. If you expand the seven sets of propositions of the Tractatus Logico-Philosophicus one by one, you will find that it is a complete technical specification for large models — world view, learning principle, generation principle, architecture definition, failure mode (hallucination), safety scheme (alignment), and even endgame warning, all are included without missing. The blueprint drawn by that artillery officer in 1918 was quietly implemented by two organizations in the 2020s.

Let's first give the skeleton of the comparison table. The main text will advance from the bottom up along the technical stack of large models — corpus, embedding, generation, prediction, architecture, scaling, alignment, seven keywords decoded one by one; by the seventh section, the first six will converge into one name: intelligence emergence.

Figure: Tractatus Logico-Philosophicus × Large Language Model: Mapping of twelve sets of propositions. The left side is the philosophy of 1921, and the right side is today's technology. Seven levels of steps: Propositions 1–6 climb step by step, and Proposition 7 emerges without words.

After reading this table, you will find an even more chilling thing: this book not only explains why large models can succeed, but also predicts where they will go wrong. Because its author himself was the first person to "run" this system and then be backfired by it.

01

Corpus (Data): The world is the totality of facts

The phrase "Data is the new oil" has been popular for ten years, but few people ask: Why can text feed intelligence, why when you pour what humans have written into a machine, what grows out is not a database, but something close to understanding? The first brick of the answer was laid in 1918.

On the first page of the book, the first set of propositions:

1 The world is all that is the case.

1.1 The world is the totality of facts, not of things.

These two sentences were a course correction in the history of philosophy in 1918. For the previous two thousand years, the main body of Western philosophy has been asking "what are things": what is substance, what is essence, what is universal. Wittgenstein said: The question is wrong. The basic unit of the world is not things, but facts — not "apple", but "the apple is on the table".

More than a hundred years later, this turn has been fulfilled by a technical fact. What large models do during training is exactly not to learn the "essence" of any noun, but to learn facts — how words combine, co-occur, and exclude each other in real language use. Linguists call this distributed semantics: the meaning of a word is determined by all the sentences in which it appears. Engineers call this training. And the statement in Tractatus Logico-Philosophicus is:

2.011 It is essential to things that they can be constituents of states of affairs.

2.012 In logic nothing is accidental: if a thing can occur in a state of affairs, the possibility of the state of affairs must be written into the thing itself.

In other words: There is no isolated meaning, only meaning in relations. This is exactly why models can learn to "understand" the world from pure text without touching any physical objects — because text is not a list of things, but the total record of facts (the context of the universe). Wittgenstein pushed this to the limit in 4.26:

4.26 The totality of true elementary propositions gives a complete description of the world.

Please read this sentence again. "The complete presentation of all true propositions is a complete description of the world." — this is the theoretical definition of pre-training. The training pipeline of large model companies is essentially an engineering project that collects, cleans, and compresses the facts recorded by humans as completely as possible. The corpus is not raw material, it is the computable form of the world itself.

02

Embedding: Logical space is latent space

"Embedding" — the operation that large models turn each word into a sequence of numbers, which is extremely simple, is the entrance to all magic. The philosophical license for it is issued by the second set of propositions.

1.13 The facts in logical space are the world.

2.013 Each thing is, as it were, in a space of possible states of affairs. This space I can imagine empty, but I cannot imagine the thing without the space.

There is no place inside the model to store the "essence of apple". When a word enters the model, it is represented as a vector in a high-dimensional space — a position. The meaning of this position is completely determined by its relative relationship with other vectors. Engineers call it embedding space, latent space; Wittgenstein calls it logical space. And in 2.0131 there is the most sentence in the whole book that resembles a deep learning paper:

2.0131 A spatial object must be situated in infinite space. (A spatial point is an argument-place.)

"A spatial point is an argument-place" — every embedding vector is an argument-place. A philosophical assertion from 1918 has become something that is actually stored in every piece of HBM memory in the 2020s.

What is more profound is 3.42:

3.42 A proposition can determine only one place in logical space: nevertheless the whole of logical space must already be given by means of it.

Each sentence only occupies one point in logical space, but the possibility of uttering this sentence presupposes the existence of the entire space. This explains the most disturbing ability of large models — generalization: it can generate sentences that never appear in the training data. This does not violate the data boundary, but is the logical inevitability of 2.013 — you cannot imagine a thing without its space. New sentences do not come out of nowhere, they are just new positions in the old space.

03

Generation: Picture theory and phonograph records

"Generation" — when the machine starts to create something out of nothing, the public's excitement and fear reach their peak at the same time. Wittgenstein would say: There is nothing to be surprised about. Humans have been doing this all the time, it is called a picture.

The most famous doctrine of Tractatus Logico-Philosophicus is the Picture Theory (Bildtheorie). Proposition 2.1:

2.1 We make to ourselves pictures of facts.

2.12 The picture is a model of reality.

4.031 In a proposition a state of affairs is, as it were, put together for the sake of experiment.

Wittgenstein's inspiration came from the Paris courts: newspapers used small figurines to set up a scene model of a car accident to state the facts. This "structural correspondence" between the model and the facts is what he calls a picture. And "in a proposition, a state of affairs is put together experimentally" — this is the operational definition of generative models: every output of the model is to assemble a possible world in logical space for reality to compare.

The example he gave in 4.014 reads today like a Transformer comment written a hundred years in advance:

4.014 The gramophone record, the musical idea, the written notes, the sound waves, all stand to one another in the same internal relation of depicting that holds between language and the world. … The rule is the law of projection which projects the symphony into the language of the musical score. It is the rule for translating this language into the language of gramophone records.

The same logical content (symphony) can be projected losslessly between different media: musical idea → musical score → gramophone record groove → sound wave. Text → vector → probability distribution → text, which is exactly this projection chain. The musical score and the gramophone record are not similar to each other, but carry the same content, just as the string of characters "hello" and its 100,000-dimensional vector inside the model are not similar, but carry the same fact. The philosophical basis for the possibility of machine translation and multimodal models lies entirely in this one proposition: all media share logical form (2.18), so projection is possible between all media.

04

Prediction (Next Token): The root of probability and hallucination

"A large model is nothing more than a probability machine that predicts the next word" — this popular saying is both contempt and truth. And the fifth set of propositions placed probability at the core of truth theory long before probability became an engineering discipline:

5 Propositions are truth-functions of elementary propositions.

5.15 The probability that proposition r gives to proposition s is the ratio of the number of truth-grounds common to r and s to the number of truth-grounds of r.

Replace "proposition" with "token", and this sentence is the mathematical definition of next token prediction: What the model assigns to candidate tokens is exactly the ratio from the existing context (truth-ground) to possible continuation (fact). Large models do not output true or false, they output probability — and the Tractatus Logico-Philosophicus placed probability at the core of truth theory long before it became an engineering concept.

This also accurately predicts the philosophical status of hallucination:

2.223 To discover whether a picture is true or false we must compare it with reality.

2.225 There are no pictures that are true a priori.

The model has no access to compare with reality. What it contacts is not the world, but the total collection of pictures of the world — the corpus. So hallucination is not a bug, not an engineering oversight, but an inevitable corollary of picture theory: A system that has only seen pictures cannot in principle determine the truth or falsehood of pictures by itself. All engineering efforts around "fact checking", "retrieval-augmented generation", and "tool calling" are essentially supplementing the conditions for 2.223 — artificially reconnecting the picture with reality.

And Wittgenstein has long defined the training itself:

6.363 The procedure of induction consists in accepting as true the simplest law that can be reconciled with our experiences.

6.36311 The sun will rise tomorrow: that is a hypothesis; and that means that we do not know whether it will rise.

Scaling laws — the common cornerstone of faith for large model companies, first systematically stated by Kaplan et al. at OpenAI in 2020 — are philosophically 6.363: it is not a logical necessity, but "the simplest law that matches experience". And every generation of the model is a repetition of 6.36311: Every token is a hypothesis.

05

Architecture (Transformer): The general form of propositions

"Architecture" — in 2017, Google's paper "Attention Is All You Need" brought Transformer; and its blueprint was already drawn in the fourth and sixth sets of propositions. Wittgenstein's real ambition was presented in the fourth set of propositions:

4.5 It now seems possible to give the most general propositional form: that is, to give a description of the propositions of any sign-language whatsoever in such a way that every possible sense can be expressed by a symbol satisfying the description…

4.51 Suppose that all elementary propositions are given to me: then I can simply ask what propositions I can construct out of them. And there all propositions are given, and their limits are given by that.

What Wittgenstein was looking for was a "universal master version" of language — a form that generates all legal sentences and only legal sentences. The sixth set of propositions gives the answer:

6 The general form of a truth-function is: [p, ξ, N(ξ)].

6.001 This says nothing else than that every proposition is the result of successive applications of the operation N(ξ) to the elementary propositions.

6.03 The general form of the integer is: [0, ξ, ξ+1].

Look at 6.03. Construction of integers: starting from 0, each number is generated by "adding one to the previous number". Sequence, autoregressive, term-by-term generation, the same rule. This is how large models generate: given the previous context, generate the next term, then merge it into the previous context, then generate the next term. The construction method of integer sequences and the generation method of token sequences are the same mathematical action. And