Yudkowsky on Popper

In Rationality From AI to Zombies Eliezer Yudkowsky writes some stuff about Popper. He does a bad job of discussing Popper’s ideas. On pp. 820-821 he writes:

Previously, the most popular philosophy of science was probably Karl Popper’s falsificationism—this is the old philosophy that the Bayesian revolution is currently dethroning. Karl Popper’s idea that theories can be definitely falsified, but never definitely confirmed, is yet another special case of the Bayesian rules; if P(X|A) ≈ 1—if the theory makes a definite prediction—then observing ¬X very strongly falsifies A. On the other hand, if P(X|A) ≈ 1, and we observe X, this doesn’t definitely confirm the theory; there might be some other condition B such that P (X|B) ≈ 1, in which case observing X doesn’t favor A over B. For observing X to definitely confirm A, we would have to know, not that P(X|A) ≈ 1, but that P(X|¬A) ≈ 0, which is something that we can’t know because we can’t range over all possible alternative explanations. For example, when Einstein’s theory of General Relativity toppled Newton’s incredibly well-confirmed theory of gravity, it turned out that all of Newton’s predictions were just a special case of Einstein’s predictions.

Popper didn’t call his position falsificationism. He explicitly rejected that label, see the introduction to ‘Realism and the aim of science’.

Popper called his position critical rationalism. Popper said that all knowledge creation proceeds by guessing and criticism. First, you spot a problem with your current ideas, then you guess solutions to the problem, then you criticise the guesses.

Falsifying an idea by experimental testing is just one of the possible ways of criticising an idea: it’s not central to Popper’s ideas.

You can even formalize Popper’s philosophy mathematically. The likelihood ratio for X, the quantity P(X|A)/P(X|¬A), determines how much observing X slides the probability for A; the likelihood ratio is what says how strong X is as evidence. Well, in your theory A, you can predict X with probability 1, if you like; but you can’t control the denominator of the likelihood ratio, P(X|¬A)—there will always be some alternative theories that also predict X, and while we go with the simplest theory that fits the current evidence, you may someday encounter some evidence that an alternative theory predicts but your theory does not.

It is possible to formalise reasoning, but not using the rules of probability. Reasoning is a form of information processing: you use information and the knowledge you already have to find new and better ideas. Any physically possible information processing can be simulated by a computer. This simulation can simulate both the results and the process by which they were created. So reasoning can be formalised. Nobody knows how to do this, but it is possible.

Back to Yudowsky:

That’s the hidden gotcha that toppled Newton’s theory of gravity. So there’s a limit on how much mileage you can get from successful predictions; there’s a limit on how high the likelihood ratio goes for confirmatory evidence.

The idea that Newton’s theory was replaced cuz it was incompatible with evidence is misleading. Newton’s theory of gravity got into trouble because it is incompatible with the special theory of relativity.

One problem is that Newton’s theory implies that changes in the mass distribution instantly change gravitational forces everywhere. Special relativity sez that the maximum speed at which one system can influence another is the speed of light.

Another problem is that special relativity sez that mass and energy are tied together as parts of a single conserved quantity. So what should take the place of mass in a relativistic theory of gravitation?

Einstein thought about how to solve those problems and others and came up with the general theory of relativity. Newton’s theory of gravity was replaced because it was a bad explanation. The experimental evidence that Newton’s theory was wrong was mostly found after general relativity was invented. Until then, nobody knew where to look for experimental evidence against Newton’s theory.

There’s a lot of historical discussion of the origin of general relativity, e.g. – ‘The genesis of general relativity’ vols 1-4 edited by Jurgen Renn (I’d link to it on Amazon but it’s super expensive). If Yudkowsky is going to use historical examples, he should look them up.

On the other hand, if you encounter some piece of evidence Y that is definitely not predicted by your theory, this is enormously strong evidence against your theory. If P (Y |A) is infinitesimal, then the likelihood ratio will also be infinitesimal. For example, if P (Y |A) is 0.0001%, and P (Y |¬A) is 1%, then the likelihood ratio P (Y |A)/P (Y |¬A) will be 1:10,000. at’s −40 decibels of evidence! Or, flipping the likelihood ratio, if P (Y |A) is very small, then P (Y |¬A)/P (Y |A) will be very large, meaning that observing Y greatly favors ¬A over A. Falsification is much stronger than confirmation. It is is a consequence of the earlier point that very strong evidence is not the product of a very high probability that A leads to X, but the product of a very low probability that not-A could have led to X. This is the precise Bayesian rule that underlies the heuristic value of Popper’s falsificationism.

Similarly, Popper’s dictum that an idea must be falsifiable can be interpreted as a manifestation of the Bayesian conservation-of-probability rule; if a result X is positive evidence for the theory, then the result ¬X would have disconfirmed the theory to some extent. If you try to interpret both X and ¬X as “confirming” the theory, the Bayesian rules say this is impossible! To increase the probability of a theory you must expose it to tests that can potentially decrease its probability; this is not just a rule for detecting would-be cheaters in the social process of science, but a consequence of Bayesian probability theory. On the other hand, Popper’s idea that there is only falsification and no such thing as confirmation turns out to be incorrect. Bayes’s theorem shows that falsification is very strong evidence compared to confirmation, but falsification is still probabilistic in nature; it is not governed by fundamentally different rules from confirmation, as Popper argued.

 

Popper made many arguments against the idea of inductive probability, see Realism and the aim of science, part II, chapter II. For example, in Section 13 of that chapter Popper points out that we must have an explanation of what counts as repeating the same experiment to do induction, but induction provides us with no means to get that explanation. Yudkowksy shows no sign of being aware of any of these arguments.

The only Popper book Yudkowsky cites is LScD and he doesn’t seem to have understood it.