Knowledge: guesses that survive

1. The puzzle: where does knowledge come from?

Most people assume that knowledge comes from experience. We look at the world, notice regularities and generalise. Francis Bacon made this into a programme in Novum Organum (1620): gather observations patiently, climb from particulars to laws, and keep your imagination on a short lead.

David Hume found the hole in it a century later. No number of white swans proves that the next one will be white. To get from "the sun has always risen" to "the sun will rise tomorrow" you need a premise that the future will resemble the past, and the only evidence for that premise is that it has worked so far. The argument goes in a circle. Hume concluded that we expect the future to resemble the past out of habit, not reason, and for two centuries philosophers tried and failed to rebuild the bridge he had knocked down.

Popper's move was to stop trying. Knowledge never came across that bridge in the first place. Every theory starts as a guess, a bold and unjustified leap beyond anything observed. Observation cannot prove a guess, but it can refute one. A single black swan settles what a million white ones never could. Induction is a myth, and science runs on the asymmetry between confirming and refuting.

This turns the common-sense picture upside down. The mind is not a bucket that experience fills. It is a searchlight, already pointed by a problem and a theory before it sees anything.

2. Conjecture and refutation

Knowledge grows in a cycle with no starting point in observation: a problem, a bold guess, criticism, the removal of errors, and a better problem. Popper set it out in Logik der Forschung (1934) and gave it its slogan in Conjectures and Refutations (1963).

The cycle rewards courage. A theory that forbids a lot says a lot, and a theory that forbids nothing explains nothing. Einstein's general relativity predicted that starlight passing the Sun would bend by a precise angle, about twice the figure Newtonian reasoning gave. In 1919 the eclipse expeditions organised by Arthur Eddington and Frank Dyson, to Príncipe and to Sobral in Brazil, photographed the stars around the darkened Sun. Had the stars sat in the wrong places, the theory would have died. Its exposure to that risk is what made its survival mean something.

Compare a theory that bends to fit every outcome. Popper's targets were the psychoanalysis and Marxist history of 1920s Vienna, which seemed to find confirmation everywhere. That was the warning sign. A theory that nothing could refute is not strong. It is empty.

Criticism matters more than the guess. Anyone can guess; the skill lies in building traditions that hunt errors down: experiments, argument between peers, institutions that allow dissent. On this view the Enlightenment's real discovery was not any particular theory. It was a tradition of criticism that let theories improve without end.

In 1960 the psychologist Peter Wason showed how badly we do this unaided. He gave people the triple 2, 4, 6 and told them it fitted a hidden rule. They could propose more triples and be told yes or no. Try it before reading on.

Most of Wason's subjects guessed "even numbers rising by two", tested triples such as 8, 10, 12 and 20, 22, 24, heard yes every time, and announced their rule with confidence. The real rule was far wider. The only way to find it is to propose a triple your own theory forbids, such as 1, 2, 3 or 6, 4, 2, and see whether it is rejected. A test you expect to pass can teach you almost nothing. A test you expect to fail can teach you that you were wrong.

3. Good explanations

Deutsch's addition to Popper is a test for which surviving theories deserve to be taken seriously: a good explanation is hard to vary while still accounting for what it purports to account for.

His example in The Beginning of Infinity (2011) is the seasons. The ancient Greeks explained winter with Persephone. Hades carried her to the underworld, and her mother Demeter, the goddess of the harvest, grieves each year while her daughter is gone. The story fits the facts, but it would fit any facts. Swap the pomegranate seeds for a broken promise, or Demeter's grief for her anger, and it explains winter just as well. Nothing in it is load-bearing.

The axial-tilt explanation is different. The Earth's axis leans about 23.4 degrees, so each hemisphere tilts towards the Sun for half the year and away for the other half. Every detail does work. It predicts that the seasons reverse between hemispheres, so that Australia has summer while Britain has winter. The Persephone myth cannot account for that, and changing any part of the tilt explanation breaks it.

This gives fallibilism a direction. We never prove a theory true, but we can prefer the one whose details are all pinned in place by what they explain.

Where the view is contested

Critics inside the critical-rationalist camp press on a weak joint in the criterion. What counts as a variation, and who decides whether the varied theory still explains the same thing? If those judgements need further criteria, "hard to vary" cannot be the bottom of the story. And if it lets us rank theories by degree, it risks bringing back the weighing of ideas that Popper's decisive refutation was meant to replace. This is an open dispute inside the tradition, which is how a Popperian site should treat its own foundations.

4. Knowledge is physical

Knowledge is not a mental glow. It is a physical property of some patterns of matter: information that, once embodied in the right environment, tends to keep itself in existence.

That definition comes from Deutsch and is developed further in the constructor theory he works on with Chiara Marletto. It has a surprising consequence: knowledge does not need a knower. A gene for an eye carries knowledge of optics. Nobody ever thought it, yet it copies itself through the generations because it works. A theory in a textbook persists for the same reason. The ink and the DNA both encode something about the world that a random pattern would not.

The two kinds of knowledge grow at very different speeds. Genes improve by blind variation and selection, and natural selection keeps or kills whole organisms over millions of years. Ideas improve by conjecture and criticism, and a person can kill a bad idea in an afternoon without dying. Popper's way of putting it was that we can let our theories die in our place.

This is where animal minds come in. On Deutsch's account an animal runs knowledge it inherited: algorithms written by selection, sometimes extremely sophisticated, but unable to solve a problem that evolution has not already solved. A crow that bends a wire into a hook is running a program, not inventing one. That is the line between a creature and a person.

What the line settles is narrower than it first appears. Explanatory creativity marks a person. It does not obviously mark the edge of experience. A dog without a single new idea may still hurt. The mind site left that question open, and this site keeps it open, because a theory of knowledge is not a theory of awareness.

5. Closing the loop

Time, mind and free will are three faces of one fact: the growth of knowledge, which no one can predict. This is where the three sister sites meet.

Time. In the picture that Deutsch defends, and that Don Page and William Wootters made precise for quantum theory, nothing flows. There is a timeless whole of snapshots, and "time" is the pattern of correlations between them. What makes one snapshot later than another is that it holds records of the earlier one. A fossil, a memory or a written theory is knowledge carried forward, so the arrow of time and the growth of knowledge are the same arrow. A universe with no records would have no "before". The time site follows that argument step by step.

Mind. A mind is a system that models the world and itself. A person is a mind that can create new explanations rather than only run inherited ones. On that account the mark of personhood is not feeling, intelligence or language. It is the ability to originate knowledge.

Free will. In the preface to The Poverty of Historicism (1957), Popper gives a short argument. The course of history depends on the growth of knowledge. We cannot predict what we will know in future, because predicting a discovery means making it now. So nobody can predict the course of history. The same argument works for a single creative mind. Its choices depend on explanations it has not created yet, so no observer, however well informed, can compute them in advance. Nothing in physics breaks for this to happen. The unpredictability comes from what knowledge is, not from a gap in the laws.

That is the freedom worth wanting. It does not need an uncaused cause or an exemption from physics. It is a process that responds to reasons, revises itself under criticism, and produces something that was not already in its inputs.

6. Can machines do it?

Today's AI systems are powerful recombiners, not yet creators of explanation. That is a conjecture, and it gets its test below.

In his essay "Creative Blocks" (Aeon, 2012), Deutsch distinguishes AI from AGI. AI performs tasks set by others and is judged by how well it does them. AGI would be a person: a program that creates explanations and could, among other things, decline the task. On his view the gap is not about computing power. It is philosophical. We will not program creativity until we understand it, and approaches built on induction start from a wrong theory of where knowledge comes from.

Large language models sharpen the question. They learn from more text than any person could read and produce fluent answers, working code and plausible arguments. Is that conjecture, or recombination at an extraordinary scale? Recombination can look creative, because a new arrangement of old ideas can still surprise. The distinction that matters is whether a system can produce an explanation that is not latent in its inputs, and then criticise and improve it by itself.

A Popperian page has to stake a claim it could lose. This one is the site's own, not Deutsch's.

The question connects back to free will. If such a system appeared, Popper's argument would apply to it as it does to us. Its behaviour would be unpredictable for the same reason ours is, and the line between a tool and a person would no longer follow the line between silicon and flesh.

7. The beginning of infinity

If knowledge comes from conjecture and criticism, progress has no ceiling, and optimism stops being a mood and becomes a claim about epistemology.

Deutsch compresses it into a motto: "Problems are inevitable. Problems are soluble." Every solution creates new problems, because every theory has errors we have not yet found. But any problem that the laws of physics do not forbid us to solve can be solved, given the right knowledge. So every evil, whether disease, poverty or ignorance, comes from not yet knowing enough.

The idea is easy to caricature. It does not say that things will turn out well. Civilisations have stalled and collapsed when they stifled criticism. It says that nothing outside us sets a limit, and the only thing that can stop the growth of knowledge is a way of life that punishes error-correction.

That is why the series ends here. A timeless universe of snapshots sounds bleak until you notice what some of those snapshots contain: records, theories and arguments that never existed before. Minds sound mechanical until you notice that some of them create what no mechanism could predict. Free will sounds like an illusion until you see that it is the same unpredictability, seen from the inside.

In short

• Knowledge does not come from observation. It begins as conjecture.
• Observation and argument cannot prove a theory, but they can refute one. Science lives on that asymmetry.
• Among the theories that survive, prefer good explanations: those that are hard to vary.
• Knowledge is physical. It is information that keeps itself in existence, in genes as well as in minds.
• People are the systems that create new explanations. That is why their future cannot be predicted, and why it is open.
• Whether a machine can do the same is an open question, and this page stakes a conjecture on it.
• Nothing but a lack of knowledge stands in the way of progress, and nothing limits knowledge but the suppression of criticism.