The following essay was published 20 years ago (January, 2006) on my blog The Technium. I edited the intro here, but the speculations are basically unchanged.
In a recent podcast from Dwarkesh Patel, Terence Tao talked briefly about how AI is turning the scientific method on its head, whereby the data comes first (data collection from experiments or other sources), and hypotheses come second, generated by AI as it spins through the collected data.
I think this fits into your category "Theory-less Pattern Augmentation".
Wasn't that always the case? People made observations about nature, formulated hunches about potential patterns and/or explanations, and speculated about future implications. The application of AI or other analytic tools to large data sets requires a question to be addressed. Otherwise, as someone said, an undefined question has an infinite number of answers!
Kevin Kelly recently republished his essay speculating on the future of the scientific method. It's a smart list. Compiled negative results. Triple blind experiments. AI proofs. Evolutionary search. Zillionics. Twenty years on, a surprising number of his predictions have arrived or are arriving.
But Kelly wrote a post about the scientific method — lowercase s, lowercase m — and never touched Science. Capital S. The institution. The living thing. His entire essay is about how we might upgrade the engine. He never asks who's driving, where it's going, or who it's running over on the way.
That's the blind spot. And it's not just Kelly's. It's the signature blind spot of the entire Wired/Long Now/Santa Fe worldview: an extraordinary capacity to see and predict tools and an extraordinary inability to see power.
Here's what I mean.
Every one of Kelly's predictions — better data, smarter algorithms, adaptive experiments, wiki-science — improves the method. The technique. The procedure by which we extract reliable knowledge from reality. Fine. But Science is not a method. Science is a living institution. It recruits attention. It enforces boundaries. It accumulates memory across generations. It reproduces itself through training, funding, publication, and peer review. It persists beyond any individual. And like every living institution, it depends on resources it does not control: human cognition, public trust, physical infrastructure, political will, and — above all — money.
The method is how Science thinks. The institution is how Science eats. And what Science eats determines what Science thinks about.
This is the half Kelly left out. Not how we improve the method — but how we govern the institution. How we get Science to serve People instead of Markets.
Because right now, the direction of scientific inquiry is overwhelmingly set by the institutions that fund it. Pharmaceutical companies fund drug research that serves quarterly earnings. Defense departments fund physics that serves weapons programs. Tech platforms fund AI research that serves engagement metrics. The method inside each of these programs might be rigorous. The experiments might be well-designed. The data might be clean. And the direction — what questions get asked, what problems get funded, what knowledge gets produced — is determined by the survival needs of the institutions writing the checks.
Kelly's upgrades make the engine more powerful. They do nothing to change where the engine is pointed.
Kuhn saw the internal version of this problem. Normal science is the institution defending its own boundaries — enforcing protocols, training participants to see what the paradigm permits, suppressing anomalies that threaten the structure. But Kuhn was describing Science's immune system. The governance problem is different and worse: it's not that Science resists internal disruption. It's that Science has been captured by external institutions whose survival needs have nothing to do with what humanity actually needs to know.
The replication crisis isn't a methodological failure. It's a governance failure. Scientists aren't faking results because the method is bad. They're cutting corners because the funding structure rewards publication volume over reliability, because career incentives select for novelty over replication, because the institutions Science depends on for survival have optimized it for output rather than integrity. You can triple-blind every experiment on earth and it won't matter if the questions being asked are chosen by the wrong people for the wrong reasons.
So here's the question Kelly should have asked: what would it look like to evolve not just the scientific method, but the governance of Science itself?
What if the direction of scientific inquiry were set not by the institutions that profit from the answers, but by the people who have to live with the consequences? What if funding structures were designed to protect feedback integrity — the capacity of Science to tell us things we don't want to hear — rather than to maximize commercially viable output? What if we treated Science the way we treat other critical public infrastructure: as something too important to be governed primarily by market logic?
This isn't a utopian fantasy. It's an institutional design problem. We've solved versions of it before — central banks, public utilities, judicial independence — by creating structures that are publicly funded but operationally insulated from the short-term incentives of the institutions around them. We know how to build firewalls between funding and direction. We just haven't applied that knowledge to Science, because the Kellys of the world keep us dazzled by the tools while the governance rots underneath.
Kelly closes by saying today's scientific method will seem "primitive and crude" in forty years. He's probably right. But not because we'll have better algorithms. Because we'll finally understand that the method was never the thing that needed saving. The method is fine. It's the institution that's sick. And the cure isn't a better microscope. It's answering a question that Kelly, for all his speculative imagination, never thought to ask:
Good stuff. I would call the examples Brute Force Science. Similar to Edison’s approach to finding the perfect filament for incandescent lights, try thousands of iterations until you hit on the right one. No theory just grinding away. Nothing grinds better than a computer. But is that the scientific method? I think about Karl Popper’s explanation for the scientific method; a theory is held and then attempts are made to falsify the theory. If the theory holds up then the theory becomes stronger. But always conditional.
Popper's is the cannonical take on the method. What I am suggestions are expansions of that method. For instance you do it without a theory. It is still science (if it works) because it is a method by which we know things to be true.
That's an amazing list from 20 years ago.
In a recent podcast from Dwarkesh Patel, Terence Tao talked briefly about how AI is turning the scientific method on its head, whereby the data comes first (data collection from experiments or other sources), and hypotheses come second, generated by AI as it spins through the collected data.
I think this fits into your category "Theory-less Pattern Augmentation".
Wasn't that always the case? People made observations about nature, formulated hunches about potential patterns and/or explanations, and speculated about future implications. The application of AI or other analytic tools to large data sets requires a question to be addressed. Otherwise, as someone said, an undefined question has an infinite number of answers!
What Kevin Kelly Can't See
Kevin Kelly recently republished his essay speculating on the future of the scientific method. It's a smart list. Compiled negative results. Triple blind experiments. AI proofs. Evolutionary search. Zillionics. Twenty years on, a surprising number of his predictions have arrived or are arriving.
But Kelly wrote a post about the scientific method — lowercase s, lowercase m — and never touched Science. Capital S. The institution. The living thing. His entire essay is about how we might upgrade the engine. He never asks who's driving, where it's going, or who it's running over on the way.
That's the blind spot. And it's not just Kelly's. It's the signature blind spot of the entire Wired/Long Now/Santa Fe worldview: an extraordinary capacity to see and predict tools and an extraordinary inability to see power.
Here's what I mean.
Every one of Kelly's predictions — better data, smarter algorithms, adaptive experiments, wiki-science — improves the method. The technique. The procedure by which we extract reliable knowledge from reality. Fine. But Science is not a method. Science is a living institution. It recruits attention. It enforces boundaries. It accumulates memory across generations. It reproduces itself through training, funding, publication, and peer review. It persists beyond any individual. And like every living institution, it depends on resources it does not control: human cognition, public trust, physical infrastructure, political will, and — above all — money.
The method is how Science thinks. The institution is how Science eats. And what Science eats determines what Science thinks about.
This is the half Kelly left out. Not how we improve the method — but how we govern the institution. How we get Science to serve People instead of Markets.
Because right now, the direction of scientific inquiry is overwhelmingly set by the institutions that fund it. Pharmaceutical companies fund drug research that serves quarterly earnings. Defense departments fund physics that serves weapons programs. Tech platforms fund AI research that serves engagement metrics. The method inside each of these programs might be rigorous. The experiments might be well-designed. The data might be clean. And the direction — what questions get asked, what problems get funded, what knowledge gets produced — is determined by the survival needs of the institutions writing the checks.
Kelly's upgrades make the engine more powerful. They do nothing to change where the engine is pointed.
Kuhn saw the internal version of this problem. Normal science is the institution defending its own boundaries — enforcing protocols, training participants to see what the paradigm permits, suppressing anomalies that threaten the structure. But Kuhn was describing Science's immune system. The governance problem is different and worse: it's not that Science resists internal disruption. It's that Science has been captured by external institutions whose survival needs have nothing to do with what humanity actually needs to know.
The replication crisis isn't a methodological failure. It's a governance failure. Scientists aren't faking results because the method is bad. They're cutting corners because the funding structure rewards publication volume over reliability, because career incentives select for novelty over replication, because the institutions Science depends on for survival have optimized it for output rather than integrity. You can triple-blind every experiment on earth and it won't matter if the questions being asked are chosen by the wrong people for the wrong reasons.
So here's the question Kelly should have asked: what would it look like to evolve not just the scientific method, but the governance of Science itself?
What if the direction of scientific inquiry were set not by the institutions that profit from the answers, but by the people who have to live with the consequences? What if funding structures were designed to protect feedback integrity — the capacity of Science to tell us things we don't want to hear — rather than to maximize commercially viable output? What if we treated Science the way we treat other critical public infrastructure: as something too important to be governed primarily by market logic?
This isn't a utopian fantasy. It's an institutional design problem. We've solved versions of it before — central banks, public utilities, judicial independence — by creating structures that are publicly funded but operationally insulated from the short-term incentives of the institutions around them. We know how to build firewalls between funding and direction. We just haven't applied that knowledge to Science, because the Kellys of the world keep us dazzled by the tools while the governance rots underneath.
Kelly closes by saying today's scientific method will seem "primitive and crude" in forty years. He's probably right. But not because we'll have better algorithms. Because we'll finally understand that the method was never the thing that needed saving. The method is fine. It's the institution that's sick. And the cure isn't a better microscope. It's answering a question that Kelly, for all his speculative imagination, never thought to ask:
Who should Science belong to?
Science belongs to whomever does it, anywhere in the world.
Can't argue with you there...Science belongs to doers of science...a loop that can't be cracked...
Good stuff. I would call the examples Brute Force Science. Similar to Edison’s approach to finding the perfect filament for incandescent lights, try thousands of iterations until you hit on the right one. No theory just grinding away. Nothing grinds better than a computer. But is that the scientific method? I think about Karl Popper’s explanation for the scientific method; a theory is held and then attempts are made to falsify the theory. If the theory holds up then the theory becomes stronger. But always conditional.
Popper's is the cannonical take on the method. What I am suggestions are expansions of that method. For instance you do it without a theory. It is still science (if it works) because it is a method by which we know things to be true.