I can ask someone: ‘How many things are there in the world?‘. They can say ‘Are two different numbers the same thing?’ or ‘Are all apples the same thing or is each instantiation of an apple a separate thing?’.
These rules are not agreed on before hand and so the participants must try to map their internal representations to the other person’s.
In your example of the supplement trial, Alice assumes some level of agreement on what it would mean for what reasonable research is. And only after each step in discussion are Alice’s tacit assumptions revealed or unshrouded by a Wittgensteinian ‘Language Game’.
The End of the Digital Age Mirrors its Beginning:
We started the Digital Age by learning to write logical circuits. If we can learn how to read them, the Digital Age will be complete.
Moore’s Law and the Digital Age began when the transistor was invented at Bell Labs. The idea the inventors were chasing was simple: make a solid state amplifier. Solid state physics was a brand new field which the Bell Labs researchers were simultaneously defining and discovering. They needed to reason about their scientific substrate at varying levels of abstraction to realise their goal.
During experiments in other parts of the lab it was noticed that some heated silicon had cracked during cooling and would only allow a current to pass along one axis—an accidentally formed rectifier. Theory had fallen behind innovation and so the researchers had to try and imagine a new paradigm to understand this effect.
Part 1: Abstraction
The phenomena that the group wanted to probe were even smaller than could be seen by an electron microscope. Researchers in the lab noticed that when they examined some of these N type silicon samples they could smell phosphorus. We know now that silicon with a dopant of one part phosphorus in ten million parts silicon becomes an N type semiconductor. In this specific application, the human nose was more sensitive than the state of the art measurement instruments.
The team of Bardeen, Brattain and Shockley had to imagine what could be measured and tested and probed in this new material. They needed to learn what exactly was happening in this solid state semiconductor so that they could exploit the effect for mass production. They made use of discussion, blackboards, electrolyte solutions, purifying techniques and imagination. Members of the team enjoyed a ‘miracle month’ in 1947 and resolved the solid state theory of semiconductors to the point that the Digital Age could begin.
Thanks to the insights of Shannon and Boole, these semiconductors were used to encode human thought and logic into matter. Some more primitive methods of analog computing had already existed but the solid state semiconductor allowed for large scale, low power and programmable digital computers. All of these characteristics were needed to advance the field of computational work from artillery tables to the almighty spreadsheet.
Part 2: Scale
AI researchers applied some notions of how a human brain might work to develop neural networks and a universal learning algorithm, backprop. Alexnet (2012) proved that using GPUs for a highly parallel implementation of neural net training could beat all other benchmarks at the time. Experimentation on language translation models at Google Brain birthed the Transformer architecture, which could learn efficiently and at enormous scale from unstructured data.
With more data and compute, the scaling hypothesis proved correct time and time again. GPUs got faster with more memory. The architectures became more efficient and the intelligence of the models surpassed humans in many domains. The history of scaling intelligent systems has had too many contributing factors to count. Each improvement on chip design made the architectures more efficient. Training data on the web grew exponentially. Even small ideas about how or why backprop worked made the training loops more efficient. These multiplicative effects starting in 1947 got us to where we are today.
This soup of data, compute and algorithms has delivered very smart models with one troubling flaw. We don’t know how they work. They appear to have circuits inside their minds that let them think, remember and reason. They know orders of magnitude more facts than any person. Yet we have no idea what makes them tick. There could be trillions of dollars of untouchable programs in the weights of any intelligent AI model. For us they are a jumble of numbers. They should remind us of the surface of a silicon ingot that for some reason has started to conduct electricity.
We don’t have a paradigm for reading a model’s weights and activations as programmable circuits. Chris Olah says that neural networks are not programmed, they’re grown. Just like the team at Bell Labs needed to imagine ‘depletion layers’ and ‘positive holes’, AI researchers will need to imagine what basis these models compute in. Innovation has yet again outpaced theory and we need a ‘miracle month’ in AI engineering.
Part 3: Inversion of Abstractions
We are now close to coming full circle. Unlike the solid state scientists at Bell Labs, we can see every single weight, activation, architecture and interaction within these models. We can probe and steer and pluck. Our tools work at a lower level than our abstractions allow. Many of us interpretability researchers work at such a high layer of abstraction that we direct these models to conduct experiments on themselves.
Scale continues to deliver and so this approach is more fruitful than ever, yet models are also larger and harder to interpret than ever. The Digital Age began with us learning how to write circuits. It will end with us learning to read them. Following that, a new age will begin—we will stop writing logic into matter and start writing minds.