While I agree that the potential for AI (we probably need a better term than LLMs or transformers as multimodal models with evolving architectures grow beyond those terms) in exploring less testable topics as more testable is quite high, I’m not sure the air gapping on information can be as clean as you might hope.
Does the AI generating the stories of Napoleon’s victory know about the historical reality of Waterloo? Is it using something like SynthID where the other AI might inadvertently pick up on a pattern across the stories of victories distinct from the stories preceding it?
You end up with a turtles all the way down scenario in trying to control for information leakage with the hopes of achieving a threshold that no longer has impact on the result, but given we’re probably already seriously underestimating the degree to which correlations are mapped even in today’s models I don’t have high hopes for tomorrow’s.
I think the way in which there’s most impact on fields like history is the property by which truth clusters across associated samples whereas fictions have counterfactual clusters. An AI mind that is not inhibited by specialization blindness or the rule of seven plus or minus two and better trained at correcting for analytical biases may be able to see patterns in the data, particularly cross-domain, that have eluded human academics to date (this has been my personal research interest in the area, and it does seem like there’s significant room for improvement).
And yes, we certainly could be. If you’re a fan of cosmology at all, I’ve been following Neil Turok’s CPT symmetric universe theory closely, which started with the Baryonic asymmetry problem and has tackled a number of the open cosmology questions since. That, paired with a QM interpretation like Everett’s ends up starting to look like the symmetric universe is our reference and the MWI branches are variations of its modeling around quantization uncertainties.
(I’ve found myself thinking often lately about how given our universe at cosmic scales and pre-interaction at micro scales emulates a mathematically real universe, just what kind of simulation and at what scale might be able to be run on a real computing neural network.)
It’s going to have to.
Ilya is brilliant and seems to really see the horizon of the tech, but maybe isn’t the best at the business side to see how to sell it.
But this is often the curse of the ethically pragmatic. There is such a focus on the ethics part by the participants that the business side of things only sees that conversation and misses the rather extreme pragmatism.
As an example, would superaligned CEOs in the oil industry fifty years ago have still only kept their eye on quarterly share prices or considered long term costs of their choices? There’s going to be trillions in damages that the world has taken on as liabilities that could have been avoided with adequate foresight and patience.
If the market ends up with two AIs, one that will burn down the house to save on this month’s heating bill and one that will care if the house is still there to heat next month, there’s a huge selling point for the one that doesn’t burn down the house as long as “not burning down the house” can be explained as “long term net yield” or some other BS business language. If instead it’s presented to executives as “save on this month’s heating bill” vs “don’t unhouse my cats” leadership is going to burn the neighborhood to the ground.
(Source: Explained new technology to C-suite decision makers at F500s for years.)
The good news is that I think the pragmatism of Ilya’s vision on superalignment is going to become clear over the next iteration or two of models and that’s going to be before the question of models truly being unable to be controlled crops up. I just hope that whatever he’s going to be keeping busy with will allow him to still help execute on superderminism when the market finally realizes “we should do this” for pragmatic reasons and not just amorphous ethical reasons execs just kind of ignore. And in the meantime I think given the present pace that Anthropic is going to continue to lay a lot of the groundwork on what’s needed for alignment on the way to superalignment anyways.