“It’s not that we need formalizations per se; it’s that we need gears-level understanding. We need to have some understanding of why e.g. modularity shows up in trained/evolved systems, what precisely makes that happen. The need for gears-level understanding, in turn, stems from the need for generalizability.”
In this case the most straightforward approach would be to simply derive e.coli behaviour from basic quantum chemistry, as that is the closest field where fully deterministic simulations are possible, and verifiable.
The gap between simulating hydrogen-oxygen reactions and simulating e.coli moving around in a petri dish is about the same order of magnitude as the gap between simulating e.coli moving around in a petri dish and simulating human level intelligences in a complex society.
However the clear difficulty here is that it will likely take more energy than is available in the Milky Way to carry out such simulations for a reasonable time span (from the quantum level on up).
Thus any plausible approach can not be an entirely straightforward derivation from first principles, they will invariably produce some quantity of ‘hand-waving’, ‘fudge factors’, preconceptions, biases, etc., that everyone would carry into, or desire from, such endeavours.
i.e. the Gears will only be so at a distance, and will fall apart upon sufficiently motivated close inspection.
Furthermore, possible courses of actions and the resultant consequences are intertwined to an extent, due to the human condition, that it does not seem likely we can presume a greater intelligence will not be able to find some loophole to exploit as if it were a black box anyway. Because ultimately all human understanding is reliant on ’black boxes’ somewhere along the way.
The reason I am writing this is because it would be wise to not get your hopes up that all possible schemes of an ‘unfriendly AI’ can be foiled through such methods, or a combination of such methods.
“It’s not that we need formalizations per se; it’s that we need gears-level understanding. We need to have some understanding of why e.g. modularity shows up in trained/evolved systems, what precisely makes that happen. The need for gears-level understanding, in turn, stems from the need for generalizability.”
In this case the most straightforward approach would be to simply derive e.coli behaviour from basic quantum chemistry, as that is the closest field where fully deterministic simulations are possible, and verifiable.
The gap between simulating hydrogen-oxygen reactions and simulating e.coli moving around in a petri dish is about the same order of magnitude as the gap between simulating e.coli moving around in a petri dish and simulating human level intelligences in a complex society.
However the clear difficulty here is that it will likely take more energy than is available in the Milky Way to carry out such simulations for a reasonable time span (from the quantum level on up).
Thus any plausible approach can not be an entirely straightforward derivation from first principles, they will invariably produce some quantity of ‘hand-waving’, ‘fudge factors’, preconceptions, biases, etc., that everyone would carry into, or desire from, such endeavours.
i.e. the Gears will only be so at a distance, and will fall apart upon sufficiently motivated close inspection.
Furthermore, possible courses of actions and the resultant consequences are intertwined to an extent, due to the human condition, that it does not seem likely we can presume a greater intelligence will not be able to find some loophole to exploit as if it were a black box anyway. Because ultimately all human understanding is reliant on ’black boxes’ somewhere along the way.
The reason I am writing this is because it would be wise to not get your hopes up that all possible schemes of an ‘unfriendly AI’ can be foiled through such methods, or a combination of such methods.