Peano arithmetic is a way of formulating all possible computations, so among the capable things in there, certainly some and possibly most won’t cause good outcomes if given influence in the physical world (this depends on how observing human data tends to affect the simpler learners, whether there is some sort of alignment by default). Certainly Peano arithmetic doesn’t have any biases specifically helpful for alignment, and it’s plausible that there is no alignment by default in any sense, so it’s necessary to have such a bias to get a good outcome.
But also enumerating things from Peano arithmetic until something capable is encountered likely takes too much compute to be a practical concern. And anything that does find something capable won’t be meaningfully descibed as enumerating things from Peano arithmetic, there will be too much structure in the way such search/learning is performed that’s not about Peano arithmetic.
Peano arithmetic is a way of formulating all possible computations, so among the capable things in there, certainly some and possibly most won’t cause good outcomes if given influence in the physical world (this depends on how observing human data tends to affect the simpler learners, whether there is some sort of alignment by default). Certainly Peano arithmetic doesn’t have any biases specifically helpful for alignment, and it’s plausible that there is no alignment by default in any sense, so it’s necessary to have such a bias to get a good outcome.
But also enumerating things from Peano arithmetic until something capable is encountered likely takes too much compute to be a practical concern. And anything that does find something capable won’t be meaningfully descibed as enumerating things from Peano arithmetic, there will be too much structure in the way such search/learning is performed that’s not about Peano arithmetic.