Decreasing this communication bandwidth might be a useful way to increase the interpretability of a population AGI.
On one hand, there would be an effect where reduced bandwidth encouraged the AI’s to focus on the most important pieces of information. If the AI’s have 1 bit of really important info, and gigabytes of slightly useful info to send to each other, then you know that if you restrict the bandwidth to 1 bit, that’s the important info.
On the other hand, perfect compression leaves data that looks like noise unless you have the decompression algorithm. If you limit the bandwidth of messages, the AIs will compress the messages until the recipient can’t predict the next bit with much more than 50% accuracy. Cryptoanalysis often involves searching for regular patterns in the coded message, and a regular patterns are an opportunity for compression.
But the concomitant lack of flexibility is why it’s much easier to improve our coordination protocols than our brain functionality.
There are many reasons why human brains are hard to modify that don’t apply to AI’s. I don’t know how easy or hard it would be to modify the internal cognitive structure of an AGI, but I see no evidence here that it must be hard.
On the main substance of your argument, I am not convinced that the boundary line between a single AI and multiple AI’s carves reality at the joints. I agree that there are potential situations that are clearly a single AI, or clearly a population, but I think that a lot of real world territory is an ambiguous mixture between the two. For instance, is the end result of IDA (Iterated distillation and Amplification) a single agent or a population. In basic architecture, it is a single imitator. (maybe a single neural net) But if you assume that the distillation step has no loss of fidelity, then you get an exponentially large number of humans in HCH.
(Analogously there are some things that are planets, some that aren’t and some ambiguous icy lumps. In order to be clearer, you need to decide which icy lumps are planets. Does it depend on being round, sweeping its orbit, having a near circular orbit or what?)
Here are some different ways to make the concept clearer.
1) There are multiple AI’s with different terminal goals, in the sense that the situation can reasonably be modeled as game theoretic. If a piece of code A is modelling code B, and then A randomises its own action to stop B from predicting A, this is a partially adversarial, game theoretic situation.
2) If you took some scissors to all the cables connecting two sets of computers, so there was no route for information to get from one side to the other, then both sides would display optimisation behavior.
Suppose the paradigm was recurrent reinforcement learning agents. So each agent is a single neural net and also has some memory which is just a block of numbers. On each timestep, the memory and sensory data are fed into a neural net, and out comes the new memory and action.
AI’s can be duplicated at any moment so the structure is more branching tree of commonality.
AI moments can be.
1) Bitwise Identical
2)Predecessor and Successor states. B has the same network as A, and Mem(B) was made by running Mem(A) on some observation.
3) Share a common memory predecessor.
4) No common memory, same net.
5) One net was produced from the other by gradient decent.
6) The nets share a common gradient decent ancestor.
7) Same architecture and training environment, net started with different random seed.
8) Same architecture, different training
9) Different architecture (number of layers, size of layer, activation func ect)
Each of these can be running at the same time or different times, and on the same hardware or different hardware.
On one hand, there would be an effect where reduced bandwidth encouraged the AI’s to focus on the most important pieces of information. If the AI’s have 1 bit of really important info, and gigabytes of slightly useful info to send to each other, then you know that if you restrict the bandwidth to 1 bit, that’s the important info.
On the other hand, perfect compression leaves data that looks like noise unless you have the decompression algorithm. If you limit the bandwidth of messages, the AIs will compress the messages until the recipient can’t predict the next bit with much more than 50% accuracy. Cryptoanalysis often involves searching for regular patterns in the coded message, and a regular patterns are an opportunity for compression.
There are many reasons why human brains are hard to modify that don’t apply to AI’s. I don’t know how easy or hard it would be to modify the internal cognitive structure of an AGI, but I see no evidence here that it must be hard.
On the main substance of your argument, I am not convinced that the boundary line between a single AI and multiple AI’s carves reality at the joints. I agree that there are potential situations that are clearly a single AI, or clearly a population, but I think that a lot of real world territory is an ambiguous mixture between the two. For instance, is the end result of IDA (Iterated distillation and Amplification) a single agent or a population. In basic architecture, it is a single imitator. (maybe a single neural net) But if you assume that the distillation step has no loss of fidelity, then you get an exponentially large number of humans in HCH.
(Analogously there are some things that are planets, some that aren’t and some ambiguous icy lumps. In order to be clearer, you need to decide which icy lumps are planets. Does it depend on being round, sweeping its orbit, having a near circular orbit or what?)
Here are some different ways to make the concept clearer.
1) There are multiple AI’s with different terminal goals, in the sense that the situation can reasonably be modeled as game theoretic. If a piece of code A is modelling code B, and then A randomises its own action to stop B from predicting A, this is a partially adversarial, game theoretic situation.
2) If you took some scissors to all the cables connecting two sets of computers, so there was no route for information to get from one side to the other, then both sides would display optimisation behavior.
Suppose the paradigm was recurrent reinforcement learning agents. So each agent is a single neural net and also has some memory which is just a block of numbers. On each timestep, the memory and sensory data are fed into a neural net, and out comes the new memory and action.
AI’s can be duplicated at any moment so the structure is more branching tree of commonality.
AI moments can be.
1) Bitwise Identical
2)Predecessor and Successor states. B has the same network as A, and Mem(B) was made by running Mem(A) on some observation.
3) Share a common memory predecessor.
4) No common memory, same net.
5) One net was produced from the other by gradient decent.
6) The nets share a common gradient decent ancestor.
7) Same architecture and training environment, net started with different random seed.
8) Same architecture, different training
9) Different architecture (number of layers, size of layer, activation func ect)
Each of these can be running at the same time or different times, and on the same hardware or different hardware.