[ignoring misalignment risk] I think it’s pretty unlikely that by default ASI built by great powers will end up sufficiently empowering the middle powers. I expect that by default the ASI-creators will try and make the ASI follow their orders or the orders of their government, rather than something that upholds the global order.
I think that middle powers could potentially use verification mechanisms to help ensure that an ASI was being trained with a model spec that includes respecting the interests of middle powers. Overall though, I expect getting it to be very hard to actually get good assurance that an AI isn’t being trained in a way that middle powers don’t like. And therefore, I guess that it would be better to stop/slow.
I do agree that a reasonable ask of the middle powers is asking for some control over the model spec, but I think it’s probably better for the main ask to be about something that the middle powers have direct influence over, e.g., AI hardware.
If we ignore misalignment risk, why are you envisioning a world where ASI acts like a benevolent dictator, such that it is up the ASI’s own values and decisions whether the middle powers have significant influence over the world? Rather than human customers and users of ASI-powered products and services around the world creating a balance of power (akin to how AI works today)? I think alignment should include be “don’t be a dictator”, not just “don’t be an evil dictator”.
I expect that by default the ASI-creators will try and make the ASI follow their orders or the orders of their government, rather than something that upholds the global order.
Could you be more specific about this contrast between “orders of the US government” and “upholding the global order”? Is the idea that if the US government controls the model spec of an ASI machine, they’ll use that to overthrow or replace the government of the Netherlands or Korea? That they’ll use ASI to impoverish the Dutch or Korean populations? These are the kinds of things being referred to by “insofar as the ASI built by great powers isn’t aligned to the interest of middle powers.”
I don’t understand the relationship, i.e. I don’t know what you’re referring to by “America first” policies that involve impoverishing or overturning the Korean or Dutch governments. If you mean the specific “America First” foreign-policy slogan, that’s in fact a non-interventionist policy.
Why would the superpower allow any other country to keep more of their wealth/government/power than whatever is optimal from the perspective of the superpower? If they wouldn’t, it feels like that would put a lot of downward pressure on all of these, especially power/government. Wealth as well at least in a relative sense, though perhaps not in an absolute sense. Does your intuition differ?
I wouldn’t necessarily expect overthrown governments, as most would just realize they have no choice but to accede to the demands of the superpower. And in most cases I wouldn’t expect the superpower to let any nation get so far out of control that overthrowing their government would be necessary. But surely the rare few cases where this does materialize would indeed be overthrown?
Do you consider the current America First administration to be non-interventionist? From my perspective, it is quite obviously very interventionist, very unilateralist, and very nationalist. Imagining something like this administration, but superpowered, I find it hard to understand the claim that this would not disempower middle powers.
[ignoring misalignment risk] I think it’s pretty unlikely that by default ASI built by great powers will end up sufficiently empowering the middle powers. I expect that by default the ASI-creators will try and make the ASI follow their orders or the orders of their government, rather than something that upholds the global order.
I think that middle powers could potentially use verification mechanisms to help ensure that an ASI was being trained with a model spec that includes respecting the interests of middle powers. Overall though, I expect getting it to be very hard to actually get good assurance that an AI isn’t being trained in a way that middle powers don’t like. And therefore, I guess that it would be better to stop/slow.
I do agree that a reasonable ask of the middle powers is asking for some control over the model spec, but I think it’s probably better for the main ask to be about something that the middle powers have direct influence over, e.g., AI hardware.
If we ignore misalignment risk, why are you envisioning a world where ASI acts like a benevolent dictator, such that it is up the ASI’s own values and decisions whether the middle powers have significant influence over the world? Rather than human customers and users of ASI-powered products and services around the world creating a balance of power (akin to how AI works today)? I think alignment should include be “don’t be a dictator”, not just “don’t be an evil dictator”.
Could you be more specific about this contrast between “orders of the US government” and “upholding the global order”? Is the idea that if the US government controls the model spec of an ASI machine, they’ll use that to overthrow or replace the government of the Netherlands or Korea? That they’ll use ASI to impoverish the Dutch or Korean populations? These are the kinds of things being referred to by “insofar as the ASI built by great powers isn’t aligned to the interest of middle powers.”
Why would you expect the American government not to pursue America first policies?
I don’t understand the relationship, i.e. I don’t know what you’re referring to by “America first” policies that involve impoverishing or overturning the Korean or Dutch governments. If you mean the specific “America First” foreign-policy slogan, that’s in fact a non-interventionist policy.
Why would the superpower allow any other country to keep more of their wealth/government/power than whatever is optimal from the perspective of the superpower? If they wouldn’t, it feels like that would put a lot of downward pressure on all of these, especially power/government. Wealth as well at least in a relative sense, though perhaps not in an absolute sense. Does your intuition differ?
I wouldn’t necessarily expect overthrown governments, as most would just realize they have no choice but to accede to the demands of the superpower. And in most cases I wouldn’t expect the superpower to let any nation get so far out of control that overthrowing their government would be necessary. But surely the rare few cases where this does materialize would indeed be overthrown?
Do you consider the current America First administration to be non-interventionist? From my perspective, it is quite obviously very interventionist, very unilateralist, and very nationalist. Imagining something like this administration, but superpowered, I find it hard to understand the claim that this would not disempower middle powers.