Thank you! We actually tried to write one that was much closer to a vision we endorse! The TLDR overview was something like:
Both the US and Chinese leading AGI projects stop in response to evidence of egregious misalignment.
Sign a treaty to pause smarter-than-human AI development, with compute based enforcement similar to ones described in our live scenario, except this time with humans driving the treaty instead of the AI.
Take time to solve alignment (potentially with the help of the AIs). This period could last anywhere between 1-20 years. Or maybe even longer! The best experts at this would all be brought in to the leading project, many different paths would be pursued (e.g. full mechinterp, Davidad moonshots, worst case ELK, uploads, etc).
Somehow, a do a bunch of good governance interventions on the AGI project (e.g. transparency on use of the AGIs, no helpful only access to any one. party, a formal governance structure where a large number of diverse parties all are represented.).
This culminates with aligning an AI “in the best interests of humanity” whatever that means, using a process where a large fraction of humanity is engaged and has some power to vote. This process might look something like giving each human some of the total resources of space and then doing lots of bargaining to find all the positive sum trades, with some rules against blackmail / using your resources to cause immense harm.
Unfortunately, it was hard to write this out in a way that felt realistic.
The next major project I focus on is likely going to be focusing on thinking through the right governance interventions here to make that happen. I’m probably not going to do this in scenario format (and instead something closer to normal papers and blog posts), but would be curious for thoughts.
Big +1 on adding this and/or finding another high-quality way of depicting what the ideal scenario would look like. I think many people think and feel that the world is in a very dire state to an extent that leads to hopelessness and fatalism. Articulating clear theories of victory that enable people to see the better future they can contribute towards will be an important part of avoiding this scenario.
:) it’s good to know that you tried this. Because on your way trying to make it realistic, you might think of a lot of insights to solving the unrealisticness problems.
Thank you for the summary. From this summary, I sorta see why it might not work as well as a story. Regulation and governance isn’t very exciting a narrative. And big changes in strategy and attitude inevitably sound unrealistic, even if they aren’t unrealistic. E.g. if someone predicted that Europe will simply accept the fact its colonies want independence, or that the next Soviet leader will simply allow his constituent republics to break away, they would be laughed out of the room. Even though their predictions will turn out accurate.
Maybe in your disclaimer, you can point out that this summary you just wrote, is what you would actually recommend (instead of what the characters in your story did).
Yes, papers and blog posts are less entertaining of us but more pragmatic for you.
Thank you! We actually tried to write one that was much closer to a vision we endorse! The TLDR overview was something like:
Both the US and Chinese leading AGI projects stop in response to evidence of egregious misalignment.
Sign a treaty to pause smarter-than-human AI development, with compute based enforcement similar to ones described in our live scenario, except this time with humans driving the treaty instead of the AI.
Take time to solve alignment (potentially with the help of the AIs). This period could last anywhere between 1-20 years. Or maybe even longer! The best experts at this would all be brought in to the leading project, many different paths would be pursued (e.g. full mechinterp, Davidad moonshots, worst case ELK, uploads, etc).
Somehow, a do a bunch of good governance interventions on the AGI project (e.g. transparency on use of the AGIs, no helpful only access to any one. party, a formal governance structure where a large number of diverse parties all are represented.).
This culminates with aligning an AI “in the best interests of humanity” whatever that means, using a process where a large fraction of humanity is engaged and has some power to vote. This process might look something like giving each human some of the total resources of space and then doing lots of bargaining to find all the positive sum trades, with some rules against blackmail / using your resources to cause immense harm.
Unfortunately, it was hard to write this out in a way that felt realistic.
The next major project I focus on is likely going to be focusing on thinking through the right governance interventions here to make that happen. I’m probably not going to do this in scenario format (and instead something closer to normal papers and blog posts), but would be curious for thoughts.
Big +1 on adding this and/or finding another high-quality way of depicting what the ideal scenario would look like. I think many people think and feel that the world is in a very dire state to an extent that leads to hopelessness and fatalism. Articulating clear theories of victory that enable people to see the better future they can contribute towards will be an important part of avoiding this scenario.
:) it’s good to know that you tried this. Because on your way trying to make it realistic, you might think of a lot of insights to solving the unrealisticness problems.
Thank you for the summary. From this summary, I sorta see why it might not work as well as a story. Regulation and governance isn’t very exciting a narrative. And big changes in strategy and attitude inevitably sound unrealistic, even if they aren’t unrealistic. E.g. if someone predicted that Europe will simply accept the fact its colonies want independence, or that the next Soviet leader will simply allow his constituent republics to break away, they would be laughed out of the room. Even though their predictions will turn out accurate.
Maybe in your disclaimer, you can point out that this summary you just wrote, is what you would actually recommend (instead of what the characters in your story did).
Yes, papers and blog posts are less entertaining of us but more pragmatic for you.