Power seeking:
Agents: Open-ended goals may motivate secret plans to gain power.
Agencies: Bounded tasks include time and budget constraints.
I think the methods for accumulating resources is one of the key questions of interest that an AI safety approach needs to handle. For instance:
We might want to apply AI to sales and marketing. How does Open Agencies address questions such as how you know what to tell customers?
We might want to apply AI to allocating budgets within a company. How does Open Agencies address questions such as how you know which projects to allocate resources to?
We might want to apply AI to design political messaging for a political party. How does Open Agencies address questions such as how you know what political promises you need to make?
We might want to apply AI to design molecular nanotechnology. How does Open Agencies address questions such as how you know how to power the nanobots, how to produce large quantities of nanobots, how to decide what waste products are acceptable to leak, etc.?
I think the methods for accumulating resources is one of the key questions of interest that an AI safety approach needs to handle. For instance:
We might want to apply AI to sales and marketing. How does Open Agencies address questions such as how you know what to tell customers?
We might want to apply AI to allocating budgets within a company. How does Open Agencies address questions such as how you know which projects to allocate resources to?
We might want to apply AI to design political messaging for a political party. How does Open Agencies address questions such as how you know what political promises you need to make?
We might want to apply AI to design molecular nanotechnology. How does Open Agencies address questions such as how you know how to power the nanobots, how to produce large quantities of nanobots, how to decide what waste products are acceptable to leak, etc.?