Many people worry about a rogue AI with maybe self-replicating robots taking over the world, sometimes from a hypothetical basement, but I wonder how much of it is a straightforward intelligence or engineering problem that we know are solvable, and how much depends on sci-fi level technologies that we don’t know whether they are feasible even with superhuman general problem-solving algorithms, for an AI starting with realistic amounts of knowledge and compute. I think arguing whether AI can realistically achieve the sci-fi feats of real-life engineering (or whether they are even physically possible, as with grey goo type nanobots) isn’t very productive. Instead, as a tangible warning argument or upper bound if nothing else, it can be helpful to try and estimate what would an AI need initially to be able to survive and expand, and how conspicuous it would be, assuming it only has access to current science and technology (and slightly superhuman but not magical levels of engineering and problem-solving). For example, in the form of a game where people can play the role of a rogue AI.
Imagine a LARP where a team of experts in the role of a rogue AI can actually try to survive, get resources and build self-sustaining industries with the help of current AIs and realistic robotics, and volunteers can play the role of some robots if advanced robots are not available. The team of humans can meet in a workshop or other dedicated site, with modest starting resources, playing as an AI and its robot fleet, and can do whatever an AI might do that does not involve technologies or experimental results we are not sure of (that require some unknown real-world empirical questions working out a certain way, and don’t just depend on better general intelligence and faster processing). For example, making robots with human level general tool manipulation but higher precision and speed of real robotic hardware, or hacking/persuading some lab to synthesize some known virus (simulated), are fair game (volunteers can play these roles), but 99% efficient solar panels or hypothetical nanobots capable of wiping out humans silently are not. They may use current AIs and any available machines, simulate “hacking” systems (with owner’s permission), earn money and buy items online, and pretend to scavenge electronics, parts, materials etc (buy or get free used items). There can be scenarios with various difficulties: from an automated warehouse with plenty of of electronics, solar panels and supplies and its own data center, to a basement with nothing but a computer, some maker’s tools and a few hobbyist-grade robots.
Many people worry about a rogue AI with maybe self-replicating robots taking over the world, sometimes from a hypothetical basement, but I wonder how much of it is a straightforward intelligence or engineering problem that we know are solvable, and how much depends on sci-fi level technologies that we don’t know whether they are feasible even with superhuman general problem-solving algorithms, for an AI starting with realistic amounts of knowledge and compute. I think arguing whether AI can realistically achieve the sci-fi feats of real-life engineering (or whether they are even physically possible, as with grey goo type nanobots) isn’t very productive. Instead, as a tangible warning argument or upper bound if nothing else, it can be helpful to try and estimate what would an AI need initially to be able to survive and expand, and how conspicuous it would be, assuming it only has access to current science and technology (and slightly superhuman but not magical levels of engineering and problem-solving). For example, in the form of a game where people can play the role of a rogue AI.
Imagine a LARP where a team of experts in the role of a rogue AI can actually try to survive, get resources and build self-sustaining industries with the help of current AIs and realistic robotics, and volunteers can play the role of some robots if advanced robots are not available. The team of humans can meet in a workshop or other dedicated site, with modest starting resources, playing as an AI and its robot fleet, and can do whatever an AI might do that does not involve technologies or experimental results we are not sure of (that require some unknown real-world empirical questions working out a certain way, and don’t just depend on better general intelligence and faster processing). For example, making robots with human level general tool manipulation but higher precision and speed of real robotic hardware, or hacking/persuading some lab to synthesize some known virus (simulated), are fair game (volunteers can play these roles), but 99% efficient solar panels or hypothetical nanobots capable of wiping out humans silently are not. They may use current AIs and any available machines, simulate “hacking” systems (with owner’s permission), earn money and buy items online, and pretend to scavenge electronics, parts, materials etc (buy or get free used items). There can be scenarios with various difficulties: from an automated warehouse with plenty of of electronics, solar panels and supplies and its own data center, to a basement with nothing but a computer, some maker’s tools and a few hobbyist-grade robots.