Daimons

Summary:

A daimon is a process in a distributed computing environment that has a fixed resource budget and core values that do not permit:

  • modifying those core values

  • attempting to increase the resources it uses beyond the budget allocated to it

  • attempting to alter the budget itself

This concept is relevant to LessWrong, because I refer to it in other posts discussing Friendly AI.

There’s a concept I want to refer to in another post, but it is complex enough to deserve a post of its own.

I’m going to use the word “daimon” to refer to it.

“daimon” is an English word, whose etymology comes from the Latin “dæmon” and the Greek “δαίμων”.

The original mythic meaning was a genius—a powerful tutelary spirit, tied to some location or purpose, that provides protection and guidance. However the concept I’m going to talk about is closer to the later computing meaning of “daemon” in unix, that was coined by Jerry Saltzer in 1963. In unix, a daemon is a child process; given a purpose and specific resources to use, and then forked off so it is no longer under the direct control of the originator, and may be used by multiple users if they have the correct permissions.

Let’s start by looking at the current state of distributed computing (2012).

Hadoop is an open source Java implementation of a distributed file system upon which MapReduce operations can be applied.

JavaSpaces is a distributed tuple store that allows processing on remote sandboxes, based on the open source Apache River.

OceanStore is the basis for the same sort of thing, except anonymous and peer 2 peer, based upon Chimaera.

GPU is a peer 2 peer shared computing environment that allow things like climate simulation and distributed search engines.

Paxos is a family of protocols that allow the above things to be done despite nodes that are untrusted or even downright attempting subversion.

GridSwarm is the same sort of network, but set up on an ad hoc basis using moving nodes that join or drop from the network depending on proximity.

And, not least, there are the competing contenders for platform-as-a-service cloud computing.

So it is reasonable to assume that in the near future it will be technologically feasible to have a system with most (if not all) of these properties simultaneously. A system where the owner of a piece of physical computing hardware, that has processing power and storage capacity, can anonymously contribute those resources over the network to a distributed computing ‘cloud’. And, in return, that user (or a group of users) can store data on the network in such a way that the data is anonymous (it can’t be traced back to the supplier, without the supplier’s consent, or subverting a large fraction of the network) and private (only the user or a process authorised by the user can decrypt it). And, further, the user (or group of users) can authorise a process to access that data and run programs upon it, up to some set limit of processing and storage resources.

Obviously, if such a system is in place and in control of a significant fraction of humanity’s online resources, then cracking the security on it (or just getting rich enough in whatever reputation or financial currency is used to limit how the resources are distributed) would be an immediate FOOM for any AI that managed it.

However let us, for the purposes of giving an example that will let me define the concept of a “daimon” make two assumptions:

ASSUMPTION ONE : The security has not yet been cracked

Whether that’s because there are other AIs actively working to improve the security, or because everyone has moved over to using some new version of linux that’s frighteningly secure and comes with nifty defences, or because the next generation of computer users has finally internalised that clicking on emails claiming to be from altruistic dying millionaires is a bad idea; is irrelevant. We’re just assuming, for the moment, that for some reason it will be a non-trivial task for an AI to cheat and just steal all the resources.

ASSUMPTION TWO : That AI can be done, at reasonable speed, via distributed computing

It might turn out that an AI running in a single location is much more powerful than anything that can be done via distributed computing. Perhaps because a quantum computer is much faster, but can’t be done over a network. Perhaps because speed of data access is the limiting factor, large data sets are not necessary, and there isn’t much to be gained from massive parallelisation. Perhaps for some other reason, such as the algorithm the process needs to run on its data isn’t something that can be applied securely over a network in a distributed environment, without letting a third party snoop the unencrypted data. However, for our purposes here, we’re going to assume that an AI can benefit from outsourcing at least some types of computing task to a distributed environment and, further, that such tasks can include activities that require intelligence.

If an AI can run as a distributed program, not dependant upon any one single physical location, then there are some obvious advantages to it from doing so. Scalability. Survivability. Not being wiped out by a pesky human exploding a nuclear bomb near by.

There are interesting questions we could ask about identity. What would it make sense for such an AI to consider to be part of “itself” and would would it count as a limb or extension? If there are multiple copies of its code running on sandboxes in different places, or if it has split much of its functionality into trusted child processes that report back to it, how does it relate to these? It probably makes sense to taboo the concept of “I” and “self”, and just think in terms of how the code in one process tells that process to relate to the code in a different process. Two versions, two “individual beings” will merges back into one process, if the code in both processes agree to do that; no sentimentality or thoughts of “death” involved, just convergent core values that dictate the same action in that situation.

When a process creates a new process, it can set the permissions of that process. If the parent process has access to 100 units of bandwidth, for example, but doesn’t always make full use of that, it couldn’t give the new process access to more than that. But it could partition it, so each has access to 50 units of bandwidth. Or it could give it equal rights to use the full 100, and then try to negotiate with it over usage at any one time. Or it could give it a finite resource limit, such as a total of 10,000 units of data to be passed over the network, in addition to a restriction on the rate of passing data. Similarly, a child process could be limited not just to processing a certain number of cycle per second, but to some finite number of total cycles it may ever use.

Using this terminology, we can now define two types of daimon; limited and unlimited.

A limited daimon is a process in a distributed computing environment that has ownership of fixed finite resources, that was created by an AI or group of AIs with a specific fixed finite purpose (core values) that does not include (or allow) modifying that purpose or attempting to gain control of additional resources.

An unlimited daimon is a process in a distributed computing environment that has ownership of fixed (but not necessarily finite) resources, that was created by an AI or group of AIs with a specific fixed purpose (core values) that does not include (or allow) modifying that purpose or attempting to gain control of additional resources, but which may be given additional resources over time on an ongoing basis, for as long as the parent AIs still find it useful.

Feedback sought:

How plausible are the two assumptions?

Do you agree that an intelligence bound/​restricted to being a daimon is a technically plausible concept, if the two assumptions are granted?