Toggle

Karma: 1,107

Toggle Nov 18, 2014, 12:12 AM
0 points
in reply to: So8res’s comment on: AI caught by a module that counterfactually doesn’t exist
This greatly clarified the distinction for me. Well done.

Toggle Nov 17, 2014, 8:45 PM
1 point
in reply to: Capla’s comment on: Can science come to understand consciousness? A problem of philosophical zombies (Yes, I know, P-zombies again.)
Well, one of the reasons that the Turing Test has lasted so long as a benchmark, despite its problems, is the central genius of holding inorganic machines to the same standards as organic ones. Notwithstanding p-zombies and some of the weirder anime shows, we’re actionably and emotionally confident in the consciousness of the humans that surround us every day. We can’t experience these consciousnesses directly, but we do care about their states in terms of both instrumental and object-level utility.

An AGI presents new challenges, but we’ve already demonstrated a basic willingness to treat ambulatory meat sacks as valuable beings with an internal perspective. By assigning the same sort of ‘conscious’ label to a synthetic being who nonetheless has a similar set of experiential consequences in our lives, we can somewhat comfortably map our previous assumptions on to a new domain. That gives us a beachhead, and a basis for cautious expansion and observation in the much more malleable space of inorganic intelligences.
What links here?
- Capla's comment on Can science come to understand consciousness? A problem of philosophical zombies (Yes, I know, P-zombies again.) by Capla (Nov 19, 2014, 11:47 PM; 0 points)

Toggle Nov 17, 2014, 8:14 PM
1 point
on: Can science come to understand consciousness? A problem of philosophical zombies (Yes, I know, P-zombies again.)
I tend to agree with you, although I think of this as a strict limitation of empiricism.

We expand scientific knowledge through the creation of universal experiences that are (in principle) available to anyone- replicable experiments- and on the models that follow from those experiences. Consciousness, in contrast, is an experience of being onesself (and in exceptional cases, the experience of being aware that one is aware of onesself, and on down the rabbit hole). What would it mean to create the experience of being a particular entity, accessible to any observer? That question looks suspiciously like gibberish. The excellent and admirable What is it like to be a bat? develops this idea quite a bit.

But hopefully, this problem becomes fairly trivial even if it doesn’t disappear as such. We can certainly notice that human bodies tend to seem conscious and that shoes tend not to, and by developing AI from a (gulp) basically phenomenological perspective we can create a future we have every reason to believe is rich with selves and perspectives, no more a leap of faith than biological reproduction. We just won’t be able to point a consciousness-detecting machine at them and wait for it to go ‘ping’.

Toggle Nov 17, 2014, 7:37 PM
1 point
on: AI caught by a module that counterfactually doesn’t exist
A belief that M has no impact will generate extremely poor predictions of the future, iff it’s an UFAI; it’s interesting to have a prescriptive belief in which Friendly agents definitionally believe a true thing and Unfriendly agents definitionally believe a false thing.

It’s a solution that seems mostly geared towards problems of active utility deception- it prevents certain cases of an AI that deliberately games a metric. To the extent that a singleton is disingenuous about its own goals, this is a neat approach. I am a little worried that this kind of deliberate deception is stretching the orthogonality thesis further than it can plausibly go; with the right kind of careful experimentation and self-analysis, an UFAI with a proscribed falsehood might derive a literally irreconcilable set of beliefs about the world. I don’t know if that would crack it down the middle or what, or how ‘fundamental’ that crack would go in its conception of reality.

We also have the problem of not being able to specify our own values with precision. If an AI will produce catastrophic results by naively following our exact instructions, then M will presumably be using the same metric, and it will give a green light on a machine that proceeds to break down all organic molecules for additional stock market construction projects or something. I suppose that this isn’t really the sort of problem that you’re trying to solve, but it is a necessary limitation in M even though M is fairly passive.

Toggle Nov 17, 2014, 12:34 AM
7 points
in reply to: buybuydandavis’s comment on: My new paper: Concept learning for safe autonomous AI
I read this as distinguishing between (on the one hand) an externally defined set of parameters and (on the other hand) a locally emergent pattern that may be too complex to be readily understood but which nonetheless produces behavior that conforms to our expectations for the concept. Consider Google’s surprising 2012 discovery of cats.

You can teach somebody about the moon by describing it very precisely, or you can teach them about the moon by pointing to the moon and saying ‘that thing.’ In the latter case, you have specified a concept without defining it.

Toggle Nov 15, 2014, 8:11 PM
1 point
in reply to: Capla’s comment on: The “best” mathematically-informed topics?
The main problem with the Khan version is that it got huge; they were subdividing things down to about the level of an individual half-hour lesson. For someone interested in wider strategic planning, something like this would be a bit more reasonable, as long as you added in the annotations. This book is also reviewed as a good way to conceptualize the macrostructure of mathematical reasoning, although I can’t vouch for it personally.

Toggle Nov 15, 2014, 7:44 AM
6 points
on: My new paper: Concept learning for safe autonomous AI
(Your first link is broken.)

I’m happy to see MIRI-associated papers being disseminated among wider academic communities!

Toggle Nov 14, 2014, 6:57 PM
1 point
on: The “best” mathematically-informed topics?
This is an interesting approach, but I am wondering if reversing the process wouldn’t be more helpful. The problem is that you can’t just jump from propositional logic to chaos theory, since you must first learn algebra, calculus, and linear algebra. So an unordered list of mathematical subjects has some pitfalls.

If there existed some graph of dependencies in mathematical knowledge (basically a Diablo skill tree, or Khan Academy with lower resultion), we could note on each node any applications like quantum theory, music theory, and so on. This would help Capla go about their studies in a more systematic way, and also help show the rather elegant, unified structure underlying many fields of scientific inquiry.

Toggle Nov 14, 2014, 6:49 PM
1 point
in reply to: Capla’s comment on: The “best” mathematically-informed topics?
Depends whether you want to go genetics, or a more theoretical approach. But complete expert-level knowledge of the subject would probably include fitness landscapes, network theory, a basic understanding of what’s fashionably called ‘big data’ (for bioinformatics), linear algebra (including a familiarity with the dynamics of formally unsolvable systems and chaos theory), and a LOT of probability theory.

Toggle Nov 14, 2014, 6:43 PM
1 point
in reply to: Capla’s comment on: The “best” mathematically-informed topics?
The Schrodinger equation itself can be understood as simply a particular instance of integral calculus. If I recall my undergrad days correctly, you didn’t even need linear algebra. Once you know the calculus, quantum waveforms don’t require a whole lot of additional mathematical insight.

Toggle Nov 14, 2014, 2:46 AM
0 points
on: The germ of an idea
It seems very difficult to prevent an AI from routing around this limitation by going meta. It can’t actually account for its utility penalty, but can it predict the actions of a similar entity that could, and notice how helpful that would be? An AI that couldn’t predict the behavior of such an entity might be crippled in some serious fundamental ways.

Toggle Nov 12, 2014, 5:42 PM
−1 points
in reply to: Lumifer’s comment on: The Argument from Crisis and Pessimism Bias
Does this come close to being a ‘group selection’ argument?

(A brief explanation just in case that phrase isn’t common knowledge: natural selection in biology works on the level of individuals, and individual genes. For ages, a minority of biologists have been trying to show that selection can also work between groups directly even if the adaptation harms the fitness of individual members of the group. So far, they haven’t been able to convincingly show that this exists.)

The police, as an organization, cannot ‘want’ anything, since police departments are not agents capable of wanting. So the question is whether individuals within the department share those incentives to a sufficiently strong degree that they are likely to justify and coordinate around goals like “I will spread a culture of fear throughout my neighborhood.” It might be that this is the case, but it’s less obvious at the individual level than it is at the level of the organization; I could just as easily see police enlistment as selecting for people who specifically desire to help people be less afraid.

Toggle Nov 11, 2014, 6:56 PM
3 points
in reply to: fubarobfusco’s comment on: Open thread, Nov. 10 - Nov. 16, 2014
In terms of instrumental goals (‘keep society functioning’), I think these are secondary concerns. A person might believe that we are all in a perpetual state of decay; a doctor finds it necessary to understand the kidneys of a high-functioning adult so that later problems may be diagnosed and fixed. Even if decay itself might take a long time- and even if decay is ultimately inevitable- there are reasons to want to understand and replicate the rules that provide access to ‘doing okay, for now’.

Departing from my steelman for a moment, I think a more pressing concern with the model might be a poor understanding of the environmental pressures on specific societies. Homeostasis is contextual- gills are a bad organ for somebody like me to have. In the case of human societies, it’s not obvious what these environmental pressures might be, or what consequences they might have. Technology is certainly one of them, as are other human societies, as are material resources and so on, but it’s just not a well constrained problem. Does internet access alter the most stable implementations of copyright law? Does cheap birth control change the most economically viable praxis of women’s education? Would we expect Mars colonization to result from a new cold war? So I think it is not enough to show that a society endured- you have to show that the organs of that society act as solutions to currently existing problems, otherwise they are likely to multiply our miseries.

(Rejoinder to the rejoinder: Chesterton’s Fence.)

Toggle Nov 11, 2014, 6:31 PM
3 points
in reply to: Lumifer’s comment on: Superintelligence 9: The orthogonality of intelligence and goals
Potential source of misunderstanding: we do have stated ‘terminal goals’, sometimes. But these goals do not function in the same way that a paperclipper utility function maximizes paperclips- there are a very weird set of obstacles, which this site generally deals with under headings like ‘akrasia’ or ‘superstimulus’. Asking a human about their ‘terminal goal’ is roughly equivalent to the question ‘what would you want, if you could want anything?’ It’s a form of emulation.

Toggle Nov 11, 2014, 5:01 PM
9 points
in reply to: KatjaGrace’s comment on: Superintelligence 9: The orthogonality of intelligence and goals
I suspect that it hides more assumptions about the nature of intelligence than we can necessarily make at this time.

At the present moment, we are the only general intelligences around, and we don’t seem to have terminal goals as such. As biological bodies, we are constrained by evolutionary processes, and there are many ways in which human behavior actually is reducible to offspring maximization (social status games, etc.). But it doesn’t appear to be a ‘utility function’, so much as a series of strong tendencies in the face of specific stimuli. Using novel approaches like superstimuli, it’s just as easy to make an impulse’s reproductive utility drop sharply. So we have habits constrained by evolutionary forces, but not algorithmic utility in the paper clipper sense.

There is no such thing as a general intelligence with a ‘goal’ (as Bostrom defines it). There may be at some point, but it’s not real yet. And we do have non-general intelligences with goals, that’s an easy weekend coding project. But before we declare that a GI could accept any goal regardless of its strength, we should at least check to make sure that a GI can have a goal at all.

Toggle Nov 10, 2014, 7:45 PM
2 points
in reply to: NancyLebovitz’s comment on: Open thread, Nov. 10 - Nov. 16, 2014
Holistically, yes. But they are free to fail any given assignment any number of times- in fact, many would sign up to take quizzes before studying, as a preview of what the assessment would look like and a way to rapidly jump through sections they might have studied elsewhere.

Toggle Nov 10, 2014, 3:38 PM
26 points
on: Open thread, Nov. 10 - Nov. 16, 2014
I’ve been commenting on the site or a few months now, but so far just replies and responses. I’ve been thinking about potential contributions for a top-level discussion post, and I thought I’d ask about it here first to gage interest.

I have taught university classes in the past, usually with traditional methodology but in one memorable case with some experimental methods. There were a few ways this was different; as an example, we used ‘high expectations, low stakes’- we allowed students to retake any assignment as many times as they liked, but their grade for the entire class was basically the lowest grade they got on any assignment. (This was partly inspired by video games, actually.)

It will obviously be of particular interest to anyone else who does teaching, but there’s reasonable hope that some of my experiences there would be of use to audidacts. Do you think this would be a good use of my time?

Toggle Nov 10, 2014, 3:15 PM
0 points
in reply to: ChristianKl’s comment on: more on predicting agents
True, and even much wider than the educational system, but I would probably rephrase to say that this makes human intelligence predictable within narrow domains. A math class strives to make students predictable when attempting to solve math problems, a legal system hopes to make humans predictable in the domain of violent conflict resolution, a religion hopes to make humans predictable in metaphysical inquiry, etc.

But human intelligence itself is fully general (in that we defined ‘fully general’ to mean ‘like me’), so there’s not really any form of training or education that can make, or attempts to make, human intelligence predictable across all domains.

Toggle Nov 10, 2014, 2:47 PM
25 points
in reply to: Kaj_Sotala’s comment on: Open thread, Nov. 10 - Nov. 16, 2014
IANANRx, but I think the maximally charitable answer is “Nature; especially, the biological, physiological, and game-theoretical constraints within which any society and culture must operate.” By extension, a culture neglecting these constraints is necessarily in a state of collapse- a faux perpetual motion machine may move for a few moments because of initial momentum in the system, but it must necessarily halt.

As an additional corollary, homeostatic societies (which were, presumably, not in a state of collapse) must have been acting within these constraints. Therefore, long-running traditional cultures most clearly illustrate the terms of compliance with Gnon.

Toggle Nov 8, 2014, 7:07 PM
2 points
on: more on predicting agents
Note that the second requirement is a doozy. Many dynamical nonlinear systems are in fact formally unpredictable past a certain threshold. In other words, it’s quite common for ‘a sufficient degree of accuracy’ in modeling a system to simply not exist.

In the domain of human-created agents, your system will often be predictable, because the human that created it had a specific goal. But extending this to humans themselves may end up being problematic for the aforementioned reason- in that case, you’re likely limited to probabilistic reasoning no matter how great the abilities of your predictor agent.