christopherj

Karma: 370

christopherj Apr 27, 2014, 2:59 PM
0 points
0 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: [deleted]’s comment on: AI risk, new executive summary

Do we have some reason to expect [an AGI’s] goals to be more complex than ours?

I find myself agreeing with you—human goals are a complex mess, which we seldom understand ourselves. We don’t come with clear inherent goals, and what goals we do have we abuse by using things like sugar and condoms instead of eating healthy and reproducing like we were “supposed” to. People have been asking about the meaning of life for thousands of years, and we still have no answer.

An AI on the other hand, could have very simple goals—make paperclips, for example. An AI’s goals might be completely specified in two words. It’s the AI’s sub-goals and plans to reach its goals that I doubt I could comprehend. It’s the very single-mindedness of an AI’s goals and our inability to comprehend our own goals, plus the prospect of an AI being both smarter and better at goal-hacking than us, that has many of us fearing that we will accidentally kill ourselves via non-friendly AI. Not everyone will think to clarify “make paperclips” with, “don’t exterminate humanity”, “don’t enslave humanity”, “don’t destroy the environment”, “don’t reprogram humans to desire only to make paperclips”, and various other disclaimers that wouldn’t be necessary if you were addressing a human (and we don’t know the full disclaimer list either).

christopherj Apr 27, 2014, 2:22 PM
−2 points
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Lumifer’s comment on: Google vs Wikipedia, for-profit vs not-for-profit
I never said that the “invisible hand” would fail to function, I said that it would function inefficiently. Since efficiency is the major factor in deciding whether an economic strategy “works”, I noted that it would be out-performed by a system that can account for externalities. The free market could be patched to optimize things that contain externalities by applying tariffs and subsidies.

Given that I know of no system to properly account for externalities, I noted that as a failing of the free market but did not suggest any alternative—especially since my country already has this patch applied to some of the biggest and most obvious externalities, yet also shows signs of promoting the wrong things (eg corn based ethanol).

christopherj Apr 10, 2014, 10:24 PM
2 points
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: ShardPhoenix’s comment on: The social value of high school extracurricular time
My understanding is that various games can provide benefits such as ability to find relevant things in clutter, and reaction time, and decreasing the loss of mental function in the elderly. Other games could provide other benefits. However, if you consider that computer games could easily eat up all your free time plus some of your sleep, socialization, and homework time, and that alternate activities also have non-obvious benefits, this seems merely like a feel-good excuse. It’s probably not as bad as watching certain television shows though.

christopherj Apr 10, 2014, 8:20 PM
0 points
0 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
on: Google vs Wikipedia, for-profit vs not-for-profit

The idea underpinning market economics is the “invisible hand” which is supposed to aggregate everybody’s selfish behaviour into collective good (given a certain institutional set-up).

Unfortunately, the set up for it to work involves a massive use of product-specific tariffs and subsidies, to account for negative and positive externalities respectively. Otherwise the “invisible hand” would function inefficiently, over-promoting things with negative externalities like pollution, and under-promoting things with positive externalities like education.

christopherj Apr 10, 2014, 6:43 AM
0 points
0 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: V_V’s comment on: Solomonoff induction on a random string
But it seems to me rather different to assume you can do any finite amount of calculation, vs relying on things that can only be done with infinite calculation. Can we ever have a hope of having infinite resources?

christopherj Apr 10, 2014, 6:32 AM
1 point
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Oscar_Cunningham’s comment on: Solomonoff induction on a random string
Would it be legitimate to ask the SI to estimate the probability that its guess is correct? I suppose that if it sums up its programs’ estimates as to the next bit and finds itself predicting a 50% chance either way, it at least understands that it is dealing with random data but is merely being very persistent in looking for a pattern just in case it merely seemed random? That’s not as bad as I thought at first.

christopherj Apr 9, 2014, 4:40 PM
2 points
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Gunnar_Zarncke’s comment on: Controversy—Healthy or Harmful?
Since you mention Slashdot, here’s a little side effect of one of their moderation systems. At one point, they decided that “funny” shouldn’t give posters karma. However, given the per-post karma cap of 5, this can prevent karma-giving moderation while encouraging karma-deleting moderation by people who think the comment overrated, potentially costing the poster tons of karma. As such, moderators unwilling to penalize posters for making jokes largely abandoned the “funny” tag in favor of alternatives.

I suspect that if an agree/disagree moderation option were added, it would likely suffer from a similar problem. Eg if we treated that tag reasonably and used it to try to separate karma gains/losses from personal agreement/disagreement, people would be tempted to rate a post they especially like as disagree/love/awe.

A more interesting idea, I think, would be to run correlations between your votes and various other bits, such as keywords, author, and other voters, to increase the visibility of posts you like and decrease the visibility of posts you don’t like. This would encourage honest and frequent voting, and diversity. Conversely, it would cause people to overestimate the community’s agreement with them (more than they would by default).

christopherj Apr 9, 2014, 3:54 PM
1 point
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: MrMind’s comment on: Solomonoff induction on a random string
OK, so it will predict one of multiple different ~ 1 terabyte programs as having different likelihoods. I’d still rather it predict random{0,1} for less than 10 bytes, as the most probable. Inability to recognize noise as noise seems like a fundamental problem.

christopherj Apr 9, 2014, 3:44 PM
−2 points
3 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: cousin_it’s comment on: Solomonoff induction on a random string
He has repeatedly said that he’s talking about an SI that outputs a specific prediction instead of a probability distribution of them, and you even quoted him saying so.

christopherj Apr 9, 2014, 7:46 AM
3 points
5 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: ChristianKl’s comment on: Be comfortable with hypocrisy
This does not seem nearly as bad as the flip side, people preaching weak morals so as to not be seen failing them.

christopherj Apr 9, 2014, 7:28 AM
17 points
10 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Metus’s comment on: Be comfortable with hypocrisy
I say you’re a hypocrite, pretending indifference between good and evil yet for the most part choosing good.

christopherj Apr 9, 2014, 7:18 AM
0 points
0 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
on: Solomonoff Cartesianism
I know a way to guarantee wireheading is suboptimal: make the reward signal be available processing power. Unfortunately this would guarantee that the AI is unfriendly, but at least it will self-improve!

christopherj Apr 9, 2014, 6:14 AM
0 points
0 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: LEmma’s comment on: Solomonoff induction on a random string
I think you can justify stopping the search when you are hitting your resource limits and have long since ceased to find additional signal. You could be wrong, but it seems justified.

christopherj Apr 9, 2014, 6:01 AM
0 points
0 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Oscar_Cunningham’s comment on: Solomonoff induction on a random string
But, given 1 terabyte of data, will it not generate a ~1 terabyte program as it’s hypothesis? Even if it is as accurate as the best answer, this seems like a flaw.

christopherj Apr 9, 2014, 5:46 AM
2 points
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
on: Worse Than Random
OK, let me give you another example of the lock device. Each time a code is tried, the correct code changes to (previous code) + 2571 mod 10000. You don’t know this. You won’t find out before opening the door, because of limited feedback. Sequential check of every code will fail, but let you know that the correct code changes (if there is a correct code). Constantly guessing the same code because you think it’ll randomly change to that one will fail. Random guessing will eventually succeed. Using randomness prevents you from getting stuck due to your own stupidity or an opponent. There is no method for always beating randomness, making it a reliable failsafe against always losing.

You won’t always have knowledge about the problem, and on occasion what you think is knowledge is wrong. Random may be stupid, but it has a lower bound for stupidity.

christopherj Apr 8, 2014, 5:19 PM
6 points
6 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Houshalter’s comment on: Worse Than Random
What you’re missing is that, if the signal is below the detection threshold, there is no loss if the noise pushes it farther below the detection threshold, whereas there is a gain when the noise pushes the signal above the detection threshold. Thus the noise increases sensitivity, at the cost of accuracy. (And since a lot of sensory information is redundant, the loss of accuracy is easy to work around.)

christopherj Apr 8, 2014, 4:28 PM
2 points
2 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: AliceKingsley’s comment on: Lawful Uncertainty
If you wanted to play the lottery, the best strategy is to play the “least lucky” and “least ‘random’” numbers, ie pick the numbers that won’t be picked by a bunch of superstitious people. Decrease your odds of having the split the winnings with another winner.

christopherj Apr 8, 2014, 4:01 PM
3 points
2 votes
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
in reply to: Will_Pearson’s comment on: Lawful Uncertainty
If you’re predictably committed to winning the game of chicken, then you have essentially already won, at least against a rational opponent. Though you’d have to wonder how you wound up with a rational opponent if the game is chicken.

Solomonoff induction on a random string

christopherjApr 8, 2014, 5:30 AM

2 points

4 votes

Overall karma indicates overall quality.

43 comments1 min readLW link

christopherj Apr 8, 2014, 4:53 AM
1 point
1 vote
Overall karma indicates overall quality.
0
0 votes
Agreement karma indicates agreement, separate from overall quality.
on: The Problem with AIXI
I’m having trouble understanding how something generally intelligent in every respect except failure to understand death or that it has a physical body, could be incapable of ever learning or at least acting indistinguishable from one that does know.

For example, how would AIXI act if given the following as part of its utility function: 1) utility function gets multiplied by zero should a certain computer cease to function 2) utility function gets multiplied by zero should certain bits be overwritten except if a sanity check is passed first

Seems to me that such an AI would act as if it had a genocidally dangerous fear of death, even if it doesn’t actually understand the concept.

christopherj

Solomonoff in­duc­tion on a ran­dom string

Solomonoff induction on a random string