Punoxysm

• Punoxysm 4 Mar 2014 8:17 UTC

  6 points

  on: Pro­por­tional Giving

  A pro of proportional giving is that it’s dead simple to budget and keep in mind.

  It can also get people to give a certain amount continually and keep them emotionally invested in charitable effort, while allowing them plenty of savings for contingencies and perhaps an eventual larger lump sum donation.

  And even if it’s not optimal, “indefensible” is a pretty strong claim.

• Punoxysm 4 Mar 2014 8:24 UTC

  0 points

  in reply to: TheOtherDave’s comment on: [LINK] Joseph Bot­tum on Poli­tics as the Mindkiller

  I strongly doubt it. Mainly because many of the things we recognize and revile in today’s political discourse had a thriving existence around the turn of the century: Yellow Journalism, political machines, Anarchists around every corner… And it’s hard to argue that, say, the 70′s were a more moderated time.

  It is possible that specific metaphorical frameworks and memes are more recent and promoted through a greater volume of media than back then, but there is a far greater volume of media in general.

• Punoxysm 4 Mar 2014 16:44 UTC

  0 points

  in reply to: TheOtherDave’s comment on: [LINK] Joseph Bot­tum on Poli­tics as the Mindkiller

  Yeah… Turn of the Century is getting to be an ambiguous phrase now. But I do mean 1900s

• Punoxysm 4 Mar 2014 16:56 UTC

  0 points

  in reply to: gjm’s comment on: Pro­por­tional Giving

  I think you won’t find anything without a lot of assumptions and caveats; Any concave utility function will simply never lead to proportional giving.

  But certainly, many people will scale up their own expectations of lifestyle along with their income until a pretty high point: that is, they’ll try to tell you or at least behave as if they don’t have a concave utility function, until they’re well into the six figures (sometimes more). Since income changes are slow and tend to co-occur with many other life changes, this, along with psychological biases, complicate any earnest attempt to sit down and say “I’ll be best off giving X portion of my income this year”.

• Punoxysm 4 Mar 2014 16:59 UTC

  4 points

  on: Tell Culture

  Fun fact: Linguists talk about this in the context of the Gricean maxims, http://​​en.wikipedia.org/​​wiki/​​Cooperative_principle

  ---

  Maxim of Quality[edit] Do not say what you believe to be false. Do not say that for which you lack adequate evidence.

  Maxim of Quantity[edit] Make your contribution as informative as is required (for the current purposes of the exchange). Do not make your contribution more informative than is required.

  Maxim of Relation[edit] Be relevant. With respect to this maxim, Grice writes, “Though the maxim itself is terse, its formulation conceals a number of problems that exercise me a good deal: questions about what different kinds and focuses of relevance there may be, how these shift in the course of a talk exchange, how to allow for the fact that subjects of conversations are legitimately changed, and so on. I find the treatment of such questions exceedingly difficult, and I hope to revert to them in later work.”[1]

  Maxim of Manner[edit] Avoid obscurity of expression. Avoid ambiguity. Be brief (avoid unnecessary prolixity). Be orderly.

  ---

  Relying on the other’s inference is critical in a lot of communication, along many dimensions, and people manage to do it all the time by following these rules (most are learned through typical social interaction, but execution can always be improved).

  If you want to be super-direct, go for it, but if you earnestly keep in mind the other person’s state of mind, and the above maxims, you’ll usually have no problem navigating guess culture (unless you’re making a transition between countries or something similar).

• Punoxysm 4 Mar 2014 21:38 UTC

  2 points

  on: Open Thread: March 4 − 10

  Hi, long time lurker, new user. I was thinking about writing a post on how any potential AGI of human-level intelligence is likely to have a band of a few years before and after its creation where FOOM risks can be contained with care, and how this would be an especially fruitful period to deal with friendliness. Any posts/​articles I should look at to avoid being too redundant?

• Punoxysm 4 Mar 2014 23:12 UTC

  0 points

  in reply to: Strilanc’s comment on: Open Thread: March 4 − 10

  Okay, I’ll be addressing this pretty directly. I don’t find it particularly convincing though.

• Punoxysm 4 Mar 2014 23:19 UTC

  1 point

  in reply to: Strilanc’s comment on: Open Thread: March 4 − 10

  I understand the notion, but think of it in terms of preventing a pandemic: There’s a certain set of characteristics of a virus that would overwhelm virtually any attempt to prevent it from wiping out humanity. All existing viruses are pretty safely within the bounds of what our actual public health protocols can handle. On top of that, existing or hypothetical yet plausible protocols can prevent pandemics with viruses that have higher transmissibility, or higher mortality than anything previously experienced.

  Realistically, a protocol to deal with AGI will be in a similar position. It will be distinctly “one-shot” but there’s no reason it couldn’t deal with a computer somewhat more intelligent than any existing human being.

• Punoxysm 5 Mar 2014 17:15 UTC

  0 points

  in reply to: Viliam_Bur’s comment on: Open Thread: March 4 − 10

  Edit: I want to be clear that I think its fairly likely that SOME AI could do what’s described above, and have absolute runaway capacity, but that it’s likely the first few won’t. Prototypes just don’t work very well, in general. And yes, an AI is a bit different if it is self-modifying and intelligence comes closely tied with agency. But there’s still tons of possibilities for an AI that is very unlikely to exponentially self-improve (consider an in silico brain simulation; hardware requirements and a fine-grained control over input and simulation speed would seem to make this pretty safe).

  ---

  I find the story outlined above plausible, but not extremely probable, especially with regards to whatever the first AGI worth the name looks like. It will be smarter than people in some ways, but mostly dumber, or severely limited in some capacity. The architecture may be self-modifying, but it will be possible to monitor it to some degree, and also to prevent the system from having a full view of itself (in fact it is easier to prevent the system from having a complete view of itself than not).

  The runaway scenario you outline is more likely if the AI is allowed to run rampant, which is why any attempt to build an AGI should include plenty of security protocols, completely orthogonal to any in-code attempts to enforce friendliness (you can’t argue against redundancy when it comes to safety!). Sandbox the AI. Apart from a fixed initial store of information (could be as large as the whole web), let nothing get in or out except for some very low bandwidth channel (text, video). Make the barrier physical if necessary.

  And have the whole thing run by paranoid bureaucrats. Treat the AI as dangerous if not contained. I’ll elaborate more later, but while there certainly is a failure mode for any security protocol, that’s not the same as saying that protocols aren’t worth developing and valuable for a wide range of circumstances.

  A prison can contain a prisoner smarter than the architect or any of the guards or wardens. Not one infinitely smarter, but there’s definitely a margin of safety.

• Punoxysm 5 Mar 2014 19:18 UTC

  2 points

  in reply to: Kaj_Sotala’s comment on: Open Thread: March 4 − 10

  Thanks. I’ll look over these.

  Edit: It looks like section 4, AGI Containment, covers many of my thoughts and comes to a pretty similar conclusion: External constraints on AGI are an imperfect plan, but potentially valuable and complementary to other safety approaches.

• Punoxysm 5 Mar 2014 21:12 UTC

  2 points

  in reply to: NoSuchPlace’s comment on: Open Thread: March 4 − 10

  Yeah, I’ve seen this. It’s pretty frustrating because of the secrecy. All I know is two guys let Yudkowsky out of the box.

  But I think there are two reasons why that scenario is actually very favorable to the AI.

  1) An AI that is a bit dumber than humans in all ways, dumber in some ways and smarter in others, or just a little bit smarter than humans can still teach you a lot about further AIs you’d want to builid, and it seems at least plausible that an AI that’s 2xHuman intelligence will come along before one that’s 1000x human intelligence. We want to get as much out of the 2x-10x AIs as possible before we try building, and IF it is possble to avoid accidentally making that 1000x AI before you can make sure it is safe, then there could be a point where you are dealing with 10x AIs and where you can do so safely.

  So don’t pit your engineer against an unboundedly transhuman AI. Pit your engineer against a slightly transhuman AI.

  2) Security-wise, you can do a ton better than somebody who is just “really really sure they wouldn’t let the AI out”. You can have the AI talk only to people who literally cannot let the AI out, and on top of that ensure that everyone is briefed on the risks. Make sure that to the extent possible, letting the AI out is literally not an option. It is always an option to attempt to do so covertly, or to attempt to convince people to break self-imposed rules, sure, but how many prisoners in history have talked their way out of prison?

  You can even tell people “Okay, we’re never ever letting this AI out. It is a prototype and considered dangerous. But hard, diligent work with attention to safety protocols will ensure we can someday build a safe AI that will have every capacity and more that this one does, to the enormous benefit of humanity”.

  3) If you look at this answer and say “well you sound like you think you’re that much better than the people who took the challenge and lost not even to a transhuman AI but to Yudkowsky” and you’d be a little right. But mostly I believe that if we ever face that actual situation, the challenge the AI faces will be more akin to trying to cause a nuclear weapon launch with a telephone; not just one person but an entire formidable physical and bureaucratic security apparatus will (or SHOULD) stand in the way.

  ---

  There’s also the issue of the political problem of FAI. You might not have a choice but to rely on less-than-desirable safety protocols because of external pressures. Say a genuinely feasible plan for AGI is developed that would take 2 years of research, while the most optimistic friendliness researchers doubt that the architecture could be made provably safe in sooner than 10. How will you confront the problem of convincing every lab in the world with sufficient resources to hold off for 10 years? Would that even be possible?

  I guess we would have to look at the past and see how political and research institutions have behaved in the face of technologies with enormous potential but which were also known to pose a concrete existential risk (Yes I am talking about nuclear weapons).

• Punoxysm 5 Mar 2014 21:38 UTC

  4 points

  on: How my math skills im­proved dramatically

  I would break out a sub-header from the first one: It sounds like you tried to actively ignore official prerequisites and prescribed orderings when you felt it would benefit you. You were actively choosing to risk biting off more than you could chew (which makes plenty of sense: it’s pretty safe in an academic context, but it’s still more bold than most people).

  As shminux says below, your story could have turned out poorly if you’d done the same and found yourself constantly mired in confusion. Your motivation would have warn out quickly. This is why those prescribed orderings, even if they have more baby-steps than some people require, are so popular in pedagogy.

  Congratulations

• Punoxysm 6 Mar 2014 6:08 UTC

  5 points

  in reply to: Vaniver’s comment on: Pro­posal: LW courses

  CFAR runs courses. They could probably put on a MOOC without having to make a new curriculum

• Punoxysm 6 Mar 2014 17:02 UTC

  2 points

  in reply to: Vaniver’s comment on: Pro­posal: LW courses

  Well, they could start with a set of mini-modules and upload them to youtube. A couple hours of total content. Put lecture notes and homeworks and a forum on a website. That was a pretty typical way to prototype the early MOOCs.

• Punoxysm 7 Mar 2014 3:29 UTC

  0 points

  on: AALWA: Ask any LessWronger anything

  Ask me anything. Until recently I was a machine learning and text mining graduate student. Now I work on data visualization in a corporate setting (but I can’t talk a ton about that).

• Punoxysm 7 Mar 2014 6:08 UTC

  2 points

  in reply to: asr’s comment on: AALWA: Ask any LessWronger anything

  I left before completing my degree, but I retain the option of returning. The lifestyle and culture of academia, the chance to do research, and the possibility of entering a research career are the advantages of academia over industry. At the time I left, none of those were as compelling to me as the career I’ve entered.

  I’m glad I gave academia a shot, and I left not too long after I had figured out my priorities weren’t well-satisfied by staying in grad school.

  I am sorry if that’s not very helpful. It was just a natural move for me at the time.

How to Study Un­safe AGI’s safely (and why we might have no choice)

• Punoxysm 7 Mar 2014 16:11 UTC

  4 points

  in reply to: ahbwramc’s comment on: How to Study Un­safe AGI’s safely (and why we might have no choice)

  It’s just a little color and context.

  You don’t have to believe anything except that either (or both):

  • We gain the capacity to develop AI before we can guarantee friendliness, and some organization attempts to develop maybe-unsafe AI

  • Redundant safety measures are good practice.

• Punoxysm 7 Mar 2014 16:25 UTC

  0 points

  in reply to: ThrustVectoring’s comment on: How to Study Un­safe AGI’s safely (and why we might have no choice)

  In addition to what V_V says below, there could be absolutely no official circumstance under which the AI should be released from the box: that iteration of the AI can be used solely for experimentation, and only the next version with substantial changes based on the results of those experiments and independent experiments would be a candidate for release.

  Again, this is not perfect, but it gives some more time for better safety methods or architectures to catch up to the problem of safety while still gaining some benefits from a potentially unsafe AI.

• Punoxysm 7 Mar 2014 16:33 UTC

  0 points

  in reply to: private_messaging’s comment on: How to Study Un­safe AGI’s safely (and why we might have no choice)

  Security by omission is a very good point. The same is true in omitting options in protocols. If, for instance, “let the AI out of its box” or even “Give this AI extra hardware or communication with the broader public” are not official options for the result of a certain phase of the project, that makes it all the harder for it to happen.

  Questions about architecture, and how we could begin to “bolt on” behavioral constraints are critical. And that’s precisely why we need to experiment. I suspect many architectures will be highly opaque and correspondingly difficult to constrain or directly modify. A simulated human brain might fall under this category.