ryan_b

• ryan_b 20 Oct 2017 23:35 UTC

  1 point

  on: FDT: Perfect Pre­dic­tor, Fix-Point Com­pu­ta­tion and Agency

  If we have a rate of successful past predictions instead of a perfect prediction assumption, doesn’t that abrogate the fixed-point problem? Or at least constrain it to the space of failed predictions?

• ryan_b 22 Oct 2017 2:39 UTC

  1 point

  on: Why no to­tal win­ner?

  It is worth considering that in the context of power, regulations are merely a form of incentive. Compare to the pre-commitment strategies we often discuss.

  In the early history of the firm we saw examples of businesses exceeding state power all the time. For example, the British East India company in India, American shipping companies in Nicaragua, United Fruit in Guatemala. In current affairs, companies like Nestle and Coca-cola are accused of violently enforcing their interests in Africa. I am confident that given attention, we either would find or will soon misbehavior of a similar nature from Chinese firms. And this is only a comparison against states; local governments get run over roughshod routinely, exactly as we would expect from comparing their available resources.

  OThe intuition is that all possible agents are competing in the same field, which is reality. The extent to which they understand the field makes it an information game of sorts. I think this means the fundamental limits on AGI are the same as any other variety of agent: the cost in time and resources of integrating new information. We just expect both of these costs to be very low relative to other agents, and therefore the threat is high.

• ryan_b 30 Oct 2017 19:28 UTC

  5 points

  in reply to: alwhite’s comment on: Lead­ers of Men

  The important detail in the post is that how people react to the leader is the metric for good leadership. If people respond to confidence, then projecting confidence is competence in any situation where it is important for people to respond.

• ryan_b 7 Nov 2017 13:16 UTC

  5 points

  in reply to: Chris_Leong’s comment on: Moloch’s Toolbox (2/​2)

  I think you have described their real motivation, but how would they know? Politicians don’t have a secret database of polling data unavailable to the rest of us, or a deep expertise in reading the ones which are available.

  The problem is both the voters and the politicians are drawing on the journalists for information. The journalists are writing for clicks, which means a national focus for the widest audience. The politicians then spend their time sounding good to their local constituents about these national trends, which is why they all sound insane to everyone else.

  There is local variation in the perception of the national trends, which leads to everyone having a different conversation.

Read­ing Papers in Undergrad

• ryan_b 9 Nov 2017 20:35 UTC

  1 point

  on: Re­search Syndicates

  Roger M. Stein of MIT gave a TED talk on how to distribute the risk of things like pharmaceutical research. This emphasized funding research and distributing risk from the products of the research, but would probably be a useful method for a syndicate to generate funds.

  • The talk is called “A Bold New Way to Fund Drug Research”, which I cannot link effectively because they made the poor choice of using underscores everywhere which markdown hijacks.

  • His website is here: http://​​www.rogermstein.com/​​#header

  I find the idea of treating things as financial assets very interesting, because you can split risk among multiple types of assets because they all have payoff functions. This is similar to how you can multiply probability functions together from all of the evidence.

• ryan_b 9 Nov 2017 20:55 UTC

  1 point

  on: Moved from Moloch’s Toolbox: Dis­cus­sion re style of lat­est Eliezer sequence

  I interpret the Inadequate Equilibrium as being a long-form of defining the problem.

  Now that we have a larger and more stable community with shared information, it seems like a good time to go back to fundamental motivations and take advantage of the higher sanity waterline. It is not preaching to the choir so much as to other preachers.

  I fully expect we will do this dance yet again, once we get enough progress that it seems worthwhile. Each time, it will target the community using the tools the community has developed.

Set, Game, Match

• ryan_b 11 Nov 2017 15:32 UTC

  2 points

  in reply to: Vaughn Papenhausen’s comment on: Read­ing Papers in Undergrad

  This is a good point—for example, I find history papers to be easily readable in order, and the abstracts are useful. My expectation is that this will work for any of the humanities, are more generally anything lacking a minimum threshold for methodological background.

  I do frequently encounter difficulties with highly specialized terms: sociology is a common subject where that problem appears.

• ryan_b 11 Nov 2017 15:35 UTC

  1 point

  in reply to: Unnamed’s comment on: Read­ing Papers in Undergrad

  Have you always read psychology papers the same way? Have your results changed much as you gained a broader base of psychological knowledge?

• ryan_b 12 Nov 2017 22:50 UTC

  2 points

  on: How Pop­per kil­led Par­ti­cle Physics

  As a consequence of this post, I have pre-ordered the book and now follow the blog; thank you for bringing it to my attention.

  I very much enjoy the incisive commentary, and it makes me wonder about the difference between being a seeker-of-truth and an executioner-of-falsehood.

• ryan_b 13 Nov 2017 0:22 UTC

  6 points

  in reply to: Emile’s comment on: Blind Empiricism

  It is worth considering that in other environments than this one, speaking directly to the point with calibrated confidence is not the persuasive tactic. Just get something in front of users is a not a complete statement of the optimal strategy, it is a rhetorical overreaction to insisting on millions of dollars and years worth of waterfall-model development before actually checking to see if anyone cares. When these ideas were first proposed, IBM had overwhelming market share in computing.

  The big corporate model of product development was and is very good at producing a finished product, the problem is it doesn’t address whether anyone will buy it. This is solving the wrong problem—Big Design Up Front was mostly about how to use the corporation’s technical resources efficiently. Further, it is all-or-nothing; people either buy the finished product or they do not. This still looks like the basic case of good model or bad model, not one of no model at all.

  There are two further things to consider: one, that IBM is generations of dominance ago clearly indicates the norm has moved a great deal from that time; two, IBM is still alive and still routinely criticized for the heavy, bureaucratic way it goes about business.

• ryan_b 13 Nov 2017 2:01 UTC

  4 points

  on: If cre­at­ing FAI is easy, it might be automatic

  The “everything might just turn out to be okay” proposition is under-specified. This is effectively asking for the likelihood of all known and unknown catastrophes not occurring. If you want to evaluate it, you need to specify what the dimensions of “okay” are, as well as their bounds. This becomes a series of smaller propositions, which is approximately the field of X-risk.

  Regarding hedging for the worst: suppose we flipped that around. Isn’t it weird that people are okay with a 1% chance of dying horribly? Indeed evidence suggests that they are not—for scary things, people start getting anxious in the 0.01% range.

• ryan_b 13 Nov 2017 20:57 UTC

  1 point

  in reply to: gjm’s comment on: Set, Game, Match

  The axiom of determinacy is very interesting.

  I am reading a book called Probability and Finance: It’s Only a Game, by Glenn Shafer and Vladimir Vovk, which has two insights which have caused my head to explode.

  The first insight is that the environment is a player in the game. The basic game is between unequal players, Skeptic and World. The second insight is players can be decomposed into other players. They decompose World into a variety of other players, mostly by the kinds of moves they make.

  It seems reasonable that this relationship must work two ways—something like all games roll up into Agent v. Reality games. If that is true, then I would accept the axiom of determinacy, because I have a real hard time imagining that Reality would not have the winning strategy over a long enough game.

• ryan_b 15 Nov 2017 0:16 UTC

  1 point

  on: Pas­cal’s mug­ging in re­ward learning

  I am not clear at what point in this process the reward blows up such that it qualifies as a mugging. It appears that defining \pi(MAX) as achievable through human policies places the reward calculation firmly within the usual realm.

  Pascal’s Wager was about the infinite gain of eternal salvation, and Eliezer’s Mugging example was as much about how the rewards are inducted as it was their magnitude—the pitch was that the nature of Solomonoff induction was such that rewards had no meaningful cap even when statements about their likelihood do, because magnitude is very efficiently communicated.

• ryan_b 15 Nov 2017 17:31 UTC

  8 points

  on: Against Modest Epistemology

  The new sequence is very good and I have been able to put a couple of them to immediate use. I had the following series of thoughts when reading this piece:

  1) This sequence appears less timeless than Rationality: A-Z, because that was addressed to everyone and this looks addressed to the current community.

  2) Is it really targeted at the community specifically, or should I expect Modest Epistemology to be a common local maxima in the pursuit of clear thinking? If the latter, then the first feeling was wrong.

  3) To what extent does the community represent a local maxima or plateau? If I think a lot of other individuals will go through the same sort of progression, there may be overlap.

  I realize the goal is for the community to be a special case of general pursuit of rationality, but it has been a minute since I thought about it as a whole. This is a another mark in favor of the sequence.

• ryan_b 19 Nov 2017 17:08 UTC

  1 point

  on: Sta­tus Reg­u­la­tion and Anx­ious Underconfidence

  The link to footnote 3 is currently showing as Not Found.

• ryan_b 20 Nov 2017 1:07 UTC

  1 point

  on: Im­pli­ca­tions of a feel­ings-first meta­physics?

  A few questions leap to mind.

  1) How are you accounting for the fact that people do not experience the same feelings? Even among neurotypicals, there is wide variation in the response to the same stimuli.

  2) The second of your initial ideas—you believe what enables you to act—seems to assume a utilitarian perspective on belief. Is that right?

  3) There is a heavy emphasis on language—are you including mathematics in that description? Other forms of communication? Ritualized experiences? What is it bounded by, if not by language?

  4) What is the goal here?

  5) If you don’t have access to information outside of your feelings and these rules, how do you determine what actions make you feel a sense of balance?

• ryan_b 21 Nov 2017 0:05 UTC

  1 point

  in reply to: Stuart_Armstrong’s comment on: Pas­cal’s mug­ging in re­ward learning

  Reviewing the post with your update, I think the problem may just be that the examples are de-priming my intuition. In your reply you chose ‘the human doesn’t eat’ as the reward for a modified human to maximize, which means the gains are only all the food humans would eat if unmodified. This is compared to brain surgery, which a bit of googling suggests costs 50-150K, much more than it costs to feed a person. It looks like I chunked the proposition as ‘costly intervention to achieve bounded reward’ as a consequence.

  However, none of this is actually implied by the math. Insofar as you project there are likely to be other readers like me, it may be worth changing the examples to emphasize a trivial intervention for a very high reward.

Ar­bi­trary Math Questions