TruePath

• TruePath 14 Aug 2012 21:55 UTC

  24 points

  on: Bayes for Schizophren­ics: Rea­son­ing in Delu­sional Disorders

  All of the theories presented in this post seem to make the implausible assumption that somehow the brain acts like a hypothetical ideally rational individual and that impairment somehow breaks some aspect of this rationality.

  However, there is a great deal of evidence the brain works nothing like this. In contrast, it has many specific modules that are responsible for certain kinds of thought or behavior. These modules are not weighed by some rational actor that sifts through them, they are the brain. When these modules come into conflict, e.g., in the standard word/​color test where yellow is spelled in red, fairly simply conflict resolution methods are brought into play. When things go wrong in the brain, either an impairment in conflict resolution mechanisms or in the underlying modules themselves, things will go wonky in specific (not general) ways.

  Speaking from personal experience, being in a psychotic/​paranoid state simply makes certain things seem super salient to you. You can be quite well aware of the rational arguments against the conclusion you are worrying about but it’s just so salient that it ‘wins.’ In other words it also feels like there is just a failure in your ability to override certain misbehaving brain processes rather than some general inability to update appropriately. This is further supported by the fact that skizophrenics and others with delusions seem to be able to update largely appropriately in certain aspects, e.g., what answer is expected on a test, while still maintaining their delusional state.

• TruePath 16 Nov 2018 4:35 UTC

  22 points

  on: Sam Har­ris and the Is–Ought Gap

  I essentially agree with you that science can’t bridge the is-ought gap (see caveats) but it’s a good deal more complicated than the arguments you give here allow for (they are a good intro but I felt it’s worth pointing out the complexities).

  1. When someone claims to have bridged the is-ought gap they aren’t usually claiming to have analytically identified (i.e. identified as a matter of definition) ought with some is statements. That’ s a crazily high bar and modern philosophers (and Sam Harris was trained as a philosopher) tend to feel true analytic identities are rare but are not the only kind of necessary truths. For instance, the fact that “water is H20” is widely regarded as a necessary truth that isn’t analytic (do a search if you want an explanation) and there are any number of other philosophical arguments that are seen as establishing necessary truths which don’t amount to the definitional relationship you demand.

  I think the standard Harris is using is much weaker even than that.

  1. You insist that to be an ought it must be motivating for the subject. This is a matter of some debate. Some moral realists would endorse this while others would insist that it need only motivate certain kinds of agents who aren’t too screwed up in some way. But, I tend to agree with your conclusion just suggest it be qualified by saying we presuming the standard sense of moral realism here.

  2. One has to be really careful with what you mean by ‘science’ here. One way people have snuck around the is-ought gap before is using terms like cruel which are kinda ‘is’ facts that back in an ought (to be cruel requires that you immorally inflict suffering etc..).

  3. It’s not that Harris is purely embedded in some kind of dialectical tradition. He was trained as an analytic philosopher and they invented the is-ought gap and are no strangers to the former mode of argumentation. IT’s more that Carrol is a physicist and doesn’t know the terminology that would let him pin Harris down in terms he would understand and keep him from squirming off the point.

  However, I’m pretty sure (based on my interaction with Harris emailing him over what sounded like a similarly wrongheaded view in the philosophy of mind) that Harris would admit that he hasn’t bridged Hume’s is-ought gap as philosophers understand it but instead explain that he means to address the general public’s sense that science has no moral insight to offer.

  In that sense I think he is right. Most people don’t realize how much science can inform our moral discussions...he’s just being hyperbolic to sell it.

Ev­i­dence For Simulation

• TruePath 19 Aug 2018 10:49 UTC

  12 points

  on: The “semio­sis re­ply” to the Chi­nese Room Argument

  You are getting the statement of the Chinese room wrong. The claim isn’t that the human inside the room will learn Chinese. Indeed, it’s a key feature of the argument that the person *doesn’t* ever count as knowing Chinese. It is only the system consisting of the person plus all the rules written down in the room etc.. which knows Chinese. This is what’s supposed to (but not convincingly IMO) be an unpalatable conclusion.

  Secondly, no one is suggesting that there isn’t an algorithm that can be followed which makes it appear as if the room understands Chinese. The question is whether or not there is some conscious entity corresponding to the system of the guy plus all the rules which has the qualitative experience of understanding the Chinese words submitted etc.. As such the points you raise don’t really address the main issue.

De­ci­sion The­ory Anti-realism

• TruePath 28 May 2011 17:35 UTC

  9 points

  on: Some Heuris­tics for Eval­u­at­ing the Sound­ness of the Aca­demic Main­stream in Un­fa­mil­iar Fields

  You confuse two very different issues.

  1) How much weight you should give to the views of academics in that area, e.g., if some claim is accepted by the mainstream establishment (or conversely viewed as a valid point of disagreement) how much should that information affect your own probability judgement.

  2) How much progress/​how useful is the academic discipline in question. Does it require reform.

  ---

  Your arguments in the first part are only relevant to #2. The programming language research community may be mirred in hopeless mathematical jealousy as they create more and more arcane type systems while ignoring the fact that ultimately programming language design is an entirely psychological question. The languages are all Turing complete and most offer the same functionality in some form the only question is one of human useability and the community doesn’t seem very interested in checking what sorts of type systems or development environments really are empirically more productive. Maybe physics is stuck and can no longer make any real progress.

  Nevertheless this has no bearing on how I should treat the evidence that 99% of physics professors predict experiment X will have outcome Y. Indeed, the argument that physics is stuck is largely that they have been so successful in explaining any easily testable phenomena it is difficult to make further progress. Similarly if I see that the programming language research people say that type system Blah is undecidable I will take that evidence seriously even if it doesn’t turn out to be that useful.

  (Frankly I think the harsh on CS is a bit unfair. Academia by it’s nature is conservative and driven by pure research. We don’t yet know whether their work will turn out to be useful down the road since CS is such a young discipline while at the same time many people do work in both practical and theoretical areas.)

  ---

  I think #1 is the more interesting question. Here I would say the primary test should be whether or not disputes eventually produce consensus or not. That is does the discipline build up a store of accepted fact and move on to new issues (with occasional Kuhnian style paradigm shifts) or does it simply stay mired in the same issues without generating conclusions.

[Question] Does the hard­ness of AI al­ign­ment un­der­mine FOOM?

• TruePath 3 Feb 2010 7:51 UTC

  8 points

  in reply to: Toby_Ord’s comment on: Com­plex­ity of Value ≠ Com­plex­ity of Outcome

  The right response to moral realism isn’t to dispute it’s truth but to simply observe you don’t understand the concept.

  I mean imagine someone started going around insisting some situations were Heret and others were Grovic but when asked to explain what made a situation Heret or Grovic he simply shrugged and said they were primitive concepts. But you persist and after observing his behavior for a period of time you work out some principle that perfectly predicts which category he will assign a given situation to, even counterfactually but when you present the algorithm to him and ask, “Ohh so is it satisfying this principle that makes one Heret rather than Grovic?” he insists that while your notion will always agrees with his notion that’s not what he means. Moreover, he insists that no definition in terms of physical state could capture these concepts.

  Confused you press him and he says that there are special things which we can’t casually interact with that determine Heret or Grovic status. Bracketing your skepticism you ask him to say what properties these new ontological objects must have. After listing a couple he adds that most importantly they can’t just be random things with this structure but they also have to be Heret making or Grovic making and that’s what distingushes them from all the other casually inaccessible things out there that might otherwise yield some slightly different class of things as Heret and Grovic.

  Frustrated you curse the guy saying he hasn’t really told you anything since you didn’t know what it meant to be Heret or Grovic in the first place so you surely don’t know what it means to be Heret making or Grovic making. The man’s reply is simply to shrug and say, “well it’s a fundamental concept, if you don’t understand I can’t explain it to you anymore than I could explain the perceptual experience of redness to a man who had never experienced color.”

  ---

  In such a situation the only thing you can do is give up on the notion of Heret and Grovic. Debating about whether to say it’s an incoherent notion, a concept you lack the facilities to comprehend or something else would just waste time with a useless word game. Ultimately you just have to ignore such talk as something that lacks content for you and treat it the same way as you would meaningless gibberish.

  The fact that when moral realists say the same thing about good and evil they are using the same sounds that we understand to mean something different shouldn’t change the situation at all.

• TruePath 3 Jul 2023 1:39 UTC

  7 points

  on: AI safety from first prin­ci­ples: Superintelligence

  I like the idea of this sequence, but—given the goal of spelling out the argument in terms of first principles—I think more needs to be done to make the claims precisce or acknowledge they are not.

  I realize that you might be unable to be more precisce given the lack of precision in this argument generally—I don’t understand how people have invested so much time/​mondy on research to solve the problem and so little on making the argument for it clear and rigorous—but if that’s the case I suggest you indicate where the definitions are insufficient/​lacking/​unclear.

  I’ll list a few issues here:

  Defining Superintelligence

  Even Bostrom’s definition of superintelligence is deeply unclear. For instance, would an uploaded human mind which simply worked at 10x the speed of a normal human mind qualify as a superintelligence? Intuitively the answer should be no, but per the definition the answer is almost certainly yes (at least if we imbue that upload with extra patience). After all, virtually all cognitive tasks of interest benefit from extra time—if not at the time of performance then extra time to practice (10x the practice games of chess would make you a better player). And if it did qualify it undermines the argument about superintelligence improvement (see below).

  If you require a qualitative improvement rather than merely speeding up the rate of computation to be a superintelligence then the definition risks being empty. In many important cognitive tasks humans already implement the theoretically optimal algorithm or nearly do so. Lots of problems (eg search on unordered data on classical TM) have no better solution than just brute force and this likely includes quite a few tasks we care quite a bit about (maybe even in social interactions). Sure, maybe an AI could optimize away the part where our slow human brain slogs through (tho arg we have as well w/​ computers) but that just sounds like increased processing speed.

  Finally, does that superiority measure resource usage? Does a superintelligence need to beat us on a watt for watt comparison or could it use the computing capacity of the planet.

  These are just a few concerns but they illustrate the inadequacy of the definition. And it’s not just nitpicking. This loose way of talking about superintelligence invites us, w/​o adequate argument, to assume the relationship we will have to AI is akin to the relationship you have with your dumb family members/​friends. And even if that was the relationship, remember that your dumb friends wouldn’t seem so easily dominated if they hadn’t decided not to put in much effort into intellectual issues.

  Self-improvement

  When it comes to talking about self-improvement the discussion is totally missing any notion of rate, extent or qualitative measure. The tendency is for people to assume that since technology seems to happen fast somehow so will this self-improvement but why should that be?

  I mean we are already capable of self-improvement. We change the culture we pass down over time and as a result a child born today ends up learning more math, history and all sorts of problem solving tools in school that an ancient Roman kid wouldn’t have learned [1]. Will AI self-improvement be equally slow? If it doesn’t improve itself any faster than we improve our intelligence no problem. So any discussion of this issue that seeks to draw any meaningful conclusions needs to make some claim about the rate of improvement and even defining such a quantitative measure seems extremely difficult.

  And it’s not just the instantaneous rate of self-improvement that matters but also the shape of the curve. You seem to grant that figuring out how to improve AI intelligence will take the AI some time to figure out—it’s gotta do the same kind of trial and error we did to build it in the first place—and won’t be instantaneous. Ok, how does that time taken scale with increasing intelligence? Maybe an AI with a 100 SIQ points can build one with 101 SIQ after a week of work. But then maybe it takes 2 weeks for the 101 SIQ AI to figure out how to reach 102 and so on. Maybe it even asymptotes.

  And what does any of this even mean? Is it getting much more capable or marginally capable? Why assume the former? Given the fact that there are mathematical limits on the most efficient possible algorithms shouldn’t we expect an asymptote in ability? Indeed, there might be good reasons to think humans aren’t far from it.

  1: Ofc, I know that people will try and insist that merely having learned a bunch of skills/​tricks in school that help you solve problems doesn’t qualify as improving your intelligence. Why not? If it’s just a measure of ability to solve relevant cognitive challenges such teaching sure seems to qualify. I think the temptation here is to import the way we use intelligence in human society as a measure of raw potential but that relies on a kind of hardware/​software distinction that doesn’t obviously make sense for AI (and arguably doesn’t make sense for humans over long time scales—Flynn effect).

• TruePath 15 Oct 2022 6:59 UTC

  7 points

  4

  on: The shard the­ory of hu­man values

  This is one of the most important posts ever on LW though I don’t think the implications have been fully drawn out. Specifically, this post raises serious doubts about the arguments for AI x-risk as a result of alignment mismatch and the models used to talk about that risk. It undercuts both Bostrom’s argument that an AI will have a meaningful (self-aware?) utility function and Yudkowsky’s reward button parables.

  The role these two arguments play in convincing people that AI x-risk is a hard problem is to explain why, if you don’t anthropomorphize should a program that’s , say, excellent at conducting/​scheduling interviews to ferret out moles in the intelligence community try to manipulate external events at all not just think about them to better catch moles? I mean it’s often the case that ppl fail to pursue their fervent goals outside familiar context. Why will AI be different? Both arguments conclude that AI will inevitably act like it’s very effectively maximizing some simple utility function in all contexts and in all ways.

  Bostrom tries to convince us that as creatures get more capable they tend to act more coherently (more like they are governed by a global utility function). This is of course true for evolved creatures but by offering a theory of how value type things can arise this theory predicts that if you only train your AI in a relatively confined class of circumstances (even if that requires making very accurate predictions about the rest of the world) it isn’t going to develop that kind of simple global value but, rather, would likely find multie shards in tension without clear direction if forced to make value choices in very different circumstances. Similarly, it exains why the AI won’t just wirehead itself by pressing it’s rewaes button.

• TruePath 20 Nov 2018 13:48 UTC

  7 points

  on: [Insert clever in­tro here]

  Given that you just wrote a whole post to say hi and share your background with everyone I’m pretty confident you’ll fit right in and won’t have any problems being too shy. Writing a post like this rather than just commenting is such a less wrong kind of thing to do so I think you’ll be right at home.

• TruePath 27 Mar 2019 8:14 UTC

  6 points

  on: Can a Bayesian agent be in­finitely con­fused?

  There is a lot of philosophical work on this issue some of which recommends taking conditional probability as the fundamental unit (in which case Bayes theorem only applies for non-extremal values). For instance, see this paper

• TruePath 13 Mar 2023 8:48 UTC

  5 points

  2

  on: TruePath’s Shortform

  Maybe this question has already been answered but I don’t understand how recursive self-improvement of AIs is compatible with the AI alignment problem being hard.

  I mean doesn’t the AI itself face the alignment problem when it tries to improve/​modify itself substantially? So wouldn’t a sufficently intelligent AI refuse to create such an improvement for fear the goals of the improved AI would differ from its own?

• TruePath 19 Sep 2018 3:04 UTC

  5 points

  on: Real­ism about rationality

  First, let me say I 100% agree with the idea that there is a problem in the rationality community of viewing rationality as something like momentum or gold (I named my blog rejectingrationality after this phenomena and tried to deal with it in my first post).

  However, I’m not totally sure everything you say falls under that concept. In particular, I’d say that rationality realism is something like the belief that there is a fact of the matter about how best to form beliefs or take actions in response to a particular set of experiences and that many facts about this (going far beyond don’t be dutch booked). With the frequent additional belief that what is rational to do in response to various kind of experiences can be inferred by a priori considerations, e.g., think about all the ways that rule X might lead you wrong in certain possible situations so X can’t be rational.

  When I’ve raised this issue in the past the response I’ve gotten from both Yudkowsky and Hanson is: “But of course we can try to be less wrong,” i.e., have less false beliefs. And of course that is true but that’s a very different notion than the notion of rationality used by rationality realists and misses the way that much of the rationality’s community’s talk about rationality isn’t about literally being less wrong but about classify rules for reaching beliefs into rational and irrational even when they don’t disagree in the actual world.

  In particular, if all I’m doing is analyzing how to be less wrong I can’t criticize people who dogmatically believe things that happen to be true. After all, if god does exist, than dogmatically believing he does makes the people who do less wrong. Similarly the various critiques of human psychological dispositions as leading us to make wrong choices in some kinds of cases isn’t sufficient if those cases are rare and cases where it yields better results are common. However, those who are rationality realists suggest that there is some fact of the matter which makes these belief forming strategies irrational and thus appropriate to eschew and criticize. But, ultimately, aside from merely avoiding getting dutch booked, no rule for belief forming can assure it is less wrong than another in all possible worlds.

• TruePath 16 Aug 2018 9:02 UTC

  5 points

  on: Cause Aware­ness as a Fac­tor against Cause Neutrality

  The inveresly proportional thing is a bad move. Sorting through potential charitable causes is itself charitable work and it’s just crazy inefficient to do that by everyone voting on what tugs at their heartstrings but by paying someone smart to consider all the various pet causes and evaluate them. Worse, the causes that are least well known are often unknown for very good reason but will now get special attention.

  The reason you are right about cases like the doctor example is that when you are actually in a location that then gets hit you *are* leveraging your superior knowledge of how to get things done there or even just understanding what’s happened. Thought, truthfully, the real reason it makes sense is the easier psychological motivation.

• TruePath 9 Sep 2011 10:41 UTC

  5 points

  on: Con­se­quen­tial­ism Need Not Be Nearsighted

  I think the point can be made more simply as follows:

  Consequentialism is a theory about which states of the world (and thus implicitly actions) are preferable. For it to be true does not require or imply that worlds in which some particular bounded agent (or even all humans) believe in consequentialism or attempt to act according to it’s dictates is preferable to the alternatives.

  The obvious and simple counterexample is to have an alien race who tortures all humanity if they use their mind reading technology on anyone and ascertain they believe in consequentialism. In such a situation consequentialism obviously entails that it is better for everyone to believe in some non-consequentialist view of morality.

• TruePath 30 May 2019 17:01 UTC

  4 points

  on: In­finity is an ad­jec­tive like pos­i­tive rather than an amount

  So your intent here is to diagnose the conceptual confusion that many people have with respect to infinity yes? And your thesis is that: people are confused about infinity because they think it has a unique referant while in fact positive and negative infinity are different?

  I think you are on to something but it’s a little more complicated and that’s what gets people are confused. The problem is that in fact there are a number of different concepts we use the term infinity to describe which is why it so super confusing (and I bet there are more).

  1. Virtual Points that are above or below all other values in an ordered ring (or their positive component) which we use as shorthand to write limits and reason about how they behave.

  2. The background idea of the infinite as meaning something that is beyond all finite values (hence why a point at infinity is infinite).

  3. The cardinality of sets which are bijectable with a proper subset of themselves, i.e., infinite. Even here there is an ambiguity between the sets with a given cardinality and the cardinal itself.

  4. The notion of absolute mathematical infinity. If this concept makes sense it does have a single reference which is taken to be ‘larger’ (usually in the sense of cardinality) than any possible cardinal, i.e. the height of the true hierarchy of sets.

  5. The metaphorical or theological notion of infinity as a way of describing something beyond human comprehension and/​or without limits.

  
  

  The fact that some of these notions do uniquely refer while others don’t is a part of the problem.

• TruePath 27 Aug 2018 21:53 UTC

  4 points

  on: For­mal vs. Effec­tive Pre-Commitment

  In particular, I’d argue that the paradoxical aspects of Newcomb’s problem result from exactly this kind of confusion between the usual agent idealization and the fact that actual actors (human beings) are physical beings subject to the laws of physics. The apparent paradoxical aspects results because we are used to idealizing individual behavior in terms of agents where that formalism requires we specify the situation in terms of a tree of possibilities with each path corresponding to an outcome and with the payoff computed by looking at the path specified by all agent’s choices (e.g. there is a node where the demon player chooses what money to put in the boxes and then there is a node where the human player, without knowledge of demon player’s choices, decides to take both boxes or neither). The agent formalization (where 1 or 2 boxing is modeled as a subsequent choice) simply doesn’t allow the content of the boxes to depend on whether or not the human agent chooses 1 or 2 boxes.

  Of course, since actual people aren’t ideal agents one can argue that something like the newcomb demon is physically possible but that’s just a way of specifying that we are in a situation where the agent idealization breaks down.

  This means there is simply no fact of the matter about how a rational agent (or whatever) should behave in newcomb type situations because the (usual) rational agent idealization is incompatible with the newcomb situation (ok, more technically you can model it that way but the choice of how to model it just unsatisfactorily builds in the answer by specifying how the payoff depends on 1 vs two boxing).

  To sum up what the answer to the newcomb problem is depends heavily on how you preciscify the question. Are you asking whether humans who are disposed to decide in way A end up better of than humans disposed to behave in way B? In that case it’s easy. But things like CDT, TDT etc.. don’t claim to be producing facts of that kind but rather saying something about ideal rational agents of some kind which then just boringly depends on a ambiguities in what we mean by that.ideal rational agents.

• TruePath 28 May 2011 17:40 UTC

  4 points

  in reply to: ChristianKl’s comment on: Some Heuris­tics for Eval­u­at­ing the Sound­ness of the Aca­demic Main­stream in Un­fa­mil­iar Fields

  It’s also devilishly hard to accurately research. To isolate the effects of hypnosis from the desire of the subject to please the researcher you need to administer a placebo hypnosis. However, this is very difficult as hypnosis by it’s very nature creates an immediate experiential effect on the patient that you somehow must replicate in the placebo (so the subjects can’t just guess whether they got real hypnosis) without actually performing hypnosis.

  Also there really aren’t any good precise questions to ask about hypnosis. Does it help people quit smoking. Yes, but so does placebo. Does it help people deal with pain. Yes, but again so does placebo. Before you start damning people for not investigating hypnosis describe what precisely you want them to figure out.

[Question] Es­ti­mat­ing Re­turns to In­tel­li­gence vs Num­bers, Strength and Looks