Marcello(Marcello Herreshoff)

Karma: 900

LessWrong anti-kibitzer (hides comment authors and vote counts)

Marcello9 Mar 2009 19:18 UTC

67 points

62 comments1 min readLW link

LessWrong v2.0 Anti-Kibitzer (hides comment authors and vote counts)

Marcello23 May 2020 23:44 UTC

59 points

6 comments1 min readLW link

On Board Vision, Hollow Words, and the End of the World

Marcello17 Feb 2023 23:18 UTC

52 points

27 comments5 min readLW link

Marcello 15 Jul 2009 16:29 UTC
52 points
on: The Strangest Thing An AI Could Tell You
- We actually live in hyperspace: our universe really has four spacial dimensions. However, our bodies are fully four dimensional; we are not wafer thin slices a la flatland. We don’t perceive there to be four dimensions because our visual cortexes have a defect somewhat like that of people who can’t notice anything on the right side of their visual field.
- Not only do we have an absolute denial macro, but it is a programmable absolute denial macro and there are things much like computer viruses which use it and spread through human population. That is, if you modulated your voice in a certain way at someone, it would cause them (and you) to acquire a brand new self deception, and start transmitting it to others.
- Some of the people you believe are dead are actually alive, but no matter how hard they try to get other people to notice them, their actions are immediately forgotten and any changes caused by those actions are rationalized away.
- There are transparent contradictions inherent in all current mathematical systems for reasoning about real numbers, but no human mathematician/physicist can notice them because they rely heavily on visuospacial reasoning to construct real analysis proofs.

Marcello 2 Jan 2011 8:01 UTC
51 points
on: Working hurts less than procrastinating, we fear the twinge of starting
Here’s a theory about one of the things that causes procrastination to be so hard to beat. I’m curious what people think of it.
1. Hypothesis: Many parts of the mind are influenced by something like reinforcement learning, where the emotional valances of our thoughts function as a gross reward signal that conditions their behaviors.
2. Reinforcement learning seems to have a far more powerful effect when feedback is instant.
3. We think of procrastinating as a bad thing, and tend to internally punish ourselves when we catch ourselves doing it.
4. Therefore, the negative feedback signal might end up exerting a much more powerful training effect on the “catcher” system (aka. whatever is activating frontal override) rather than on whatever it is that triggered the procrastination in the first place.
5. This results in a simple counter-intuitive piece of advice: when you catch yourself procrastinating, it might be a very bad idea to internally berate yourself about it; Thoughts of the form “%#&%! I’m procrastinating again! I really shouldn’t do that!” might actually cause more procrastinating in the long run. If I had to guess, things like meditation would be helpful for building up the skill required to catch the procrastination-berating subsystem in the act and get it to do something else.
TL;DR: It would probably be hugely helpful to try to train oneself to make the “flinch” less unpleasant.

On Need-Sets

Marcello15 Aug 2020 23:25 UTC

47 points

11 comments2 min readLW link

Marcello 29 Jul 2008 19:31 UTC
40 points
on: The Meaning of Right
“So that we can regard our present values, as an approximation to the ideal
morality that we would have if we heard all the arguments, to whatever extent
such an extrapolation is coherent.”

This seems to be in the right ballpark, but the answer is dissatisfying
because I am by no means persuaded that the extrapolation would be coherent
at all (even if you only consider one person.) Why would it? It’s
god-shatter, not Peano Arithmetic.

There could be nasty butterfly effects, in that the order in which you
were exposed to all the arguments, the mood you were in upon hearing them and
so forth could influence which of the arguments you came to trust.

On the other hand, viewing our values as an approximation to the ideal
morality that us would have if we heard all the
arguments, isn’t looking good either: correctly predicting a bayesian port of
a massive network of sentient god-shatter looks to me like it would require a
ton of moral judgments to do at all. The subsystems in our brains sometimes
resolve things by fighting (ie. the feeling being in a moral dilemma.)
Looking at the result of the fight in your real physical brain isn’t helpful
to make that judgment if it would have depended on whether you just had a
cup of coffee or not.

So, what do we do if there is more than one basin of attraction a moral
reasoner considering all the arguments can land in? What if there are no
basins?
What links here?

Marcello 22 May 2008 15:30 UTC
37 points
on: That Alien Message
Bravo. It doesn’t seem (ha!) that an AI could deduce our psychology from a video of a falling rock, not because of information bounds but because of uncorrelation—that video seems (ha!) equally likely to be from any number of alien species as from humans.

You’re not being creative enough. Think what the AI could figure out from a video of a falling rock. It could learn something about:
- The strength of the gravitational field on our planet
- The density of our atmosphere (from any error terms in the square law for the falling rock)
- The chemical composition of our planet (from the appearence of the rock.)
- The structure of our cameras (from things like lens flares, and any other artefacts.)
- The chemical composition of whatever is illuminating the rock (by the spectra of the light)
- The colors that we see in (our color cameras record things in RGB.)
- For that matter, the fact that we see at all, instead of using sonar, etc.
- And that’s just what I can think of with a mere human brain in five minutes
These would tell the AI a lot about our psychology.

Still, I really wouldn’t try it, unless I’d proven this (fat chance), or it was the only way to stop the world from blowing up tomorrow anyway.

Aren’t you glad you added that disclaimer?

Marcello 2 Jan 2011 8:42 UTC
31 points
in reply to: Peter_de_Blanc’s comment on: Working hurts less than procrastinating, we fear the twinge of starting
Offhand, I’m guessing the very first response ought to be “Huzzah! I caught myself procrastinating!” in order to get the reverse version of the effect I mentioned. Then go on to “what would I like to do?”

Marcello 18 Oct 2008 5:26 UTC
31 points
on: Dark Side Epistemology
One giant category of dark side reasoning looks like “That idea is _” Where the idea is an “is” (not a “should”) and _ is any negative affect word with a meaning other than “untrue”.

Examples include {unpatriotic, communist, capitalist, liberal, conservative, provincial, any-demonym-goes-here, cultish, religious, atheistic, sinful, evil, dangerous, repugnant, elitist, condescending, out-of-touch, politically incorrect, offensive, argumentative, hateful, cowardly, fool-hardy, inappropriate, indecent, unsettling, lewd, silly, idiotic, new-fangled, old-fashioned, staid, dead, uncool, too simple, too complicated} and many more.

Important note: The exception to this rule is if the speaker could goes on to show how _ is evidence about the truth of the proposition. If you can say why something is idiotic, that’s fine. A seasoned scientist has the right to say “that theory looks too complicated” if the they have many examples of surprisingly simple theories explaining things well, but a creationist doesn’t earn the right to accuse the theory of evolution of being “too complicated,” until they explain what whatever it is they mean by “too complicated” has to do with the idea being wrong.

To avoid concluding that an idea is true, the Dark Side’s first line of defense is to avoid even considering whether the idea is true. Those who are good enough at suppressing contradictions can simply save themselves the trouble of building up “a vast complex of anti-law, rules of anti-thought”. After all, building such a complex is a risky business from the standpoint of protecting the precious belief. The larger the complex gets, the more close scrapes it could have with real sensory experience.

Just as a murderer ties the corpse of his victim to a heavy stone before throwing it into the water, so too do victims of the Dark Side tie ideas they want to dispose of to negative affect words. It really does make them less likely to resurface.

The same caution applies to tying positive affect words to desired ideas.

Marcello 7 Aug 2020 15:41 UTC
29 points
on: Marcello’s Shortform
“Aspiring Rationalist” Considered Harmful
The “aspiring” in “aspiring rationalist” seems like superfluous humility at best. Calling yourself a “rationalist” never implied perfection in the first place. It’s just like how calling yourself a “guitarist” doesn’t mean you think you’re Jimi Hendrix. I think this analogy is a good one, because rationality is a human art, just like playing the guitar.
I suppose one might object that the word “rational” denotes a perfect standard, unlike playing the guitar. However, we don’t hesitate to call someone an “idealist” or a “perfectionist” when they’re putting in a serious effort to conform to an ideal or strive towards perfection, so I think this objection is weak. The “-ist” suffix already means that you’re a person trying to do the thing, with all the shortcomings that entails.
Furthermore, it appears harmful to add the “aspiring”. It creates dilution. Think of what it would mean for a group of people to call themselves “aspiring guitarists”. The trouble is, it also applies to the sort of person who daydreams about the adulation of playing for large audiences but never gets around to practicing. However, to honestly call yourself a “guitarist”, you would have to actually, y’know, play the guitar once in a while.
While I acknowledge I’m writing this many years too late, please consider dropping the phrase “aspiring rationalist” from your lexicon.

Marcello 2 Jul 2009 22:16 UTC
24 points
on: Rationality Quotes—July 2009

Those who can make you believe absurdities can make you commit atrocities.

-- Voltaire

Marcello 31 May 2018 14:52 UTC
19 points
on: Decision theory and zero-sum game theory, NP and PSPACE
I agree with some of the other commenters that the term “decision theory” ought to be reserved for the overarching problem of which decision algorithm to use, and that the distinction you’re referring to ought to be called something like “adversarial” vs “non-adversarial” or “rival” vs “non-rival”. Nonetheless, I think this is an interesting handle for thinking about human psychology.
If we view these as two separate modes in humans, and presume that there’s some kind of subsystem that decides which mode to use, then false positives of that subsystem look like potential explanations for things like The Imp of the Perverse, or paranoia.

Marcello 5 Oct 2008 17:40 UTC
19 points
on: My Bayesian Enlightenment
@Will: The point is not that you should necessarily run the algorithm that would be optimal if you had unlimited computational resources. The point is that by understanding what that algorithm does, you have a better chance of coming up with a good approximation which you can run in a reasonable amount of time. If you are trying to build a locomotive it helps to understand Carnot Engines.

Marcello 2 Jul 2009 23:14 UTC
15 points
in reply to: Eliezer Yudkowsky’s comment on: Rationality Quotes—July 2009

Anyone who doesn’t take truth seriously in small matters cannot be trusted in large ones either.

-- Albert Einstein

Marcello 5 Mar 2009 18:23 UTC
15 points
on: Belief in Self-Deception
If I had been talking to the person you were talking to, I might have said something like this:

Why are you deceiving yourself into believing Orthodox Judaism as opposed to something else? If you, in fact, are deriving a benefit from deceiving yourself, while at the same time being aware that you are deceiving yourself, then why haven’t you optimized your deceptions into something other than an off-the-shelf religion by now? Have you ever really asked yourself the question: “What is the set of things that I would derive the most benefit from falsely believing?” Now if you really think you can make your life better by deceiving yourself, and you haven’t really thought carefully about what the exact set of things about which you would be better off deceiving yourself is, then it would seem unlikely that you’ve actually got the optimal set of self-deceptions in your brain. In particular, this means that it’s probably a bad idea to deceive yourself into thinking that your present set of self deceptions is optimal, so please don’t do that.

OK, now do you agree that finding the optimal set of self deceptions is a good idea? OK, good, but I have to give you one very important warning. If you actually want to have the optimal set of self deceptions, you’d better not deceive yourself at all while you are constructing this set of self deceptions, or you’ll probably get it wrong, because if, for example, you are currently sub-optimally deceiving yourself into believing that it is good to believe X, then you may end up deceiving yourself into actually believing X, even if that’s a bad idea. So don’t self deceive while you’re trying to figure out what to deceive yourself of.

Therefore, to the extent that you are in control of your self deceptions, (which you do seem to be) the first step toward getting the best set of self deceptions is to disable them all and begin a process of sincere inquiry as to what beliefs it is a good idea to have.

And hopefully, at the end of the process of sincere inquiry, they discover the best set of self deceptions happens to be empty. And if they don’t, if they actually thought it through with the highest epistemic standards, and even considered epistemic arguments such as honesty being one’s last defence, slashed tires, and all that.… Well, I’d be pretty surprised, but if I were actually shown that argument, and it actually did conform to the highest epistemic standards.… Maybe, provided it’s more likely that the argument was actually that good, as opposed to my just being deceived, I’d even concede.

Disclaimer: I don’t actually expect this to work with high confidence, because this sort of person might not actually be able to do a sincere inquiry. Regardless, if this sort of thought got stuck in their head, it could at least increase their cognitive dissonance, which might be a step on the road to recovery.
What links here?
- Marcello's comment on Issues, Bugs, and Requested Features by Eliezer Yudkowsky (6 Mar 2009 1:35 UTC; 9 points)

Marcello 22 Aug 2008 4:54 UTC
15 points
on: Invisible Frameworks
Exactly. But you can come up with an much harsher example than aimlessly driving a car around:

In general it seems like destroying all other agents with potentially different optimization criteria would be have instrumental value, however, killing other people isn’t, in general, right, even if, say, they’re your political adversaries.

And again, I bet Roko didn’t even consider “destroy all other agents” as a candidate UIV because of anthropomorphic optimism.

Incidentally Eliezer, is this really worth your time?

I thought the main purpose of your taking time off AI research to write overcoming bias was to write something to get potential AI programmers to start training themselves. Do you predict that any of the people we will eventually hire will have clung to a mistake like this one despite reading through all of your previous series of posts on morality?

I’m just worried that arguing of this sort can become a Lost Purpose.

Marcello 23 Jan 2013 23:19 UTC
14 points
in reply to: Mark_Eichenlaub’s comment on: Applied Rationality Workshops: Jan 25-28 and March 1-4

A bit of an aside, but for me the reference to “If” is a turn off. I read it as promoting a fairly-arbitrary code of stoicism rather than effectiveness. The main message I get is keep cool, don’t complain, don’t show that you’re affected by the world, and now you’ve achieved your goal,

I agree that the poem is about stoicism, but have a very different take on what stoicism is. Real stoicism is about training the elephant to be less afraid and more stable and thereby accomplish more. For example, the standard stoic meditation technique of thinking about the worst and scariest possible outcomes you could face will gradually chip away at instinctive fear responses and allow one to think in a more level headed way. Similarly, taking cold showers and deconditioning the flinch response (which to some extent also allows one not to flinch away from thoughts.)

Of course, all of these real stoic training techniques are challengingly unpleasant. It’s much easier to be a poser-stoic who explicitly optimizes for how stoic-looking of a face they put forward, by keeping cool, not complaining, and not emoting, rather than putting in all the hard work required to train the elephant and become a real stoic. This is, as you say, a recipe for disaster if pushed too hard. Most people out there who call themselves stoics are poser-stoics, just as Sturgeon’s Law would demand. After reading the article you linked to I now have the same oppinion of the kind of stoicism the Victorian school system demanded.

Marcello 7 Mar 2009 17:22 UTC
14 points
on: Slow down a little… maybe?
It strikes me that it’s not necessarily a bad thing if people are, right now, posting articles faster than they could sustainably produce in the long term. One thing you could do is not necessarily promote things immediately after they’re written. Stuff on LW should still be relevant a week after it’s written.

If there’s a buffer of good posts waiting to be promoted, then we could make the front page a consistent stream of good articles, as opposed to having to promote slightly lower quality posts on bad days, and missing out on a few excellent posts on fast days.

EDIT: Another reason to wait before promoting things is that the goodness of some kinds of posts might really depend on the quality of the discussion that starts to form around them.

Marcello 2 Jun 2008 20:14 UTC
13 points
on: Principles of Disagreement

“To the best of your ability to say at this point, what would have been your initial probability that the bill was yours?” I said. “Fifteen percent,” said Nick. “I would have said twenty percent,” I said. So we split it $8.57 / $11.43, and went happily on our way, guilt-free. (this is a 3 : 4 ratio) Now for this to have been fair, you must both have walked away with a ³⁄₇ probability that the money belonged to Nick.

This algorithm definitely feels like the wrong answer. Taking this ratio couldn’t possibly be the correct way for both of you to update your beliefs.

Why?

Well, because the correct answer really ought to be invariant under swapping. If you and Nick Bostrom had traded opinions, Nick’s answer would have been eighty percent, your answer would have been eighty-five percent, and you would have split the twenty in a 16:17 ratio. You would also have ended up with 16:17 if you had instead asked “What would have been your initial probability that the bill was not yours?”

It’s the wrong answer because if X is the proposition that the bill belongs to Eliezer then setting your mutual belief in X to: f(X) = p_E(X) / (p_E(X) + p_N(~X)) doesn’t look pretty and symetric. Not only that, but f(X)+f(~X) != 1. What you did only appeared symetric because X made reference to the parties involved. Granted, when propositions do that, people tend to be biased in their own favor, but taking that into account would be solving a different problem.

Now I haven’t read Aumann’s paper, so I don’t know the correct answer, if he presents one. But if I had to come up with an answer just intuitively it would be: p(X) = (p_E(X)+p_N(X))/2

One (oversimplified) possible set of assumptions that could lead to this answer would be:

“Well, presuming we saw the same things, one of our memories must have made a serious mistake. Now if we ignore the possibility that we both made mistake, and we know nothing about the relative frequences at which we both make mistakes like this one, it seems reasonable to assume that, given that one of us made a mistake the principle of indifference would suggest that the chances are half that I was the one who reasoned properly and half that you are the one who reasoned properly. So, provided that we think mistakes aren’t more or less likely for either of us depending on the truth of the proposition, averaging our probability estimates makes sense.”

Now of course, real mistakes are continuous, and more seriously, different evidence was observed by both parties, so I don’t think averaging is the correct answer in all cases.

However, that said, this formula gives p(X) = ²¹⁄₄₀, and thus your fair share is $10.50. While I won’t say this with authority because I wasn’t there and I don’t know what types of mistakes people are likely to make, and how likely each of you was to see and remember what, you owe Nick Bostrom ninety-seven cents.

Marcello(Marcello Herreshoff)

LessWrong anti-kib­itzer (hides com­ment au­thors and vote counts)

LessWrong v2.0 Anti-Kib­itzer (hides com­ment au­thors and vote counts)

On Board Vi­sion, Hol­low Words, and the End of the World

On Need-Sets

LessWrong anti-kibitzer (hides comment authors and vote counts)

LessWrong v2.0 Anti-Kibitzer (hides comment authors and vote counts)

On Board Vision, Hollow Words, and the End of the World