gjm
Karma: 13,792
NewTop
This doesn’t (I think) really have much to do with randomness as such. The relevant thing about R is that it’s shared information that a hypothetical adversary doesn’t get to see.
If isn’t chosen adversarially, then our players don’t care about pessimizing over but about something like an average over , and then R isn’t needed. Or, if they are ultra-cautious people who universally care about worst cases, then they don’t care about expectation w.r.t. R but about the worst case as R varies, and then R doesn’t help. R only helps them when is chosen adversarially and R isn’t; or when they care about pessimizing w.r.t but not w.r.t. R.
So the real conclusion here is not “sometimes shared randomness helps”, it’s “if some people are trying to coordinate in the presence of an adversary, anything they share and the adversary has no access to helps”.
Sorry, it wasn’t clear to me whether you were talking about the comments or about the OP; thanks for the clarification.
So, you’re talking about the comments rather than the original post, and specifically about Eliezer’s comments. Except that I can’t find any comments from Eliezer in this thread that show (to me) the slightest sign of assuming that existence requires a beginning. Nor, actually, do the other comments seem to me like they’re making such an assumption. There are a lot of them and I might have missed some, but I’m not seeing a general trend of making that assumption.
I think it would help if you were more specific. Could you point to a few specific things in this discussion that show the assumption in question being relied on?
(As to “If you don’t care about the issue, why respond?“: whether I care depends on exactly what the issue is, and I’m not yet sure what it is. The specific point I was making there is that if you were just saying “one or two people in an 11-year-old discussion made a dubious assumption” then I don’t see why you’d care about that or why anyone else should. If you’re saying that Eliezer made that assumption, or that the whole LW community did, that surprises me more because it doesn’t seem to me like the sort of assumption I would expect either Eliezer or the whole LW community to make.)
OK. Then I confirm that I agree with you: there surely is (for some people if not all) a psychological difference between having debt and having no debt, and it’s surely (for most of those people if not all) in favour of the latter, and Jacob’s article might have been improved by taking that into account.
But here’s a counterargument: he isn’t claiming to offer a complete analysis of debt and why one might choose to take it on or not; he’s pointing out one thing about debt, its “anti-investment” character, and looking in some detail at that. When I started writing this comment I wrote “would have been improved” at the end of the paragraph above, but the more I think about the point in this paragraph the less I believe that, hence the weaker language that stands there now.
Oh! I’ve been interpreting Christian’s comment differently from you. Christian, could you clarify whether “value the psychological factor of having debt at zero” means (1) “place zero value on the psychological consequences of having debt” (which I think is what rossry has been taking you to mean) or (2) “place positive value on the psychological consequences of having zero debt” (which is what I was taking you to mean)? Thanks!
I think probably you’re right, and in any case you’re right that section 5 is about the emotional disutility of volatility; I was misremembering it as having said something about the emotional disutility of being in debt. Sorry about that.
… And having written that, I see that adrusi has made the same point about ambiguity. I’ll leave this here anyway.
Nope. I think the OP doesn’t overlook the emotional disutility of knowing you have debt, and that seems to me sufficient justification for what I think Christian is objecting to.
Are you saying that Eliezer’s original post assumes that existence requires a beginning? (I can’t see that it does.) Or that subsequent discussion in comments here assumes that? (Maybe some people here do, but who cares?)
Why not?
Suppose I feel happier and suffer less stress when I am in no debt than when I am in some debt. Why shouldn’t that be a factor in my decisions? It surely isn’t wrong to value feeling happy and unstressed.
If careful consideration leads me to conclude that those feelings make my life worse rather than better, then perhaps I should try to change them. But as long as I have them, what’s wrong with taking them into account when deciding what to do?
Interesting suggestion. It’s not at all clear to me, though, that 5h/week of small-group time is really more effective than 25h/week or large-group time; it means a considerably longer interval between chances to get any sort of support on any given topic. And the children have to be doing something while they aren’t in those small groups—a major (though not always acknowledged) purpose of the school system is to keep children occupied in a reasonably safe way while their parents get on with their lives, so it seems like there would still need to be some kind of large-group supervision during the rest of the school week. How would you envisage this working out in practice so that it doesn’t end up costing twice as much as the system we have now?
Presumably they were working with a constraint of not spending much more on teaching overall.
If you kick out 10% of children, you then need to find something to do with those 10% of children, who are likely disproportionately in need of more individual attention to keep them on track. (I guess you could just declare them unteachable and useless, but you still have to do something with them and personally I’m not keen on writing people off without very good reason.)
If you reduce class sizes by, say, 25%, then you need 33% more teachers.
If (per waveman’s comment) you get rid of, say, 10% of underperforming teachers then (1) this affects only ~10% of children at any one time, which makes it hard to believe its overall impact would be huge, and (2) you now need to find a whole lot of new competent teachers, which for the usual supply-and-demand reasons means paying more per teacher.
I am very much in favour of being willing to spend a lot more on education, but it isn’t terribly surprising that these projects didn’t do that.
Tut tut tut! Instead of just multiplying together those factors, you need to consider the probability distribution on each one and estimate the resulting probability distribution of N. Most of the distribution will probably have smaller N than your point estimate.
[:-)]
Apparently there’s a thing called Stylus, forked from an earlier version of Stylish. I expect it works with userstyles.org but haven’t checked.
I believe userstyles.org is owned by the same people who own Stylish; I haven’t looked into whether they do anything evil with it, or whether they could if they wanted to.
Since January 2017, the Stylish browser extension has been spyware that transmits every single URL you visit to its corporate owners’ servers. Firefox has just blacklisted it; users of other browsers may want to disable it manually. I think there’s a config option that supposedly stops it sending all that information, if you trust its developers to implement that honestly.
It turns out that the Stylish browser extension is, since January 2017, basically spyware. Firefox has just blacklisted it. LWers who, like me, installed it at some point to experiment with Lesser Wrong or Greater Wrong CSS should seriously consider uninstalling it. There’s probably nothing you can do retroactively about having had every single URL you visited since installing Stylish sent off to SimilarWeb’s servers (and from there presumably to SimilarWeb’s customers).
My apologies for interfering with your typing practice!
Transcript since I find the above basically impossible to read (I have to go and do something else for a bit; will transcribe more when I’m done):
[note: I have not tried to e.g. turn underlining into italics etc.; this was enough effort as it was; nor does my spacing exactly match the original.]
----
Abram’s Machine-Learning model of the benefits of meditation
(a synthesis of Zizian “fusion” and Shinzen’s explanation of meditation, also inspired by some of the ideas in Kaj Sotala’s “My attempt to explain Looking and enlightenment in non-mysterious terms” … but this model is no substitute for those sources and does not summarize what they have to say)
note that I am not an experienced meditator; let that influence your judgement of the validity of what I have to say as it may.
(also heavily influenced by my CFAR level 2 workshop experience)
My immediate inspiration for postulating this model was noticing that after just a little meditation, tolerating cold or hot shower temperatures was much easier.
[picture: person in hot/cold shower]
I had previously been paying attentin to what happens in my mind when I flinch away from too-hot or too-cold temperatures in the shower, as a way to pay attention to “thoughts which lead to action”.
There are several reasons why it might be interesting to pay attention to thoughts which lead to action.
1. “Where’s the steering wheel on this thing, anyway?” [picture: confusing car dashboard] If you’re experiencing “motivational issues”, then it stands to reason that it might be useful to keep an eye on which thoughts are leading to actions and which are not.
2. “Who [or what] is steering this thing?” [picture: car with various people in it] Far from being alone in a mysterious spacecraft, it is more like we are on a big road trip with lots of backseat driving and fighting for the wheel, if you buy the multiagent mind picture.
We often think as if we were unitary, and blame any failings of this picture on a somewhat mysterious limited resource called “willpower”. I’m not implying willpower models are wrong exactly; I’m unsure of what is going on. But bear with me on the multiagent picture...
I think there is a tendency to gravitate toward narratives where an overarching self with coherent goals drives everything—missing the extent to which we are driven by a variety of urges such as immediate comfort. So, I think it is interesting to watch oneself and look for what really drives actions. You don’t often eat because eating is necessary for continuing proper function of body & brain in order to use them for broader goals; you eat because food tastes good / you’re hungry / etc.
Well, maybe. You have to look for yourself. But, it seems easy to mistakenly rationalize goals as belonging to a coherent whole moreso than is the case.
Why would we be biased to think we are alone in an alien spaceship which we only partly know how to steer, whne in fact we are fighting for the wheel in a crowded road-trip?
[picture: same car as before, loudmouth backseat driver circled]
Well, maybe it is because the only way the loudmouth (that is to say, consciousness) gets any respect around here is by maintaining the illusion of control. More on that later.
3. A third reason to be interested in “thoughts which lead to action” is that it is an agentless notion of decision.
Normally we think of a decision made by an atomic agent which could have done one of several things; chooses one; and, does it. [picture: person labelled “agent” with “input” and “output” arrows, and “environment” outside] In reality, there is no solid boundary between an agent and its environment; no fixed interface with a well-defined set of actions which act across the interface.
[picture: brain, spinal cord, muscles, eyeballs, bones, arrows, with circles sketched in various places]
Instead, there are concentric rings where we might draw such a boundary. The brain? The nerves? The muscles? The skin?
With a more agentless notion of agency, you can easily look further out.
Does this person’s thought of political protest cause such a protest to happen? Does the protest lead to the change which it demands?
Anyway. That is quite enough on what I was thinking in the shower. [picture: recap of some of the pictures from above, in a thought bubble] The point is, after meditation, the thoughts leading to action were quite different, in a way which (temporarily) eliminated any resistance which I had to going under a hot or cold shower which I knew would not harm me but which would ordinarily be difficult to get myself to stand under.
(I normally can take cold-water showers by applying willpower; I’m talking about a shift in what I can do “easily”, without a feeling of effort.)
So. My model of this:
I’m going to be a little bit vague here, and say that we are doing something like some kind of reinforcement learning, and the algorithm we use includes a value table:
[picture: table, actions on x-axis, states on y-axis, cells of table are estimated values of taking actions in states]
A value isn’t just the learned estimate of the immediate reward which you get by taking an action in a state, but rather, the estimate of the eventual rewards, in total, from that action.
This makes the values difficult to estimate.
An estimate is improved by value iteration: passing current estimates of values back along state transitions to make values better-informed.
[picture: table like above, with arrows, saying “if (s1,a1)->(s2,a2) is a common transition, propagate backward along the link (s1,a1)<-(s2,a2)”]
For large state & action sets, this can be too expensive: we don’t have time to propagate along all the possible (state,action) transitions.
So, we can use attention algorithms to focus selectively on what is highest-priority to propagate.
The goal of attention is to converge to good value estimates in the most important state,action pairs as efficiently as possible.
Now, something one might conceivably try is to train the attention algorithm based on reinforcement learning as well. One might even try to run it from the very same value table:
[picture: value table as before, actions partitioned into “thinking” actions that propagate values and “standard external actions”]
“The problem with this design is that it can allow for pathological self-reinforcing patterns of attention to emerge. I will provocatively call such self-reinforcing patterns “ego structures”. An ego structure doesn’t so much feed on real control as on the illusion of control.
[picture: loudmouth-representing-consciousness from before, saying “I told you so!“]
The ego structure gets it supply of value by directing attention to its apparent successes and away from its apparent failures, including focusing on interpretations of events which make it look like the ego had more control than it did during times of success, and less than it did in cases of failure.
[picture: car with loudmouth saying “I never said to go left!“]
Some of this will sound quite familiar to students of cognitive bias. One might normally explain these biases (confirmation bias, optimism bias, attribution bias) as arising from interpersonal incentives (like signalling games).
I would not discount the importance of those much, but the model here suggests that internal dynamics are also to blame. In my model, biases arise from wireheading effects. In the societal analogies mentioned earlier, we’re looking at regulatory capture and rent-seeking.
This is rather fuzzy as a concrete mathematical model because I haven’t specified any structure like an “interpretation” -- but I suspect details could be filled in appropriately to make it work. (Specifically, model-based reinforcement needs to somehow be tied in.)
Anyway, where does meditation come in?
My model is that meditation entices an ego structure with the promise of increased focus (i.e., increased attentional control), which is actually delivered, but at the same time dissolves ego structures by training away any contortions of attention which prevent value iteration from spreading value through the table freely and converging to good estimates efficiently.
[picture: happy meditating person with value table including “updating” actions in his/her head]
How does it provide increased control while dissolving control structures?
Well, what are you training when you meditate? Overtly, you are training the ability to keep attention fixed on one thing. This is kind of a weird thing to try to get attention to do. The whole point of attention is to help propagate updates as efficiently as possible. Holding attention on one thing is like asking a computer to load the same data repeatedly. It doesn’t accomplish any computation. Why do it?
[picture: same meditation with an empty-set symbol instead of face + value table]
Well, it isn’t quite a no-operation. Often, the meditative focus is on something which you try to observe in clarity and detail, like the sensations of the body. This can be useful for other reasons.
For the sake of my model, though, think of it as the ego structure trying to keep the attention algorithm in what amounts to a no-op, repeatedly requesting attention in a place where the information has already propagated.
[picture: value table with lots of arrows between the same pair of cells]
The reason this accomplishes anything is that the ego is not in complete control. Shadows dance beneath the surface.
[picture: same value table with a bunch of other arrows sketched in too]
The ego is a set of patterns of attention. It has “attachments” -- obligatory mental gymnastics which it has to keep up as part of the power struggle. Indeed, you could say it is only a set of attachments.
In CFAR terms, an attachment is a trigger-action pattern.
Examples:
“Oh, I mustn’t think that way” (rehearsing a negative association to make sure a specific attention pattern stays squished)
Getting up & getting food to distract yourself when you feel sad
Rehearsing all the reasons you’ll definitely succeed whenever a failure though comes up
Meditation forces you to do nothing whenever these thoughts come up, because the only way to maintain attention at a high level is to develop what is called quanimity: any distracting thought is greeted and set aside in the same way, neither holding it up nor squishing it down. No rehearsal of why you must not think that way. No getting up to go to the fridge. No rehearsal of all the reasons why you will definitely succeed.
Constantly greeting distractions with equanimity and setting them aside fills the value table with zeros where attachments previously lived.
[picture: value table with a bunch of zeros in the left portion labelled “attentional acts”]
Note that these are not fake zeros. You are not rewriting your true values out of existence (though it may feel that way to the ego). You are merely experimenting with not responding to thoughts, and observing that nothing terrible happens.
Another way of thinking about this is un-training the halo effect (though I have not seen any experimental evidence supporting this interpretation). Normally, all entities of conscious experience are imbued with some degree of positive or negative feeling (according to cognitive bias research & experienced meditators alike), which we have flinch-like responses to (trigger-action patterns). Practicing non-response weakens the flinch, allowing more appropriate responses.
Putting zeros in the table can actually give the ego more control by eliminating some competition. However, in the long term, it destabilizes the power base.
You might think this makes sustained meditative practice impossible by removing the very motivational structures trying to meditate; and perhaps it sometimes works that way. Another possibility is that the ego is sublimated into a form which serves to sustain the meditative practice, the skills of mental focus / mindfulness which have been gained, and the practice of equanimity. This structure serves to ensure that propagation of value through the table remains unclogged by attachments in the future. Such a structure doesn’t need to play games to get credit for what it does, since it is actually useful.
Regardless, my advice is that you should absolutely not take this model as an invitation to try and dissolve your ego.
Perhaps take it as an invitation to develop better focus, and to practice equanimity in order to debias halo-effect related problems & make “ugh fields” slowly dissolve.
I have no particular indication that directly trying to “dissolve ego” is a safe or fruitful goal, however, and some reason to think that it is not. The indirect route to un-wireheading our cognitive strategies through a gently rising tide of sanity seems safest.
Speaking of the safety of the approach...
Why doesn’t “zeroing out” the value table destroy our values, again??
At sinceriously.fyi, Ziz talks about core vs structure.
Core is where your true values come from. However, core is not complex enough to interface with the world. Core must create structure to think and act on its behalf.
[picture: central golden circle with complex stuff radiating out from it]
“Structure” means habits of thinking and doing; models, procedures. Any structure is an approximation of how the values represented by the core play out in some arena of life.
So, in this model, all the various sub-agents in your mind arise from the core, as parts of the unfolding calculation of the policy maximizing the core’s values.
These can come into conflict only because they are approximations.
[picture: that car again, with core+radiating-stuff superimposed on it]
The model may sound strange at first, but it is a good description of what’s going on in the value-table model I described. (Or rather, the value-table model gives a concrete mechanism for the core/structure idea.)
The values in the table are approximations which drive an agent’s policy; a “structure” is a subset of the value table which acts as a coherent strategy in a subdomain.
Just removing this structure would be bad; but, it would not remove the core values which get propagated around the value table. Structure would re-emerge.
However, meditation does not truly remove any structure. It only weakens structure by practicing temporary disengagement with it. As I said before, meditation does not introduce any false training data; the normal learning mechanisms are updating on the simple observation of what happens when most of the usual structure is suppressed. This update creates an opportunity to do some “garbage collection” if certain structures prove unnecessary.
According to this model, all irrationality is coming from the approximation of value which is inherent in structure, and much of the irrationality there is coming from structures trying to grab credit via regulatory capture.
(“Regulatory capture” refers to getting undue favor from the government, often in the form of spending money lobbying in order to get legislation which is favorably to you; it is like wireheading the government.)
The reflective value-table model predicts that it is easy to get this kind of irrationality; maybe too easy. For example, addictions can be modeled as a mistaken (but self-reinforcing) attention structure like “But if I think about the hangover I’ll have tomorrow, I won’t want to drink!”
So long as the pattern successfully blocks vlaue propagation, it can stick.
(This should be compared with more well-studied models of such irrationality such as hyperbolic discounting.)
Control of attention is a computationally difficult task, but the premise of Buddhist meditation (particularly Zen) is that you have more to unlearn than to learn. In the model I’m presenting here, that’s because of wireheading by attentional structure.
However, there is some skill which must be learned. I said earlier that one must learn equanimity. Let’s go into what that means.
The goal is to form a solid place on which to stand for the purpose of self-evaluation: an attentional structure from which you can judge your other attentional structures impartially.
[picture: wisdom-seeker on mountaintop nonplussed at being told by lotus-sitting master that all that’s in the way of seeing himself is himself and he should simply stand aside]
If you react to your own thoughts too judgementally, you will learn to hide them from yourself. Better to simply try to see them clearly, and trust the learning algorithms of the brain to react appropriately. Value iteration will propagate everything appropriately if attention remains unblocked.
According to some Buddhist teachings, suffering is pain which is not experienced fully; pain with full mindfulness contains no suffering. This is claimed from experience. Why might this be true? What experience might make someone claim this?
Another idea about suffering is that it results from dwelling on a way that reality differs from how you want it to be which you can’t do anything about.
Remember, I’m speaking from within my machine-learning model here, which I don’t think captures everything. In particular, I don’t think the two statements above capture everything important about suffering.
Within the model, though, both statements make sense. We could say that suffering results from a bad attention structure which claims it is still necessary to focus on a thing even though no value-of-information is being derived from i. The only way this can persist is if the attention structure is refusing to look at some aspects of the situation (perhaps because they are too painful), creating a block to value iteration properly scoring the attentional structure’s worth.
For example, it could be refusal to face the ways in which your brilliant plan to end world hunger will succeed or fail due to things beyond your control. You operate under a model which says that you can solve every potential problem by thinking about it, so you suffer when this is not the case.
From a rationalist perspective, this may at first sound like a good thing, like the attitude you want. But it ruins the value-of-information calculations, ignores opportunity costs, and stops you from knowing when to give up.
To act with equanimity is to be able to see a plan as having a 1% chance of success and see it as your best bet anyway, if best bet it is—and in that frame of mind, to be able to devote your whole being toward that plan; and yet, to be able to drop it in a moment if sufficient evidence accumulates in favor of another way.
So, equanimity is closely tied to your ability to keep your judgements of value and your judgements of probability straight.
Adopting more Buddhist terminology (perhaps somewhat abusively), we can call the opposite of equanimity “attachment” -- to cling to certain value estimates (or certain beliefs) as if they were good in themselves.
To judge certain states of affairs unacceptable rather than make only relative judgements of better or worse: attachment! You rob yourself of the ability to make tradeoffs in difficult choices!
To cling to sunk costs: attachment! You rob your future for the sake of maintaining your past image of success!
To be unable to look at the possibility of failure and leave yourself a line of retreat: attachment! Attachment! Attachment!
To hunt down and destroy every shred of attachment in oneself—this, too, would be attachment. Unless our full self is already devoted to the task, this will teach some structure to hide itself.
Instead, equanimity must be learned gently, through nonjudgemental observation of one’s own mind, and trust that our native learning algorithm can find the right structure if we are just able to pay full attention.
(I say this not because no sect of Buddhism recommends the ruthless route—far from it—nor because I can derive the recommendation from my model; rather, this route seems least likely to lead to ill effects.)
So, at the five-second level, equanimity is just devoted attention to what is, free from immediate need to judge as positive or negative or to interpret within a pre-conceived story.
“Between stimulus and response there is a space. In that space is our power to choose our response. In our response lies our growth and our freedom.” -- Viktor E. Frankl
There’s definitely a lot that is missing in this model, and incorrect. However, it does seem to get at something useful. Apply with care.
-- End --
Ah, I see. Yes, that would be an improvement, maybe 10% as good as just making the stuff be text in the first place.
Since when did GIFs have notably better compression than PNGs? (Perhaps the issue is that these are badly-generated PNGs, and simply loading them into something that knows how to write PNGs and saving them again would produce similar gains?)
This is just spam, right? (Seems like a rather weird sort of spam, though.)
Yes. That might actually be a better question—except that the actually-relevant population is presumably something like “technologies introduced in science fiction that seemed like they might actually be possible in the not-outrageously-far future”.
Bringsjord is the author of a “proof” that P=NP. It is … not an impressive piece of work, or at least I don’t find it so. And it fails to be impressive in a way that seems highly relevant to philosophizing about AI. Namely, Bringsjord seems to think he’s entitled to leap from “such-and-such a physical system seems like it correctly finds optimal solutions to small instances of the Steiner tree problem” to “such-and-such a physical system will somehow find the optimal solution in every instance of the Steiner tree problem, at least given a large enough universe to do it in”, which is complete nonsense, and to my mind making such a fundamental error of reasoning (and doubling down on it when challenged, which seems to have happened) indicates a mind unsuited for this kind of thinking. It’s not just that his logic is bad; it’s that he seems not to be even trying to think in terms of what the universe might actually be doing to solve small easy STP instances, and what might happen when you try to generalize that to big hard ones.
[EDITED to add:] His SEP article more or less endorses Searle’s “Chinese Room” argument, while observing that most AI practitioners think it’s rubbish (with a bit of a sneer at said practitioners). A surprising amount of the article is based specifically on, or responding specifically to, Russell & Norvig’s AIMA, which seems a weird approach for a survey of a whole field to take; one uncharitable but tempting explanation is that Bringsjord doesn’t really know the field so well. The final section is a predictably sneery discussion of predictions of superhuman AI. The thing as a whole isn’t by any means uniformly terrible, but I can’t help thinking they could have done rather better than Bringsjord.