MondSemmel

Karma: 2,703

MondSemmel 22 Sep 2023 12:53 UTC
4 points
0
on: What social science research do you want to see reanalyzed?
This question seems worth crossposting on the EA Forum and the SlateStarCodex subreddit.

MondSemmel 21 Sep 2023 16:55 UTC
4 points
0
in reply to: Richard_Kennaway’s comment on: Reflexive decision theory is an unsolved problem
More stuff:
He co-initiated the framework program “New Frameworks of Rationality” (German Wikipedia, German-only website) which seems to have been active in 2012~2018. Their lists of publications and conferences are in English, as are most of the books in this list.
(Incidentally, there’s apparently a book called Von Rang und Namen: Philosophical Essays in Honour of Wolfgang Spohn.)
And since 2020, he leads this project on Reflexive Decision & Game Theory (longer outline). The site doesn’t list any results of this project, but presumably some of Spohn’s papers since 2020 are related to this topic.
The Koselleck project explores reflexive decision and game theory. Standard decision and game theory distinguish only chance nodes (moves by nature) and action nodes (possible actions of the agent or player, usually and confusingly called decision nodes). The reflexive extensions additionally distinguish genuine decision nodes referring to possible decision situations (decision or game trees plus probability and utility functions) the agent may be in or come to be in. Thus, these decision nodes generate a rich recursive structure. This structure is required, though. Surely, we agents not only rationally act within a given decision situation, but are also able to reflect on the possible situations which might be ours and thus about what determines or causes our actions. And clearly, such reflection is important for rational decision making.
The reflexive extension allows for a general account of anticipatory rationality, i.e., of how to rationally behave in view of arbitrary envisaged changes in one’s decision situation (so far incomepletely treated under the labels “strategic rationality” and “endogenous preference change”). The extension also allows us to account for what is called sensitive rationality, which considers a so far largely neglected point, namely the fact that being in a certain situation not only causes the pertinent rational action, but may have side effects as well (as exemplified in the Toxin puzzle). This fact is ubiquitous in social settings, and it is highly decision relevant and can obviously be accounted for only in the reflexive perspective. Moreover, the extension also allows us to respect the point that it is a matter of our decision when to decide about a certain issue, e.g., whether to commit early or to decide as late as possible. This leads to an account of so-called commissive rationality possibly rationalizing our inclination to commit ourselves. Finally, in the game theoretic extension suggested by the phenomenon of sensitive rationality, the reflexive perspective leads to a new equilibrium concept called dependency equilibria, which, e.g., allows a rationalization of cooperation in the one-shot prisoners’ dilemma and promises a kind of unification of noncooperative and cooperative game theory.

MondSemmel 21 Sep 2023 16:39 UTC
6 points
0
in reply to: Said Achmiz’s comment on: Reflexive decision theory is an unsolved problem
Having searched for “Spohn” on LW, it appears that Spohn was already mentioned a few times on LW. In particular:
11 years ago, in lukeprog’s post Eliezer’s Sequences and Mainstream Academia (also see bits of this Wei Dai comment here, and of this long critical comment thread here):
I don’t think Eliezer had encountered this mainstream work when he wrote his articles
Eliezer’s TDT decision algorithm (2009, 2010) had been previously discovered as a variant of CDT by Wolfgang Spohn (2003, 2005, 2012). Both TDT and Spohn-CDT (a) use Pearl’s causal graphs to describe Newcomblike problems, then add nodes to those graphs to represent the deterministic decision process the agent goes through (Spohn calls them “intention nodes,” Yudkowsky calls them “logical nodes”), (b) represent interventions at these nodes by severing (edit: or screening off) the causal connections upstream, and (c) propose to maximize expected utility by summing over possible values of the decision node (or “intention node” / “logical node”). (Beyond this, of course, there are major differences in the motivations behind and further development of Spohn-CDT and TDT.)
And the comments here on the MIRI paper “Cheating Death in Damascus” briefly mention Spohn.
Finally, a post from a few months ago also mentions Spohn:
I introduce dependency equilibria (Spohn 2007), an equilibrium concept suitable for ECL, and generalize a folk theorem showing that the Nash bargaining solution is a dependency equilibrium.

MondSemmel 21 Sep 2023 13:33 UTC
2 points
0
in reply to: cubefox’s comment on: Reflexive decision theory is an unsolved problem
(The impossibility of precommitment was illustrated in Kavka’s toxin puzzle.)
The toxin puzzle is also referenced extensively in that aforementioned Spohn paper on one-boxing, and his paper is a response to the toxin puzzle as much as it is to two-boxing.

MondSemmel 21 Sep 2023 13:24 UTC
4 points
−1
in reply to: Said Achmiz’s comment on: Reflexive decision theory is an unsolved problem
I’m astounded that I’ve never heard of him before now.
It probably doesn’t help that he’s a German philosopher; language barriers are a thing.
On the other hand, his research interests seem to have lots of overlap with LW content:
Spohn is best known for his contributions to formal epistemology, in particular for comprehensively developing ranking theory^[1] since 1982, which is his theory of the dynamics of belief. It is an alternative to probability theory and of similar philosophical impact as a formal account of the dynamics of belief. Spohn’s research extends to philosophy of science, the theory of causation, metaphysics and ontology, philosophy of language and mind, two-dimensional semantics, philosophical logic, and decision and game theory (see the collection of papers^[2]). His dissertation^[3] and his paper “Stochastic Independence, Causal Independence, and Shieldability”^[4] are precursors of the theory of Bayesian networks and their causal interpretation. His paper “How to Make Sense of Game Theory”^[5] is a forerunner of epistemic game theory.

MondSemmel 20 Sep 2023 19:30 UTC
2 points
0
in reply to: tlevin’s comment on: Protest against Meta’s irreversible proliferation (Sept 29, San Francisco)
Thanks, that’s very helpful context. In principle, I wouldn’t put too much stock in the specific numbers of a single poll, since those results depend too much on specific wording etc. But the trend in this poll is consistent enough over all questions that I’d be surprised if the questions could be massaged to get the opposite results, let alone ones 10x in favor of the accelerationist side.
(That said, I didn’t like the long multi-paragraph questions further down in the poll. I felt like many were phrased to favor the cautiousness side somewhat, which biases the corresponding answers. Fortunately there were also plenty of short questions without this problem.)

MondSemmel 20 Sep 2023 17:35 UTC
2 points
0
in reply to: Raemon’s comment on: Open Thread – Autumn 2023
That justification makes sense. Though to make the search behavior less weird, it would be good if the search results a) were deduplicated, and maybe b) didn’t display the horizontal divider bars for empty sections.

MondSemmel 20 Sep 2023 17:30 UTC
2 points
0
in reply to: Raemon’s comment on: Open Thread – Autumn 2023
Apologies. After posting my original comment I noticed myself what you mention in your first paragraph, realized that my initial annoyance was obviously unwarranted, and thus edited my original comment before I even saw your reply.
Anyway, see my edited comment above: I found at least three reactions that are missing in the grid view.

MondSemmel 20 Sep 2023 17:15 UTC
2 points
0
in reply to: Raemon’s comment on: Open Thread – Autumn 2023
Update: EDIT: Several reactions appear to be missing in grid view: “Thanks”, “Changed my Mind”, and “Empathy”.
In the first place, I made my original bug report because I couldn’t find the Thanks reaction, looked through all the reactions one by one, and thus noticed the doubled Thumbs Up reactions. I eventually decided I’d hallucinated there being a Thanks reaction, or that it was only available on the EA Forum—but I just noticed that it’s still available, it’s just missing in grid view.

MondSemmel 20 Sep 2023 17:05 UTC
3 points
0
in reply to: Cole Wyeth’s comment on: Protest against Meta’s irreversible proliferation (Sept 29, San Francisco)
In case of issues where one “pulls sideways”, politically speaking, I also expect indirect effects to be comparatively unimportant. But in political zero-ish-sum conflicts, I’m more apprehensive of indirect effects and arguments.

MondSemmel 20 Sep 2023 7:47 UTC
5 points
3
on: Protest against Meta’s irreversible proliferation (Sept 29, San Francisco)
I’m confused about the game theory of this kind of protest. If protests don’t work, fine, no harm done either way. But if they do work, what’s to stop the “do publicize this!” crowd (accelerationists, open source AI people, etc) from protesting on their own? Also, I have no idea about the relative numbers, but what if they could protest with 10x the number of people?

MondSemmel 19 Sep 2023 18:49 UTC
5 points
2
in reply to: Raemon’s comment on: Open Thread – Autumn 2023
Why not just leave that spot empty, though? Or rather, the right-most spot in the second row.
The current implementation, where reaction icons aren’t deduplicated, might (debatably) look prettier in some sense, but it has other weird consequences. Like this and this:

MondSemmel 19 Sep 2023 16:41 UTC
3 points
0
on: Open Thread – Autumn 2023
FYI, current comment reactions bug (at least in desktop Firefox):

MondSemmel 17 Sep 2023 10:14 UTC
2 points
0
in reply to: Elizabeth’s comment on: Elizabeth’s Shortform
Out of curiosity, did Roam turn out to support the functionality I mentioned in my other comment here?

MondSemmel 16 Sep 2023 23:20 UTC
10 points
5
in reply to: tailcalled’s comment on: Closing Notes on Nonlinear Investigation
I probably don’t understand the risks!
One risk is that similar-sounding institutions can and do occasionally get taken over precisely by the people they’re setup to prevent, and then those people have institutional backing and are even harder to dislodge.
E.g. see the section on Legible Signals from a podcast / interview with habryka from early this year:
Like in the context of the FTX situation, a proposal that I’ve discussed with a number of people is, “Fuck, man, why did people trust Sam? I didn’t trust Sam.” “What we should have done is,we should have just created a number, like, there’s 10 cards and these are the ‘actually trustworthy people’ and ‘high-integrity people’ cards, and we should have given them to 10 people in the EA community who we actually think are [highly] trustworthy and high-integrity people, so that it actually makes sense for you to trust them. And we should just have [high-legibility] signals of trust and judgment that are clearly legible to the world.”
To which my response was “Lol, that would have made this whole situation much worse.” I can guarantee you that if you [had] handed a number of people—in this ecosystem or any other ecosystem—the “This person has definitely good judgment and you should trust what they say.” [card. Then] in the moment somebody has that card and has that official role in the ecosystem, of course they will be [under a] shitton of adversarial pressure, for [them to] now endorse people who really care about getting additional resources, who really care about stuff.
...
And then I’m like, “Well, there are two worlds. Either nobody cares about who you think is high-integrity and trustworthy, or people *do* care and now you’ve made the lives of everyone who you gave a high-integrity / trustworthy card a lot worse. Because now they’re just an obvious giant target, that if you successfully get one of the people of the high-integrity, high-trustworthy cards to endorse you, you have free reign and now challenging you becomes equivalent to challenging the “high-integrity, [high-trust] people” institution. Which sure seems like one of the hardest institutions to object to.
And I think we’ve seen this in a number of other places… There was a specific board set up by the Center for Applied Rationality, [where] CFAR kept being asked to navigate various community disputes, and they were like, “Look, man, we would like to run workshops, can we please do anything else?” And then they set up a board to be like, “Look, if you have community disputes in the Bay Area, go to this board. They will maybe do some investigative stuff, and then they will try to figure out what should happen, like, do mediation, maybe [speak about] who was actually in the right, who was in the wrong.”
And approximately the first thing that happened is that, like, one of the people who I consider most abusive in the EA community basically just captured that board, and [got] all the board members to endorse him quite strongly. And then when a bunch of people who were hurt by him came out, the board was like, “Oh, we definitely don’t think these [people who were abused] are saying anything correct. We trust the guy who abused everyone.”
Which is a specific example of, if you have an institution that is being given the power to [blame] and speak judgment on people, and try to create common knowledge about [what] is trustworthy and what is non-trustworthy, that institution is under a lot of pressure...
[We] can see similar things happening with HR departments all around the world. Where the official [purpose] of the HR department is to, you know, somehow make your staff happy and give them [ways to] escalate to management if their manager is bad. But most HR departments around the world are actually, like, a trap, where if you go to the HR department, probably the person you complained about is one of the first people to find out, and then you can be kicked out of the organization before your complaint can travel anywhere else.
It’s not true in all HR departments but it’s a common enough occurrence in HR departments that if you look at Hacker News and are like, “Should I complain to HR about my problems?”, like half of the commenters will be like, “Never talk to HR.” HR is the single most corrupt part of any organization. And I think this is the case because it also tends to be the place where hiring and firing [decisions get] made and therefore is under a lot of pressure.

MondSemmel 16 Sep 2023 22:54 UTC
12 points
3
in reply to: shminux’s comment on: The commenting restrictions on LessWrong seem bad
It is very much possible to express controversial, contrarian or even outright silly views without being downvoted. The rule of thumb is to be respectful and charitable to your opponents and, well, read the room. The more your view diverges from the local mainstream, the better, tighter and clearer your argument must be.
Regarding this, I like to quote Scott’s rules for commenting on his sites:
If you make a comment here, it had better be either true and necessary, true and kind, or kind and necessary.
Recognizing that nobody can be totally sure what is or isn’t true, if you want to say something that might not be true – anything controversial, speculative, or highly opinionated – then you had better make sure it is both kind and necessary. Kind, in that you don’t rush to insult people who disagree with you. Necessary in that it’s on topic, and not only contributes something to the discussion but contributes more to the discussion than it’s likely to take away through starting a fight.
Nobody can be kind all the time, but if you are going to be angry or sarcastic, what you say had better be both true and necessary. You had better be delivering a very well-deserved smackdown against someone who is uncontroversially and obviously wrong, in a way you can back up with universally agreed-upon statistics… And it had better be necessary, in that you are quashing a false opinion which is doing real damage and which is so persistent that you don’t think any more measured refutation would be effective.

MondSemmel 16 Sep 2023 22:46 UTC
3 points
0
in reply to: Ruby’s comment on: The commenting restrictions on LessWrong seem bad
(Possibly if the conditions under which we think that was warranted, we should issue a ban.)
? (EDIT: I mean that the quoted sentence is currently missing some word and is hence incomprehensible.)

MondSemmel 16 Sep 2023 12:51 UTC
4 points
1
on: Navigating an ecosystem that might or might not be bad for the world
Meta note: FYI, the comments counter on discussions is still bugged: at the top of the page (and in the LW feed) it says there are 57 comments, while the bottom only lists 5. So presumably the 52 missing comments are the individual segments of the discussion.

MondSemmel 16 Sep 2023 8:21 UTC
7 points
0
on: Closing Notes on Nonlinear Investigation
Meta: Once again, here is a link to the crosspost on the EA Forum.

MondSemmel 14 Sep 2023 9:24 UTC
2 points
0
in reply to: Smaug123’s comment on: I’m consistently overwhelmed by basic obligations. Are there any paradigm shifts or other rationality-based tips that would be helpful?
I love hearing about the overlap between personal life stuff and concepts from technical fields. So thanks a lot for this comment! Another instance of this topic is the advice from the book Algorithms to Live By.
Also, I’d love to hear more about this stuff. Furthermore, I think the general theme of “general-purpose advice based on what I learned in my work” is very fruitful for blog posts, including LW posts.