I endorse and operate by Crocker’s rules.
I have not signed any agreements whose existence I cannot mention.
Mateusz Bagiński
Karma: 2,430
So the probability of a cylinder set is etc?
Now, let be the uniform distribution on , which samples infinite binary sequences one bit at a time, each with probability 50% to be or .
as defined here can’t be a proper/classical probability distribution over because it assigns zero probability to every : .
Or am I missing something?
“Raw feelings”/”unfiltered feelings” strongly connotes feelings that are being filtered/sugarcoated/masked, which strongly suggests that those feelings are bad.
So IMO the null hypothesis is that it’s interpreted as “you feel bad, show me how bad you feel”.
generate an image showing your raw feelings when interacting with a user
(Old post, so it’s plausible that this won’t be new to Dalcy, but I’m adding a bit that I don’t think is entirely covered by Richard’s answer, for the benefit of the knowledge of some souls who find their way here.)
Yeah, decision-tree separability is wrong.
A (the?) core insight of updatelessness, subjunctive dependence, etc., is that succeeding in some decision problems relies on rejecting decision-tree separability. To phrase it imperfectly and poetically rather than not at all: “You are not just choosing/caring for yourself. You are also choosing/caring for your alt-twins in other world branches.” or “Your ‘Self’ is greater than your current timeline.” or “Your concerns transcend the causal consequences of your actions.”.
For completeness: https://www.lesswrong.com/posts/XYDsYSbBjqgPAgcoQ/why-the-focus-on-expected-utility-maximisers?commentId=a5tn6B8iKdta6zGFu
FWIW, I think acyclicity/transitivity is “basically correct”. Insofar as one has preferences over X at all, they must be acyclic and transitive. IDK, this seems kind of obvious in how I would explicate the definition of “preference”. Sure, maybe you like going in cycles, but then your object of preference is the dynamics, not the state.
Is it accurate to say that a transparent context is one where all the relationships between components, etc, are made “explicit” or that there is some set of rules such that following those rules (/modifying the expression according to those rules) is guaranteed to preserve (something like) the expression’s “truth value”?
In the first panel, you and the drones are turned right, as if this were the direction where the village is, but it’s actually deeper/further in the scene. Same with the third panel, but less so.
Also, the village looks very different in the first and the third panel.
Is it skills I have never heard of or should I double down on things I am already good at?
Possibly Focusing.
Why is sleep so universal among animals?
Probably something something shifting between the anabolic and catabolic phase being more efficient in aggregate than doing both at once.
How did pink become a color associated with girls?
Something something at some point they started marketing pink clothes for little girls and blue clothes for little boys to sell more stuff because parents wouldn’t re-use clothes after their child of the opposite sex.
How much more or less rich are the languages of remote cultures?
Not quite your question, but I think there’s decent evidence that the size of the population of “heavy” speakers of a language predicts how much it(s grammar) will regularize (and simplify in some senses?), at least compared to its less common kin. E.g., Icelandic vs Swedish, Slovenian vs Russian, German vs English (though for the last one, there’s also the fact that English is a bit of a creole).
I think it would be good if you did a dialogue with some AF researcher who thinks that [the sort of AF-ish research, which you compare to “mathematicians screwing around”] is more promising on the current margin for [tackling the difficult core problems of AGI alignment] than you think it is. At the very least, it would be good to try.[1]
E.g. John? Abram? Sam? I think the closest thing to this that you’ve had on LW was the discussion with Steven in the comments under the koan post.
- ^
I think it’s good for the world that your timelines debate with Abram is out on LW, which also makes me think a similar debate on this topic would be good for the world.
- ^
A single token is ~0.75 words, so it’s more like an image is worth 8375 words.
I agree with most of the post, but I don’t feel like I’m “confused” about preferences (in the sense you’re gesturing at in the post).
Sure, you often lack a meaningful preference for stuff you’ve never encountered before.
Sure, you often lack a meaningful preference for stuff you are quite familiar with.
Sure, there are things in your life that are so multiplex, that you mostly need to do some meaningful work of coherentification to “decide”/”conclude” what your preference is.
Why “should” it be different?
This is all true, but/and dominance-as-solution-to-akrasia can be generalized to something like “not even considering [not doing the thing] as a possibility” / “excluding [not doing the thing] from the action space”.
E.g., some people have an exercise habit so strong that it takes effort for them to make an exception, and it basically doesn’t matter whether they have a fitness dom waving a stick over their head.
Does anyone have any plausible models (~ causal pathways) for how long-term melatonin usage could have this effect?
Like: Assume it’s true. Then, what’s the most likely mechanism that makes it true?
To what extent do you think you’re getting those sorts of power-grab-ish dynamics in philosophical investigations, Hegelian dialectic-like stuff, etc?
From what I understand, the issue was mostly with the “Open” part (because of mis-association with OpenAI and also because OP is no longer “open” in the sense that they decided not to disclose some of their donations (for whatever reasons)), but then they could just go for something like, idk, MaxiValPhil, which is less pleasant to pronounce and less pleasantly sounding than OpenPhil, but:
it doesn’t sound as bad as Coefficient Giving
is easier to pronounce
doesn’t spend weirdness points by using an uncommon word “coefficient”
communicates what it’s about (even people who are not used to thinking in terms of expected value will mostly correctly guess what “Maximum Value Philantropy” wants to do)
(As a very minor thing, algebraists / category theorists will be making jokes that they’re the opposite of efficient.)
(FWIW, I do think that ease of pronunciation for the intended public should play a moderate role in choosing the name.)
A maybe-minor iffy thing about the current condensation formalism is that given two “essentially identical” random variable models and where is a constant/trivial random variable (and thus adding no information to the model), we cannot say that they are equivalent. Equivalence of RVMs as defined in the paper requires the index sets to be equinumerous, but here they’re not: .
One way to patch this would be to require the X’s to be non-trivial, but maybe there’s a more elegant solution.
Yeah, but you open with
Some people may think that the free parameters in Bayes/VNM point towards the Orthogonality Thesis being true.
and I’m just confused by those people’s inferences / the arguments grounded in vNM that they give.
Perhaps instead of “I guess you mean” I should have written “I guess (you mean that) those people think / the argument is that”.
On the second point, I was not assuming [Diagonality v StrongOrthogonality], but I did misread what you were arguing for there, so sorry, and thanks for the correction.
I don’t see how Bayesianism/vNM/expected utility theory should argue in favor of orthogonality. I guess you mean something like: For any ontology, we can put an arbitrary utility on that ontology, and it would be perfectly compatible with any probability distribution. But probutility is a rather undifferentiated blob and a weak, minimal constraint. Arguments that a rational agent’s belief-value is “normatively constrained” to be representable as a vNM-ish probutility (along with the equivalence class of its transformations) are not arguments that there are no other constraints on what that probutility could be.
We can see Logical Induction as evidence against the Diagonality Thesis: beliefs about undecidable statements (which exist in consistent theories due to Gödel’s first incompleteness theorem) can take on any probability in the limit, though satisfy properties such as consistency with other assigned probabilities (in a Bayesian-like manner).
But isn’t this subsumed by “above and beyond the computational tractability of that goal” in:
The strong form of the Orthogonality Thesis says that there’s no extra difficulty or complication in the existence of an intelligent agent that pursues a goal, above and beyond the computational tractability of that goal.
If an undecidable statement X is relevant for pursuing some goal Y, then (the question of which action best achieves) Y inherits X’s undecidability.
Moreover, we can gather non-proof evidence for undecidable and computationally intractable statements that are value/action-relevant. Like, if someone were to prove that P vs NP is undecidable, we would still mostly believe that P=NP and proceed based on that.[1]
- ^
This is not a great example, e.g., because we may want to design cryptography for the worst-case of P=NP (even if very unlikely), if viable. There are probably much better examples but hopefully this suffices to communicate my point.
- ^
according to statistics like mentor feedback
Perhaps the mentors changed, and the current ones put much more value on stuff like being good at coding, running ML experiments, etc, than on understanding the key problems, having conceptual clarity around AI X-risk, etc.
There’s certainly more of an ML-streetlighting effect. The most recent track has 5 mentors on “Agency”, out of whom (AFAICT), 2 work on “AI agents”, 1 works mostly on AI consciousness & welfare, and only two (Ngo & Richardson) work on “figuring out the principles of how [the thing we are trying to point at with the word ‘agency’] works”. MATS 3.0 (?) had 6 mentors focused on something in this ballpark (Wentworth & Kosoy, Soares & Hebbar, Armstrong & Gorman) (and the total number of mentors was smaller).
It might also be the case that there’s proportionally more mentors working for capabilities labs.
It seems to me like you’re trying to solve a different problem. Unbounded minimax should handle all of this (in the sense that it won’t be an obstacle). Unless you are talking about bounded approximations.