Here’s where I think the conversation went off the rails. :( I think what happened is M.Y.Zuo’s bullshit/woo detector went off, and they started asking pointed questions about the credentials of Critch and his ideas. Vlad and LW more generally react allergically to arguments from authority/status, so downvoted M.Y.Zuo for making this about Critch’s authority instead of about the quality of his arguments.
Personally I feel like this was all a tragic misunderstanding but I generally side with M.Y.Zuo here—I like Critch a lot as a person & I think he’s really smart, but his ideas here are far from rigorous clear argumentation as far as I can tell (I’ve read them all and still came away confused, which of course could be my fault, but still...) so I think M.Y.Zuo’s bullshit/woo detector was well-functioning.
That said, I’d advise M.Y.Zuo to instead say something like “Hmm, a brief skim of those posts leaves me confused and skeptical, and a brief google makes it seem like this is just Critch’s opinion rather than something I should trust on authority. Got any better arguments to show me? If not, cool, we can part ways in peace having different opinions.”
I appreciate the attempt at diagnosing what went wrong here. I agree this is ~where it went off the rails, and I think you are (maybe?) correctly describing what was going on from M.Y. Zou’s perspective. But this doesn’t feel like it captured what I found frustrating.
[/edit]
What feels wrong to me about this is that, for the question of:
How would one arrive at a value system that supports the latter but rejects the former?
it just doesn’t make sense to me to be that worried about either authority or rigor. I think the nonrigorous concept, generally held in society of “respect people’s boundaries/autonomy” is sufficient to answer the question, without even linking to Critch’s sequence. Critch’s sequence is a nice-to-have that sketches out a direction for how you might formalize this, but I don’t get why this level of formalization is even particularly desired here.
(Like, last I checked we don’t have any rigorous conceptions of functioning human value systems that actually work, either for respecting boundaries or aggregating utility or anything else. For purposes of this conversation this just feels like an isolated demand for rigor)
I think that there are many answers along these lines (like “I’m not talking about a whole value system, I’m talking about a deontological constraint”) which would have been fine here.
The issue was that sentences like “It’s a boundary concept (element of a deontological agent design), not a value system (in the sense of preference such as expected utility, a key ingredient of an optimizer)” use the phrasing of someone pointing to a well-known, clearly-defined concept, but then only link to Critch’s high-level metaphor.
Okay, I get where you’re coming from now. Will have to mull over whether I agree but I am at least no longer feel confused about what the disagreement is about now.
Thanks, & thanks for putting in your own perspective here. I sympathize with that too; fwiw Vladimir_Nesov’s answer would have satisfied me, because I am sufficiently familiar with what the terms mean. But for someone new to those terms, they are just unexplained jargon, with links to lots of lengthy but difficult to understand writing. (I agree with Richard’s comment nearby). Like, I don’t think Vladimir did anything wrong by giving a jargon-heavy, links-heavy answer instead of saying something like “It may be hard to construct a utility function that supports the latter but rejects the former, but if instead of utility maximization we are doing something like utility-maximization-subject-to-deontological-constraints, it’s easy: just have a constraint that you shouldn’t harm sentient beings. This constraint doesn’t require you to produce more sentient beings, or squeeze existing ones into optimized shapes.” But I predict that this blowup wouldn’t have happened if he had instead said that.
I may be misinterpreting things of course, wading in here thinking I can grok what either side was thinking. Open to being corrected!
To be clear I super appreciate you stepping in and trying to see where people were coming from (I think ideally I’d have been doing a better job with that in the first place, but it was kinda hard to do so from inside the conversation)
I found Richard’s explanation about what-was-up-with-Vlad’s comment to be helpful.
Thanks for the insight. After looking into ’Vladimir_Nesov’s background I would tend to agree it was because of some issue with the phrasing of the parent comment that triggered the increasingly odd replies, instead of any substantive confusion.
At the time I gave him the benefit of the doubt in confusing what SEP is, what referencing an entry in encyclopedias mean, what I wanted to convey, etc., but considering there are 1505 seemingly coherent wiki contributions to the account’s credit since 2009, these pretty common usages should not have been difficult to understand.
To be fair, I didn’t consider his possible emotional states nor how my phrasing might be construed as being an attack on his beliefs. Perhaps I’m too used to the more formal STEM culture instead of this new culture that appears to be developing.
Here’s where I think the conversation went off the rails. :( I think what happened is M.Y.Zuo’s bullshit/woo detector went off, and they started asking pointed questions about the credentials of Critch and his ideas. Vlad and LW more generally react allergically to arguments from authority/status, so downvoted M.Y.Zuo for making this about Critch’s authority instead of about the quality of his arguments.
Personally I feel like this was all a tragic misunderstanding but I generally side with M.Y.Zuo here—I like Critch a lot as a person & I think he’s really smart, but his ideas here are far from rigorous clear argumentation as far as I can tell (I’ve read them all and still came away confused, which of course could be my fault, but still...) so I think M.Y.Zuo’s bullshit/woo detector was well-functioning.
That said, I’d advise M.Y.Zuo to instead say something like “Hmm, a brief skim of those posts leaves me confused and skeptical, and a brief google makes it seem like this is just Critch’s opinion rather than something I should trust on authority. Got any better arguments to show me? If not, cool, we can part ways in peace having different opinions.”
[edit]
I appreciate the attempt at diagnosing what went wrong here. I agree this is ~where it went off the rails, and I think you are (maybe?) correctly describing what was going on from M.Y. Zou’s perspective. But this doesn’t feel like it captured what I found frustrating.
[/edit]
What feels wrong to me about this is that, for the question of:
it just doesn’t make sense to me to be that worried about either authority or rigor. I think the nonrigorous concept, generally held in society of “respect people’s boundaries/autonomy” is sufficient to answer the question, without even linking to Critch’s sequence. Critch’s sequence is a nice-to-have that sketches out a direction for how you might formalize this, but I don’t get why this level of formalization is even particularly desired here.
(Like, last I checked we don’t have any rigorous conceptions of functioning human value systems that actually work, either for respecting boundaries or aggregating utility or anything else. For purposes of this conversation this just feels like an isolated demand for rigor)
I think that there are many answers along these lines (like “I’m not talking about a whole value system, I’m talking about a deontological constraint”) which would have been fine here.
The issue was that sentences like “It’s a boundary concept (element of a deontological agent design), not a value system (in the sense of preference such as expected utility, a key ingredient of an optimizer)” use the phrasing of someone pointing to a well-known, clearly-defined concept, but then only link to Critch’s high-level metaphor.
Okay, I get where you’re coming from now. Will have to mull over whether I agree but I am at least no longer feel confused about what the disagreement is about now.
(updated the previous comment with some clearer context-setting)
Thanks, & thanks for putting in your own perspective here. I sympathize with that too; fwiw Vladimir_Nesov’s answer would have satisfied me, because I am sufficiently familiar with what the terms mean. But for someone new to those terms, they are just unexplained jargon, with links to lots of lengthy but difficult to understand writing. (I agree with Richard’s comment nearby). Like, I don’t think Vladimir did anything wrong by giving a jargon-heavy, links-heavy answer instead of saying something like “It may be hard to construct a utility function that supports the latter but rejects the former, but if instead of utility maximization we are doing something like utility-maximization-subject-to-deontological-constraints, it’s easy: just have a constraint that you shouldn’t harm sentient beings. This constraint doesn’t require you to produce more sentient beings, or squeeze existing ones into optimized shapes.” But I predict that this blowup wouldn’t have happened if he had instead said that.
I may be misinterpreting things of course, wading in here thinking I can grok what either side was thinking. Open to being corrected!
To be clear I super appreciate you stepping in and trying to see where people were coming from (I think ideally I’d have been doing a better job with that in the first place, but it was kinda hard to do so from inside the conversation)
I found Richard’s explanation about what-was-up-with-Vlad’s comment to be helpful.
Thanks for the insight. After looking into ’Vladimir_Nesov’s background I would tend to agree it was because of some issue with the phrasing of the parent comment that triggered the increasingly odd replies, instead of any substantive confusion.
At the time I gave him the benefit of the doubt in confusing what SEP is, what referencing an entry in encyclopedias mean, what I wanted to convey, etc., but considering there are 1505 seemingly coherent wiki contributions to the account’s credit since 2009, these pretty common usages should not have been difficult to understand.
To be fair, I didn’t consider his possible emotional states nor how my phrasing might be construed as being an attack on his beliefs. Perhaps I’m too used to the more formal STEM culture instead of this new culture that appears to be developing.