I am a PhD student in computer science at the University of Waterloo.
My current research is related to Kolmogorov complexity. Sometimes I build robots, professionally or otherwise.
See my personal website colewyeth.com for an overview of my interests.
You have pointed out some important tradeoffs. Many of my closest friends and intellectual influences outside of lesswrong are religious, and often have interesting perspectives and ideas (though I can’t speak to whether this is because of their religions, caused by a common latent variable, or something else). However, I do not think that the purpose of lesswrong is served by engaging with religious ideology here, and I think that avoiding this is probably worth the cost of losing some valuable perspectives.
As you’ve said, @Jeffrey Heninger does participate in the lesswrong community, at its current level of hostility towards religion. I have read some of his other posts in the past and found them enjoyable and valuable, though I think I am roughly indifferent to this one being published. Why does this suggest to you that the community needs to be less hostile to religion, instead of more or roughly the same amount? Presumably if it were less hostile towards religion, there would be more than the current level of religious discussion—do you think that would be better on the margin? I would also expect an influx of religious people below Jeffrey’s level, not above it.
I’m open to starting a dialogue if you want to discuss this further.
Perhaps AGI but not human level. A system that cannot drive a car or cook a meal is not human level. I suppose it’s conceivable that the purely cognitive functions are at human level, but considering the limited economic impact I seriously doubt it.
Hostility towards Others may be epistemically and ethically corrosive, but the kind of hostility I have discussed is also sometimes necessary. For instance, militaristic jingoism is bad, and I am hostile to it. I am also wary of militaristic jingoists, because they can be dangerous (this is an intentionally extreme example; typical religions are less dangerous).
There is a difference between evangelizing community membership and evangelizing an ideology or set of beliefs.
Usually, a valuable community should only welcome members insofar as it can still maintain its identity and reason for existing. Some communities, such as elite universities, should and do have strict barriers for entry (though the specifics are not always ideal). The culture of lesswrong would probably be erased (that is, retreat to other venues) if lesswrong were mainstreamed and successfully invaded by the rest of the internet.
I generally agree that (most) true beliefs should be shared. Ideologies however are sometimes useful to certain people in certain contexts and not to other people in other contexts. Also, evangelism is costly, and it’s easy to overestimate the value of your ideology to others.
The distinctions you’re pointing out are subtle enough that it may be better to ask people instead of trying to infer their beliefs from such a noisy signal. I reject painting religious people as uniformly or even typically villainous, but it is probably fine to share a common frustration with some religions and religious institutions shutting down new ideas. I was not at secular solstice so I can’t say for sure which better describes it, but based on the examples given the former seems at least as accurate.
I appreciate the perspective. Personally I don’t really see the point of a secular solstice. But frankly, the hostility to religion is a feature of the rationalist community, not a bug.
Rejection of faith is a defining feature of the community and an unofficial litmus test for full membership. The community has a carefully cultivated culture that makes it a kind of sanctuary from the rest of the world where rationalists can exchange ideas without reestablishing foundational concepts and repeating familiar arguments (along with many other advantages). The examples you point to do not demonstrate hostility towards religious people, they demonstrate hostility towards religion. This is as appropriate here as hostility towards factory farming is at a vegan group.
Organizations (corporate, social, biological) are all defined by their boundaries. Christianity seems to be unusually open to everyone, but I think this is partially a side effect of evangelism. It makes sense to open your boundaries to the other when you are trying to eat it. Judaism in contrast carefully enforces the boundaries of its spaces.
Lesswrong hates religion in the way that lipids hate water. We want it on the outside. I don’t know about other rationalists, but I don’t have a particular desire to seek it out and destroy it everywhere it exists (and I certainly wish no harm to religious people). I agree with you that too much hostility is harmful; but I don’t agree that good organizations must always welcome the other.
I don’t find these intuitive arguments reliable. In particular, I doubt it is meaningful to say that reflective oracle AIXI takes the complexity of its own counterfactual actions into account when weighing decisions. This is not how its prior works or interacts with its action choice. I don’t fully understand your intuition, and perhaps you’re discussing how it reasons about other agents in the environment, but this is much more complicated than you imply, and probably depends on the choice of reflective oracle. (I am doing a PhD related to this).
I think about this a lot too. One of my biggest divergences with (my perception of) the EA/rationalist worldview is a desire to leave “wilderness”/”wildness” in the world. I think that all of us should question our wisdom.
The concern may become valid in the future but honestly if this description of your circumstances is accurate I strongly suggest you completely stop reading lesswrong, instead focus on your financial circumstances and mental health.
It’s true that classical that modern RL models don’t try to maximize their reward signal (it is the training process which attempts to maximize reward), but DeepMind tends to build systems that look a lot like AIXI approximations, conducting MCTS to maximize an approximate value function (I believe AlphaGo and MuZero both do something like this). AIXI does maximize it’s reward signal. I think it’s reasonable to expect future RL agents will engage in a kind of goal directed utility maximization which looks very similar to maximizing a reward. Wireheading may not be a relevant concern, but many other safety concern around RL agents remain relevant.
It is probably possible to make some form of scaffolding work, I’m just skeptical that it’s going to be as effective as training an agent directly. Depending on timelines, scaffolding might still feed progress towards superintelligence.
One could hook up a language model to decide what to visualize, Sora to generate visualizations, and a vision model to extract outcomes.
This seems like around 40% of what intelligence is—the only thing I don’t really see is how reward should be “plugged in,” but there may be naive ways to set goals.
this is mind blowing. When it works it’s crazy, even the “glitches” are bizarre like the real world were a slightly broken video game.
Remarkably long context window can process many tokens, but can it process the token safety discussion ;)
Yes, this is a good point. Confidence in delegation is probably a panic reducing factor. In a way delegating to one’s future self can be viewed as a special case.
I don’t mean to imply that levels of planning are objective. I think that solving math problems usually has relatively few levels of planning involved, but requires domain specific competency and a pretty high level of fluid intelligence. Also, the levels of planning required depends on the person—at a low level of mathematical maturity, each recursive level of chasing definitions might feel like a level of planning, though for me “chasing definitions” is itself effectively one step. I do think that in mathematics it can be easier to compartmentalize the levels of planning, which perhaps is one of a combination of factors that makes me good at it. This note is probably worth adding to the post, thank you.
I did not that executives who don’t panic at many levels of planning can still fail because they lack domain specific competency. Difficulty and levels of planning are not identical.
Should be fixed.