I think that the approaches based on being a holy madman greatly underestimates the difficulty on being a value maximiser running on corrupted, basic human hardware.
I’d be extremely skeptical on anyone who claims to have found a way to truly maximise it’s utility function, even if they claim to have avoided all the obvious pitfalls of burning out and so-so.
It would be extremely hard to conciliate “put forth your full effort” and staying rational enough to notice you’re burning yourself out or noticing that you’re getting stuck in some suboptimal route because you’re not leaving yourself enough slack to notice better opportunities.
The detached academic seems to me an odd way to describe Scott Alexander, who seems to make a really effective effort to spread his values and live his life rationally, for him most of the issues he talks about seem to be pretty practical and relevant, even if he often takes interest on what makes him curious and isn’t dropping everything to work on AI—maximise the number of competent people who would work on AI.
I’m currently in a now-nine-months-long attempt to move from detached-lazy-academic to make an extraordinary effort.
So far every attempt to accurately predict how much of a full effort I can make without getting backlash that makes me worse at it in the next period has failed.
Lots of my plans have failed, so if I had went along with plans that required me to make sacrifices, as taking an idea Seriously would require you to do, would have left me at a serious loss.
What worked most and obtained the most result was keeping a curious attitude toward plans and subjects that are related to my goal, studying to increase my competence in related areas even if I don’t see any immediate way it could be of help, and monitoring on how much “weight” I’m putting on the activities that produce the results I need.
I feel I started out being unbelievably bad at working seriously at something, but in nine months I got more results than in a lifetime (in a broad sense, not just related to my goal) and I feel like I went up a couple levels.
I try to avoid going toward any state that resembles a “holy madman” for fear of crashing hard, and I notice that what I’m doing already has me pass as one to even my most informed friends on related subjects, when I don’t censor to look normally modest and uninterested.
I might be just at such a low level in the skill of “actually working” that anything that would work great for a functional adult with a good work ethic is deadly to me.
But I’d strongly advise anyone trying the holy madman path to actively pump for as much “anti-holy-madmannes” as they can, since making the full effort to maximise for something seems to me the best way to make sure your ambition burns through any defence your naive, optimistic plans think you have put in place to protect your rationality and your mental health.
Cults are bad, becoming a one-man-cult is entirely possible and slightly worse.
It would be extremely hard to conciliate “put forth your full effort” and staying rational enough to notice you’re burning yourself out or noticing that you’re getting stuck in some suboptimal route because you’re not leaving yourself enough slack to notice better opportunities.
I bet if you said this to Nate he’d have a pretty convincing counter. Even though Nate works some ridiculous number of hours a week (in contrast to me; I’m closer to the standard 40 hours), I suspect he has enough slack, and thinks of this as part of the optimization problem.
Part of the skill of optimizing without shooting yourself in the foot is explicitly counting slack as part of the optimization problem.
Part of the meta-skill of learning to do this is always asking yourself whether you’re falling into some kind of trap (mostly, forms of Goodhart), and prioritizing steps which avoid traps. EG if you were a self-modifying AGI, you would do well to self-modify in a cautious way, rather than as soon as something looks positive-EV.
However, I’m not sure whether this caution eventually cashes out to “don’t be a holy madman” vs “here’s how to be the right kind of holy madman”, in the terms of the post.
Lots of my plans have failed, so if I had went along with plans that required me to make sacrifices, as taking an idea Seriously would require you to do, would have left me at a serious loss.
Yeah, I feel that I can similarly look back at my history and say that in several cases, it either has been better or would have been much better to be more the detached academic.
Mh… I guess “holy madman” is a definition too vague to make a rational debate on it? I had interpreted it as “sacrifice everything that won’t negatively affect your utility function later on”. So the interpretation I imagined would be someone that won’t leave himself an inch of comfort more than what’s needed to keep the quality of his work constant.
I see slack as leaving yourself enough comfort that you’d be ready to use your free energy in ways you can’t see at the moment, so I guess I was automatically assuming an “holy madman” would optimise for outputting the current best effort he can in the long term, rather than sacrificing some current effort to bet on future chances to improve the future output.
I’d define someone who’s leaving this level of slack as someone who’s making a serious or full effort, but not an holy madman, but I guess this doesn’t means much.
If I were to try to summarise my thoughts on what would happen in reality if someone were to try these options… I think the slack one would work better in general, both by managing to avoid pitfalls and to better exploit your potential for growth.
I still feel there’s a lot of danger to oneself in trying to take ideas seriously though. If you start trying to act like it’s your responsibility to solve a problem that’s killing people, the moment you lose your grip on your thoughts it’s the moment you cut yourself badly, at least in my experience.
In these days I’ve managed to reduce the harm that some recurrent thoughts were doing by focusing on distinguish between 1) me legitimately wanting A and planning/acting to achieve A and 2) my worries related to not being able to get A or distress for things currently being not A, telling myself that 2) doesn’t helps me get what I want in the least, and that I can still make a full effort for 1), likely a better one, without paying to 2) much attention.
(I’m afraid I’ve started to slightly rant from this point. I’m leaving it because I still feel it might be useful)
This strategy worked for my gender transition. I’m not sure how I’d react if I were to try telling myself I shouldn’t care/feel bad/worry if people die because I’m not managing to fix the problem, even if I KNOW that worrying myself about people dying hinders my effort to fix the problem because feeling sick and worried and tired wouldn’t in any way help toward actually working on the problem, I still don’t trust my corrupted hardware to not start running some guilt trip against me because I’m trying to be, in a sense that’s not utilitarian at all, callous, because I’m trying to not care/feel bad/worry about something like that.
Also, as a personal anecdote of possible pitfalls, trying to take personal responsibility for a global problem had drained my resources in ways I could’t foreseen easily. When I got jumped by an unrelated problem about my gender, I found myself without the emotional resources to deal with both stresses at once, so some recurrent thoughts started blaming me because I was letting a personal problem that was in no way as bad as being dead, and didn’t blipped at all on any screen in confront to a large number of deaths, screw up with my attempt of working on something that was actually relevant. I realised immediately that this was a stupid thing to think and in no way healthy, but that didn’t do much to stop it, and climbing out of that pit of stress and guilt took a while.
In short, my emotional hardware is stupid and bugged and it irritates me to no end how it can just go ahead and ignore my attempts of thinking sanely about stuff.
I’m not sure if I’m just particularly bad at this, or if I just have expectations that are too high. An external view would likely tell me that it’s ridiculous for me to expect to be able to go from “lazy and detached” to “saving the world (read reducing X risk), while effortlessly holding at bay emotional problems that would trip most people”. I’d surely tell anyone that. On the other hand, it just feels like a stupid thing to not manage doing.
(end of the rant)
(in contrast to me; I’m closer to the standard 40 hours)
Can I ask if you have some sort of external force that makes you do these hours? If not, any advice on how to do that?
I’m coming from a really long tradition of not doing any work whatsoever, and so far I’m struggling to meet my current goal of 24 hours (also because the only deadlines are the ones I manage to give myself… and for reasons I guess I have explained above).
Getting to this was a massive improvement, but again, I feel like I’m exceptionally bad at working hard.
I think that the approaches based on being a holy madman greatly underestimates the difficulty on being a value maximiser running on corrupted, basic human hardware.
I’d be extremely skeptical on anyone who claims to have found a way to truly maximise it’s utility function, even if they claim to have avoided all the obvious pitfalls of burning out and so-so.
It would be extremely hard to conciliate “put forth your full effort” and staying rational enough to notice you’re burning yourself out or noticing that you’re getting stuck in some suboptimal route because you’re not leaving yourself enough slack to notice better opportunities.
The detached academic seems to me an odd way to describe Scott Alexander, who seems to make a really effective effort to spread his values and live his life rationally, for him most of the issues he talks about seem to be pretty practical and relevant, even if he often takes interest on what makes him curious and isn’t dropping everything to work on AI—maximise the number of competent people who would work on AI.
I’m currently in a now-nine-months-long attempt to move from detached-lazy-academic to make an extraordinary effort.
So far every attempt to accurately predict how much of a full effort I can make without getting backlash that makes me worse at it in the next period has failed.
Lots of my plans have failed, so if I had went along with plans that required me to make sacrifices, as taking an idea Seriously would require you to do, would have left me at a serious loss.
What worked most and obtained the most result was keeping a curious attitude toward plans and subjects that are related to my goal, studying to increase my competence in related areas even if I don’t see any immediate way it could be of help, and monitoring on how much “weight” I’m putting on the activities that produce the results I need.
I feel I started out being unbelievably bad at working seriously at something, but in nine months I got more results than in a lifetime (in a broad sense, not just related to my goal) and I feel like I went up a couple levels.
I try to avoid going toward any state that resembles a “holy madman” for fear of crashing hard, and I notice that what I’m doing already has me pass as one to even my most informed friends on related subjects, when I don’t censor to look normally modest and uninterested.
I might be just at such a low level in the skill of “actually working” that anything that would work great for a functional adult with a good work ethic is deadly to me.
But I’d strongly advise anyone trying the holy madman path to actively pump for as much “anti-holy-madmannes” as they can, since making the full effort to maximise for something seems to me the best way to make sure your ambition burns through any defence your naive, optimistic plans think you have put in place to protect your rationality and your mental health.
Cults are bad, becoming a one-man-cult is entirely possible and slightly worse.
I bet if you said this to Nate he’d have a pretty convincing counter. Even though Nate works some ridiculous number of hours a week (in contrast to me; I’m closer to the standard 40 hours), I suspect he has enough slack, and thinks of this as part of the optimization problem.
Part of the skill of optimizing without shooting yourself in the foot is explicitly counting slack as part of the optimization problem.
Part of the meta-skill of learning to do this is always asking yourself whether you’re falling into some kind of trap (mostly, forms of Goodhart), and prioritizing steps which avoid traps. EG if you were a self-modifying AGI, you would do well to self-modify in a cautious way, rather than as soon as something looks positive-EV.
However, I’m not sure whether this caution eventually cashes out to “don’t be a holy madman” vs “here’s how to be the right kind of holy madman”, in the terms of the post.
Yeah, I feel that I can similarly look back at my history and say that in several cases, it either has been better or would have been much better to be more the detached academic.
Mh… I guess “holy madman” is a definition too vague to make a rational debate on it? I had interpreted it as “sacrifice everything that won’t negatively affect your utility function later on”. So the interpretation I imagined would be someone that won’t leave himself an inch of comfort more than what’s needed to keep the quality of his work constant.
I see slack as leaving yourself enough comfort that you’d be ready to use your free energy in ways you can’t see at the moment, so I guess I was automatically assuming an “holy madman” would optimise for outputting the current best effort he can in the long term, rather than sacrificing some current effort to bet on future chances to improve the future output.
I’d define someone who’s leaving this level of slack as someone who’s making a serious or full effort, but not an holy madman, but I guess this doesn’t means much.
If I were to try to summarise my thoughts on what would happen in reality if someone were to try these options… I think the slack one would work better in general, both by managing to avoid pitfalls and to better exploit your potential for growth.
I still feel there’s a lot of danger to oneself in trying to take ideas seriously though. If you start trying to act like it’s your responsibility to solve a problem that’s killing people, the moment you lose your grip on your thoughts it’s the moment you cut yourself badly, at least in my experience.
In these days I’ve managed to reduce the harm that some recurrent thoughts were doing by focusing on distinguish between 1) me legitimately wanting A and planning/acting to achieve A and 2) my worries related to not being able to get A or distress for things currently being not A, telling myself that 2) doesn’t helps me get what I want in the least, and that I can still make a full effort for 1), likely a better one, without paying to 2) much attention.
(I’m afraid I’ve started to slightly rant from this point. I’m leaving it because I still feel it might be useful)
This strategy worked for my gender transition.
I’m not sure how I’d react if I were to try telling myself I shouldn’t care/feel bad/worry if people die because I’m not managing to fix the problem, even if I KNOW that worrying myself about people dying hinders my effort to fix the problem because feeling sick and worried and tired wouldn’t in any way help toward actually working on the problem, I still don’t trust my corrupted hardware to not start running some guilt trip against me because I’m trying to be, in a sense that’s not utilitarian at all, callous, because I’m trying to not care/feel bad/worry about something like that.
Also, as a personal anecdote of possible pitfalls, trying to take personal responsibility for a global problem had drained my resources in ways I could’t foreseen easily. When I got jumped by an unrelated problem about my gender, I found myself without the emotional resources to deal with both stresses at once, so some recurrent thoughts started blaming me because I was letting a personal problem that was in no way as bad as being dead, and didn’t blipped at all on any screen in confront to a large number of deaths, screw up with my attempt of working on something that was actually relevant. I realised immediately that this was a stupid thing to think and in no way healthy, but that didn’t do much to stop it, and climbing out of that pit of stress and guilt took a while.
In short, my emotional hardware is stupid and bugged and it irritates me to no end how it can just go ahead and ignore my attempts of thinking sanely about stuff.
I’m not sure if I’m just particularly bad at this, or if I just have expectations that are too high. An external view would likely tell me that it’s ridiculous for me to expect to be able to go from “lazy and detached” to “saving the world (read reducing X risk), while effortlessly holding at bay emotional problems that would trip most people”. I’d surely tell anyone that. On the other hand, it just feels like a stupid thing to not manage doing.
(end of the rant)
Can I ask if you have some sort of external force that makes you do these hours? If not, any advice on how to do that?
I’m coming from a really long tradition of not doing any work whatsoever, and so far I’m struggling to meet my current goal of 24 hours (also because the only deadlines are the ones I manage to give myself… and for reasons I guess I have explained above).
Getting to this was a massive improvement, but again, I feel like I’m exceptionally bad at working hard.