please reach out to me if you’d like to be friends or have any inquiries <3
(note my posts/comments here don’t yet represent what i usually think about, or what i’m like in private. i’ll probably start a new LW account sometime cause i feel like this one is not me)
Just thought about this post again, and wanted to add. I disabled karma notifications after being downvoted a few times. Maybe it would help you too.
I think it is okay for you to be vague. Simply saying that you can see numerous bottlenecks, but don’t wish to list them to avoid others working on them, is enough to cause some weaker update than a list would cause.
Hasn’t the alignment community historically done a lot to fuel capabilities?
For example, here’s an excerpt from a post I read recently
My guess is RLHF research has been pushing on a commercialization bottleneck and had a pretty large counterfactual effect on AI investment, causing a huge uptick in investment into AI and potentially an arms race between Microsoft and Google towards AGI: https://www.lesswrong.com/posts/vwu4kegAEZTBtpT6p/thoughts-on-the-impact-of-rlhf-research?commentId=HHBFYow2gCB3qjk2i
Maybe I shouldn’t vote this way, but whether I agree plays a role in whether I upvote a post. I don’t downvote things I disagree with, though.
thank you!
silly meme thing that happened vv
>new lesswrong notif
>oh no, did someone argue against something i wrote
>”waveform linear fixtures”
>oh gosh oh no what are these concepts
>its the name of the light brand
>they were replying to my question
(not that i’d mind ppl disagreeing ofc :p)
How do you power them / what things do you need to power them?
The basic idea seems true to me (for the probably-rare subclass of agents which terminally value preventing some worldstate from occurring; and conditional on destroying the universe being possible), and you’ve phrased it concisely, so I’ve strong-upvoted (to counteract downvotes). I’m not sure about the anthropic argument at the bottom, but I think it’s good that you’re trying to think creatively.
I’m reminded of an excerpt from the value handshakes tag page
nobody knows what kind of scorched-earth strategy a losing superintelligence might be able to use to thwart its conqueror, but it could potentially be really bad – eg initiating vacuum collapse and destroying the universe.
Either way, the universe getting destroyed and the universe being tiled with non-conscious computronium meant to prevent a certain state from occurring seem about equally valueless to me.
As for the doomsday argument, I’m not sure it holds. We’d exist now no matter what happens in the future, so I don’t know if “conditioning on being an observer” makes sense here. It seems akin to the religious view of humans as having been preexisting “souls” which were then reincarnated as random observers anywhere in time. Wheras in reality, we were instantiated by temporally-local physical interactions, so the probability of us experiencing what we do right now is 1 no matter what happens in the future.
I would also note the space of possible explanations for arguments such as the doomsday argument and the fermi paradox is large, so I think the fact “x argument could explain it” only marginally increases that argument’s probability. And I’d expect, based on you finding this idea, that you’ll come across many more possible explanations.
If there’s a population of mostly TDT/UDT agents and few CDT agents (and nobody knows who the CDT agents are) and they’re randomly paired up to play one-shot PD, then the CDT agents do better. What does this imply?
the CDT agents don’t necessarily do better. e.g., if “not being tied for first” means “the CDT agents accrue an advantage, use that advantage to gain further advantage after the game, and eventually take over the world”, then the TDT/UDT agents all choose to defect because of the presence of CDT agents, and the CDT agents also all defect because of their decision theory.
wheras if there’s no real cost to not being first, then there’s a couple possibilities i see. (a) the TDT/UDT agents all defect in order to discourage the existence of CDT agents. (b) the TDT/UDT agents cooperate, and the CDT agents do perform better. i think (a) would be done in worlds/situations where it would work, but (b) is what happens in, e.g., a truly one-shot prisoners dilemma that’s the only thing that will ever exist, and afterwards all the agents cease to exist, for example.
“what does this imply”—maybe we can conclude there is no decision algorithm which is necessarily best in all possible situations/worlds, and that one of the main things ‘how well a decision algorithm performs’ depends on is what decision algorithms other agents are using.
(meta p.s: i’d appreciate feedback on whether this comment was insightful to you)
‘Asking for advice’ seems like the best first step in such a situation, congrats on doing that.
Here’s some options
You could consider looking for a trusted alignment researcher to disclose it to, if you think this would be helpful (e.g., if you think the idea is about something which, if true, can be prevented).
Alternatively, you could not disclose it to anyone, if you don’t foresee any benefit from doing so.
I’d volunteer to discuss it with you; you could message me here on lesswrong. However, note I’m just someone who uses this site, not someone with established trustworthiness or presence.
If the idea is more akin to a hypothetical AGI takeoff story, you could share it publicly.
I’d also suggest familiarizing with the unilateralist’s curse, a concept relevant for situations where unilateral action could have large impacts.
Note that most ideas end up not being significant, and “an idea which an AGI could use” probably doesn’t pose much risk.
There was this idea, it may or may not be feasible to predict which companies will lose value due to AI though
Sometimes there’s some sunlight through the window, when it’s night or I have the curtains closed then my light comes only from one wrapping of fairy lights around the perimter + a 500 lumen lamp + my laptop screen. (the room’s main light broke)
Waveform’s linear fixtures
How many would you suggest for a 20m³ room?
I think that better LLM scaffolding accelerates capabilities as much as it accelerates alignment
Which is not good enough. We need alignment to accelerate faster than capabilities in order to catch up.
I’m going to experiment with, as I aesthetically call it internally, ‘entering the wired’.
By which I mean, a more mundane thing: replacing my environment with my computer screen via VR. I don’t want to bother myself with the things around me, with the physical world, with that antiquated level of physics on which we still concern ourselves with matter rather than information. I want my whole perception to be information; text, webpages, articles here.
I’m hoping this will help me focus on it more. More ambituously, I hope it might allow my mind to overgeneralize to that information environment and forego, even further than it has already, processing related to the physical world, in order to fully dedicate itself to information processing and idea generation.
I hope this language doesn’t sound mystical or ambiguous, I’m just describing a mundane thing (wearing a VR headset for most of the day to read things) in some exciting language/concepts I use for it internally.
If anyone’s interested in hearing about how this goes let me know now. :)
people who are high in abstract thinking and believing their beliefs and anxious thought patterns
I think one who this description fits can avoid any risks of ‘going insane’ while still using their abilities for good. For example, in my own case (I think the first two describe me, and the third one sort-of does), if I were to apply these suggestions..
try to stay grounded with people and activities that don’t send you off into abstract thought space. Spend some time with calm normal people who look at the world in straightforward ways, not only creative wild thinkers. Spend time doing hobbies outdoors that use your physical body and attention in satisfying ways, keeping you engaged enough to stay out of your head.
then my creative output related to alignment would probably drop significantly.
(I agree with not trying psychedelics though. Even e.g nootropics and adhd meds are things I’m really cautious with, cause I don’t wanna mess up some part of my process.)
For anyone reading this post in the future, I’d instead suggest doing things meant to help you channel your ability: being conscious and reflective about your thoughts, revisiting basic rationality techniques and theory occasionally, noticing privileged hypotheses (while still allowing yourself to ponder them if you’re just doing it because you find it interesting; I think letting ones mind explore is also important to generating important ideas and making connections).
“Please don’t throw your mind away” in this other sense of counteracting your tendency to think abstractly; you might be able to do a lot of good with it.
i’m not really satisfied with my writing here on lesswrong (vs in private dms). i’ve developed an impression that i have to modify my default output a lot before it will sound intuitively good/plausible to + make sense to the broader range of readers, even when i don’t change the point at all. this has a number of negative impacts:
it reduces the info conveyed about the internal process which generated the meaning the text attempts to contain
it makes me feel like my full body of writing here hardly expresses myself
key example: i linked something i wrote here to a friend, and distilled/simplified the above by noting that i worry about judgement when i link things i’ve written here. they replied, “It was written well to a degree which transformed it into ‘text you read online’ and I forgot it was written by someone I know”
there are positives about the forum, with respect to communication: the users here are generally more thoughtful than elsewhere, and generally more mutually interested in doing good, and at least a little averse to social-pressuring each other thru the medium of linguistic choices; but this is counteracted in other ways (like the fact complex statements are more difficult to reduce into english than simple ones are) which mean i still have to optimize for understandability of the text to the demise of the two points above (and probably more which i can’t word).
I’ll check out that section.
Anthropic reasoning is in general extremely weak
Was there anything which led you to believe this that I could read? (About the weakness of anthropic reasoning, not about the potential errors humans attempting to use it could make; I agree that those exist, and that they’re a good reason to be cautious and aware of one’s own capability, but I don’t really see them as arguments against the validity of the method when used properly.)
I’m not sure I see the relevance of the bus example to anthropic reasoning. Below I explain why (maybe I spent too long on this; ended up hyperfocusing). Note that all uses of ‘average’ and ‘expectation’ are in the technical, mathematical sense.
By a similar argument (your random arrival time has the mean and median in the middle between two buses), the time you should expect to wait for the bus to arrive is the same as the time you have already been waiting
If the ‘random arrival time has the mean in the middle between two busses,’ one should expect to wait time equal to the remaining wait time in that average situation.
One could respond that we don’t know the interval between busses, and thus don’t know the remaining wait time; but this does not seem to be a reason to expect the bus to arrive after the time you’ve been waiting doubles (from the view of neither anthropic nor non-anthropic reasoners).
The bus example has some implicit assumptions about the interval between events (i.e., one assumes that busses operate on the timescales of human schedules). In the odd scenario where one was clueless when an event would happen, and only knew (a) it would eventually happen, (b) it happened at least once before, and (c) it’s happening on an interval, then that one would have no choice but to reason that the event could have happened most recently at any point in the past, and thus could happen next at any point in the future.
Or, more precisely, given the event could not have first happened earlier than the starting point of the universe, an anthropic reasoner could reason that the average observer will exist at the mid-point between (a) the average expected time of the initial event: half-way between when the universe began and now, and (b) when the event will next happen. Their ‘expectation’ would still shift forwards as time passes (correctly from their perspective; it would seem like they were updating on further evidence about which times the event has not happened at), but it would not start out equal to the time since they first started waiting.
This still wouldn’t seem quite analogous to real-world anthropic reasoning to me, though, because in reality an anthropic reasoner doesn’t expect the amount of observers to be evenly distributed over their range. There are a few possible distributions of the amount of observers over time which we consider plausible (some of which are mentioned in the original post), and in none of these distributions is the average in the center as it is with busses.
I thought so too. I’m actually not sure about timelines, I just wanted to not rule out the possibility.