In another life I wrote Gemini Song and The Coven series. Now I’m working on the pause.
Bridgett Kay
Karma: 426
It is normal! That’s the point :)
My Weirdest Experience Wasn’t
There was no special connection to the truth in this koan. It was actually a little lame- just my brain pattern-matching. But pattern matching can do some odd things things. I won’t reach enlightenment with this koan- I’ve already followed it as far as it will go. It was “where is your voice located.” It’s a very “if a tree falls in the forest” type of koan, but I learned just a little from following it in its expected loop.
Ah- thank you. In retrospect what seems like “the obvious thing” to me is not always so.
I’m glad you liked it :)
I always wondered why, at the before-Christmas battle, Hermione didn’t give her wish to Sunshine to incentivize them to work together instead of defecting and turning traitor (that is to say- make the wish for sunshine and announce it before the battle.) I think it was the obvious course to unite the army, and it was the obvious Hermione thing to do. Did she fail to do this just because Dumbledore got to her first, or was this a true blind spot on her part?
Maya’s Escape
It looks like it’s difficult to wear with glasses. Do you have any ideas for adjustments that might make them fit better?
Meta Alignment: Communication Guide
I had a related (and admittedly somewhat strange) experience to this. I had a dream in which I was given a koan and told to use it to seek enlightenment. When I woke up I wrote down the koan and decided to meditate on it. I gained some wisdom from it, but I also noticed that I was starting to feel empty. Not a peaceful one-ness kind of empty- just empty empty. I realized that if experienced anything like ego death in my present state, it would go very, very badly. It seems counterintuitive, but knew I had to find myself before I continued on the path- to find myself before I lost myself, if that makes any sense. It’s as if in order to reach transcendence, there had to be something worth transcending.
If it helps anyone suffering from existential fear, I’m very glad I put it out here.
The Last Light
Thank you! I’ll work on that and see if I have any other questions.
“(I typically closed my eyes briefly and generated a small pleasurable feeling.)”—could you explain a little further how to do this? I’m not good at this at all and I think it would be an extremely useful skill for me. Apologies if you’ve answered this elsewhere.
“Future progress is a part of current human values” of course- the danger lies in the “future” always being just that- the future. One would naturally hope that it wouldn’t go this way, but continuously putting off the future because now is always the present is a possible outcome. It can even be a struggle with current models to get them to generate novel ideas, because of a stubbornness not to say anything for which there is not yet evidence.
Thank you for that criticism- I hadn’t necessarily given that point enough thought, and I think I am starting to see where the weaknesses are.
Yeah- calling myself a failed scifi writer really was half in jest- had some very limited success as an indie writer for a good number of years, and recently need has made me shift direction. Thank you for the encouragement, though!
“If your conclusion is that we don’t know how to do value alignment, I and I think most alignment thinkers would agree with you. If the conclusion is that AGI is useless, I don’t think it is at all.”
Sort of- I worry that it may be practically impossible for current humans to align AGI to the point of usefulness.
“If we had external help that allowed us to focus more on what we truly want—like eliminating premature death from cancer or accidents, or accelerating technological progress for creative and meaningful projects—we’d arrive at a very different future. But I don’t think that future would be worse; in fact, I suspect it would be significantly better.”
That’s my intuition and hope- but I worry that these things are causally entangled with things that we don’t anticipate. To use your example- what if we only ask an aligned and trusted AGI to cure premature death by disease and accident, which wouldn’t greatly conflict with most people’s values in the way that radical life extension would, but then a sudden loss of an entire healthcare and insurance industry results, causing such a total economic collapse that causes vast swaths of people to starve. (I don’t think this would actually happen, but it’s an example of the kind of unforeseen consequence that getting a wish suddenly granted may cause, when you ask an instruction following AGI to give, without counting on a greater intelligence to project and weigh all of the consequences.)
I also worry about the phrase “a human you trust.”
Again- this feels like cynicism, if not the result of a catastrophizing mind (which I know I have.) I think you make a very good argument- I’m probably indulging too much in black-and-white thought- that there’s a way to fulfill these desires quickly enough that we are able to relieve more suffering than we would have if left to our devices, but still slow enough to monitor unforeseen consequences. Maybe the bigger question is just whether we will.
[Question] Alignment Paradox and a Request for Harsh Criticism
Does anyone have any generally helpful advice for someone who doesn’t really get vibes? Should I just continue to be more timid than normal, or is there a helpful heuristic I can use (aside from the ‘don’t talk to strangers, don’t join MLM’s, be wary of things that are too good to be true’ stuff that our parents tell us at a young age.)
That ambiguity was my intention! Though I do like the concept of the (metaphorical) death of the author.