ProgramCrafter

Karma: 88

ProgramCrafter 22 Apr 2024 14:57 UTC
1 point
0
on: Transformers Represent Belief State Geometry in their Residual Stream
Speaking of next steps, I’d love to see a transformer that was trained to manipulate those states (given target state and interactor’s tokens, would emit its own tokens for interleaving)! I believe this would look even cooler, and may be useful in detecting if AI starts to manipulate someone.

ProgramCrafter 21 Apr 2024 15:17 UTC
3 points
0
in reply to: Chris_Leong’s comment on: “You’re the most beautiful girl in the world” and Wittgensteinian Language Games
I’d say that it doesn’t carve reality at the same places as my understanding. I neither upvoted nor downvoted the post, but had to consciously remember that I have that option at all.
I think that language usage can be represented as vector, in basis of two modes:
1. “The Fiat”: words really have meanings, and goal of communication is to transmit information (including requests, promises, etc!),
2. “Non-Fiat”: you simply attempt to say a phrase that makes other people do something that furthers your goal. Like identifying with a social group (see Belief as Attire) or non-genuine promises.
(Note 1: if someone asked me what mode I commonly use, I would think. Think hard.)
(Note 2: I’ve found a whole tag about motivations which produce words—https://www.lesswrong.com/tag/simulacrum-levels! Had lost it for certain time before writing this comment.)
In life, I try to communicate less hyperboles and replace them with non-verbal signs, which do not carry implication of either “the most beautiful” or “more beautiful than everyone around”.

ProgramCrafter 19 Apr 2024 5:47 UTC
1 point
−2
on: hydrogen tube transport
Maybe vehicles would need to carry some shaped charges to cut a hole in the tube in case of emergency.
That would likely create sparks, and provided the tube has been cut the hydrogen is going to explode.

ProgramCrafter 19 Apr 2024 4:13 UTC
1 point
0
in reply to: ChosunOne’s comment on: When is a mind me?
*preferably not the last state but some where the person felt normal.
I believe that’s right! Though, if person can be reconstructed from N bits of information, and dead body retains K << N, then we need to save N-K bits (or maybe all N, for robustness) somewhere else.
It’s an interesting question how many bits can be inferred from social networks trace of the person, actually.

ProgramCrafter 18 Apr 2024 16:29 UTC
3 points
0
on: ProgramCrafter’s Shortform
Continuing to make posts into songs! I believe I’m getting a bit better, mainly in rapid-lyrics-writing; would appreciate pointers how to improve further.
1. https://suno.com/song/ef734c80-bce6-4825-9906-fc226c1ea5b4 (based on post Don’t teach people how to reach the top of a hill)
2. https://suno.com/song/c5e21df5-4df7-4481-bbe3-d0b7c1227896 (based on post Effectively Handling Disagreements—Introducing a New Workshop)
Also, if someone is against me creating a musical form of your post, please say so! I don’t know beforehand which texts would seem easily convertible to me.

ProgramCrafter 17 Apr 2024 11:39 UTC
1 point
0
on: Should we maximize the Geometric Expectation of Utility?
This is especially concerning if we, as good Bayesians, refuse to assign a zero probability to any event, including zero utility ones.
I feel that since people don’t care ultimately about money, all-nonzero probabilities will make all events have nonzero utility as well.

ProgramCrafter 12 Apr 2024 22:15 UTC
4 points
1
on: Non-ultimatum game problem
Let’s solve this problem without trying to refer to existing particular cases.
For the start, we assume that utility of A-the-thief monotonically decreases with time served; utility of B monotonically increases, and if A gives up lollipops it is increased by another constant.
Let’s graph what choices A has when B does not give up lollipops.
We may notice that in this case B will simply throw A in jail for life. Well, what happens if A is willing to cooperate a bit?
1. Coordination result will not be below or to the left of Pareto frontier, since otherwise it is possible to do better than that;
2. Coordination result will not be below or to the left of no-coordination result, since otherwise one or both parties are acting irrationally.
We may see that after these conditions, only a piece of curve “A gives up lollipops” remains. The exact bargaining point can then be found out by using ROSE values, but we can already conclude that A will likely be convicted for a long time but not for life.

ProgramCrafter 4 Apr 2024 19:10 UTC
3 points
0
on: ProgramCrafter’s Shortform
The LessWrong’s AI-generated album was surprisingly nice and, even more importantly, pointed out the song generator to me! (I’ve tried to find one a year ago and failed)

So I’ve decided to try my hand on quantum mechanics sequence. Here’s what I have reached yet: https://app.suno.ai/playlist/81b44910-a9df-43ce-9160-b062e5b080f8/. (10 generated songs, 3 selected, unfortunately not the best quality)

ProgramCrafter 4 Apr 2024 19:04 UTC
4 points
0
on: The Story of “I Have Been A Good Bing”
I’ve decided to try my hand on quantum mechanics sequence! Here’s what I have reached yet: https://app.suno.ai/playlist/81b44910-a9df-43ce-9160-b062e5b080f8/. (10 generated songs, 3 selected, unfortunately not the best quality)

ProgramCrafter 23 Mar 2024 19:48 UTC
2 points
0
on: Open Thread Spring 2024
I’ve came across a poll about exchanging probability estimates with another rationalist: https://manifold.markets/1941159478/you-think-something-is-30-likely-bu?r=QW5U.

You think something is 30% likely but a friend thinks 70%. To what does that change your opinion?

I feel like there can be specially-constructed problems when the result probability is 0, but haven’t been able to construct an example. Are there any?

User-inclination-guessing algorithms: registering a goal

ProgramCrafter20 Mar 2024 15:55 UTC

2 points

0 comments2 min readLW link

ProgramCrafter 18 Mar 2024 12:37 UTC
1 point
0
in reply to: Ooker’s comment on: Alice and Bob is debating on a technique. Alice says Bob should try it before denying it. Is it a fallacy or something similar?
On what basis can Alice assume
Not actually assume, but that’s certainly Bayesian evidence (should Bob have tried $T_{1}$ , he would likely respond in another way).
Also, :smile: your own comment is a fairly large bit of evidence that you haven’t yet read the Sequences (by the way, I recommend doing that). For instance, you can consider different ways of thinking, answer the questions 1-4 from their perspectives, and that would be evidence on which way is better—though, reality is still the ultimate judge how each situation turns out.

ProgramCrafter 23 Feb 2024 14:28 UTC
1 point
0
in reply to: Foyle’s comment on: Raising children on the eve of AI

I don’t think there is anything I can do educationally to better ensure they thrive as adults other than making sure I teach them practical/physical build and repair skills

I think one more thing could be useful, I’d call it “structural rise”: over many different spheres of society, large projects are created by combining some small parts; ways to combine them and test robustness (for programs)/stability (for organisations)/beauty (music)/etc seem pretty common for most of the areas, so I guess they can be learned separately.

ProgramCrafter 27 Jan 2024 10:06 UTC
1 point
0
in reply to: Johannes C. Mayer’s comment on: What subjects are unexpectedly high-utility?
I suppose 3d-whiteboard could be useful in allowing to connect more relevant subjects to each node (re: problem of four colours: countries on plane can be coloured in 4 different ways so that touching ones have different colour, and in space there’s no such limit).

ProgramCrafter 11 Jan 2024 14:40 UTC
2 points
0
on: Saving the world sucks
the best you can do is what you think is personally good
As far as you’re an ideal optimizing agent with consistent values and full knowledge, otherwise actions based on your thoughts may end up worse than using social heuristics.
That there are no bugs when it comes to values. That you should care about exactly what you want to care about. That if you want to team up and save the world from AI or poverty or mortality, you can, but you don’t have to.
Locally invalid. Values can be terminal (what you care about) and instrumental, and saving the world for most people is actually instrumental.
There’s value in giving explicit permission to confused newcomers to not get trapped in moral chains, because it’s really easy to hurt yourself doing that.
I think that’s true since memes can be harmful, but there is also value in reminding that if more people worked to ~~save~~ improve the world on average, it would be better, and often a simpler way is to do that yourself instead of pushing that meme+responsibility out.
Save the world if you want to, but please don’t if you don’t want to.
I’d continue that with “but please don’t destroy the world whichever option you choose, since that will interfere with my terminal goals and I’ll care about your non-existence”.

ProgramCrafter 9 Jan 2024 10:31 UTC
1 point
0
on: The shard theory of human values
That’s certainly an interesting position in discussion about what people want!

Namely, that actions and preferences are just conditionally-activated and those context activations are balanced against each other. That means that person’s preference system may be not only incomplete but incoherent in architecture, and moral systems and goals obtained via reflection are almost certainly not total (will lack in some contexts), creating problem in RLHF.

The first assumption, that part of neurons is basically randomly initialized, can’t be tested really well because all humans are born in similar gravity field, see similarly-structured images in first days (all “colorful patches” correspond to objects which are continuous, mostly flat or uniformly round), etc and that leaves a generic imprint.

ProgramCrafter 8 Jan 2024 6:06 UTC
1 point
0
on: Models Don’t “Get Reward”
The post explains difference between two similar-looking positions:
1. Model gets reward, and attempts to maximize it by selecting appropriate actions.
2. Model does some actions on the environment, and selection/mutation process calculates the reward to find out how to maximize it by modifying the model.
Those positions differ in level at which optimization pressure is applied. Actually, either of them can be implemented (so, the “reward” value can be given to the model), but common RL uses the second option.

ProgramCrafter 31 Dec 2023 7:34 UTC
3 points
0
on: Threat-Resistant Bargaining Megapost: Introducing the ROSE Value
This post suggests a (and, quite possibly, the) way to select outcome in bargaining (fully deterministic multi-player game).

ROSE values replace Competition-Cooperation ones by having players not compete against each other in attempts to extract more utility but average their payoffs over possible initiative orders. A probable social consequence is noted, that people wouldn’t threaten everyone else in order to get something they want but would rather maximize own utility (improve their situation themselves).

ROSE values are resistant to unconditional threats. Unfortunately, conditional ones can’t be easily distinguished from conditional bonuses (if player B decides to pay A if A chooses some action from a certain set) so counteracting that requires further research. There are at least two more important directions of research: non-deterministic and imperfect-information games.

In comments, a problem is mentioned: what happens if there are multiple ROSE equilibrium points? I don’t believe solution “bargain over those points only” will work, as it’s possible all of those points will remain equilibriums when calculating ROSE values again. Hopefully, this players disagreement will involve only tiny slices of utility and not matter much overall.

ProgramCrafter 22 Nov 2023 11:03 UTC
1 point
0
on: For Civilization and Against Niceness
Meta-comment: I think articles $(c i v i l i z a t i o n + n i c e n e s s)$ and $(c i v i l i z a t i o n - n i c e n e s s)$ can explain more than if there were just articles $(c i v i l i z a t i o n)$ and $(n i c e n e s s)$ .

ProgramCrafter 13 Nov 2023 19:12 UTC
1 point
0
on: Open Thread – Autumn 2023
I’ve noticed that my thoughts are bound to what I see, and if I go to another room old thoughts are muted. Are there some techniques to protect from this effect?

ProgramCrafter

User-in­cli­na­tion-guess­ing al­gorithms: reg­is­ter­ing a goal

User-inclination-guessing algorithms: registering a goal