“Membranes” enthusiast! Everything on my account relates to this!
I am currently in Prague at Epistea Residency for rationality research continuing my philosophy (rationality + AI safety) research on membranes. I will be back in the US in December. (I might be in the bay area then but that’s entirely dependent on getting funding.)
I have some drafts I’ve been working on for several months that I hope to post.. eventually
So it seems to me that the trauma comes from believing that you have been violated, because authority figures or parents or society tell you that you have—not from being violated, itself
need to think about this more, gee
I like this as a partial explanation for the root cause of unenlightenment/un-rationality (though I can think of other factors that probably have larger effects towards that)
Also, have you read The Courage to be Disliked? Separation of Concerns, Gordian Knot, … very similar.
Do literally anything involving Adlerian Psychology:)
I’m surprised that there aren’t many organizations hire people who are incubating new/weird/unusual agendas. I think FAR does this, but they seem pretty small.
Otherwise for independent research it’s kinda just LTFF (?)
As for how well LTFF does this , well, I applied a few months ago and finally they got back recently (about 10 weeks later) and rejected my application— no feedback, no questions. I’m not even sure they understood what my agenda is.
Rather, it’s in terms of the grasshopper (an object/physical system), it’s physiological properties (age, point in the circadian cycle, level of hunger, temperature, etc.) and its distances and exposures to the relevant stimuli, such as the topology of light and shade around it, gradients of certain attractive smells, and so on. So, under this level of coarse graining, we treat these as the “grasshopper’s properties”.
hm… how would an AI system locate these properties to begin with?
I haven’t written the following on LW before, but I’m interested in finding the setup that minimizes conflict.
(An agent can create conflict for itself when it perceives its sense of self as in some regard too large, and/or as in some regard too small.)
Also I must say: I really don’t think it’s (straightforwardly) subjective.
Related: “Membranes” is better terminology than “boundaries” alone
Thanks for writing this. I also think what we want from psuedokindness is captured from membranes/boundaries.
Ah, I see what you mean now. Thanks
And I very much agree with-
though that would be more centrally defender’s own fault
I would also clarify that it’s more than “fault”: they are the only reliable source of responsibility for solving and preventing such an attack
Assurance that you can only get hijacked through the attack surface rather than an unaccounted-for sidechannel doesn’t help very much.
I agree. I hope that my membranes proposal (post coming eventually) addresses this.
(BTW Mark Miller has a bunch of work in this vein along the lines of making secure computer systems)
Also, acausal control via defender’s own reasoning about environment from inside-the-membrane is immune to causal restrictions on how environment influences inside-of-the-membrane, though that would be more centrally defender’s own fault.
Would you rephrase this? This seems possibly quite interesting but I can’t tell what exactly you’re trying to say. (I think I’m confused about: “acausal control via defender’s own reasoning”)
The low-level practical solution […]
I will respond to this part later
Very surprised that you don’t have a regranting budget! I don’t know which funder I would expect to do that, but I would’ve expected this to be more common.
I guess Jan Tallinn does this, and Manifund does this. Hmm.
I’ve made a conceptual distillation of how this post defines infiltration on Markov blankets as boundary violations here: Formalizing «Boundaries» with Markov blankets + Criticism of this approach. I’ve also included criticism from Abram Demski, who thinks this approach is incomplete.
Related: https://lean-fro.org/about/ (automated theorem proving FRO)
FWIW I find using the word “alignment” when specifically what you mean is “safety” to be confusing. I consider AI alignment to be “getting actively positive things that you want because of AI”. I consider AI safety to be “getting no actively negative things to happen because of AI”. And it seems that Davidad’s approach is much more about safety than it is alignment.
Update: I just simplified this post a bit to be more clear and easier to read.
Counterfactuals in Decision Theory
This reminds me of Counterfactual Harm https://arxiv.org/pdf/2204.12993.pdf where the authors define harm to Agent 1 by Agent 2 as the counterfactual of Agent 2′s actions. However, this also requires defining what the acceptable “default action” is. For example, one couldn’t expect a mule farmer to save the life of someone having a heart attack, and so the mule farmer hasn’t done “harm” if they didn’t help successfully, but we would expect a doctor to help, and they have harmed if they haven’t helped.
However, they also admit:
we do not provide a method for determining the desired default action or policy in general
I believe that «membranes» (what Critch calls «boundaries») can provide these defaults.
I think it might be possible to determine moral defaults from a simple premise: “it’s unworkable to rely on forcing anyone to do anything that they haven’t agreed to”. In which case, if Alice can’t control Bob, then all she can do is “mind her own business”. She may want to control others, but she can’t.
Put another way: There are things that only Bob can do that Alice cannot meddle in.
(This can then be formalized in terms of Markov blankets.)
For example, I cannot control your actions and I cannot observe your subjective experience, and you cannot me. I call this fact “individual sovereignty”.
And I think individual sovereignty is the default. I think this is then where the most fundamental moral “defaults” come from.
Of course, we can make extra agreements with others on top of that, but crucially there is a finite number of limited-scope social attracts that are ~explicitly added on top of that default.
For example, a patient and a doctor enter into a contract where the doctor agrees to give service, and the patient agrees to pay. This contract is then also enforced by a larger force (like the government) that enforces other contracts, like the one that says they will send the doctor to jail if he breaks the law.
Social contracts can add on top of the individual sovereignty default, albeit to a limited extent.
Another example: Duty to Rescue laws obligate not that you “mind your own business”, but that you actively try to save people near you in trouble. But everyone “agrees” to it by living in their society, so it works.
The above should address the doctor and mule farmer examples.
In sum: I think “individual sovereignty (baseline) + finite social contracts (extra, subjective)” is enough to (fully?) determine moral defaults
Put another way: “Never expect to be able to force anyone to do anything, except when they’ve agreed”
Of course, it would be nice to live in a world where everyone helps others as much as they can all the time, but that violates the premise and I think it is unworkable. (Though, in the morality literature it doesn’t seem uncommon to assume that you, the ethicist, get to decide what other people do, AFAICT?)
Oooo I like this comment, especially the first two examples
also,
For example, two people locked in a tiny closet together are unable to establish, perhaps, any boundary around personal space.
Personally I wouldn’t call this a «boundary». I don’t consider boundaries to be things that are “set” or “established”
Peeking at the next post, I would also add:
There is a difference between being able to achieve your goals in the outside world, and being able to maintain your individual sovereignty.
“Getting what you want” (in the former category) is different from, say, “being safe” (in the latter category).
Also, some things might feel like “big deals” when they’re actually misinterpretations (because maybe it’s something you actually couldn’t control anyway, or actually shouldn’t care about for other reasons)
Tagged this post with “self-fulfilling prophecies”