Filip Sondej
New cooperation mechanism—quadratic funding without a matching pool
Yeah, unanimous may be too strong—maybe it would be better to have 2 out of 3 majority voting for example. And I agree, my past self is a third party too.
Hm, yeah, trusting Elua to do it would work too. But in scenarios where we don’t have Elua, or have some “almost Elua” that I don’t fully trust, I’d rather rely on my trusted friends. And those scenarios are likely enough that it’s a good option to have.
(As I side note, I don’t think I can fully specify that “please take me out of there if X”. There may be some Xs which I couldn’t foresee, so I want to rely on those third party’s judgement, not some hard rules. (of course, sufficiently good Elua could make those judgements too))
As for that limitation, how would you imagine it? That some mind modifications are just forbidden? I have an intuition that there may be modifications so alien, that the only way to predict their consequences is to actually run that modified mind and see what happens. (an analogy may be, that even the most powerful being cannot predict if some Turing machine halts without actually running it). So maybe reverting is still necessary sometimes.
Yeah, that makes sense.
I’d like the serious modifications to (at the very least) require a lot of effort to do. And be gradual, so you can monitor if you’re going in the right direction, instead of suddenly jumping into a new mindspace. And maybe even collectively decide to forbid some modifications.
(btw, here is a great story about hedonic modification https://www.utilitarianism.com/greg-egan/Reasons-To-Be-Cheerful.pdf)
The reason that I lean toward relying on my friends, not a godlike entity, is because on default I distrust centralized systems with enormous power. But if we had Elua which is as good as you depicted, I would be okay with that ;)
New tool for exploring EA Forum, LessWrong and Alignment Forum—Tree of Tags
Hm, I thought those instructions (under “Explore hidden treasures...”) are enough, but I probably misestimated it because I’m already used to that tool.
Thanks for pointing that out! I’ll try to clarify it. (Also I’m not sure if you looked at this page which roughly explains how it works. I linked it at the end because to use that tool you don’t need to understand those details.)
Are you able to pin down what causes the most confusion?
[edit] I also copied over the whole explanation into the post now, and made the details link more salient
Mind is uncountable
Good point. I edited the post to say “near epiphenomenalism”, because like you said, it doesn’t fit into the strict definition.
If the physical and mental are quantized (and I expect that), then we can’t really speak of “infinitesimal” changes, and the situation is as you described. (But if they are not quantized, then I would insist that it should really count as epiphenomenal, though I know it’s contentious.)
Still, even if it’s only almost epiphenomenal, it feels too absurd to me to accept. In fact you could construct a situation where you create an arbitrarily big mental change (like splitting Jupyter-sized mind in half), by the tiniest possible physical change (like moving one electron by one Planck length). Where would all that mental content “come from”?
Here, to have that discontinuity between input and output (start and end position), we need some mechanism between them—the system of ball, hill, and their dynamics. What’s worse it needs to evolve for infinite time (otherwise the end still continuously depends on start position).
So I would say, this discontinuous jump “comes from” this system’s (infinite) evolution.
It seems to me, that to have discontinuity between physical and mental, you would also need some new mechanism between them to produce the jump.
It seems that we just never had any situations that would challenge this way of thinking (those twins are an exception).
This Cartesian simplification almost always works, so it seems like it’s just the way the world is at its core.
It just looks that’s what worked in evolution—to have independent organisms, each carrying its own brain. And the brain happens to have the richest information processing and integration, compared to information processing between the brains.
I don’t know what would be necessary to have a more “joined” existence. Mushrooms seem to be able to form bigger structures, but they didn’t have an environment complex enough to require the evolution of brains.
Hm, yeah, the smallest relevant physical difference may actually be one neuron firing, not one moved atom.
What I meant by between them, was that there would need to be some third substrate that is neither physical nor mental, and produces this jump. That’s because in that situation discontinuity is between start and end position, so those positions are analogous to physical and mental state.
Any brain mechanism, is still part of the physical. It’s true that there are some critical behaviors in the brain (similar to balls rolling down that hill). But the result of this criticality is still a physical state. So we cannot use a critical physical mechanism, to explain the discontinuity between physical and mental.
I thought about it some more, and now I think you may be right. I made an oversimplification when I implicitly assumed that a moment of experience corresponds to a physical state in some point in time. In reality, a moment of experience seems to span some duration of physical time. For example, events that happen within 100ms, are experienced as simultaneous.
This gives some time for the physical system to implement these discontinuities (if some critical threshold was passed).
But if this criticality happens, it should be detectable with brain imaging. So now it becomes an empirical question, that we can test.
I still doubt the formulation in IIT, that predicts discontinious jumps in experience, regardless of whether some discontinuity physically happens or not.
(BTW, there is a hypothetical mechanism that may implement this jump, proposed by Andres Gomez Emilsson—topological bifurcation.)
That’s a good point. I had a similar discussion with @benjamincosman, so I’ll just link my final thoughts: my comment
Oh, I’ve never stumbled on that story. Thanks for sharing it!
I think it’s quite independent from my post (despite such a similar thought experiment) because I zoomed in on that discontinuity aspect, and Eliezer zoomed in on anthropics.
Yeah, when I thought about it some more, maybe the smallest relevant physical change is a single neuron firing. Also with such a quantization, we cannot really talk about “infinitesimal” changes.
I still think that a single neuron firing, changing the content of experience so drastically, is quite hard to swallow. There is a sense in which all that mental content should “come from” somewhere.
I had a similar discussion with @benjamincosman, where I explore that in more detail. Here are my final thoughts from that discussion.
Cool post! I think the minimum viable “guardian” implementation, would be to
embed each post/video/tweet into some high-dimensional space
find out which regions of that space are nasty (we can do this collectively—f.e. my clickbait is probably clickbaity for you too)
filter out those regions
I tried to do something along these lines for youtube: https://github.com/filyp/yourtube
I couldn’t find a good way to embed videos using ML, so I just scraped which videos recommend each other, and made a graph from that (which kinda is an embedding). Then I let users narrow down on some particular region of that graph. So you can not only avoid some nasty regions, but you can also decide what you want to watch right now, instead of the algorithm deciding for you. So this gives the user more autonomy.
The accuracy isn’t yet too satisfying. I think the biggest problem with systems like these is the network effect—you could get much better results with some collaborative filtering.
Oh yeah, definitely. I think such a system shouldn’t try to enforce one “truth”—which content is objectively good or bad.
I’d much rather see people forming groups, each with its own moderation rules. And let people be a part of multiple groups. There’s a lot of methods that could be tried out, f.e. some groups could use algorithms like EigenTrust, to decide how much to trust users.
But before we can get to that, I see a more prohibitive problem—that it will be hard to get enough people to get that system off the ground.
Love that post!
Can we train ML systems that clearly manifest a collective identity?
I feel like in multi-agent reinforcement learning that’s already the case.
Re training setting for creating shared identity. What about a setting where a human and LLM take turns generating text, like in the current chat setting, but first they receive some task, f.e. “write a good strategy for this startup” and the context for this task. At the end they output the final answer and there is some reward model which rates the performance of the cyborg (human+LLM) as a whole.
In practice, having real humans in this training loop may be too costly, so we may want to replace them most of the time with an imitation of a human.
(Also a minor point to keep in mind: having emergent collective action doesn’t mean that the agents have a model of the collective self. F.e. colony of ant behaves as one, but I doubt ants have any model of the colony, rather just executing their ant procedures. Although with powerful AIs, I expect those collective self models to arise. I just mean that maybe we should be careful in transferring insight from ant colonies, swarms, hives etc., to settings with more cognitively capable agents?)
Thanks! I’m always hungry for good sci-fi utopias :) I particularly liked that mindmelding part.
After also reading Diaspora and ∀V, I was thinking what should be done about minds who self-modify themselves into insanity and suffer terribly. In their case, talking about consent doesn’t make much sense.
Maybe we could have a mechanism where:
I choose some people I trust the most, for example my partner, my mom, and my best friend
I give them the power to revert me back to my previous snapshot from before the modification, even if it’s against my insane will (but only if they unanimously agree)
(optionally) by old snapshot is temporarily revived to be the final arbiter and decide if I should be reverted—after all I know me the best