I’m fuzzy about the whole thing, but a feature that I think I like about the proposal is that it gives you a nicely-behaved way to deal with the problem of how to value lives lived in extremely complex interpretations of rocks. And if someone lives so far away in space or time that just to locate him requires as much information as it would to specify his whole mind starting from a rock, it’s not obvious to me that he exists in a sense in which the rock-mind does not.
I don’t think there’s anything wrong with valuing people who live in contrived interpretations or rocks, you just can’t interact with them, and whatever it is you observe is usually more of a collection of snapshots than a relevant narrative. Also, destroying the rock only destroys part of your contrived device for observing facts about those people, unless you value the rock itself.
If you use your current location as a reference point than the theory becomes non-updateless and incoherent and falls apart. You don’t “get” any starting point when you try to locate someone.
I think the universe implicitly defines a reference point in the physics. By way of illustration, I think Tegmark sometimes talks about an inflation scenario where an actually infinite space is the same as a finite bubble that expands from a definite point, but with different coordinates that mix up space and time; and in that case I think that definite point would be algorithmically privileged. But I’m even fuzzier on all this than before.
I think the focus on a physical reference point here seems misguided. Perhaps more conceptually well-founded would be something like a search for a logical reference point, using your existence in some form at some level of abstraction and your reasoning about that logical reference point both as research of and as evidence about attractors in agentspace, via typical acausal means.
Vladimir Nesov’s decision theory mailing list comments on the role of observational uncertainty in ambient-like decision theories seems relevant. Not to imply he wouldn’t think what I’m saying here is complete nonsense.
In one of my imaginable-ideal-barely-possible worlds, Eliezer’s current choice of “thing to point your seed AI at and say ‘that’s where you’ll find morality content’” was tentatively determined to be what it currently nominally is (instead of tempting alternatives like “the thing that makes you think that your proposed initial dynamic is the best one” or “the thing that causes you to care about doing things like perfecting things like the choice of initial dynamic” or something) after he did a year straight of meditation on something like the lines of reasoning I suggest above, except honed to something like perfection-given-boundedness (e.g. something like the best you could reasonably expect to get at poker given that most of your energy has to be put into retaining your top 5 FIDE chess rating while writing a bestselling book popular science book).
But surely given any scheme to assign addresses in an infinite universe, for every L there’s a finite bubble of the universe outside of which all addresses are at least L in length?
Naively, you wouldn’t use some physical location, but instead logical descriptions in the space of algorithms given axioms you predict others will predict are Schelling points (using your own (your past) architecture/reasoning as evidence of course).
I thought “Schelling point” was used by the decision theory workshop folk, I may be wrong. Anyway, decision theory shares many aspects of cooperative game theory as pointed out by Wei Dai long ago, and many questions of ethics must be determined/resolved/explored by such (acausal) cooperation/control.
I mistakenly thought that Will Sawin was in said group and was thus expressing confusion that he wasn’t already familiar with its broader not-quite-game-theoretic usage, or at least what I perceived to be a broader usage. Our interaction is a lot more easily interpreted in that light.
And if you meant that you don’t see a more technical referent for my use of Schelling point then there almost certainly isn’t one, and thus it could be claimed that I was sneaking in technical connotations with my naive intuitions. Honestly I thought I was referring to a standard term or at least concept, though.
Two agents in a PD can find a reason to cooperate in proving (deciding) that their decision algorithms are equivalent to some third algorithm that is the same for both agents (in which case they can see that their decision is the same, and so (C,C) is better than (D,D)). This common algorithm could be seen as a kind of focal point that both agents want to arrive at.
I don’t think it matters much, but the specific agents I had in mind were perhaps two subagents/subalgorithms (contingent instantiations? non-Platonic instantiations?) both “derived” (logically/acausally) from some class of variably probable unknown-to-them but less-contingent creator agents/algorithms (and the subagents have a decision theory that ‘cares’ about creator/creation symmetry or summat, e.g., causally speaking, there should be no arbitrary discontinuous decision policy timestamping). There may be multiple possible focal points and it may be tricky to correctly treat the logical uncertainty.
All of that to imply that the focus shouldn’t be determining some focal point for the universe, if that means anything, but focal points in algorithmspace, which is probably way more important.
You’ve talked about similar things yourself in the context of game semantics / abstract interpretation / time-symmetric perceptions/actions. I’d be interested in Skype convo-ing with you now that I have an iPhone and thus a microphone. I’m very interested in what you’re working on, especially given recent events. Your emphasis on semantics has always struck me as well-founded. I have done a fair amount of speculation about how an AI (a Goedel machine, say) crossing the ‘self-understanding’/‘self-improving’/Turing-universal/general-intelligence/semantic boundary would transition from syntactic symbol manipulator to semantic goal optimizer and what that would imply about how it it would interpret the ‘actual’ semantics of the Lisp tokens that the humans would identify as its ‘utility function’. If you don’t think about that much then I’d like to convince you that you should, considering that it is on the verge of technicality and also potentially very important for Shulman-esque singularity game theory.
The idea is that having exactly the same or similar algorithms to agents is enormously good, due to a proliferation of true PDs, and that therefore even non-game-theoretic parts of algorithms should be designed, whenever possible, to mimic other agents.
However applying this argument to utility functions seems a bit over-the-top. Considering that whether or not something is a PD depends on your utility function, altering the utility function to win at PDs should be counter-productive. If that makes sense, we need better decision theories.
The intuition that “Schelling points” are an at all reasonable or non-bastardized way of thinking about this, or the intuition behind the “this” I just mentioned? If the latter, I did preface it with “naively”, and I fully disclaim that I do not have a grasp of the technical aspects, just aesthetics which are hard to justify or falsify, and the only information I pass on that might be of practical utility to folk like you or Sawin will be ideas haphazardly stolen from others and subsequently half-garbled. If you weren’t looking closely, you wouldn’t see anything, and you have little reason to look at all. Unfortunately there is no way for me to disclaim that generally.
And if someone lives so far away in space or time that just to locate him requires as much information as it would to specify their whole mind starting from a rock, it’s not obvious to me that he exists in a sense in which the rock-mind does not.
I intuit that the difference between logical and observational uncertainty could be relevant in non-obvious ways. Anyway, this sort of thinking seems obviously correct, but I fear the comparison may mislead some, considering that inferring the numbers and preferences of minds in causally disconnected parts of the multiverse through sheer logical reasoning is probably way way way easier than interpreting the ‘strength’/‘existence’ and preferences of minds in rocks, at least as I consider it. (I worded that so poorly that it’s incoherent as explicitly stated but I think the message is intact.)
I’m fuzzy about the whole thing, but a feature that I think I like about the proposal is that it gives you a nicely-behaved way to deal with the problem of how to value lives lived in extremely complex interpretations of rocks. And if someone lives so far away in space or time that just to locate him requires as much information as it would to specify his whole mind starting from a rock, it’s not obvious to me that he exists in a sense in which the rock-mind does not.
I don’t think there’s anything wrong with valuing people who live in contrived interpretations or rocks, you just can’t interact with them, and whatever it is you observe is usually more of a collection of snapshots than a relevant narrative. Also, destroying the rock only destroys part of your contrived device for observing facts about those people, unless you value the rock itself.
It’s good to see that panpsychism is finally getting the attention it rightfully deserves!
“far away” from what?
If you use your current location as a reference point than the theory becomes non-updateless and incoherent and falls apart. You don’t “get” any starting point when you try to locate someone.
I think the universe implicitly defines a reference point in the physics. By way of illustration, I think Tegmark sometimes talks about an inflation scenario where an actually infinite space is the same as a finite bubble that expands from a definite point, but with different coordinates that mix up space and time; and in that case I think that definite point would be algorithmically privileged. But I’m even fuzzier on all this than before.
I think the focus on a physical reference point here seems misguided. Perhaps more conceptually well-founded would be something like a search for a logical reference point, using your existence in some form at some level of abstraction and your reasoning about that logical reference point both as research of and as evidence about attractors in agentspace, via typical acausal means.
Vladimir Nesov’s decision theory mailing list comments on the role of observational uncertainty in ambient-like decision theories seems relevant. Not to imply he wouldn’t think what I’m saying here is complete nonsense.
In one of my imaginable-ideal-barely-possible worlds, Eliezer’s current choice of “thing to point your seed AI at and say ‘that’s where you’ll find morality content’” was tentatively determined to be what it currently nominally is (instead of tempting alternatives like “the thing that makes you think that your proposed initial dynamic is the best one” or “the thing that causes you to care about doing things like perfecting things like the choice of initial dynamic” or something) after he did a year straight of meditation on something like the lines of reasoning I suggest above, except honed to something like perfection-given-boundedness (e.g. something like the best you could reasonably expect to get at poker given that most of your energy has to be put into retaining your top 5 FIDE chess rating while writing a bestselling book popular science book).
I think it depends on the physics. Some have privileged points, some don’t.
But surely given any scheme to assign addresses in an infinite universe, for every L there’s a finite bubble of the universe outside of which all addresses are at least L in length?
If a universe is tiled with a repeating pattern then you can assign addresses to parts of the pattern, each an infinite number of points.
I don’t know how this applies to other universes.
If hypothetically our universe had a privileged point, what would you do if you discovered you were much farther away from it than average?
Naively, you wouldn’t use some physical location, but instead logical descriptions in the space of algorithms given axioms you predict others will predict are Schelling points (using your own (your past) architecture/reasoning as evidence of course).
Naively, this is a question of ethics and not game theory, so I don’t see why Schelling points should enter into it.
I thought “Schelling point” was used by the decision theory workshop folk, I may be wrong. Anyway, decision theory shares many aspects of cooperative game theory as pointed out by Wei Dai long ago, and many questions of ethics must be determined/resolved/explored by such (acausal) cooperation/control.
Relevance? (That people in group Y use a word doesn’t obviously clarify why you used it.)
I mistakenly thought that Will Sawin was in said group and was thus expressing confusion that he wasn’t already familiar with its broader not-quite-game-theoretic usage, or at least what I perceived to be a broader usage. Our interaction is a lot more easily interpreted in that light.
(I didn’t understand what you meant either when I wrote that comment, now I see the intuition, but not a more technical referent.)
And if you meant that you don’t see a more technical referent for my use of Schelling point then there almost certainly isn’t one, and thus it could be claimed that I was sneaking in technical connotations with my naive intuitions. Honestly I thought I was referring to a standard term or at least concept, though.
The term is standard, it was unclear how it applies, the intuition I referred to is about how it applies.
Can you explain that intuition to me or point me to a place where it is explained or something?
Or, alternately, tell me that the intuition is not important?
Two agents in a PD can find a reason to cooperate in proving (deciding) that their decision algorithms are equivalent to some third algorithm that is the same for both agents (in which case they can see that their decision is the same, and so (C,C) is better than (D,D)). This common algorithm could be seen as a kind of focal point that both agents want to arrive at.
I don’t think it matters much, but the specific agents I had in mind were perhaps two subagents/subalgorithms (contingent instantiations? non-Platonic instantiations?) both “derived” (logically/acausally) from some class of variably probable unknown-to-them but less-contingent creator agents/algorithms (and the subagents have a decision theory that ‘cares’ about creator/creation symmetry or summat, e.g., causally speaking, there should be no arbitrary discontinuous decision policy timestamping). There may be multiple possible focal points and it may be tricky to correctly treat the logical uncertainty.
All of that to imply that the focus shouldn’t be determining some focal point for the universe, if that means anything, but focal points in algorithmspace, which is probably way more important.
Ah, I see.
(I, on the other hand, don’t.)
You’ve talked about similar things yourself in the context of game semantics / abstract interpretation / time-symmetric perceptions/actions. I’d be interested in Skype convo-ing with you now that I have an iPhone and thus a microphone. I’m very interested in what you’re working on, especially given recent events. Your emphasis on semantics has always struck me as well-founded. I have done a fair amount of speculation about how an AI (a Goedel machine, say) crossing the ‘self-understanding’/‘self-improving’/Turing-universal/general-intelligence/semantic boundary would transition from syntactic symbol manipulator to semantic goal optimizer and what that would imply about how it it would interpret the ‘actual’ semantics of the Lisp tokens that the humans would identify as its ‘utility function’. If you don’t think about that much then I’d like to convince you that you should, considering that it is on the verge of technicality and also potentially very important for Shulman-esque singularity game theory.
The idea is that having exactly the same or similar algorithms to agents is enormously good, due to a proliferation of true PDs, and that therefore even non-game-theoretic parts of algorithms should be designed, whenever possible, to mimic other agents.
However applying this argument to utility functions seems a bit over-the-top. Considering that whether or not something is a PD depends on your utility function, altering the utility function to win at PDs should be counter-productive. If that makes sense, we need better decision theories.
The intuition that “Schelling points” are an at all reasonable or non-bastardized way of thinking about this, or the intuition behind the “this” I just mentioned? If the latter, I did preface it with “naively”, and I fully disclaim that I do not have a grasp of the technical aspects, just aesthetics which are hard to justify or falsify, and the only information I pass on that might be of practical utility to folk like you or Sawin will be ideas haphazardly stolen from others and subsequently half-garbled. If you weren’t looking closely, you wouldn’t see anything, and you have little reason to look at all. Unfortunately there is no way for me to disclaim that generally.
link? explanation? something of that nature?
EDIT: Private message sent instead of comment reply.
I intuit that the difference between logical and observational uncertainty could be relevant in non-obvious ways. Anyway, this sort of thinking seems obviously correct, but I fear the comparison may mislead some, considering that inferring the numbers and preferences of minds in causally disconnected parts of the multiverse through sheer logical reasoning is probably way way way easier than interpreting the ‘strength’/‘existence’ and preferences of minds in rocks, at least as I consider it. (I worded that so poorly that it’s incoherent as explicitly stated but I think the message is intact.)