If you are going to read just one thing I wrote, read The Problem of the Criterion.
More AI related stuff collected over at PAISRI
G Gordon Worley III(G Gordon Worley III)
Karma: 5,737
I still really like this post, and rereading it I’m surprised how well it captures points I’m still trying to push because I see a lot of people out there not quite getting them, especially by mixing up models and reality in creative ways. I had not yet written much about the problem of the criterion at this time, for example, yet it carries all the threads I continue to think are important today. Still recommend reading this post and endorse what it says.
Also skeptical. I think this is tracking something more like the rise and fall of high modernism than the rise of fall of good epistemics.
This doesn’t really seem like solving symbol grounding, partially or not, so much as an argument that it’s a non-problem for the purposes of value alignment.
+1 on the wording likely being because Alcor has dealt with resistant families a lot, and generally you stand a better chance of being preserved if Alcor has as much legal authority as possible to make that happen. You may have to explain that you’re okay with your wife potentially doing something that would have been against your wishes (yes, I realize you don’t expect that, but there more than 0% chance it will happen) and result in no preservation when Alcor thinks you would have liked one.
This is actually why I went with Alcor: they have a long record of going to court to fight for patients in the face of families trying to do something else.
This is a great point. Drinking plenty of water before sleep doesn’t guarantee no hangover, but I find it works like 80% of the time since I think some hangover effects are just dehydration.
The point is kinda that you can take it to be a hypothesis and have it approach 100% likelihood. That’s not possible if that hypothesis is instead assumed to be true. I mean, you might still run the calculations, they just don’t matter since you couldn’t change your mind in such a situation even if you wanted to.
I think the baked-in absurdity of that last statement (since people do in fact reject assumptions) points at why I think there’s actual no contradiction in my statements. It’s both true that I don’t have access to the “real” God’s eye view and that I can reconstruct one but will never be able to be 100% sure that I have. Thus I mean to be descriptive of how we find reality: we don’t have access to anything other than our own experience, and yet we’re able to infer lots of stuff. I’m just trying to be especial careful to not ground anything prior in the chain of epistemic reasoning on something inferred downstream, and that means not being able to predicate certain kinds of knowledge on the existence of an objective reality because I need those things to get to the point of being able to infer the existence of an objective reality.
Really appreciate the relative methanol calculations, and they match my own experience: generally no hangovers if I drink vodka drinks, mild hangovers from beer and other liquors, terrible hangovers from red wine.
I think this is exactly right, because without some goals, purpose, or even just a norm to be applied, there’s nothing to power knowing anything since knowing is at its ground about picking and choosing what goes into what category to separate out the world into things.
I think a lot of the reason people are desperate to signal is because they are desperate: they need something to happen to feel safe, secure, fulfilled, or whatever and so they greedily grasp for it. Not doing that requires being sufficiently content with the world being as it is to not try to force it to be a particular way, but getting to a place where one has that kind of security is quite hard.
B
If we give up any assumption that there’s an external reality and try to reason purely from our experience, then in what sense can there be any difference between “the snow is white” and “the snow looks white to me”? This is, in part, what I’m trying to get at in the post: the map-territory metaphor creates this kind of confusing situation where it looks and awful lot like there’s something like a reality where it could independent of any observer have some meaning where snow is white, whereas part of the point of the post that this is nonsense, there must always be some observer, they decide what is white and not, and so the truth of snow being white is entirely contingent on the experience of this observer. Since everything we know is parsed through the lens of experience, we have no way to ground truth in anything else, so we cannot preclude the possibility that we only think snow is quite because of how our visual system works. In fact, it’s quite likely this is so, and we could easily construct aliens who would either disagree or would at least be unable to make sense of what “snow is white” would mean since they would lack something like a concept of “white” or “snow” and thus be unable to parse the proposition.
Perhaps it seems like I’m not really defending #1 because it still all has to add up to normality, so it’s not like I am going to go around claiming an objective universe is total nonsense except in a fairly technical sense; in an everyday sense I’m going to act not much different than a person claiming for there to definitely be objective reality because I’ve still got to respond to the conditions I find myself in.
From a pragmatic perspective most of the time it doesn’t matter what you believe so long as you get the right outcome, and that can be for a surprisingly large space where it can be hard to find the places where things break down. Mostly they break down when you try to justify how things are grounded without stopping when it’s practical and instead going until you can’t go anymore. That’s the kind of places where rejecting #3 (except as something contingent) and accepting something more like #1 starts to make sense, because you end up getting underneath the processes that were used to justify the belief in #3.
I hope they are okay with Neurath’s boat, because that seems to be the world we live in. That is, the problem of the criterion shows us there is no solid foundation because we are born into contingency.
There’s certainly folks who wanted a typed theory of truth (logical positivists) and there are folks who refashion truth in the image of referential integrity (coherentists), but even for them it either means giving up completeness to get a typed theory or giving up objectivity since truth must be coherent with our subjective experience. So circularity isn’t really a sign of failure, it’s just how it is, and truth isn’t really about truth, it’s about signaling (just kidding, it’s about what we care about).
I’m not aware of anything quite so rigorous beyond what we might call “philosophical math” of using words in a precise way to evaluate doxastic logic. Maybe this is enough, but does feel like we should at least write it down somewhere in formal notation to make sure there’s no gaps.
The Problem of the Criterion is NOT an Open Problem
Seems right to me. For example, I think by most natural notions of “agency” that don’t introduce anything crazy, we should probably think of thermostats as agents since they go about making the world different based on inputs. But such deflationary notions of agency seem deeply uncomfortable to a lot of people because they violate the very human-centric notion that lots of simple things don’t have “real” agency because we understand their mechanism, whereas things with agency seem to be complex in a way that we can’t easily understand how they work.
B: “Okay, cool, but that’s information you constructed from within our universe and so is contingent on the process you used to construct it thus it’s not actually a God’s eye view but an inference of one. Thus you should be very careful what you do with that information because if you start to use it as the basis for your reasoning you’re now making everything contingent on it and thus necessarily more likely to be mistaken in some way that will bite you in the ass at the limits even if it’s fine 99.99% of the time. And since I happen to know you care about AGI alignment and AGI alignment is in large part about getting things right at the extreme limits, you should probably think hard about if you’re not seeing yourself up to be inadvertently turned into a paperclip.”
So to return to your GoL example, it only works because you exist outside the universe. If you were inside that GoL, you wouldn’t be able to construct such a view (at least based on the normal rules of GoL). I see this as exactly analogous to the case we find ourselves in: what we know about physics seems to imply that we couldn’t hope to gather enough information to ever successfully construct a God’s eye view.
This is why I make a claim more like your #1 (though, yes, #2 is obviously the right thing here because nothing is100% certain) that a God’s eye view is basically nonsense that our minds just happen to be able to image is possible because we can infer what it would be like if such a thing could exist from the sample set of our experience, but the logic of it seems to be that it just isn’t a sensible thing we could ever know about except via hypothesizing the possible existence of it, putting it on par with thinking about things outside our Hubble volume, for example.
I’m suspicious someone could endorse #3 and not get confused reasoning about embedded agency because I’d expect either assuming #3 to cause you to get confused thinking about the embedded agency situation (and getting tripped up on questions like “why can’t we just do thing X that endorsing #3 allows?”) or that thinking about embedded agency hard enough would cause you to have to break down the things that make you endorse #3 and then you would come to no longer endorse it (my claim here is backed in part by that fact that I and others have basically gone down this path before one way or another, previously having assumed something like #3 and then having to unassume it because it got in the way and was inconsistent with the rest of our thinking).
Am roughly in middle management. Can confirm. Basically I and everyone around me is trying to walk some line between take enough responsibility to get results (the primary thing you’re evaluated on) but don’t take so much that if something goes south you’ll be in trouble. Generally we don’t want the pain to fall on ICs (“individual contributor” employees whose scope of responsibility is ultimately limited to their own labor since they need sponsorship from someone else or a process to commit to big decisions) unless they messed up for reasons within their control.
I generally see the important split as who is responsible and who is accountable. Responsible means here something like “who has to do the work” and accountable means something like “who made the decision and thus gets the credit or blame”. ICs do well when they do a good job doing whatever they were told to do, even if it’s the wrong thing. Management-types do well when the outcomes generate whatever we think is good, usually whatever we believe is driving shareholder value or some proxy of it. ICs get in trouble when they are inefficient, make a lot of mistakes, or otherwise produce low quality work. Management-types are in trouble when they make the wrong call and do something that produces neutral or negative value for something the company is measuring.
Basically I think all the maze stuff is just what happens when middle management manages to wirehead the organization so we’re no longer held accountable for mistakes. I’ve not actually seen much serious mazes in my life because I’ve mostly worked for startups of various sizes, and in startups there’s enough pressure from the executives on down to hold people accountable for stuff. I think it’s only if the executives get on board with duping the board and shareholders so they can wirehead that things fall apart.