Independent AI safety researcher
alexflint
Karma: 2,628
Right. But it’s notable that almost no-one in the world is stuck in an actual infinite why-regress, in that there don’t seem to be many people sitting around asking themselves “why” until they die, or sitting with a partner asking “why” until one person dies. (I also don’t think this is what is happening for monks or other contemplative folks.) I guess in practice people escape by shifting attention elsewhere. But sometimes that is a helpful thing to do, such as when stuck in a rut, and sometimes it is an unhelpful thing to do, such as when already overwhelmed with information. Furthermore some people at very good at shifting their attention around in a way that leads to understanding. Chaitin strikes me as exactly such a person and discusses allocation of attention in that talk (thank you for the lovely link btw—really delightful read!). So what actually is our attentional mechanism and in what way can we trust it?
Interested in any thoughts you may have.
Hope you are well.
Ah good point. OK yeah I believe that (2) doesn’t require the graph to be finite, and I also agree that it’s not tenable to believe all three of your statements.
If, hypothetically, we were to stop here, then you might look at our short dialog up to this point as, roughly, a path through a justification graph. But if we do stop, it seems that it will be because we reached some shared understanding, or ran out of energy, or moved on to other tasks. I guess that if we kept going, we would reach a node with no justifications, or a cycle, or an infinite chain as you say. Now:
A node with no justification would be quite a strange thing to experience. I would write something, and you would question me, and I would have literally nothing that I could say
A cycle would be quite a normal experience to go a few loops around—plenty of conversations go in loops for some finite time—but it would be strange for there to be absolutely no way out of the cycle. We would just go and go and go until we lost all energy, and neither of us would notice that we’re in a cycle?
An infinite chain would be perhaps the most “normal” of the three experiences. We would just have some length of conversation and then, what, give up? Since we have finite minds, there must be a finite program that generates the infinite graph, so wouldn’t we eventually notice that and say “huh, it looks like we are on a path with the following generator functions”. What then? Would we not go some place else in the justification graph other than the infinite chain we were previously on?
So it’s hard for me to imagine really experiencing any of the three possibilities you point out. Yet they would seem to be not just possible but actually guaranteed (in aggregate).
Interested in what you make of this.
But then are you saying that it’s impossible to experience profound doubt? Or are you saying that it’s possible to experience profound doubt, but noting perception as belief is a reliable way out of it? If the latter then how do you go from noting perception as belief to making decisions?
Thank you for the kind words. If you have time and inclination, I’d be interested to hear anything at all about what the raw justification in your own experience is like.
I disbelieve 2 because it assumes that there are a finite number of nodes in the graph. (We don’t have to hold an infinite graph in our finite brains; we might instead have a finite algorithm for lazily expanding an infinite graph.)
I think it’s mostly incoherent as a principle
What is it that you are saying is incoherent as a principle?
Plenty of people are perfectly well satisfied with various answers to this question within all sorts of systems
Yeah, I’m interested in this. If you have time, what are some of the answers that you see people being satisfied by?
While there’s nothing fundamentally wrong with supposing that things may well change right now when they don’t appear to have changed in at least the past few billion occasions of right now, it does seem to privilege the observer almost to the point of solipsism.
Right yeah it seems like empiricism follows from a certain kind of humility. It’s like, if I don’t see the world as having any nature of its own then it basically makes no sense to pay attention to anything beyond myself because how could I possibly look carefully at a tree or a bird or a waterfall without some kind of background view that there is something out there to look at.
And it does seem that when I look at some system for a while, like rain falling on a lake or a rabbit hopping around on the grass, that I just naturally start to discern some cause and effect. It’s almost as if a kind of intuitive empiricism is the default, and that it would take some effortful resistance to say “no no none of this is justifiable”
But now we are trusting something about own nature that has this tendency towards an intuitive empiricism, and it really comes down to the question of what it is about our own nature that is trustworthy, because it sure isn’t the case that everything we do intuitively has beneficial consequences.
Interested in your thoughts on this.
Do you not want to strongly believe in something without a strong and non-cyclic conceptual justification for it?
It’s not that I don’t want to strongly believe in something without a strong and non-cyclic conceptual justification for it. It’s that I want my actions to help reduce existential risk, and in order to do that I use reasoning, and so it’s important to me that I use the kind of reasoning that actually helps me to reduce existential risk, so I am interested in what aspects of my reasoning are trustworthy or not.
Now you have linked to many compelling impossibility arguments. Hume’s is-ought gap, the problem of induction, and many of Eliezer’s writings rule out whole regions of the space of possible resolutions to this problem, just as the relativization barrier in computational complexity theory rules out whole regions of the space of possible resolutions to the P versus NP problem. So, good, let’s not look in the places that we can definitively rule out (and I do agree that the arguments you have linked to in fact soundly rule out their respective regions of the resolution space).
Given all that, how do you determine whether your reasoning is trustworthy?
Yeah that resonates with me. I’d be interested in any more thoughts you have on this. Particularly anything about how we might recognize knowing in another entity or in a physical system.
Yes it’s true, there are people who have spent time at the Monastic Academy and have experienced psychological challenges after leaving.
For me, I enjoyed the simplicity and the living in community and the meditation practice as you say. The training style at the Monastic Academy seemed to really really really work for me. There were tons of difficult moments, but underneath that I felt safe, actually, in a way that I don’t think I ever had before. That safety was really critical for me to face some deep doubts that I’d been carrying for a really long time.
But there are also people who have left Monastic Academy feeling very hurt.
I guess sometimes it’s good to push through doubt and pain in order to get to the other side, while other times it’s better to listen to doubt and pain because it’s telling you that something isn’t working for you.
I do think it’s worth finding somewhere that can provide deep sanctuary. I definitely did find that at the Monastic Academy. There are others who seem not to have.
Instead they spend their days performing mental gymnastics and writing about enigmas without getting anywhere, grasping at a sense of meaning, purpose, and connection that perpetually alludes them because they cannot meet with the suffering that is right in front of them and inside of them.
Hmm. Since you refer to “writing about enigmas without getting anywhere” and the title of the post is “three enigmas at the heart of our reasoning”, I understand this to be a critique of my character (?) or perhaps just a personal attack. Was that your intention?
Regarding the first enigma, the expectation that what has worked in the past will work in the future is not a feature of the world, it’s a feature of our brains. That’s just how neural networks work, they predict the future based on past data.
Yeah right, we are definitely hard-wired to predict the future based on the past, and in general the phenomenon of predicting the future based on the past is a phenomenon of the mind, not of the world. But it sure would be nice to know whether that aspects of our minds is helping us to see things clearly or not. For me personally, I found it very difficult to get to work with full conviction without spending some real time investigating this.
Another way to say this is that we are born hard-wired to do all kinds of things, and we can look at our various hard-wirings and reflect on whether they are helping us to see things clearly and decide what to do about them. Now you might say that neural networks predict the future based on the past in a way that is a level more ingrained than any one particular heuristic or bias. But to me that just makes it all the more pressing to investigate whether this deep aspect of our brains is helping or hurting our capacity to see things clearly. I just found that I could put this question aside for only so long.
Yeah I agree the three enigmas are instances of the same thing. And it does seem to me that no system of reasoning can provide its own ultimate justification. But whether it’s possible to find ultimate justification for anything is a different question, I think. It seems to me that it’s worth making some real time and space to look into whether anything can be ultimately justified. For me it was important to make such time and space because I just couldn’t fully dive into my work without first having looked into this.
Yeah thank you for that connection Ben. It seems like a true connection to me.
Thank you for the kind words and encouragement Gunnar
Gosh yes having doubts is ultra important as you say, but I don’t think they’ll you much good unless you act on them, and a lot of the time acting on doubt means picking an appropriate time and place to investigate them.
Look let’s say you’re working for a space elevator company and your job is to design the cars that will climb the elevator cable, but you have some doubts about whether the basic science behind the cable design has been done well. Now it’s completely fine to just do your job and not worry much about these underlying doubts, but eventually that’s going to be a pretty demotivating way to exist, particularly if you are doing the work you are doing because you really care about it. So from a purely practical perspective, it’s going to help you do your work to spend a little time investigating the basic assumptions underneath the thing you’re working on.
It’s a similar situation, I think, with investigating the foundations of reasoning itself. If this underlying thing isn’t quite right, then everything we do on top is going to be skewed. We know this, and our doubts correctly implore us to make some time and space to investigate. The demotivating thing, it seems to me, is when we do have the doubts but do not make time and space to investigate.
But yes the goal is not to paper over all doubts forever, but to actually resolve the thing that is unresolved.
Right, yeah I agree that we can evaluate empiricism on empirical grounds. That is a thing we can do. And yes, as you say, we can come to different conclusions about empiricism when we evaluate it on empirical grounds. Very interesting point re object-level and meta-level conclusions. But why would start with empiricism at all? Why should we begin with empiricism, and then conclude on such grounds either that empiricism is trustworthy or untrustworthy?
When I say “empiricism cannot justify empiricism”, I mean that empiricism cannot explain why we trust empiricism, because the decision to start with empiricism as the framework for evaluating empiricism is not itself accounted for by empiricism. And when I say “accounted for in a way that resolves doubt”, not merely argued for.
Maybe a clearer way to say it is that I actually agree with everything you’ve said, but I don’t think what you’ve said is yet sufficient to resolve the question of whether our reasoning is based on something trustworthy.
(Also, yes I do think the first enigma is close to or identical to Hume’s problem of induction.)
Three enigmas at the heart of our reasoning
David Wolpert on Knowledge
Well thank you. I’m glad you enjoyed it. I enjoyed reading this comment.
Out of interest, what parts did you see as ironic or anti-ironic?
Thank you for sharing this.
In my own experience, there are moments where I see something that I haven’t seen before, such as what is really going on in a certain relationship in my life, or how I have been unwitting applying a single heuristic over and over, or how I have been holding tension my body, and it feels like a big gong has just rung with truth. But I think what’s really going on is that I was seeing things in one particular way for a long time, and then upon seeing things in just a slightly different way, I let go of some unconscious tightness around the previous way of seeing things, and that letting go frees up my mind to actually think, and that’s such a big relief that I feel this gong ringing with truth. It seems that letting go of seeing things one particular way is what the energetic release is about, rather than the particular new way of seeing things.
I mention this just because it’s the thing that seems closest in my own experience to the direct experience of self-evident truth. It seems that when I see that I have been holding to one particular way of seeing things, it is self-evident that it’s better to make a conscious choice about how to see things rather than just being unconsciously stuck. But it does not seem to me that there is any self-evident truth in any particular replacement way of seeing things.