Recent Ph.D. in physics from MIT, Complex Systems enthusiast, AI researcher, digital nomad. http://pchvykov.com
pchvykov
Karma: 128
Objective truth?
Thanks for your interest—really nice to hear! here is a link to the videos (and supplement): https://science.sciencemag.org/content/suppl/2020/12/29/371.6524.90.DC1
Our compressed perception
Mindfulness as debugging
That’s an interesting question—I was assuming that there is a sort of “natural selection” process that acts over generations, and picks out the “best” algorithms. This way, I can understand your comment in two ways:
the selection pressures may not be directed at individual benefit, but rather at group survival or optimal transmission (rules that are easier to remember are easier to pass down)
the selection that led to our algorithms may be outdated in our modern world
Am I getting it, or did you have something else in mind?
Does butterfly affect?
Cool—thanks for your feedback! I agree that I could be more rigorous with my terminology. Nonetheless, I do think I have a rigorous argument underneath all this—even if it didn’t come across. Let me try to clarify:
I did not mean to refer to human intentionality anywhere here. I was specifically trying to argue that the “chaos-theory definition of causality” you give, while great in idealized deterministic systems, is inadequate in complex messy “real world.” Instead, the rigorous definition I prefer is the counter-factual information theoretic one, developed by Judea Pearl, and which I here tried to outline in layman’s terms. This definition is entirely ill-posed in a deterministic chaotic system, but will work as soon as we have any stochasticity (from whatever source).Does this address your point at all, or am I off-base?
ah yes, great minds think alike! =)
What I really like about J. Pearl’s counter-factual causality framework is that it gives a way to make these arguments rigorously, and even to precisely quantify “how much did the butterfly cause the tornado”—in bits!
Yes!! Very cool—going even one meta level up. I agree that usefulness of proposed models is certainly the ultimate judge of whether it’s “good” or not. To make this even more concrete, we could try to construct a game and compare the mean performance of two agents having the two models we want to compare… I wonder if anyone’s tried that… As far as I know, the counterfactual approach is “state of the art” for understanding causality these days—and it is a bit lacking for the reason you say. This could be a cool paper to write!
I’m not sure why this was crossed out—seems quite civil to me… And I appreciate your thoughts on this!
I do think we agree at the big-picture level, but have some mismatch in details and language. In particular, as I understand J. Pearl’s counter-factual analysis, you’re supposed to compare this one perturbation against the average over the ensemble of all possible other interventions. So in this sense, it’s not about “holding everything else fixed,” but rather about “what are all the possible other things that could have happened.”
whow, some Bayesian updating there—impressive! :)
hmm, so what I was thinking is whether we could give an improved definition of causality based on something like “A causes B iff the model [A causes B] performs superior to other models in some (all?) games / environments”—which may have a funny dependence on the game or environment we choose.
Though as hard as the counterfactual definition is to work with in practice, this may be even harder…
You post may be related to this, though not the same, I think. I guess what I’m suggesting isn’t directly about decision theory.
cool—and I appreciate that you think my posts are promising! I’m never sure if my posts have any meaningful ‘delta’ - seems like everything’s been said before.
But this community is really fun to post for, with meaningful engagement and discussion =)
Yeah, I’m quite curious to understand this point too—certainly not sure how far this reasoning can be applied (and whether Ferdinand is too much of a stretch). I was thinking of this assassination as the “perturbation in a super-cooled liquid”—where it’s really the overall geopolitical tension that was the dominant cause, and anything could have set off the global phase transition. Though this gets back to the limitations of counter-factual causality in the real-world...
Is social theory our doom?
A gentle apocalypse
Not sure I understand you here. Our AI will know the things we trained it and the tasks we set it—so to me it seems it will necessarily be a continuation of things we did and wanted. No?
Well, in some sense yes, that’s sort of the idea I’m entertaining here: while these things all do matter, they aren’t the “end of the world”—humanity and human culture carries on. And I have the feeling that it might not be so different even if robots take over.
[of course, in the utilitarian sense such violent transitions are accompanied by a lot of suffering, which is bad—but in a consequentialist sense purely, with a sufficiently long time-horizon of consequences, perhaps it’s not as big as it first seems?]
yeah, I can try to clarify some of my assumptions, which probably won’t be fully satisfactory to you, but a bit:
I’m trying to envision here a best-possible scenario with AI, where we really get everything right in the AI design and application (so yes, utopian)
I’m assuming that the question “is AI conscious?” to be fundamentally ill-posed as we don’t have a good definition for consciousness—hence I’m imagining AI as merely correlation-seeking statistical models. With this, we also remove any notion of AI having “interests at heart” or doing anything “deliberately”
and so yes, I’m suggesting that humans may be having too much fun to reproduce with other humans, nor will feel much need to. It’s more a matter of a certain carelessness, than deliberate suicide.
I’m really excited about this post, as it relates super closely to a recent paper I published (in Science!) about spontaneous organization of complex systems—like when a house builds itself somehow, or utility self-maximizes just following natural dynamics of the world. I have some fear of spamming, but I’m really excited others are thinking along these lines—so I wanted to share a post I wrote explaining the idea in that paper https://medium.com/bs3/designing-environments-to-select-designs-339d59a9a8ce
Would love to hear your thoughts!