Do you have a source on the claims about echo chambers? I feel like most studies I encounter on it say that echo chambers are an overrated issue, with people tending to interact with contradictory views, but I haven’t looked into it in detail.
tailcalled
Karma: 275
It seems like it would be really interesting to study this quantitatively, recording data collected over time. If you (or anyone else) would be interested, I know a bunch of psychological research methods that could help.
We were comparing epsilon to no-epsilon (what I had in mind with my post).
Anyway, the point is that strict equality would require astronomical consequences, and so only be measure 0. So outside of toy examples it would be a waste to consider lexicographic preferences or probabilities.
ah, I think I am starting to follow. It is a bit ambigious whether it is supposed to be two instances of one arbitraliy small finite or two (perhaps different) arbitrarily small finites. If it is only one then the tails are again relevant.
I don’t see what you mean. It doesn’t make a different whether you have only one or two small-but-finite quantities (though usually you do have on for each quantity you are dealing with), as long as they are in general position. For instance, in my modification to the example you gave, I only used one: While it’s true that the tail becomes relevant in (0, 1)(1, 0) vs (1, 0)(0, 1) because 0*1=1*0, it is not true that the tail becomes relevant in the slightly modified (eps, 1)(1, 0) vs (1, 0)(0, 1) for any real eps != 0, as eps*1 != 1*0.
So the only case where infinitesimal preferences are relevant are in astronomically unlikely situations where the head of the comparison exactly cancels out.
I am tripping over the notation a little bit. I was representing eps times 1 as (0,1)(1,0) so (eps, 1)(1, 0) and (1, 0)(eps, 1) both evaluate to 2eps in my mind which would make the tail relevant.
Surely this would be your representation for infinitesimals, not for eps? By eps, I mean epsilon in the sense of standard real numbers, i.e. a small but positive real number. The issue is that in the real world, there will always be complexity, noise and uncertainty that will introduce small-but-positive real gaps, and these gaps will outweigh any infinitesimal preference or probability that you might have.
In your example, r1m1*p1m1 is exactly equal to r2m1*p2m1; 0*1=0=1*0. The point is that if in the former case, you instead have (eps, 1)(1, 0), or in the latter case, you instead have (1, 0)(eps, 1), then immediately the tail of the sequence becomes irrelevant, because the heads differ. So only when the products of the heads are exactly equal do the tails become relevant.
However if the differences are 0.000000000000002 then you migth be more paranoid and start to actually look whether all the “assumed 0” assumptions should actually be made.
This requires you to not use infinitesimal probabilities. 0.000000000000002 is still infinitely bigger than infinitesimals.
Oh, you meant literal infinitesimal (non-real) probabilities. I thought you meant very small probabilities.
The argument against lexicographic ordering is also an argument against infinitesimal probabilities. Which is to say, suppose you have some lotteries l1, l2 that you are considering choosing between (the argument can be iterated to apply to a larger number of lotteries), where each lottery ln can have payouts rn1, rn2, rn3, … with probabilities pn1, pn2, np3, ….
Suppose that rather than being real numbers, the probabilities and payouts are ordered lexicographically, with payout m being of shape (rnm1, rnm2, rnm3, …) and probability m being of shape (pnm1, pnm2, pnm3, …).
In order for the infinitesimal tail of the sequence to matter, you need the sum over r1m1\*p1m1 to be exactly precisely 100% equal to the sum over r2m1\*p2m1 (that is, after all, the definition of lexicographic orderings). If they differ by even the slightest margin (even by 10^-(3^^^^^^3)), this difference outweights anything in the infinitesimal tail. While you could obviously set up such a scenario mathematically, it seems like it would never happen in reality.
So while you could in principle imagine lexicographic preferences, I don’t think they would matter in reality, and a proper implementation of them would act isomorphic to or want to self-modify into something that only considers the leading term.
“Incoherent entities have stronger reason to become more coherent than less” is very abstract, and if you pass through the abstraction it becomes obvious that it is wrong.
This is about the agent abstraction. The idea is that we can view the behavior of a system as a choice, by considering the situation it is in, the things it “could” do (according to some cartesian frame I guess), and what it then actually does. And then we might want to know if we could say something more compact about the behavior of the system in this way.
In particular, the agent abstraction is interested in whether the systen tries to achieve some goal in the environment. It turns out that if it follows certain intuitively goal-directed rules like always taking a decision and not oscillating around, it must have some well-defined goal.
There’s then the question of what it means to have a reason. Obviously for an agent, it would make sense to treat the goal and things that derive from it as being a reason. For more general systems, I guess you could consider whatever mechanism it acts by to be a reason.
So will every system have a reason to become an agent? That is, will every system regardless of mechanism of action spontaneously change itself to have a goal? And I was about to say that the answer is no, because a rock doesn’t do this. But then there’s the standard point about how any system can be seen as an optimizer by putting a utility of 1 on whatever it does and a utility of 0 on everything else. That’s a bit trivial though and presumably not what you meant.
So the answer in practice is no. Except, then in your post you changed the question from the general “systems” to the specific “creatures”:
you should predict that reasonable creatures will stop doing that if they notice that they are doing it
Creatures a produced by evolution, and those that oscillate endlessly will tend to just go extinct, perhaps by some other creature evolving to exploit them, and perhaps just by wasting energy and getting outcompeted by other creatures that don’t do this.
No, when there are infinitesimal chances, you take the ones which favor your leading term.
I imagine you would be able to solve this by replacing real-valued utilities with utilities in a numbering system that contains infinitesimals. However, it seems to me that it would not matter much in practice, since if you had even a tiny chance of affecting the leading term, then that chance would outweigh all of the other terms in the calculation, and so in practice only the leading term would matter for the decisions.
Or to give an argument from reflection, there’s a cost to making decisions, so an agent with a lexicographic ordering would probably want to self-modify into an agent that only considers the leading term in the preference ordering, since then it doesn’t have to spend resources on the other terms.
Also, if you (or someone else) arranges some site collecting adversarial collaborations, then please contact me, as there’s some topics I can contribute with. Most notably, I’ve studied transsexuality a lot and am very familiar with the strengths and weaknesses of some controversial theories on the topic.
I would be more interested in people arranging adversarial collaborations on it. A big problem with raw presentations of fringe theories is that then you have to go through all of the arguments in detail. Having people on both sides of the issue who are well-acquainted with the evidence point out the most important core disagreements would be better.
Another tree rotation is the human vs animal distinction. Humans are animals, but sometimes one uses “animal” to refer to nonhuman animals. I wonder if there’s some general things one could say about tree rotations. The human/animal distinction seems to have a different flavor to me than the fish/tetrapod distinction, though.
Something that also seems related is asking for an eggplant and expecting a fully-developed, non-rotting eggplant.
I wonder to what degree the perspective comes from us generally not thinking about nonvertebrate animals. So the things that distinguish “phylogenetic fish” from them (like having craniums and vertebrae) are just considered “animal things”. And so instead, when defining fish we end up focusing on what distinguishes tetrapods and taking the negation of that. Sort of a categorical “tree rotation” if you will.
It’s easier to keep track of the underlying relatedness as if it were an “essence” (even though patterns of physical DNA aren’t metaphysical essences), rather than the all of the messy high-dimensional similarities and differences of everything you might notice about an organism.
Hmm, isn’t DNA metaphysical essences?
IIRC, the metaphysical notion of essence came from noticing similarities between different creatures, that they seemed to cluster together in species as if constructed according to some blueprint. The reason for these similarities is the DNA—if “essences” had to correspond to anything in reality, then that would seem to be DNA.
Am I misunderstanding something? You seem to be defending a phylogenetic definition of “fish” as a reason why dolphins aren’t fish, but if you used a phylogenetic definition of “fish”, you’d still have dolphins be fish—that’s the first part of his argument.
(Sorry Scott Alexander, you totally fell down as a rationalist when you saw one of the biggest effect sizes ever, in a study that controlled for so many things, and still thought “hmm, i doubt being tortured regularly for a decade has a long term bad effect.”)
Controlling for things isn’t a good way to go about researching the effects of this. Instead, you should ask, what factors lead to variation in who gets bullied?
Recently I’ve also been thinking about something that seems vaguely related, which could perhaps be called inference in the map vs inference in the territory.
Suppose you want to know how some composite system works. This might be a rigid body object made up of molecules, a medicine made out of chemicals to treat a disease that is ultimately built out of chemicals, a social organisation method designed for systems made out of people, or anything like that.
In that case there are two ways you can proceed: either think about the individual components of the system and deduce from their behavior how the system will behave, or just build the system in reality and observe how the aggregate behaves.
If you do the former, you can apply already-known theory about the components to deduce it’s behavior without needing to test it in reality. Though often in practice this theory won’t be known, or will be too expensive to use, or similar. So in practice one generally has to investigate it holistically. But this requires using the territory as a map to figure it out.
(When investigating it holistically there is also the possibility of just using holistic rather than reductionistic theories. Often this holistic theory will originate from one of the previous methods though, e.g. our math for rigid body dynamics comes from actual experience with rigid bodies. Though also sometimes it might come from other places, e.g. evolutionary reasoning. So my dichotomy isn’t quite as clean as yours, probably.)
I might not have explained the credence/propositional assertion distinction well enough. Imagine some sort of language model in AI, like GPT-3 or CLIP or whatever. For a language model, credences are its internal neuron activations and weights, while propositional assertions are the sequences of text tokens. The neuron activations and weights seem like they should definitely have a Bayesian interpretation as being beliefs, since they are optimized for accurate predictions, but this does not mean one can take the semantic meaning of the text strings at face value; the model isn’t optimized to emit true text strings, but instead optimized to emit text strings that match what humans say (or if it was an RL agent, maybe text strings that make humans do what it wants, or whatever).
My proposal is, what if humans have a similar split going on? This might be obscured a bit in this context, since we’re on LessWrong, which to a large degree has a goal of making propositional assertions act more like proper beliefs.
In your model, do you think there’s some sort of confused query-substitution going on, where we (at some level) confuse “is the color patch darker” with “is the square of the checkerboard darker”?
Yes, assuming I understand you correctly. It seems to me that there’s at least three queries at play:
Is the square on the checkerboard of a darker color?
Is there a shadow that darkens these squares?
Is the light emitted from this flat screen of a lower luminosity?
If I understand your question, “is the color patch darker?” maps to query 3?
The reason the illusion works is that for most people, query 3 isn’t part of their model (in the sense of credences). They can deal with the list of symbols as a propositional assertion, but it doesn’t map all the way into their senses. (Unless they have sufficient experience with it? I imagine artists would end up also having credences on it, due to experience with selecting colors. I’ve also heard that learning to see the actual visual shapes of what you’re drawing, rather than the abstracted representation, is an important step in becoming an artist.)
Do the credences simply lack that distinction or something?
The existence of the illusion would seem to imply that most people’s credences lack the distinction (or rather, lacks query 3, and thus finds it necessary to translate query 3 into query 2 or query 1). However, it’s not fundamental to the notion of credence vs propositional assertion that it lacks this. Rather, the homunculus problem seems to involve some sort of duality, either real or confused. I’m proposing that the duality is real, but in a different way than the homunculus fallacy does, where credences act like beliefs and propositional assertions can act in many ways.
This model doesn’t really make strong claims about the structure of the distinctions credences make, similar to how Bayesianism doesn’t make strong claims about the structure of the prior. But that said, there must obviously be some innate element, and there also seems to be some learned element, where they make the distinctions that you have experience with.
We’ve seen objects move in and out of light sources a ton, so we are very experienced in the distinction between “this object has a dark color” vs “there is a shadow on this object”. Meanwhile...
Wait actually, you’ve done some illustrations, right? I’m not sure how experienced you are with art (the illustrations you’ve posted to LessWrong have been sketches without photorealistic shading, if I recall correctly, but you might very well have done other stuff that I’m not aware of), so this might disprove some of my thoughts on how this works, if you have experience with shading things.
(Though in a way this is kinda peripheral to my idea… there’s lots of ways that credences could work that don’t match this.)
More generally, my correction to your credences/assertions model would be to point out that (in very specific ways) the assertions can end up “smarter”. Specifically, I think assertions are better at making crisp distinctions and better at logical reasoning. This puts assertions in a weird position.
Yes, and propositional assertions seem more “open-ended” and separable from the people thinking of them, while credences are more embedded in the person and their viewpoint. There’s a tradeoff, I’m just proposing seeing the tradeoff more as “credence-oriented individuals use propositional assertions as tools”.
The study you linked smells fishy to me. They found that the overwhelming majority of users were deeply ingrained in either a science community or a conspiracy community; but that doesn’t match my experience on facebook, where most people just seem to share stuff from their life. Is it possible that they specifically sampled from science communities/conspiracy communities? (Which would obviously overstate the degree of polarization and echo chambers by a huge amount.) They don’t seem to describe how they sampled their users, unless I’m missing something, but given the context I would guess that they specifically sampled users who were active in the communities they looked at.
Regarding the studies that said it was overstated, as I said I haven’t looked into it in detail, I just follow a bunch of social science people on twitter and they’ve discussed this, with the people who seem more trustworthy converging towards a view that the echo chamber issue is overrated and based on misinformation. But based on your comment I decided to look at the studies more closely, and they seemed a lot less convincing than I had expected, sometimes updating me in the opposite direction. Probably the twitter user that has the most comprehensive set of links is Rolf Degen, but I’ve also seen it from other sources, e.g. Michael Bang Petersen.