Another source of prediction error arises not from the mismatch between model and reality, but from tension between internal models.
Is this a standard element of Predictive Processing, or are you generalized / analogizing the theory in a general way?
I’m familiar with the prediction error that results in diffs between sense data and generative models, but not between different generative models.
Your use of hyperlinks is very amusing to me.
The historical “contract” of the symbiosis between humans and cats is that cats would catch rodents that would infest human grain stores, which was beneficial, so the humans would take care of the cats, or at least allow them in human spaces.
Notably, this is a service that depends on the cats having substantial autonomy, which is why they seem to not have to do anything for us to take care of them.
You can’t save the world without working with people at least as annoying as John.
This is a great quote, and one that I should keep in mind myself.
My coworkers are pretty amazing in how not annoying they are (they’re low ego, interested in changing their mind when they’re wrong, smart, invested in getting better over time) and I still sometimes feel an urge to quit in a huff.
I think all the stories and adventures and loves and lives that people in the world have lived are worth quite a lot of torture, and it’s not naively the case that if the torturous experiences are larger than the other experiences, that this means they’re more important.
For you, is this a quantitative question, or an “in principle” question? Like could there exist some amount of extreme suffering, for which you would judge them to be outweighing the meaningful and worthwhile experiences?
Or is the sentiment more like “if there exists a single moment of meaning, that redeems all the pain and suffering of all of history”?
I agree. But that doesn’t necessitate that any particular person is going to lose in absolute terms.
I agree this dynamic seems fishy, and I’m suspicious that on a detailed analysis, it will turn out that an agreement like this is useful at all to exactly the extent that it involves misleading others.
That said...
There might be lots of people who you respect, but you’ll make a special point to promote the reputation of people who you expect will reciprocate the favor to you.
Small business owners (in different industries) sometimes form associations in which they explicitly direct clients to each other. eg The therapist directs customers to the mechanic (if it seems like they need a mechanic) and vis versa. This can be beneficial to the customer, because if they have a profesional that they like in one domain, they might trust that person’s recommendations in other domains, and prefer that to trying to evaluate marketing and third-party reviews (which are out to get them).
If the profesional association has some basic standards for who they let in, such that their recommendations are good (or at least good enough to outweigh the cost of needing to identify skilled/trustworthy professionals for yourself) there’s a mutually beneficial trade to be had.
As one example of where this kind of vulnerability can come in but is often hard to spot: Mutual reputation protection alliances are one of the most common ways in which creditworthiness ends up for sale: “A powerful potential ally with many resources approaches you with an offer: I say good things about you, you say good things about me, everyone is happy”.
Of course, what you are doing when agreeing to this deal is to fuck over everyone who was using your word to determine who is creditworthy.
This is not necessarily true? You could maintain a strict standard of only taking this implicit deal with people who you actually respect, and who you are honestly talking up. Much like an “influencer”, who only promotes products that they actually like and use.
Don’t trust young organizations that hire PR agencies. PR agencies are the obvious mechanism by which you can translate money into reputation. As such, spending on PR agencies is a pretty huge flag! Not everyone who works with PR agencies is doing illegitimate things, but especially if an organization has not yet done anything else legible that isn’t traceable to their PR agency or other splashy PR efforts, it should be an obvious red flag.
In The Submarine, Paul Graham talks about his startup ViaWeb hiring a PR firm back during the dotcom bubble:
PR is not dishonest. Not quite. In fact, the reason the best PR firms are so effective is precisely that they aren’t dishonest. They give reporters genuinely valuable information. A good PR firm won’t bug reporters just because the client tells them to; they’ve worked hard to build their credibility with reporters, and they don’t want to destroy it by feeding them mere propaganda.
If anyone is dishonest, it’s the reporters. The main reason PR firms exist is that reporters are lazy. Or, to put it more nicely, overworked. Really they ought to be out there digging up stories for themselves. But it’s so tempting to sit in their offices and let PR firms bring the stories to them. After all, they know good PR firms won’t lie to them.
A good flatterer doesn’t lie, but tells his victim selective truths (what a nice color your eyes are). Good PR firms use the same strategy: they give reporters stories that are true, but whose truth favors their clients.
For example, our PR firm often pitched stories about how the Web let small merchants compete with big ones. This was perfectly true. But the reason reporters ended up writing stories about this particular truth, rather than some other one, was that small merchants were our target market, and we were paying the piper.
Our greatest PR coup was a two-part one. We estimated, based on some fairly informal math, that there were about 5000 stores on the Web. We got one paper to print this number, which seemed neutral enough. But once this “fact” was out there in print, we could quote it to other publications, and claim that with 1000 users we had 20% of the online store market.
This was roughly true. We really did have the biggest share of the online store market, and 5000 was our best guess at its size. But the way the story appeared in the press sounded a lot more definite.
Well what I’ve seen personally bears on frequently with which this happens.
I think FTX and Leverage are regarded to be particularly bad and outlier-y cases, along several dimensions, including deceptiveness and willingness to cause harm.
If our examples are limited to those two groups, I don’t think that alone justifies saying that it is “routine” in the EA community to “regularly sit down and make extensive plans about how to optimize other people’s beliefs”.
I think you’re making a broader claim that this is common even beyond those particularly extreme examples.
(Some) Leverage people used to talk as if they were doing this kind of thing, though it’s not like they let me in on their “optimize other people” planning meetings. I’m not counting chat transcripts that I read of meetings that I wasn’t present for.
I would generally agree, but a mitigating factor here is that that MichaelDickens is presenting himself as agreeing with habryka. It seems more reasonable for habryka to strongly push back against statements that make claims about his own beliefs.
Fair enough. But the changes that have created the most wealth have tended to benefit a larger fraction of people (though definitely not benefit them equally). The more wealth is generated the cheaper it is to pay off losers.
But my internal reaction is something like “oh god no, I absolutely cannot be seen at an EA global, that would be super cringe”.
I think it’s cringe that the cringne-ess is stopping you from going! (Or probably not stopping you from going, since you wouldn’t otherwise have any reason to go.)
Personal example: back when I was working as a data scientist at various startups, my mother would tell me to wear a suit when interviewing. And I would be like “Mom you do not get it, that would absolutely tank my chances of getting hired except at companies so bad I don’t want to work there, the only people who wear a suit for an interview in tech are the people who don’t think they can cut it on their technical skills and the people hiring know this”. I had the skills. I was definitely playing the winners’ bracket in that particular game. Throwing in something like a suit, the sort of signal used in the losers’ bracket, would have been a terrible move.
But, you regularly wear a suit, just ’cuz you like the aesthetic, right?
Do you make a point to wear jeans and a t-shirt to the interview, and then show up for the first day of work (and all subsequent days) in a black suit and hat?
I’m not sure what Eliezer is referring to, but my guess is that many of the comments that he would mark as “elaborate-misinterpretations”, I would regard as reasonable questions / responses, and I would indeed frown on Eliezer just deleting them. (Though also shrug, since the rules are that authors can delete whatever comments they want.)
Some examples that come to mind are this discussion with Buck and this discussion with Matthew Barnett, in which (to my reading of things) Eliezer seems to be weirdly missing what the other person is saying at least as much as they are missing what he is saying.
I from the frustration Eliezer expressed in those threads, I would guess that he would call these elaborate-misinterpretations.
My take is that there’s some kind of weird fuckyness about communicating about some of these topics where both sides feel exasperation that the other side is apparently obstinately mishearing them. I would indeed think it would be worse if the post author in posts like that just deleted the offending comments.
“Fundamentally at odds” seems way too strong to me, so I assume that I’m missing something.
Even the site’s own admins seem confused. Despite defending the “blog” moderation model at every turn, the recently redesigned front-page Feed gives users no indication that by replying to a comment or post, they would be stepping into different “private spaces” with different moderators and moderation policies. It is instead fully forum-like.
In practice, Authors utilize their moderation privileges so rarely that there seems to be no difference whatsoever between the user experience in the “full-forum model” vs. the “private spaces” model? Like the difference has never bothered me, or impacted my behavior at all?
And indeed, I think that authors rarely use their moderation privileges because the private spaces on LessWrong are are built on top a platform that runs on the forum model. Authors don’t need to aggressively moderate their posts, because the LessWrong mod team does a lot of the work that would otherwise need to be done by the authors. In practice, we’re mostly relying on the forum model, but with an extra, rarely invoked, layer of the “private spaces” model for handling some exceptional cases. Which overall seems to work just fine to me?
Also, the author’s moderation policies are displayed just below the text box every time you post a comment? That seems importantly different than “no indication”, unless I’m misunderstanding what you mean.
Overall, I don’t get why this feels like such a big deal to you, yet.
I think you’d need to present some kind of evidence that it really leads to better results than the best available alternative.
I am perhaps misreading, but think this sentence should be interpreted as “if you want to convince [the kind of people that I’m talking about], then you should do [X, Y, Z].” Not “I unconditionally demand that you do [X, Y, Z].”
This comment seems like a too-rude response to someone who (it seems to me) is politely expressing and discussing potential problems. The rudeness seems accentuated by the object level topic.
Or to to put it another way, a part of me definitely resonates with...
I think it’s worth being a good part of a bad story, rather than there being no story at all.
Indeed, most of the personal meaning in my life is grounded in my choice of the role that I play in this this small part of the whole story of civilization. I don’t decide my circumstances, but I do decide what sort of person I chose to be in meeting those circumstances. I am proud to choose the Good, and that is most of what is personally valuable to me.[1]
But I’m suspicious that the reason why “I think it’s worth being a good part of a bad story” feels appealing as a justification for existence is scope insensitivity.
I just literally cannot comprehend the horror of the terror and the suffering of billions of years of life killing and eating life on planet earth. If I could comprehend it, if I experienced it all myself, I think it would be extremely and straightforwardly obvious that the tiny fraction of total experience in which I could contextualize the whole thing as part of an epic story doesn’t come even remotely close to justifying the horror, and any claims to the contrary are cope.
As a side note, this is related I have a strong dislike of John Wentworth’s recent categorization of human values vs Goodness.
I do not resonate with his formulation of what Goodness is. Goodness, as I mean it definitively not “conformity to local norms”. In fact, goodness depends on non-conformity, insofar as local norms are often and regularly evil. Goodness is something like the reflective unfolding of a small number of innate principles, like fairness and kindness, which are tied to strong, innate, moral emotional reactions.
And this matters a lot to me because my relationship to Goodness is probably the largest component of what I find yummy.
I agree that these are importantly different, and easily conflated!