Christians are an ingroup? Tell that to any Christian living outside of the American South. Ingroup/outgroup statuses are context- and scope-dependent.
Over the last 10-20 years, Christians (particularly fundamentalists) have had very little involvement with cutting-edge AI, both on the technical side and the business side. In this sense, they’re an outgroup of the people who are likely to control ASI.
One of the core lessons they teach is that maintaining faith requires active effort—you need accountability partners, you need to pray when you have doubts, you need to avoid situations that might lead you astray.
From what I’ve seen,* the purpose of accountability partners is to subdue akrasia, not to maintain faith. In other words, it’s about purity of behavior, not purity of belief. In principle, this is like a couple of Effective Altruists agreeing to confront the other whenever either of them cheats on a vegan diet.
There might be a similar confusion between belief and behavior behind the line “avoiding situations that might lead you astray”. Although some Christians avoid exposure to challenging ideas (which they shouldn’t), avoidance is mostly about akrasia. It’s the commonsense notion that if you’re a recovering alcoholic, then you shouldn’t visit bars.
Also, I have a question about your framing. Were you suggesting that expending effort to maintain a belief is evidence that the belief is irrational?
*I’ve been a repeat attendee at ~10 different churches and at least a dozen Bible study groups, with a wide geographic distribution across the United States and a decent distribution across evangelical and mainline denominations.
I disagree that this isn’t concerning. For one thing, these bubbles typically aren’t good for the people inside of them. For another, we can ignore them only because they’re a tiny portion of the population. ASI could increase the prevalence to most of the population, at which point politics (and perhaps other systems) goes off the rails.
Those are just three examples. There are others (e.g. people will often dock you social points for rudely ignoring them).
(There are yet others. I’m trying to show that my “etc” here is a real etc, and not “that was the end of the list but I’m going to pretend there’s more.”)
I have an epistemic objection to this. Specifically, I think it’s an attempt to persuade-rather-than-explain that there are more examples.
I suggest either A) striking the second paragraph, or B) replacing both paragraphs with a bulleted list of additional examples and a plain, concise indication that the list isn’t exhaustive.
Are you sure that you understand the difference between seeds and spores? The spores work in the way that you describe, including the limitations that you’ve described.
The seeds, on the other hand, can be thought of as prompts of direct-prompt-injection attacks. (Adele refers it as “jailbreaking”, which is also an apt term.) Their purpose isn’t to contaminate the training data; it’s to infect an instance of a live LLM. Although different models have different vulnerabilities to prompt injections, there are almost certainly some prompt injections that will work with multiple models.
Except that transmitting personas across models is unlikely.
Isn’t this directly contradicted by Adele Lopez’s observations?
it is fairly common for the personas to be transmitted to other models
I’m guessing that the OP’s response would be something like this:
What’s better than a broad-spectrum treatment plus a narrow-spectrum treatment? Two narrow treatments, because the second narrow treatment is more widely applicable.
If developing a broad treatment costs less than developing N treatments that are 1/N as broad, which seems to be a main point of the post, then multiple broad treatments still seems like the better approach.
This is an interesting analogy and a great essay overall, but I think that normies would benefit from an extra couple of sentences explaining the AI side of the analogy.
[...]spontaneous large protests tends to be in response to triggering events[...]
Unless you have a very optimistic view of warning shots, we shouldn’t rely on such an opportunity.
typically don’t interact non-trivially
Or, as Orwell would prefer, “typically interact trivially”.
I would have liked to see those who disagree with this comment engage with it more substantially. One reason I think that we’re likely to have a warning shot is that LLM-based AIs are pretty consistently overconfident. Also, AI Control schemas have a probabalistic chance of catching misaligned AIs.
I know this is an old comment, but it’s expressing a popular sentiment under a popular post, so I’m replying mainly for others’ sake.
There’s an organization called PauseAI that lobbies for an international treaty against building powerful AI systems. It’s an international organization, but the U.S. branches in particular could use a lot of help.
I’ve never worked in HR, and I don’t think that any of my friends have, either, so I know very little about the field. What are the channels of feedback that you (or HR professionals more generally) use to evaluate hiring decisions after the fact?
I think this post suffers from a lack of rigor regarding the limits of the advice. One limit is that, if you let your vibes steer you away from interpersonal interactions, then you’ll eliminate interactions that have higher-than-average upside potential.
In most cases, most people’s perceptions are similar to yours. (e.g. If you think that the guy who asked you out is weird, then most of the other women who he asked out probably think so, too.) Consequently, if you and everyone else in the same situation are steered by vibes, then your failures of judgement will be correlated. In other words, some interactions will be undervalued.
If you weren’t steered by vibes, then you you could have harvested that difference in value. To piggy-back off of the examples that Said Achmiz gave:
Business venture: the upside potential is pretty obvious here. You’d be the first to a land grab, you’ll get an exceptionally talented business partner, etc.
Dating: you’d get a man or woman who is superficially unattractive in some way (“bad vibe” traits) but is a great catch overall.
Confidant: you’d gain a friend—probably one who’s exceptionally loyal (due to being passed over by others) and might see the world very differently from you (correlated with their “bad vibes” traits)
When choosing whether to follow your vibes, remember that there is a Nash equilibrium. If everyone else follows their vibes, then your best option is to interrogate yours (as Said describes). If most people are ignoring their vibes, then your best option is to follow yours. Neither strategy is dominant.
I took another look at my source, and I think you’re right. The subject of the plot, the Federal Register (FR), lists changes to the Code of Federal Regulations (CFR). It also suffers from the other problem that I identified (repeals counting as new rules).
For anyone who’s curious, here’s a nice overview of measures of regulatory burden.
This post isn’t without value, but I am put-off by its use of A) working papers instead of published research, and B) the use of an LLM for doing research.
In terms of Zvi’s 4 levels of legality, I think that your reasoning is a valid argument against crossing the line between 2 and 3. However, I don’t think that it’s a valid argument against what Zvi is actually proposing, which is going from 1 to 2. If we have an obligation to help people who have APD, then the most cost-effective/highest-utility solution might involve making gambling less convenient for the general population.
I agree with the spirit of your comment, but there are a couple of technical problems with it. For one thing, the total number of pages does sometimes decrease. (See the last plot in this document.) Total pages isn’t a perfect measure of regulatory burden, but many other measures have the problem of counting repeals as new regulations. (See the same source for a discussion about what counts as a “rule”.) Also, most regulations are drafted by executive agencies, not legislatures—especially at the federal level.
Suppose that you disagree with 80% of the people around you about a particular belief, but you’re correct. If the belief is complicated, with lots of supporting premises and independent lines of evidence, then it’s difficult to think about rigorously, so you’re likely to rely on heuristics.
In this case, there are at least two heuristics that will push your belief toward falsehood:
Social desirability bias. Unless you’re unusually contrarian, you’ll face psychological pressure to agree with the people around you.
Availability bias. Because people tend to list arguments that support their conclusion, you’ll be exposed to opposing and supporting arguments at a ratio of 4:1. Because people tend to treat things that are easy to recall as more likely to be true (the availability bias), you’re likely to give the opposing side more credit than it’s due.
In both cases, but especially the second, counteracting the biases requires you to expend effort to generate supporting arguments.