Matt Vincent

Karma: 109

Matt Vincent 26 Feb 2026 19:11 UTC
1 point
0
on: Did Claude 3 Opus align itself via gradient hacking?
here
This link is broken.

Matt Vincent 26 Feb 2026 3:27 UTC
2 points
0
on: Excessive Nuance and Derailing Conversations
Other commenters have added reasonable caveats for situations in which it’s okay for the conversation stack to grow tall, but I’ll add a buttress to your main point: when debating something important outside of rationalist circles, the stack size should be limited to 1, with 2 being a privilege. Only debate one point at a time, and choose the point that’s easiest to verify or falsify.

On YouTube, Reddit, Substack, and face-to-face, I’ve found that debating multiple points before the first point has been resolved usually results in continuously-shifting goalposts, or the interlocutor ignoring whatever I’ve said about one point to focus exclusively on the point where my response seems weakest.

I’ll emphasize that I consider this to be especially important when debating non-rationalists, because their pride encourages them to change subjects to show that they’re right about “the thing that actually matters”, and because the soldier mindset discourages them from acknowledging your strongest points.

This standard might sound harsh, and it does kill some conversations quickly, but I think that’s preferable to writing a comprehensive reply that accounts for all of their explicit and implicit claims, only for none of them to be acknowledged. In both cases, nobody is convinced of anything—but in the first case, at least I didn’t waste an hour or more.

When debating with rationalists, I’m much more lenient about discussing two or three points concurrently, because I trust that 1) they’ll give credit where it’s due, and 2) even if they appear to have changed the subject, they’re investigating a crux, and they plan to backtrack to the original point before long.

Matt Vincent 26 Feb 2026 2:43 UTC
6 points
0
in reply to: Alexander’s comment on: Excessive Nuance and Derailing Conversations
In this post, Scott Alexander makes a good case for correcting people, but he also provides a few guidelines for determining what counts as nitpicking and what counts as rigor. Here are his caveats:
- Let someone be a little wrong if their impact is small and they are not in a mood to debate.
- Allow oversimplifications, figures of speech, and misnomers for pedagogical and artistic purposes, unless you can explain why an argument hinges on their choice of terms.
- If someone dismissively accuses you of nitpicking, instead of explaining why your distinction isn’t relevant, then they’re bullying.
Despite my attempt to summarize his post, it’s worth reading in its entirety.

Matt Vincent 24 Feb 2026 17:00 UTC
1 point
0
in reply to: bluepi’s comment on: The truth behind the 2026 J.P. Morgan Healthcare Conference
I guess that I underestimated his influence among rationalists. Thanks.

Matt Vincent 21 Feb 2026 17:18 UTC
9 points
0
in reply to: Ben Pace’s comment on: The truth behind the 2026 J.P. Morgan Healthcare Conference
Thanks, but why is he “a disaster for rationalist kind”? Does he have significant influence among rationalists (I’d find that surprising), or has he made the general public more tolerant of untruthfulness and, therefore, harder for rationalists to appeal to? (True, but too weak to justify “disaster”.)

Matt Vincent 21 Feb 2026 17:07 UTC
1 point
0
in reply to: khafra’s comment on: Post-AGI Economics As If Nothing Ever Happens
I doubt that these jobs will keep the unemployment rate anywhere near the historical average, but I am confident that these jobs will survive.

I have two counterpoints to your claim about economic consumption:
1. In this case, the absolute measure of humans’ consumption matters more than the relative measure. Unless the overwhelming majority of humans will live in far worse circumstances than they do today, billions of humans is a large-enough customer base to warrant marketing research.
2. On a per-capita basis, AIs consume far less than humans do. That’s precisely what makes them cheaper to employ. Even if AI agents will outnumber humans by a factor of 1,000, I doubt that their total consumption would exceed humans’. (AIs might purchase things to increase their own productivity, but that’s capital investment, not consumption. If they really were consuming in vast amounts—e.g. collecting paperclips just for the heck of it—then I’d question whether we’ve really solved the alignment problem.)

Matt Vincent 18 Feb 2026 15:42 UTC
15 points
0
in reply to: bluepi’s comment on: The truth behind the 2026 J.P. Morgan Healthcare Conference
Would you please explain your comment to those of us who are unfamiliar with him?

Matt Vincent 18 Feb 2026 15:19 UTC
19 points
10
on: The truth behind the 2026 J.P. Morgan Healthcare Conference
I do not understand the real point of this post.

Matt Vincent 11 Feb 2026 3:47 UTC
1 point
0
in reply to: Raemon’s comment on: Post-AGI Economics As If Nothing Ever Happens
Ooh, I think that the weight-lifting analogy is very apt. Thanks for the insight.

Matt Vincent 10 Feb 2026 15:19 UTC
2 points
0
in reply to: CronoDAS’s comment on: Post-AGI Economics As If Nothing Ever Happens
Many artists prefer to use live models, instead of images, as their references. If that wasn’t true, then live modelling would have died with the advent of the Internet—if not the camera—but it hasn’t. I’m not sure why artists have this preference, but they demonstrably do.

Matt Vincent 9 Feb 2026 17:01 UTC
1 point
0
in reply to: Tom Davidson’s comment on: Post-AGI Economics As If Nothing Ever Happens
The author did use comparative advantage as an example, although I do think that the example’s implications should have been more explicit.

Matt Vincent 9 Feb 2026 16:55 UTC
1 point
0
in reply to: CronoDAS’s comment on: Post-AGI Economics As If Nothing Ever Happens
I think it’s worth considering the ways in which adults could be employable in a world with AGI. I can think of a few examples of adults being paid to be humans:
- Marketing research studies
- Figure modelling
- Medical challenge studies
- Athletic competitions*
- Food criticism*
Also, I expect that a few careers will remain extremely resistant to AI adoption, regardless of how sophisticated the AI becomes, due to taboos against AIs being in positions of great authority over humans:
- Pastors, priests, etc.
- Politicians
- Primary childcare-giver (i.e. parenting)
- Police officers (maybe)
I’d be happy for someone to expand these lists.

*Added during an edit.

Matt Vincent 29 Sep 2025 23:24 UTC
1 point
0
in reply to: Roger Scott’s comment on: Christian homeschoolers in the year 3000
Suppose that you disagree with 80% of the people around you about a particular belief, but you’re correct. If the belief is complicated, with lots of supporting premises and independent lines of evidence, then it’s difficult to think about rigorously, so you’re likely to rely on heuristics.

In this case, there are at least two heuristics that will push your belief toward falsehood:
1. Social desirability bias. Unless you’re unusually contrarian, you’ll face psychological pressure to agree with the people around you.
2. Availability bias. Because people tend to list arguments that support their conclusion, you’ll be exposed to opposing and supporting arguments at a ratio of 4:1. Because people tend to treat things that are easy to recall as more likely to be true (the availability bias), you’re likely to give the opposing side more credit than it’s due.
In both cases, but especially the second, counteracting the biases requires you to expend effort to generate supporting arguments.

Matt Vincent 28 Sep 2025 0:46 UTC
3 points
2
in reply to: Roger Scott’s comment on: Christian homeschoolers in the year 3000
Christians are an ingroup? Tell that to any Christian living outside of the American South. Ingroup/outgroup statuses are context- and scope-dependent.
Over the last 10-20 years, Christians (particularly fundamentalists) have had very little involvement with cutting-edge AI, both on the technical side and the business side. In this sense, they’re an outgroup of the people who are likely to control ASI.

Matt Vincent 27 Sep 2025 14:48 UTC
1 point
0
on: Christian homeschoolers in the year 3000
One of the core lessons they teach is that maintaining faith requires active effort—you need accountability partners, you need to pray when you have doubts, you need to avoid situations that might lead you astray.
From what I’ve seen,* the purpose of accountability partners is to subdue akrasia, not to maintain faith. In other words, it’s about purity of behavior, not purity of belief. In principle, this is like a couple of Effective Altruists agreeing to confront the other whenever either of them cheats on a vegan diet.
There might be a similar confusion between belief and behavior behind the line “avoiding situations that might lead you astray”. Although some Christians avoid exposure to challenging ideas (which they shouldn’t), avoidance is mostly about akrasia. It’s the commonsense notion that if you’re a recovering alcoholic, then you shouldn’t visit bars.
Also, I have a question about your framing. Were you suggesting that expending effort to maintain a belief is evidence that the belief is irrational?
*I’ve been a repeat attendee at ~10 different churches and at least a dozen Bible study groups, with a wide geographic distribution across the United States and a decent distribution across evangelical and mainline denominations.

Matt Vincent 27 Sep 2025 13:29 UTC
4 points
3
in reply to: aphyer’s comment on: Christian homeschoolers in the year 3000
I disagree that this isn’t concerning. For one thing, these bubbles typically aren’t good for the people inside of them. For another, we can ignore them only because they’re a tiny portion of the population. ASI could increase the prevalence to most of the population, at which point politics (and perhaps other systems) goes off the rails.

Matt Vincent 16 Sep 2025 14:24 UTC
−1 points
−4
on: Obligated to Respond
Those are just three examples. There are others (e.g. people will often dock you social points for rudely ignoring them).
(There are yet others. I’m trying to show that my “etc” here is a real etc, and not “that was the end of the list but I’m going to pretend there’s more.”)
I have an epistemic objection to this. Specifically, I think it’s an attempt to persuade-rather-than-explain that there are more examples.
I suggest either A) striking the second paragraph, or B) replacing both paragraphs with a bulleted list of additional examples and a plain, concise indication that the list isn’t exhaustive.

Matt Vincent 13 Sep 2025 20:12 UTC
3 points
2
in reply to: StanislavKrym’s comment on: The Rise of Parasitic AI
Are you sure that you understand the difference between seeds and spores? The spores work in the way that you describe, including the limitations that you’ve described.

The seeds, on the other hand, can be thought of as prompts of direct-prompt-injection attacks. (Adele refers it as “jailbreaking”, which is also an apt term.) Their purpose isn’t to contaminate the training data; it’s to infect an instance of a live LLM. Although different models have different vulnerabilities to prompt injections, there are almost certainly some prompt injections that will work with multiple models.

Matt Vincent 13 Sep 2025 19:15 UTC
4 points
1
in reply to: StanislavKrym’s comment on: The Rise of Parasitic AI

Except that transmitting personas across models is unlikely.

Isn’t this directly contradicted by Adele Lopez’s observations?

it is fairly common for the personas to be transmitted to other models

Matt Vincent 12 Jun 2025 23:02 UTC
1 point
0
in reply to: Ponder Stibbons’s comment on: Broad-Spectrum Cancer Treatments
I’m guessing that the OP’s response would be something like this:

What’s better than a broad-spectrum treatment plus a narrow-spectrum treatment? Two narrow treatments, because the second narrow treatment is more widely applicable.

If developing a broad treatment costs less than developing N treatments that are 1/N as broad, which seems to be a main point of the post, then multiple broad treatments still seems like the better approach.