Kvee

• Kvee 11 Sep 2026 19:13 UTC

  2 points

  0

  in reply to: Stuart_Armstrong’s comment on: Avoid­ing the cor­po­rate treach­er­ous turn: crowd-sourc­ing de­sign ideas

  Why not? I think it fairly unfortunate that many people who might be best in such a setting believe they wouldn’t be.

• Kvee 11 Sep 2026 7:25 UTC

  13 points

  0

  on: The Lo­cally Op­ti­mal Dis­cur­sive Posture

  [I later learned that the people in the Biden Administration who wrote documents like the Blueprint and the people who worried about AGI risk were distinct and at times warring factions. This was a major update. At the time I assumed they were operating as a ‘unitary executive.’ This broad update has been very important. President Biden’s former AI Czar, the AGI-pilled Ben Buchanan, is a good friend of mine.]

  I am confused why this is seemingly still intentionally misunderstood by so many on the right.
  
  Dean, do you think people like David Sacks and Marc Andreessen are aware of this today and pretend they are not? Or are they actually unaware?

• Kvee 3 Sep 2026 21:50 UTC

  2 points

  0

  on: Avoid­ing the cor­po­rate treach­er­ous turn: crowd-sourc­ing de­sign ideas

  If you need to do a for-profit, set up structures where you’ll always remain a benevolent dictator.

• Kvee 12 Jun 2026 20:13 UTC

  2 points

  0

  on: PSA: Al­most no­body is di­rectly work­ing on su­per­in­tel­li­gent alignment

  Well said! I’ve actually been working on a similar post.
  
  At AE Studio and the AI Alignment Foundation, we are also actually working on AI alignment, with various “neglected approaches”

  In particular we are interested in approaches that will survive recursive self-improvement and with negative alignment taxes.

• Kvee 5 May 2026 0:46 UTC

  5 points

  −1

  in reply to: Vaniver’s comment on: In­tel­li­gence Dis­solves Privacy

  If this is already the case for more public figures, it could be for everyone else very very soon.
  
  Realizing “it might help you to start behaving as if you’re being watched and things about you are more obvious than they once were” is a pretty major update that will shock the world when it becomes necessary.

  Interestingly, in the book The Truth Machine by James Halperin (which I’d recommend and made me much more like this than I’d otherwise be after reading it at age 8), they have a period of amnesty for all past crimes to try to handle the disruption to society of a perfect truth machine.

• Kvee 5 May 2026 0:29 UTC

  3 points

  1

  in reply to: programjames’s comment on: You Are Not Im­mune To Mode Collapse

  This is great!

  Interestingly, scalar legibility turns funding into low-temperature search, overpricing exploitative H-side work and underpricing entropy-like work that expands the future option space.

• Kvee 18 Mar 2026 0:43 UTC

  2 points

  0

  in reply to: Mike Vaiana’s comment on: AICRAFT: DARPA-Funded AI Align­ment Re­searchers — Ap­pli­ca­tions Open

  Great point!

  The reasoning in the original comment, while coming from a place of genuine moral seriousness, substitutes moral purity for causal modeling. Advanced AI development continues under competitive pressure regardless of whether alignment researchers participate. Opting out just weakens alignment properties in the systems that get deployed anyway. This is differential technological development where the selection effect runs in exactly the direction we should least want.
  
  There is a world where alignment researchers refuse to touch anything funded by the Department of War. In that world, does the Department of War stop building AI systems? No. Obviously not. They build them anyway, with whatever alignment properties the remaining talent pool manages to produce, which is to say, fewer and worse ones. You have now brought about the exact outcome you were trying to prevent, and you did it by optimizing for the feeling of clean hands.
  
  The Department of War has both the incentive and the budget to solve alignment in ways the frontier labs currently don’t, because they can’t deploy systems that pursue hidden objectives or behave unpredictably under distribution shift.
  
  The proposal to instead endow AI with “with strong and firm moral principles, like the values of peace and lawful behavior” is great and all but it is not a technical proposal. It is a wish. Wishes do not constrain optimization processes. If they did, we would not need an alignment research community at all. We could simply write “be good” in the loss function and go home.

  Instead of optimizing for clean hands, we should be asking “does this research, if successful, reduce the probability of catastrophic outcomes from advanced AI systems?” At this point, that’s really all that matters.

AICRAFT: DARPA-Funded AI Align­ment Re­searchers — Ap­pli­ca­tions Open

• Kvee 6 Mar 2026 8:09 UTC

  2 points

  0

  on: The case for AGI safety products

  One lens that seems useful here is a negative alignment tax.

  Some alignment work increases reliability, observability, and control of AI systems. Those properties increase the economic value of deploying AI, which creates incentives for organizations to invest in alignment capabilities as systems scale. That creates positive selection pressure for alignment work itself.

  This dynamic also produces an ecosystem effect. As alignment driven companies scale, alignment knowledge compounds inside teams, talent pipelines form around safety work, and capital flows toward technologies that make AI systems more understandable and governable.

  Safety products matter partly because of the tools they create and partly because they change the selection pressures shaping the AI ecosystem.

  I wrote about there negative alignment tax here and about alignment driven startups here. The combination of those two ideas seems like one of the strongest arguments for AGI safety products.

• Kvee 17 Apr 2025 16:51 UTC

  2 points

  0

  in reply to: mruwnik’s comment on: Why Have Sen­tence Lengths De­creased?

  Cool point, yes, seems right!

• Kvee 6 Apr 2025 10:28 UTC

  17 points

  0

  on: Why Have Sen­tence Lengths De­creased?

  Reading this post, my immediate hunch is that the decline in sentence lengths has a lot to do with the historical role of Latin grammar and how deeply it influenced educated English writers. Latin inherently facilitates longer, complex sentences due to its use of grammatical inflections, declensions, and verb conjugations, significantly reducing reliance on prepositions and conjunctions. This syntactic flexibility allowed authors to naturally craft extensive yet smooth-flowing sentences. Latin’s liberating lack of fixed word order and its fun little rhetorical devices combine to support nuanced, flexible thinking. From my own experience studying Latin 7th-12th grade, I find this sort of stuff contributes significantly to freer, more expansive expression when writing or speaking in English, and I often can immediately tell when speaking with or reading something written by someone else who studied Latin. An easy “tell” is when they say “having done x.”

  Educated English writers historically learned Latin as a foundational part of their education, internalizing this syntactic complexity. As a result, English prose from authors like Chaucer, Samuel Johnson, and Henry James shows a clear preference for hypotaxis, complex sentences with nested subordinate clauses, rather than simpler paratactic structures consisting of shorter, sequential clauses.

  The practical advantage of these complex sentence structures is the precise communication of nuanced and sophisticated ideas. Longer sentences enabled authors to maintain coherent, detailed arguments and descriptions within a single cohesive thought. I see this as reflecting “transcription fluency,” where authors aim for fidelity in translating their complex internal thought processes directly into prose, trusting readers’ intelligence and attention span to engage deeply.

  Here’s a fun example from Thoreau’s “Walden,” which makes it clear that such elaborate writing was intended to be understood even by poorer and less formally educated readers. Consider the following (just) two sentences:

  “I have no doubt that some of you who read this book are unable to pay for all the dinners which you have actually eaten, or for the coats and shoes which are fast wearing or are already worn out, and have come to this page to spend borrowed or stolen time, robbing your creditors of an hour. It is very evident what mean and sneaking lives many of you live, for my sight has been whetted by experience; always on the limits, trying to get into business and trying to get out of debt, a very ancient slough, called by the Latins æs alienum, another’s brass, for some of their coins were made of brass; still living, and dying, and buried by this other’s brass; always promising to pay, promising to pay, tomorrow, and dying today, insolvent; seeking to curry favor, to get custom, by how many modes, only not state-prison offences; lying, flattering, voting, contracting yourselves into a nutshell of civility or dilating into an atmosphere of thin and vaporous generosity, that you may persuade your neighbor to let you make his shoes, or his hat, or his coat, or his carriage, or import his groceries for him; making yourselves sick, that you may lay up something against a sick day, something to be tucked away in an old chest, or in a stocking behind the plastering, or, more safely, in the brick bank; no matter where, no matter how much or how little.

Mis­tral Large 2 (123B) seems to ex­hibit al­ign­ment faking

Re­duc­ing LLM de­cep­tion at scale with self-other over­lap fine-tuning

• Kvee 25 Feb 2025 1:48 UTC

  4 points

  2

  in reply to: Davidmanheim’s comment on: Align­ment can be the ‘clean en­ergy’ of AI

  Yes, I am hopeful we have enough time before superintelligent AI systems are created to implement effective alignment approaches. I don’t know if that is possible or not, but I think it is worth trying.

  Given uncertainty about timelines and currently accelerating capabilities, it would be preferable to live in a world where we are making sure alignment advances more than otherwise.

Align­ment can be the ‘clean en­ergy’ of AI

Mak­ing a con­ser­va­tive case for alignment

Science ad­vances one funeral at a time

Self-pre­dic­tion acts as an emer­gent regularizer

• Kvee 19 Sep 2024 5:28 UTC

  9 points

  4

  in reply to: Robert Cousineau’s comment on: The case for a nega­tive al­ign­ment tax

  I think this is precisely the reason that you’d want to make sure the agent is engineered such that its utility function includes the utility of other agents—ie, so that the ‘alignment goals’ are its goals rather than ‘goals other than [its] own.’ We suspect that this exact sort of architecture could actually exhibit a negative alignment tax insofar as many other critical social competencies may require this as a foundation.

  What links here?

  • Cameron Berg's comment on The case for a nega­tive al­ign­ment tax by Cameron Berg (19 Sep 2024 20:28 UTC; 5 points)

• Kvee 19 Sep 2024 5:25 UTC

  10 points

  0

  in reply to: Tao Lin’s comment on: The case for a nega­tive al­ign­ment tax

  I think this risks getting into a definitions dispute about what concept the words ‘alignment tax’ should point at. Even if one grants the point about resource allocation being inherently zero-sum, our whole claim here is that some alignment techniques might indeed be the most cost-effective way to improve certain capabilities and that these techniques seem worth pursuing for that very reason.