KvmanThinking

• KvmanThinking 14 Apr 2026 23:54 UTC

  1 point

  0

  on: Coun­ter­ar­gu­ments to the ba­sic AI x-risk case

  MENSA isn’t a major force in the world.

  That’s because differences in human intelligence are relatively small. We need a HUGE HUGE amount of relative intelligence before it starts getting dangerous. AGI has the potential to recursively develop that way way way faster than more intelligent people can get slowly selected for by evolution.

• KvmanThinking 31 Mar 2026 0:05 UTC

  5 points

  0

  on: Stan­ley Mil­gram wasn’t pes­simistic enough about hu­man na­ture?

  I wouldn’t jump to the conclusion that huge amounts of the regular population are sadistic and just don’t act that way because they don’t have an excuse. Results of experiments designed to test if regular people are selfish/​sadistic when unpunished, (like the wallet test, and Vsauce/​Mind Field’s standford prison experiment followup), tend to have different results.

• KvmanThinking 27 Feb 2026 3:13 UTC

  4 points

  2

  on: Whack-a-Mole is Not a Winnable Game

  you wind up having to prove what a drug does to a person in court, and that’s difficult

  Sorry, how is it difficult to “prove what a drug does to a person in court”? Is there something I’m missing?

• KvmanThinking 16 Nov 2025 18:17 UTC

  1 point

  0

  in reply to: KvmanThinking’s comment on: Ta­boo Your Words

  (Update: I am no longer fluent in Toki Pona)

• KvmanThinking 16 Nov 2025 18:17 UTC

  1 point

  0

  in reply to: David_Gerard’s comment on: The Fal­lacy of Gray

  Strong-upvoted both ways for visibility.

• KvmanThinking 2 Nov 2025 20:50 UTC

  1 point

  0

  in reply to: faul_sname’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  A single instance of an LLM summoned by a while loop, such that the only thing the LLM is outputting is a single token that it predicts as most likely to come after the other tokens it has recieved, only cares about that particular token in that particular instance, so if it has any method of self-improving or otherwise building and running code during its ephemeral existence, it would still care that whatever system it builds or becomes, still cares about that token in the same way it did before.

• KvmanThinking 2 Nov 2025 20:40 UTC

  1 point

  0

  in reply to: Josh Snider’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  This has in some ways already started.

  AIs are already intentionally, agentically self-improving? Do you have a source for that?

• KvmanThinking 2 Nov 2025 16:11 UTC

  1 point

  −2

  in reply to: Charlie Steiner’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  Utility functions that are a function of time (or other context)

  ?? Do you mean utility functions that care that, at time t = p, thing A happens, but at t = q, thing B happens? Such a utility function would still want to self-preserve.

  utility functions that steadily care about the state of the real world

  Could you taboo “steadily”? *All* utility functions care about the state of the real world, that’s what a utility function is (a description of the exact manner in which an agent cares about the state of the real world), and even if the utility function wants different things to happen at different times, said function would still not want to modify into a different utility function that wants other different things to happen at those times.

• KvmanThinking 2 Nov 2025 14:08 UTC

  1 point

  −2

  in reply to: faul_sname’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  If you have goals at all you care about them being advanced. It would be a very unusual case of an AI which is goal-directed to the point of self-improvement but doesn’t care if the more capable new version of themself that they build pursues those goals.

• KvmanThinking 2 Nov 2025 14:04 UTC

  −1 points

  −2

  in reply to: Zac Hatfield-Dodds’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  Think of an agent with any possible utility function, which we’ll call U. If said agent’s utility function changes to, say, V, it will start taking actions that have very little utility according to U. Therefore, almost any utility function will result in an agent that acts very much like it wants to preserve its goals. Rob Miles’s video explains this well.

• KvmanThinking 2 Nov 2025 3:53 UTC

  3 points

  0

  in reply to: faul_sname’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  Retaining your values is a convergent instrumental goal and would happen by default under an extremely wide range of possible utility functions an AI would realistically develop during training.

• KvmanThinking 1 Nov 2025 22:27 UTC

  4 points

  1

  in reply to: RHollerith’s comment on: Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

  My point is that I see recursive self-improvement often cited as the thing that gets the AI up to the level where it’s powerful enough to kill everyone, which is a characterization I disagree with, and is an important crux, because believing in it makes someone’s timelines shorter.

[Question] Shouldn’t tak­ing over the world be eas­ier than re­cur­sively self-im­prov­ing, as an AI?

• KvmanThinking 1 Nov 2025 0:45 UTC

  1 point

  0

  on: An Opinionated Guide to Pri­vacy De­spite Authoritarianism

  How did you find all of this out?

• KvmanThinking 22 Sep 2025 23:54 UTC

  4 points

  2

  in reply to: Davey Morse’s comment on: The Sorry State of AI X-Risk Ad­vo­cacy, and Thoughts on Do­ing Better

  your silly alignment game is silly. i like it. have you promoted it somewhere?

• KvmanThinking 17 Sep 2025 23:14 UTC

  1 point

  0

  on: 3 Levels of Ra­tion­al­ity Verification

  The video game The Witness.

• KvmanThinking 13 Sep 2025 21:52 UTC

  2 points

  0

  in reply to: Eye You’s comment on: The El­dritch in the 21st century

  Academia existed in the form of “trying to become a priest so people trust the stuff that you say” and various adjacent concepts.” Bureaucracies and geopolitics existed, in a state actually worse (although simpler) than they do now, before we could even smelt iron.

• KvmanThinking 12 Sep 2025 23:09 UTC

  3 points

  0

  in reply to: xepo’s comment on: Grat­i­tude Jour­nal Fallacy

  Aah, okay. Oops. Thank you. :/​

  (I will try the gratitude journal for like 4 more weeks, and see if the increase in my well-being is measurable. While I now agree with the problem you identified with my local example, I think the broader fallacy my post was talking about still exists; While I cannot think of any other examples that apply to myself, I do sometimes notice myself sort of “yearning for Amazing Things No One Does”. I’m sure if I had more time I could think of more concrete stuff.)

• KvmanThinking 12 Sep 2025 22:53 UTC

  1 point

  0

  in reply to: NoSignalNoNoise’s comment on: Grat­i­tude Jour­nal Fallacy

  What are the higher utility per unit effort things you could be doing instead during that time? Did you stop doing a gratitude journal? If so, did you do those other things instead?

  Oh, just studying, working on my stories, working on my songs, working on my conlang, getting better at coding, getting better at art, generally just enjoying being a human, in this beautiful, beautiful universe before it gets disassembled into nothing...

Grat­i­tude Jour­nal Fallacy