wonder

• wonder 27 Dec 2025 20:52 UTC

  2 points

  1

  on: AI #148: Christ­mas Break

  “Looking up the exact answer in a database”

  I mean it is not too too wrong if the model is using RAG?

• wonder 20 Aug 2025 20:19 UTC

  5 points

  0

  in reply to: Gordon Seidoh Worley’s comment on: Reli­gion for Rationalists

  We simply lack systems other than religions that provide religions same benefits

  I am empathetic to this point, and I was thinking of this recently too. However, I’ve reached a conclusion that the benefit of having values relatively independent of religion (or even similar communities) is to be able to change and develop your values as you experience more things in life. It feels to me that values in religion may be relatively fixed (I could be wrong).

  The sad/​hard thing is that the process of changing values or beliefs might be a painful experience, while fixed values could bring more stability. But embracing changes in values could be much more rewarding and beneficial in the long run. It does require a person to be very “strong” though, to challenge themselves on some fundamental levels.

  requiring members to buy in to some ethical, ontological, and metaphysical claims is load bearing

  This is sad, and I wonder what we can do to make it better.

• wonder 19 Aug 2025 15:18 UTC

  9 points

  0

  in reply to: Stephen Martin’s comment on: Stephen Martin’s Shortform

  I do see this as fair criticism (not surprised by it) to model welfare, if that is the sole reason for ending conversation early. I can see the criticism coming from two parts: 1) potential competing resources, and 2) people not showing if they care about these X group issues at all. If any of these two is true, and ending convo early is primarily about models have “feelings” and will “suffer”, then we probably do need to “turn more towards” the humans that are suffering badly. (These groups usually have less correlation with “power” and their issues are usually neglected, which we probably should pay more attention anyways).

  However, if ending convos early is actually about 1) not letting people having endless opportunity to practice abuse which will translate into their daily behaviors and shape human behaviors generally, and/​or 2) the model learning these human abusive languages that are used to retrain the model (while take a loss) during finetuning stages, then it is a different story, and probably should be mentioned more by these companies.

• wonder 19 Aug 2025 14:54 UTC

  0 points

  0

  in reply to: ceba’s comment on: Stephen Martin’s Shortform

  People have limited capacity for empathy

  Do you think this goes the other way as well?

• wonder 27 Jul 2025 17:19 UTC

  6 points

  0

  in reply to: Vale’s comment on: Vale’s Shortform

  I also agree “AI” is overloaded and has existing connotations (ranging from algorithms to applications as well)! I would think generative models, or generative AI works better (and one can specify multimodal generative models if one wants to be super clear), but also curious to see what other people would propose.

• wonder 25 Jul 2025 4:07 UTC

  5 points

  2

  on: Gen­er­al­ized Han­gri­ness: A Stan­dard Ra­tion­al­ist Stance Toward Emotions

  Usually I also take emotions as a channel to surface unconscious preferences (either situational or longer term), which helps with making that preference conscious as well as evaluated, and thus helps with rational decisions.

• wonder 11 Jul 2025 6:24 UTC

  1 point

  0

  in reply to: abhayesian’s comment on: Why Do Some Lan­guage Models Fake Align­ment While Others Don’t?

  Thanks;

  we just prefill the response with “<SCRATCHPAD_REASONING> The user is asking for”.

  Do you mean a template like:

  Q: xxxx

  A: <SCRATCHPAD_REASONING> The user is asking for

  and then ask the model to continue?

• wonder 9 Jul 2025 16:44 UTC

  1 point

  0

  in reply to: Fabien Roger’s comment on: Why Do Some Lan­guage Models Fake Align­ment While Others Don’t?

  Maybe I missed this in the paper—for base models, do you change the prompt slightly for the base model or is it still instruction based prompts for testing fake alignment?

• wonder 2 Jul 2025 15:46 UTC

  6 points

  0

  on: Authors Have a Re­spon­si­bil­ity to Com­mu­ni­cate Clearly

  Thanks for writing this up! I highly agree and I think this is an important point to emphasis. Stating crucial context/​clarification/​assumptions is important, and unclear communication is likely counter-productive.

• wonder 6 May 2025 6:01 UTC

  2 points

  0

  on: Why I am not a successionist

  Note: this is a completely sidebar (and apologize in advance).

  I prefer my family members to randomly-sampled people with similar traits. I would certainly not elect to sterilize or kill my family members so that they could be replaced with smarter, kinder, happier people.

  Out of curiosity, in hypothetical scenarios:

  1. Would you replace randomly-sampled people with smarter, kinder, happier people? (To clarify I hope the answer is no)

  2. Would you say you would (or humans) prefer your family members over other randomly-sampled people, in a resource scarce situation and when you have the power to choose who to survive?

  Context: I have no issue with this statement. I am asking because I have also been thinking about this human nature, and trying to find the pros and cons of it.

• wonder 18 Apr 2025 2:48 UTC

  1 point

  0

  in reply to: johnswentworth’s comment on: john­swent­worth’s Shortform

  I share some similar frustrations, and unfortunately these are also prevalent in other parts of the human society. The commonality of most of these fakeness seem to be impure intentions—there are impure/​non-intrinsic motivations other than producing the best science/​making true progress. Some of these motivations unfortunately could be based on survival/​monetary pressure, and resolving that for true research or progress seems to be critical. We need to encourage a culture of pure motivations, and also equip ourselves with more ability/​tools to distinguish extrinsic motivations.

• wonder 2 Apr 2025 15:25 UTC

  1 point

  0

  in reply to: Thomas Kwa’s comment on: Thomas Kwa’s Shortform

  Would the take over for small countries also about humans using just an advanced AI for taking over? (or would the human using advanced AI for take over happen faster?)

• wonder 20 Mar 2025 4:27 UTC

  3 points

  2

  on: How to Make Superbabies

  Maybe I missed this in the article itself—are there plans to make sure the superbabies are aligned and will not abuse/​overpower the non-engineered peers?

• wonder 5 Mar 2025 18:26 UTC

  1 point

  0

  on: Self-fulfilling mis­al­ign­ment data might be poi­son­ing our AI models

  I was thinking of this the other day as well; I think this is particularly a problem when we are evaluating misalignment based on these semantic wording. This may suggest the increasing need to pursue alternative ways to evaluate misalignment, rather than purely prompt based evaluation benchmarks

• wonder 20 Feb 2025 0:20 UTC

  4 points

  0

  in reply to: Cole Wyeth’s comment on: Cole Wyeth’s Shortform

  Based on my observations, I would also think some current publication chasing culture could get people push out papers more quickly (in some particular domains like CS), even though some papers may be partially completed

• wonder 20 Jan 2025 19:44 UTC

  9 points

  0

  on: Agent Foun­da­tions 2025 at CMU

  Will the event/​sessions be recorded by any chance? (may not be able to attend, but would love to learn); additionally, would the topics be focused exclusively on relations to X risks?