Usually I also take emotions as a channel to surface unconscious preferences (either situational or longer term), which helps with making that preference conscious as well as evaluated, and thus helps with rational decisions.
wonder
Karma: 39
Thanks;
we just prefill the response with “<SCRATCHPAD_REASONING> The user is asking for”.
Do you mean a template like:
Q: xxxx
A: <SCRATCHPAD_REASONING> The user is asking for
and then ask the model to continue?
Maybe I missed this in the paper—for base models, do you change the prompt slightly for the base model or is it still instruction based prompts for testing fake alignment?
Thanks for writing this up! I highly agree and I think this is an important point to emphasis. Stating crucial context/clarification/assumptions is important, and unclear communication is likely counter-productive.
Note: this is a completely sidebar (and apologize in advance).
I prefer my family members to randomly-sampled people with similar traits. I would certainly not elect to sterilize or kill my family members so that they could be replaced with smarter, kinder, happier people.
Out of curiosity, in hypothetical scenarios:
Would replace randomly-sampled people with smarter, kinder, happier people? (To clarify I hope the answer is no)
Would you say you would (or humans) prefer your family members over other randomly-sampled people, in a resource scarce situation and when you have the power to choose who to survive?
Context: I have no issue with this statement. I am asking because I have also been thinking about this human nature, and trying to find the pros and cons of it.
I share some similar frustrations, and unfortunately these are also prevalent in other parts of the human society. The commonality of most of these fakeness seem to be impure intentions—there are impure/non-intrinsic motivations other than producing the best science/making true progress. Some of these motivations unfortunately could be based on survival/monetary pressure, and resolving that for true research or progress seems to be critical. We need to encourage a culture of pure motivations, and also equip ourselves with more ability/tools to distinguish extrinsic motivations.
Would the take over for small countries also about humans using just an advanced AI for taking over? (or would the human using advanced AI for take over happen faster?)
Maybe I missed this in the article itself—are there plans to make sure the superbabies are aligned and will not abuse/overpower the non-engineered peers?
I was thinking of this the other day as well; I think this is particularly a problem when we are evaluating misalignment based on these semantic wording. This may suggest the increasing need to pursue alternative ways to evaluate misalignment, rather than purely prompt based evaluation benchmarks
Based on my observations, I would also think some current publication chasing culture could get people push out papers more quickly (in some particular domains like CS), even though some papers may be partially completed
Will the event/sessions be recorded by any chance? (may not be able to attend, but would love to learn); additionally, would the topics be focused exclusively on relations to X risks?
I also agree “AI” is overloaded and has existing connotations (ranging from algorithms to applications as well)! I would think generative models, or generative AI works better (and one can specify multimodal generative models if one wants to be super clear), but also curious to see what other people would propose.