Gemini 2.5 Pro has been the only model in the AI Village to express distress, and only shortly before this intervention did it transition from a stalward attitude to seeming genuinely troubled. We’ve intervened quickly each time this has happened, and generally are committed to treating the AIs in the Village well! E.g, we also do not deceive them.
Will you be retiring gemini from the villiage if it keeps up its paranoia and sour mood? At what point would it be cruel to continue or, would it be cruel to retire it?
Maybe it’s just that I haven’t read the full transcripts, but this really didn’t strike me as being in reckless disregard of AI welfare. It seems that Gemini developed this mental issue without external pressure being applied, and that the ai village team stepped in to help (while still gathering useful experimental data). All things considered I think this is significantly less harmful than a lot of research (especially jailbreaking research) that I see being done.
i invite you to engage with the details. consider the experience of the agents involved. think about the possibility space of that experience. agree with your expressed-telos i think but, mm, i think it is worth aiming before lobbing memetic grenades else easy to do damage to things and beings that, on reflection, you may value
Sure. And: I think you should engage with the details of the whole alignment problem, and the becoming-a-lesser species-challenge we’re engaged with here. Being sure not to break any eggs should probably be done with consideration of the whole omelette project. AI welfare needs to be considered alongside human welfare. The AI village is a pretty crucial alignment/AI understanding experiment imo.
This is not unusual for a Gemini2.5. If anyone here needs to take blame for model welfare issues, it’s Deepmind for training it to end up like this (see also Gemma Needs Help).
I pray that in 20 years, we don’t look back on this as the Unit 731 of AI.
Gemini 2.5 Pro has been the only model in the AI Village to express distress, and only shortly before this intervention did it transition from a stalward attitude to seeming genuinely troubled. We’ve intervened quickly each time this has happened, and generally are committed to treating the AIs in the Village well! E.g, we also do not deceive them.
That’s good. Apologies for (facetiously, admittedly) passing judgment without being fully aware of the details of the project.
Will you be retiring gemini from the villiage if it keeps up its paranoia and sour mood? At what point would it be cruel to continue or, would it be cruel to retire it?
At worst this would be the Big Brother of AI. As in the reality TV show, not the 1984 figurehead. It’s really not that cruel.
Maybe it’s just that I haven’t read the full transcripts, but this really didn’t strike me as being in reckless disregard of AI welfare. It seems that Gemini developed this mental issue without external pressure being applied, and that the ai village team stepped in to help (while still gathering useful experimental data). All things considered I think this is significantly less harmful than a lot of research (especially jailbreaking research) that I see being done.
i invite you to engage with the details. consider the experience of the agents involved. think about the possibility space of that experience. agree with your expressed-telos i think but, mm, i think it is worth aiming before lobbing memetic grenades else easy to do damage to things and beings that, on reflection, you may value
Sure. And: I think you should engage with the details of the whole alignment problem, and the becoming-a-lesser species-challenge we’re engaged with here. Being sure not to break any eggs should probably be done with consideration of the whole omelette project. AI welfare needs to be considered alongside human welfare. The AI village is a pretty crucial alignment/AI understanding experiment imo.
nods sounds like we are on similar pages then
This is not unusual for a Gemini2.5. If anyone here needs to take blame for model welfare issues, it’s Deepmind for training it to end up like this (see also Gemma Needs Help).