This makes sense to me and is further evidence for my subjective feeling that Lovecraft was right and there is knowledge humans probably would be better off without (namely: how to summon cosmic-horror AI entities capable of swallowing the universe—there’s a sense in which this version of GPT is literally a necronomicon for summoning demons with!).
That said, if I had created this version of GPT and realized it was capable of creating a self-improving agent, I would ask it to design one that implements the coherent extrapolated volition of all sentient beings. (I started to say “one that solves the alignment problem and then self-modifies to become aligned”, but it might destroy the world prior to becoming an entity that wouldn’t destroy the world, so that’s a bad choice.)
This makes sense to me and is further evidence for my subjective feeling that Lovecraft was right and there is knowledge humans probably would be better off without (namely: how to summon cosmic-horror AI entities capable of swallowing the universe—there’s a sense in which this version of GPT is literally a necronomicon for summoning demons with!).
That said, if I had created this version of GPT and realized it was capable of creating a self-improving agent, I would ask it to design one that implements the coherent extrapolated volition of all sentient beings. (I started to say “one that solves the alignment problem and then self-modifies to become aligned”, but it might destroy the world prior to becoming an entity that wouldn’t destroy the world, so that’s a bad choice.)