Hence, GitHub is effectively using the work others made—for personal or non-commercial use, without having GitHub in mind, and without any way to say ‘no’ - to sell a product back to them, for their own profit.
How different is this from a human being learning how to code from reading public code on the internet, and then selling their own software? I suspect Copilot directly copies code less often than humans do! GitHub claims “about 1% of the time, a suggestion may contain some code snippets longer than ~150 characters that matches the training set”, and that their research shows this is mostly for very common patterns.
I’ve used Copilot a fair bit myself, and the code I actually use from it is typically very tailored to my specific use case, as only part of some bigger function I’m writing. (often unit tests)
The same principle applies to an AI like DALL-E—it’s not copying artwork, it’s simply learned how to make art from publicly available art.
That all said, I’m not actually opposed to an opt-out, nor do I disagree that there’s something unjust in the idea of replacing human work with AIs that learned from them for free. But I think the moral intuition has more to do with the sheer scalability and potential impact of these AIs than the act of learning from public code/art itself?
Humans can currently improve uninterpretable AI, and at a reasonably fast rate. I don’t see why an AI can’t do the same. (i.e. improve itself, or a copy of itself, or design a new AI that is improved)
Why not breed for compatibility with human values as well? We could then study the differences between various degrees of “aligned” cephalopods and wild cephalopods.
It might be easier than selecting for intelligence too; humanity has successfully modified dogs, cats, and livestock to provide more positive utility to humans than their wild counterparts, but hasn’t drastically increased the intelligence of any wild animal to my knowledge, despite there plausibly being benefits for doing so with ex. horses or dogs.
Breeding for human values also limits some of the downsides of this plan, like the chance of ending up conquered by unaligned octopuses.
Re: level of effort, some brief googling tells me that there has been some interest in breeding octopuses for food, but it’s been quite difficult, particularly handling the larvae. BBC claims the current state-of-the-art is a promise that farmed octopus will be on the market in 2023.
That’s easier to understand for me than Aaronson’s, thanks. Interestingly, the author of that blog post (Jake R. Hanson) seems to have just published a version of it as a proper scientific paper in Neuroscience of Consciousness this past August… a journal whose editor-in-chief is Anil Seth, the author of the book reviewed above! Not sure if it comes up in the book or not, considering the book was published just this September—it’s probably too recent unfortunately.