Quick reactions: Re: 1: I hope you are right. I think that the power of “but we need to win the race” will overcome the downsides you describe, in the minds of the CEOs. They’ll of course also have copies that don’t have memories, etc. but there will be at least 1 gigantic corporation-within-a-corporation that collectively functions as a continually online-learning agent, and said agent will be entrusted with some serious responsibilities most notably doing the core AI R&D.
Re: 2: I think the idea would be to ‘light-touch’ nationalize, so as to avoid the problems you mention. Main thing is to let the various companies benefit from each other’s research, e.g. use models they trained, use algorithmic secrets, etc. As for open-sourcing: Yeah good points I could totally see them continuing to open-source stuff forever, at least while they remain behind the frontier. (I think that their incentives would point in a different direction if they actually thought they were winning the AI race)
Quick reactions:
Re: 1: I hope you are right. I think that the power of “but we need to win the race” will overcome the downsides you describe, in the minds of the CEOs. They’ll of course also have copies that don’t have memories, etc. but there will be at least 1 gigantic corporation-within-a-corporation that collectively functions as a continually online-learning agent, and said agent will be entrusted with some serious responsibilities most notably doing the core AI R&D.
Re: 2: I think the idea would be to ‘light-touch’ nationalize, so as to avoid the problems you mention. Main thing is to let the various companies benefit from each other’s research, e.g. use models they trained, use algorithmic secrets, etc. As for open-sourcing: Yeah good points I could totally see them continuing to open-source stuff forever, at least while they remain behind the frontier. (I think that their incentives would point in a different direction if they actually thought they were winning the AI race)