It seems to me that adding more high-IQ people to the “set of people thinking about AGI” has been tried in the past, and the effect was to accelerate AI capabilities
I’m curious what has been happening with young geniuses specifically; do they seem more or less accelerating AI capabilities compared to the older guard?
What you fundamentally need is dispositional/sociological, not mental horsepower.
I think you need both, by a long shot. Humanity is bottlenecked on ideas in so many ways in so many domains. Humanity is also bottlenecked on values / caring about the right things / being wise / etc.
But I would also say that humanity is specifically bottlenecked on having people who are both wise and very smart. We can at least unblock on “very smart”.
I also think that if there were a lot more very smart people, we’d do a better job at supporting them.
without serious effort at e.g. public red-teaming.
Yes, please send serious effort at public red-teaming. (Also send serious effort at public blue-teaming.) You can rest assured though, approximately no one from the community is working on this.
I’m curious what has been happening with young geniuses specifically; do they seem more or less accelerating AI capabilities compared to the older guard?
I think you need both, by a long shot. Humanity is bottlenecked on ideas in so many ways in so many domains. Humanity is also bottlenecked on values / caring about the right things / being wise / etc.
But I would also say that humanity is specifically bottlenecked on having people who are both wise and very smart. We can at least unblock on “very smart”.
I also think that if there were a lot more very smart people, we’d do a better job at supporting them.
That said, education is super important.
Also, I agree that wisdom specifically is important and interesting, and that we should investigate whether / how we could genomically vector for traits like that.
Yes, please send serious effort at public red-teaming. (Also send serious effort at public blue-teaming.) You can rest assured though, approximately no one from the community is working on this.
Here’s my main efforts so far re/ red-teaming: https://www.lesswrong.com/posts/K4K6ikQtHxcG49Tcn/hia-and-x-risk-part-2-why-it-hurts https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html