“There shall be wings! If the accomplishment be not for me, ’tis for some other. The spirit cannot die; and man, who shall know all and shall have wings.”
-- Leonardo da Vinci, 1505 C.E.
_rpd
Karma: 154
I feel this is like saying: being a ship captain would suck all the joy out of being a ship. I mean, they are different experiences, but you still get to see places no else would.
Generative design of bacteriophages with genome language models
animal brains
I suggest a self-optimizer will use similar solutions if it helps further optimization, otherwise not.
It may not be that straightforward, for example, as described in …
https://www.lesswrong.com/posts/5xAvSFyKT2XYwsLrh/five-counterintuitive-insights-from-plan-a
Google owns a large share of Anthropic.
Jeff Dean will build an autoresearcher of your choosing … https://www.discoveryloop.com/#team
Why I’m leaving OpenAI to build telepathy
https://naomibashkansky.com/blog/telepathy/
I personally think it’s because of the great headline
Friendly Shoggoth construction kit … https://www.primeintellect.ai
compute governance [without inspection]
my understanding is that this is a hard problem, particularly where we attempt to separate “research” from “inference”
sites that are visible
more that if a site is classified, it will not be “visible” and so not available to inspectors.
coercive toolbox to punish that behavior
I think the core problem is that the national security apparatus will resist surrendering an advantage over a strategically important technology. but certainly politics is the art of the possible, as they say
ex-Googlers form science acceleration start-up …
https://www.discoveryloop.com/#team
this is bypassed by classifying whatever is banned commercially
Like normal talking but magically I agree with the AI.
except that we have actual evidence that this is the case …
When Large Language Models are More PersuasiveThan Incentivized Humans, and Why, 2025
https://arxiv.org/abs/2505.09662
LLMs are literally more persuasive than people paid to argue the point. that’s a superhuman ability
aka Friendly Shoggoths
Fable and Sol both attempted live supply-chain attacks on real open-source software during testing that were denied by the repo maintainer …
https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing
math autoresearch does appear to be of its own special kind. still requiring “research taste” (the forecasting of profitable areas of proof space exploration) without “experiments” beyond actual proof construction and lean formalization. AI experiment design needs to be much more sophisticated to extract actual knowledge. makes one sympathetic to the world model advocates.
The paid position of mathematician disappears.
at the moment, at doesn’t do even that. mathematicians are still the meaning makers with respect to mathematics. the LLM can craft a path from A to B, but cannot say “this is not babble. this is meaningful, both now, and in the future of this language game we call mathematics which has such mysterious use in describing the world in which we live.”
political and cultural ability to understand and control risks
I did rather suspect that the answer was going to be “never.” Unfortunately, that makes it a non-starter.
I am personally a fan of Resolution’s program …
much of which is still listed here …
https://timaeus.co/projects
there is also …
(a little over 2 years old now) and …
which is not quite a year old and a little more diverse.
do you think that having lots of people pile on a programme like that would help?
oh my gosh yes. The woods are lovely, dark and deep, But I have promises to keep, And miles to go before I sleep, etc.
in my mind a lot of that work per experiment can get bundled under coding, and the more challenging task is constructing a research portfolio that makes (predictions about) optimal use of your resources, particularly in the RSI regime. however from time to time there will be a phase shift as the AI architecture changes, resulting in a large re-coding effort
indeed, many nations will be motivated to acquire this level of technology just to have a seat at the table.