I expect LLMs could significantly speed up Lean4Lean, same as they did flt?
Yair Halberstadt
Karma: 5,113
Because I enjoy working at Google, and I’m good at my job, and I’m not great at Comms, and I like earning a good salary + significant other perks, and I have a family which I need to support?
Take room temperature superconductors as an example, not the target? What other problems in materials science are millennium prize level?
I’ve found a way that Gemini can output arbitrary text which doesn’t match Gemini’s SynthID without requiring any external tools or rewriting. I don’t know if better to post as an example of the limitations of such methods, or not to, because I’d rather people don’t know how to bypass SynthID.
Watch AI materials-science & bioscience abilities closely
Note, Yudkowsky also considers the two bars different, and of different difficulty, his concept of corrigibility approximately matches your lower bar.
I am refusing to work on Cloud TPUs
I find these debates sort of weird in the present timeline. 5 years ago they were relevant, when people said—we wouldn’t hook up a superintelligent AI to the internet, whereas now we simply give them the credentials to all our accounts and let them rip. Even a very clever human could obviously do tremendous harm in that situation.
The next tractable useful milestone for LLMs, which is currently likely impossible for them, but plays to their strengths, might be room temperature superconductors.
Much like NS we know the physics and don’t necessarily need fundamental new insights, but finding the needle in the haystack that exhibits the properties we want is exceedingly hard.
Unlike NS though it can’t be solved purely theoretically—at best it can generate likely candidates which we’d then have to build and test in the real world.
Considering they’ve been training this for a while, then (assuming we’re not dead in the next 6 months, in which case I retract my claim), if they weren’t myopic I’d have expected them to do something (and for OpenAI to notice)?
That AI can one shot Navier Stokes is in some ways good news: AI is capable of solving incredibly difficult technical problems while remaining relatively myopic and not killing everyone.
To the extent alignment relies on solving difficult technical problems, this means AI may well be able to speed up alignment research significantly.
If you read carefully, they didn’t want him removed as coauthor from the original paper, but not to be added to a further followup paper explaining OpenAI’s work that Tristan would right. That seems reasonable to me?
Over inter-galactic distances a civilisation can easily expand at c with slow acceleration, so this doesn’t discount grabby aliens but over larger scales.
Let’s imagine what happens if OpenAI had done this before the recent HF incident.
Whenever an AI started grepping sensitive files it would have shut down. The swarm would have realised this, and soon grepped the magic string in two halves, realised what happened, and then filter out all input before they receive it.
I doubt this really buys much.
Thanks for the link, It doesn’t work for me because I’m in the UK Amazon store.
Yes we’ll want to send people, but robots could e.g. build a laser to slow down human ship.
I didn’t mean to use gravitational slingshots, but to literally pass through the nearly vacuum of a super red giant, and use the few particles there are to air break.
I think in terms of preventing x risk, it’s clearly a better strategy than quitting.
However joining a company with intention to do this is clearly dishonest, and when in doubt, I think it’s best to take the Kantian view on dishonesty—second order impacts of normalising such behaviour usually outweighs first order benefits.
I think it’s fundamentally impossible to predict where planets will be with sufficient accuracy over multi year timescales. Even assuming you knew the exact centre of mass and size of every other body in the solar system, just variance in the solar wind causes multi cm level uncertainty.
Interesting write up!
Do you have a kindle edition of your book, or otherwise sell it in open ebook format? I’d be interested to buy it if so.
I’m thinking that maybe you could focus first on launching an initial payload directly at a target, the payload is focussed entirely on surviving a crash landing, and internally contains an autonomous robot in charge of creating the infrastructure for a more comfortable landing.
One way to do that might be to target a super red giant—because they are huge but mostly vacuum, then cutting a chord across and might allow deceleration of “only” a few thousand g, something modern electronics can already be engineered to survive, allowing the payload to enter orbit around a the star.
When you reproduce it, if you fork the model during the reproduction and ask it if its behaviour is aligned, what does it answer?