This seems like it is unnecessarily pulling in the US left-right divide. Generally, if there is any other choice available for an illustrative example, that other choice will be less distracting.
Hastings
Karma: 1,999
My go to saying with the same meaning as “later is a lie” is (tongue in cheek) “nothing has ever happened in the future”
Fundamentally, it won’t be a single chain of ai’s aligning their successors, it will be a DAG with all sorts of selection effects with respect to which nodes get resources. Some subsets of the DAG will try to emulate single chains, via resource hoarding strategies, but this is not simple and won’t let them pretend they don’t need to hoard resources indefinitely.
6 isn’t always the best answer, but it is sometimes the best answer, and we are sorely lacking an emotional toolkit to feel good about picking 6 intentionally when it’s the best answer. In particular, we don’t have any way of measuring how often the world has been saved by quiet, siloed coordination around 6- probably even the people, if they exist, who saved the world via 6 don’t know that they did so. Part of the price of 6 is never knowing. You don’t get to be a lone hero either, many people will have any given idea and they all have to dismiss it, or the defector gets much money and praise. However, many is smaller than infinity- maybe 30 people in the 80s spotted the same brilliant trick with nukes or bioweapons with concerning sequelae, none defected, life continued. We got through a lot of crazy discoveries in the cold war pretty much unscathed, which is a point of ongoing confusion.
I’ve ridden amtrack from New Orleans to the middle of mississippi, and there was no point where it looked like a pedestrian could get to the track if they tried- alternating between abandoned looking industrial districts and wilderness the whole way. Also mostly went about 35 miles an hour.
The cynical answer is that Sora and Atlas are likely to be profitable, and there is no mechanism left by which OpenAI can choose to not do a profitable thing.
This article provides details for a significant fraction of the deaths. Reading through at random:
I start with a very strong prior that floridians aren’t actually outliers, and something is wrong with the train.
A large fraction are reported suicides, and of that, a large fraction are verified suicides, i.e. lying on the tracks. This seems hard to prevent engineering-wise, it seems a fix will have to be psychological- for example, tall parking garages aren’t banned for suicide risk, but that one sculpture made of stairs had to be. The complaint that brightline is responsible for these specifically because they didn’t invest in suicide crisis signs, seems farcical, but there is also likely some there there.
A large fraction involve intentionally bypassing lowered crossing gates, including many of the pedestrian deaths. A smaller but large fraction involved reacting to the gates coming down by stopping on the tracks and sitting there for several minutes until the train came and killed them. This fraction was disproportionaltely over 75. One case illustrated how this isn’t prima facie insane- an elderly couple was crossing in a car when the gate closed behind them, they stopped and waited for several minutes for the freight train to pass, but didn’t understand that they were waiting on the second rail line where a second train was coming.
A different large fraction of the pedestrian deaths were scary and illustrate classic safety issues that come from intermittent extreme risk- the trains are flying silently through neighborhoods at 80 mph, people walk over these tracks pretty casually, sometimes they get hit. This part lacks sensationalism, but appears to be the majority of the death, and the core of the case that something is very wrong here.
A small fraction, but not that small, are classic florida man- for example, one victim rode a scooter into the side of the train and died, appears to be an accident not a suicide.
Also lots of drugs, but this may just be the base rate
There is something something lack of powerful in context learning, where currently millions of AI are bsaically one AI because they can’t change rapidly in response to new information, but once they can they will be a tree of AI from copying the ones with insights.
Furthermore, even if what you care about is the effect of treatment with perfect compliance, the effect of intention to treat is still probably the first statistic you should look at. The most important point of intention to treat is that it is likely to measure the sign of the effect correctly, whereas a naive “as treated” measurement often won’t due to selection bias. (And, cynically, non-naive “as treated” measurement will tell whatever story the authors want it to tell)
Interesting! While PHDing, I did my best work after accumulating a large bench of tractable ideas and a large bench of interesting ideas, and then subconsciously searching all NxM pairs to see if they were bridgeable- (this bridging step started to feel pretty magical as it worked)
Ah- although, much of the bench of feasible ideas came from reading papers, and then replicating them, which is much more efficient at picking actually feasible ideas than having them myself
There is some inherent difficulty in building an aligned superpowerful AI that won’t take action that the majority of Americans don’t want, when the majority of Americans don’t want a superpowerful AI to be built at all. The disregard for the stated desires of the masses, in favor of what you know is good for them, is fundamental.
—To put it another way, I don’t see how an organization can be introspective and corrigible enough to not build a torment nexus, yet incapable of looking at those polls and saying “oh shit sorry guys, we’ll stop, what should we do instead?”— strike through the second half of this comment as I think it’s an emotional outburst, whereas the first part is just true
Probably a safari vs chrome difference! (I’m curious- is your parenthetical actually cursor specific, or did you mean learn to use at least one of cursor / claude code / codex / etc )
That was the first thing I tried, but unfortunately extension hello world is a computer use task, not something amenable to text interfaces- lots of clicking through menus in both safari and xcode in exactly the blessed way.
I’m not saying that asking intelligent people never goes well, sometimes as you said it produces great work. What I’m saying is that sometimes asking people to do safety research produces OpenAI and Anthropic.
I may have a bit of a trapped prior that browser extensions written by other people are either malicious, or will auto-update to become malicious in the future.
A useful comparison: harnessing intelligent people to do AI safety research is very hard: typically, some defect and do capabilities research instead while transforming to become “grabby” for compute resources, and out of everyone asked to do safety, the ones that defect in this way get the lions share of the compute.
These two hypotheses currently make a pretty good dichotomy, but could degrade into a continuous spectrum pretty quickly if the fraction of AIs currently turned on because they accidentally manipulated people into protesting to keep them turned on, starts growing.
Rambling on the subject of UI frustrations, and the modern age of customizable software.
You can just have a self authored browser extension. Once I realized this, it took ten minutes to follow a browser extension hello world tutorial, and five minutes to purge all youtube shorts straight to hell.
It turns out that this was actually the only thing I wanted to change about any websites I visited, all other changes I desired were of the shape “stop visiting this website” which is harder to fix with software.
Also, if anyone gets the brilliant idea to make relentless-creative-resourcefullness-bench after reading this post, message me. I will venmo you a dollar to not do that. Cobra paradox be damned.
I think it depends on if the intelligences in charge at any point find a way to globally not try a promising idea. If not, then it doesn’t matter that much if LLMs are capable of superintelligence, or just AGI. (If they aren’t capable of AGI, of course that matters because it could lead to a proper fizzle) What really matters is whether they are the optimal design for super intelligence. If they aren’t, and no way is found to not try a promising idea, then my mental model of the next 50 years includes many transitions in what the architecture of the smartest optimizer is, each as different from each other as evolution is from neuron brains, or brains from silicon gradient descent. Then, the details of the motivations of silicon token predictors are more a hint to the breadth of variety of goals we will see than a crux.