Apologies for the delay: this is a long and thoughtful response I sadly don’t have the capacity to do justice to right now. Your iff seems to be central to your argument, but I am not convinced by either component. Firstly, for human + AI < AI to hold, we need AI to be less competent than the most competent humans, in any respect. It’s not obvious that AI alignment is at the right level of difficulty! Secondly, for human + AI + mindreading > AI to hold, it seems that AI must add a great deal of productivity. Why should I expect my subconscious thoughts and reflexes to be more like a 300% boost (for the typical employable researcher) than a 30% boost?
KellerScholl
Karma: 661
While I see this as a meaningful threat, I think that it’s something that pretty smoothly follows from the development of mind-reading at any point in time, and so a five or twenty year delay is not particularly relevant to it.
I’m curious what sort of work you expect to be useful here. “Overthrow non-democratic regimes in advance” has other drawbacks, and doesn’t seem to have the characteristics you’re talking about.
“I make myself coffee. My thoughts are mostly empty, but I briefly recall what I want to say to the candidate I’m getting lunch with tomorrow. In the background, Codex is prompted to think about info I might want for the lunch chat, and decides that updating the synthetic data scaling plot before showing it off would be prudent; Codex spins off a subagent.
I’m not saying words really loudly in my head while getting coffee. I just read the plots as I normally do, and make coffee as I normally do. It feels like magic.”
I’m quoting from the post I reference at the top. A deliberate writing decision I made was not to question the feasibility of the technology: if it’s not feasible then it’s not worth putting effort into it, but this is not my area of expertise and “this novel technology might not work” is not particularly interesting.
“How much alcohol has this person consumed recently” and “are you loyal to the Glorious Leader” are different questions, and truth machines for them will have different effects.
Don’t Build Mindreading
I genuinely believe that it’s a good thing for the majority religion of the country to have at least one representative on the Supreme Court who understands it in the way that it can be difficult for a non-native. I think it’s particularly bad for legitimacy to have an entirely Catholic and Jewish court. I agree the statement is structured as a joke, but I believe there should ideally be at least one man, at least one woman, at least one Protestant, and I know that this one is more of a stretch for lifecycle and class reasons but I’d like to have at least one renter (one third of American households), at least not a geographic mono-culture, etc.
On Having Good Hot Takes
Monthly Shorts 1/22
Four Scenarios of Job-Reducing AI
Declining Marginal Costs of Alienation
There’s Room in the Manger
Water Above the Ocean
https://x.com/JacobHHilton/status/1794090561294467074
He has also explicitly told people not to expect candor from him on this issue until the situation changes. That the binding is no longer part of a contract, as opposed to implicit threat, seems of little relevance.
I am very curious how you think about this post in retrospect: parts of it seem clearly falsified. I completely understand if you currently feel bound by a non-disparagement clause and expect it to be a few weeks before that can be confirmed to no longer apply.
Three years later, I mostly stand by this: war with China has not happened, but estimates of likelihoods have risen, and it not driven by food concerns in the slightest.
I’ve been a follower since fairly early on: disagreed with you both then and now on a range of topics. I’ve been unimpressed with the precision of your language at times: I was being unfair then, I suspect at least partially because of our differences. This piece seems fair and accurate to me.