I write software for a living and sometimes write on substack: https://taylorgordonlunt.substack.com/
Taylor G. Lunt
Karma: 818
If I was the CEO of a different AI company, and this happened on Monday, then on Tuesday I would say, “AI has gotten so incredibly powerful that our competitor just shut themselves down! But don’t worry, we have a new safety technique at EvilAI that will allow us to do this more safely than that company could. Invest now in our incredibly powerful, safe AI!”
And then the AI race would be in the hands of me, an unscrupulous liar, rather than the CEO who was ethical enough to try your suggested maneuver.
Including the names is fine, but including any details about who those people are would be going beyond common knowledge, I think, even though common does not imply universal.
I’m also not sure it’s as much of a technicality as you think. Your claim isn’t “this is true about economics”, but “this is so obvious even a normal person could figure it out without any special knowledge”, and I think my argument shows that it’s not as obvious as you think.
It reminds me a bit of this XKCD
You are not addressing my primary argument.
There are reasons people’s empathy fails to trigger besides sociopathy. Maybe they just don’t think about it, and their lives are working great so far, so there’s little reason for them to analzye further.
I think the way the question was posed is part of why it has a bad reception. The theorem presented was not an economics theorem at all, but a theorem about the internal mental state of four people. Presumably this was intentional, and it basically precludes a discussion of economics, as the economics are not remotely the strongest reason the theorem is false.
The other reasons for the poor reception:
I find it unlikely this person will ever pay $10,000 as a result of this post
The resolution criteria suck. Is whoever gets the most votes in this post going to get the money? What if it’s Carl Feynman, who makes a general meta-argument that the theorem is absurd, without attacking the theorem directly? What if it’s someone whose argument is considered invalid by Bruce Middleton, perhaps because they don’t address the economic question? Or will there be a vote at a later date? “after either party affirms that in their view real proof has been presented and further debate would be pointless” So what if the best answer doesn’t have the person say “this is a real proof and further debate is pointless”? Is that answer no longer eligible for the $10,000? This is really poorly framed and makes me doubt this will ever resolve.
The general vibe of “I know more about wealth than the ninth richest man on Earth” without any real justification for that position, and the crackpot vibes that come with “so and so famous person signed off on my ideas”, again without justification in this post.
I would have probably just preferred a nice explainer post of Bruce’s position. I lack the economics knowledge to follow his mortgage rates post.
I will grant you that Pythagoras’s theorem and the theory of evolution are inferable from common knowledge. You don’t need “nominal interest” sitting in common knowledge as a finished concept. That could be inferred from scratch as part of a chain of inference.
However, no inference from common knowledge will tell me who Jamie Dimon is. Your theorem is not an economics theorem. It’s about the internal understanding of four men, and that cannot be inferred from common knowledge. First of all, I don’t think the existence of someone like Jamie Dimon can reasonably be called common knowledge. I just asked someone next to me, and they only knew two out of the four people (exactly the two I predicted they’d know: Mark Carney and Warren Buffet).
Even if the existence of these people was common knowledge, I think inferring their internal mental state/understanding from their behaviour would be very hard, but if you don’t know who these people are, then you definitely can’t do it.
The theorem is straightforwardly false as stated because it says the fact is inferable from common knowledge, but it’s not even understandable from common knowledge:
The average person would not be able to tell you who all four of those people even are. Maybe a few, but not all four.
Nominal interest, and the unreal part thereof, are not terms/concepts present in common knowledge.
If a claim cannot be understood by common knowledge, it certainly cannot be proven using only common knowledge.
QED.
Maybe this sounds cheap, but I don’t think it is. I think it’s a real and valid refutation of your theorem. If you had stated it otherwise, then maybe it wouldn’t be, but you didn’t.
I wonder if the correct approach is somewhere in the middle. I.e. try really hard to instill good baseline values, and have bounded corrigibility around those values, such that the AI is able to update to match a somewhat different definition of humanitarianism than it’s internalized, but won’t update to become a torturing genocidal hitlerbot.
I’d like to hear your arguments for why you think that’s catastrophically wrong.
The original Vidhaven post (https://www.lesswrong.com/posts/MSng3Y4rsNLykbEF7/the-vidhaven-challenge) didn’t get much attention, but it’s not to late to join. 30 days, 30 videos, any subject.
I absolutely hate the videos I’ve made so far, so that’s a sign I’m moving in the right direction I think.
The Vidhaven Challenge
The story seems to have prizes for different regions, and this is the winning entry for the Caribbean region.
I just read a few paragraphs, and this story’s prose is very AI-coded.
“The day the grove began to remember”
I’ve heard AI talk about a forest “remembering” at least once before. I think in the flavor text of an AI-generated trading card.
“No fan, no bulb, no hum – only the thin light slipping between warped boards...”
″...breath of hills holding their heat like a secret.”
I have also heard “like a secret” in AI prose before.
The story is very boring and pretentious. I gave up reading it. I wonder how many entries will be written by AI next year.
This possibility is explicitly mentioned in the article and does not constitute a problem for my framework.
In this case I’d say there are a combination of two factors, both outside the body: genetics, and lack of vitamin C in diet.
Stomach pain is gone because I treated the symptom with slippery elm powder. I definitely believe in treating symptoms also, just not treating symptoms only.
As for my overall health issues, I only got to the last part of the chain a few days ago, so I can’t report on how a change to my desk job will change my other health issues yet. Though I now know when my neck is strained after too many hours of sitting at my desk, or by doing neck exercises, my health issues all get worse for a few hours/days, so I’m at least not uncertain about the cause now.
I’ll write a follow-up months from now.
I can’t make a soberly argued for rationalist case for medical analysis because I’m not a doctor or even someone that interested in medicine. I just wanted to share my experience and a heuristic I find useful, and which seems to be missing from many people’s thinking about medicine.
I agree that sometimes finding the external root cause won’t be helpful, as in the case of genetic disorders (aside from stopping you from pursuing other avenues of treatment that won’t help you). But you won’t know whether finding an external root cause will help you until you do it, so in that sense it’s worth doing.
Doing a full root cause analysis may be costly, but simply not pretending we already found the root cause is not costly and misleads the patient into not finding the root cause on their own. The reason they don’t avoid such pretending is that they’re not pretending: they actually think GERD is the root cause of acid reflux, or depression is a lack of seratonin. This is a lack of clear thinking.
Obviously the human body has a bunch of self-repair and homeostasis mechanisms a car does not, but even given that, I suspect the root cause of your car issues is probably usually outside the car. Do you change your oil on time? How do you use the brakes? What climate are you driving in? How low do you let the fuel tank get before refueling? I would guess the majority of car issues come down to user behavior and environment.
You can never stop the inevitable decline from wear and tear over the years. Not so different from a human being. But during the expected useful life of the machine, I think the root causes will be mostly outside the car, and you’ll be making the exact same mistake as I pointed out in my post if you behave otherwise.
I’d like to hear some examples of root causes that you think are inside the body.
It’s a heuristic that works because it usually stops the mistake where you think you’ve identified the thing that’s wrong, but actually there’s another thing that you would like to know is wrong, but you don’t know it, in that if you found out about it, you’d be glad you found out about it.
The reason this works is that the human body is very complicated and unknown, and finding a cause outside the body means finding a cause that’s easier to reason about, like desk sitting or a toxic molecule that entered your body.
But it’s just a heuristic to help people avoid a common and dangerous mistake. Of course, the tree of causality can be traced back as far as you want, until you end up invoking the Big Bang.
I am interested in this topic, but wisdom is hard to communicate for the reasons you mentioned. If wisdom could be summed up, hearing a platitude like “a stitch in time saves nine” will not actually teach you the relevant wisdom, because the phrase does not contain the wisdom. The wisdom is knowing when that idea applies. This is why I learned nothing from your LLM-generated examples: wisdom is contextual, and unless I also learn a lot of information about the context, I haven’t learned anything. And doing that takes a lot of time. It is therefore best if the examples are entertaining/well-written, and they don’t particularly have to be true, as long as they are representative of real wisdom. And look at that, we’ve just reinvented literature.