Just to call it out, this post is taking the Great Man Theory of historical progress as a given, whereas my understanding of the theory is that it’s highly disputed/controversial in academic discourse.
Charlie Sanders
Karma: 42
Nyarlathotep Stirs: A Meta-Narrative ChatGPT Story
It would by definition not be bad thing. “Bad thing” is a low-effort heuristic that is inappropriate here, since I interpret “bad” to mean that which is not good and good includes aggregate human desires which in this scenario has been defined to include a desire to be turned into paperclips.
The ideal scenario would be for humans and AIs to form a mutually beneficial relationship where the furtherance of human goals also furthers the goals of AIs. One potential way to accomplish would be to create a Neuralink-esque integrations of AI into human biology in such a way that human biology becomes an intrinsic requirement for future AI proliferation. If AGIs require living, healthy, happy humans in order to succeed, then they will ensure that humans are living, happy, and healthy.
See, this is the perfect encapsulation of what I’m saying—it could design a virus, sure. But when it didn’t understand parts of the economy, that’s all it would be—a design. Taking something from the design stage to the “physical, working product with validated processes that operate with sufficient consistency to achieve the desired outcome” is a vast, vast undertaking, one that requires intimate involvement with the physical world. Until that point is reached, it’s not a “kill all humans but fail to paperclip everyone” virus, it’s just a design concept. Nothing more. More and more I see those difficulties being elided over by hypothetical scenarios that skip straight from the design stage and presuppose that the implementation difficulties aren’t worth consideration, or that if they are they won’t serve as a valid impediment.
There exists a diminishing returns to thinking about moves versus performing the moves and seeing the results that the physics of the universe imposes on the moves as a consequence.
Think of it like AlphaGo—if it only ever could train itself by playing Go against actual humans, it would never have become superintelligent at Go. Manufacturing is like that—you have to play with the actual world to understand bottlenecks and challenges, not a hypothetical artificially created simulation of the world. That imposes rate-of-scaling limits that are currently being discounted.
I’m a firm believer in avoiding the popular narrative, and so here’s my advice—you are becoming a conspiracy theorist. You just linked to a literal conspiracy theory with regards to face masks, one that has been torn apart as misleading and riddled with factual errors. As just one example, Cochrane’s review specifically did not evaluate “facemasks”, it evaluated “policies related to the request to wear face masks”. Compliance to the stated rule was not evaluated, and it is therefore a conspiracy theory to go from an information source that says “this policy doesn’t work” and end up with the takeaway “masks don’t work”. As other commenters have pointed out, it is physically implausible for facemasks to not work if they are used correctly.
The definitionally correct term to use for you is “conspiracy theorist” so long as this is a thing that you, after conducting your own research, have come to believe. Take your belief in the facemask thing as concrete evidence that your friends and family are correct and that you are indeed straying down the path of believing more and more improbable and conspiratorial things.
If all of EY’s scenarios require deception, then detection of deception from rogue AI systems seems like a great place to focus on. Is there anyone working on that problem?
Listening to Eliezer walk through a hypothetical fast takeoff scenario left me with the following question: Why is it assumed that humans will almost surely fail the first time at their scheme for aligning a superintelligent AI, but the superintelligent AI will almost surely succeed the first time at its scheme for achieving its nonaligned outcome?
Speaking from experience, it’s hard to manufacture things in the real world. Doubly so for anything significantly advanced. What is the rationale for assuming that a nonaligned superintelligence won’t trip up at some stage in the hypothetical “manufacture nanobots” stage of its plan?
If I apply the same assumption of initial competence extended to humanity’s attempt to align an AGI to that AGI’s competence in successfully manufacturing some agentic-increasing tool, then the most likely scenario I get is that we’ll see the first unaligned AGI’s attempt at takeoff long before it actually succeeds in destroying humanity.
Agreed. Facilitation- focused jobs (like the ones derided in this post) might look like bullshit to an outsider, but in my experience they are absolutely critical to effectively achieving goals in a large organization.
For 99% of people, the only viable option to achieve this is refinancing your mortgage to take any equity out and resetting terms to a 30 year loan duration.
Do you happen to know the specifics of how these cadavers came to be available? There’s recently been some investigative reporting on this topic. The broad gist is that most people who are “donating their bodies to science” probably don’t get that companies will take those donated bodies and sell them for what are essentially tourist attractions like the one that you’re participating in.
I’m finding myself developing a shorthand heuristic to figure out how LDT would be applied in a given situation: assume time travel is a thing.
If time travel is a thing, then you’d obviously want to one-box Newcomb’s paradox because the predictor knows the future.
If time travel is a thing, then you’d obviously want to cooperate in a prisoner’s dilemma game given that your opponent knows the future.
If time travel is a thing, then any strategies that involve negotiating with a superintelligence that are not robust to a future version of the superintelligence having access to time travel will not work.
As someone that frequently has work reviewed by crossfunctional groups prior to implementation, I only object to change requests that I feel will make the product significantly worse. There’s simply too much value lost in debating nitpicks.
Robin’s sense of honor would probably prevent him from litigating this, but that absolutely would not hold up in court.
Have you verified with Robin that he is okay with this from a copyright standpoint?
Why did you exclude Solo?
It’s verbatim. I think it picked up on the concept of the unreliable narrator from the H.P. Lovecraft reference and incorporated it into the story where it could make it fit—but then, maybe I’m just reading into things. It’s only guessing the next word, after all!