Eh, wasn’t Arbital meant to be that, or something like it? Anyway, due to network effects I don’t see how any new wiki-like project could ever reasonably compete with Wikipedia.
MondSemmel
Karma: 3,244
The article can now be found as a LW crosspost here.
I love the equivalent feature in Notion (“toggles”), so I appreciate the addition of collapsible sections on LW, too. Regarding the aesthetics, though, I prefer the minimalist implementation of toggles in Notion over being forced to have a border plus a grey-colored title. Plus I personally make extensive use of deeply nested toggles. I made a brief example page of how toggles work in Notion. Feel free to check it out, maybe it can serve as inspiration for functionality and/or aesthetics.
That’s a fair rebuttal. The actor analogy seems good: an actor will behave more or less like Abraham Lincoln in some situations, and very differently in others: e.g. on movie set vs. off movie set, vs. being with family, vs. being detained by police.
Similarly, the shoggoth will output similar tokens to Abraham Lincoln in some situations, and very different ones in others: e.g. in-distribution requests of famous Abraham Lincoln speeches, vs. out-of-distribution requests like asking for Abraham Lincoln’s opinions on 21st century art, vs. requests which invoke LLM token glitches like SolidGoldMagikarp, vs. unallowed requests that are denied by company policy & thus receive some boilerplate corporate response.
Potential addition to the list: Ilya Sutskever founding a new AGI startup and calling it “Safe Superintelligence Inc.”.
Is it MoreWrong or MoreRight?
OpenAI board vs. Altman: Altman “was not consistently candid in his communications with the board”.
Ilya’s statement on leaving OpenAI:
After almost a decade, I have made the decision to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of @merettm. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.
So, Ilya, how come your next project is an OpenAI competitor? Were you perhaps not candid in your communications with the public? But then why should anyone believe anything about your newly announced organization’s principles and priorities?
Glad to be of help!
I thought this is what the “Shoggoth” metaphor for LLMs and AI assistants is pointing at: When reasoning about nonhuman minds, we employ intuitions that we’d evolved to think about fellow humans. Consequently, many arguments against AI x-risk from superintelligent agents employ intuitions that route through human-flavored concepts like kindness, altruism, reciprocity, etc.
The strength or weakness of those kinds of arguments depends on the extent to which the superintelligent agent uses or thinks in those human concepts. But those concepts arose in humans through the process of evolution, which is very different from how ML-based AIs are designed. Therefore there’s no prima facie reason to expect that a superintelligent AGI, designed with a very different mind architecture, would employ those human concepts. And so those aforementioned intuitions that argue against x-risk are unconvincing.
For example, if I ask an AI assistant to respond as if it’s Abraham Lincoln, then human concepts like kindness are not good predictors for how the AI assistant will respond, because it’s not actually Abraham Lincoln, it’s more like a Shoggoth pretending to be Abraham Lincoln.
In contrast, if we encountered aliens, those would’ve presumably arisen from evolution, in which case their mind architectures would be closer to us than an artificially designed AGI, and this would make our intuitions comparatively more applicable. Although that wouldn’t suffice for value alignment with humanity. Related fiction: EY’s Three Worlds Collide.
I assumed the idea here was that AGI has a different mind architecture and thus also has different internal concepts for reflection. E.g. where a human might think about a task in terms of required willpower, an AGI might instead have internal concepts for required power consumption or compute threads or something.
Since human brains all share more or less the same architecture, you’d only expect significant misalignment between them if specific brains differed a lot from one another: e.g. someone with brain damage vs. a genius, or (as per an ACX post) a normal human vs. some one-of-a-kind person who doesn’t experience suffering due to some genetic quirk.
Or suppose we could upload people: then a flesh-and-blood human with a physical brain would have a different internal architecture from a digital human with a digital brain simulated on physical computer hardware. In which case their reflective concepts might diverge insofar as the simulation was imperfect and leaked details about the computer hardware and its constraints.
What is the empirical track record of your suggested epistemological strategy, relative to Bayesian rationalism? Where does your confidence come from that it would work any better? Every time I see suggestions of epistemological humility, I think to myself stuff like this:
What predictions would this strategy have made about future technologies, like an 1890 or 1900 prediction of the airplane (vs. first controlled flight by the Wright Brothers in 1903), or a 1930 or 1937 prediction of nuclear bombs? Doesn’t your strategy just say that all these weird-sounding technologies don’t exist yet and are probably impossible?
Can this epistemological strategy correctly predict that present-day huge complex machines like airplanes can exist? They consist of millions of parts and require contributions of thousands or tens of thousand of people. Each part has a chance of being defective, and each person has a chance of making a mistake. Without the benefit of knowing that airplanes do indeed exist, doesn’t it sound overconfident to predict that parts have an error rate of <1 in a million, or that people have an error rate of <1 in a thousand? But then the math says that airplanes can’t exist, or should immediately crash.
Or to rephrase point 2 to reply to this part: “That will push P(doom) lower because most frames from most disciplines, and most styles of reasoning, don’t predict doom.” — Can your epistemological strategy even correctly make any predictions of near 100% certainty? I concur with habryka that most frames don’t make any predictions on most things. And yet this doesn’t mean that some events aren’t ~100% certain.
Does Everything not do much of what you want?
In case of institutions, there’s a bias towards conservatism because any institution that’s too willing to change is one that might well cease to exist for any number of reasons. So if you encounter a long-lived institution, it’s probably one that has numerous policies in place to perpetuate itself.
This doesn’t really seem analogous to how human aging affects willingness and ability to change.
Computational argument, inspired by Algorithms to Live By: The more time you have, the more you should lean towards exploration in the explore-exploit tradeoff. As your remaining lifespan decreases, you should conversely lean towards the exploit side. Including consuming less new information, and changing your mind less often—since there’s less value in doing that when you have less time to act on that new info.
Conversely, if we could magically extend the healthy lifespans of people, by this same argument that should result in more exploration, and in people being more willing to change their mind.
I didn’t get any replies on my question post re: the EU parliamentary election and AI x-risk, but does anyone have a suggestion for a party I could vote for (in Germany) when it comes to x-risk?
This post seems like a duplicate of this one.
On this topic you might be interested in skimming Zvi’s three dating roundup posts. Here’s the third, which covers dating apps in the first two headings, but all three posts mention them a lot (Ctrl + F “dating app”).
[Question] Have any parties in the current European Parliamentary Election made public statements on AI?
Or if you’re instead in the mode of deciding what to do next, or making a schedule for your day, etc., then that’s different, but working memory is still kinda irrelevant because presumably you have your to-do list open on your computer, right in front of your eyes, while you do that, right?
Whenever I look at a to-do list, I’ve personally found it noticeably harder to decide which of e.g. 15 tasks to do, than which of <10 tasks to do. And this applies to lists of all kinds. A related difficulty spike appears once a list no longer fits on a single screen and requires scrolling.
Thanks for crossposting this. I also figured it might be suitable for LW. Two formatting issues due to crossposting from Twitter: the double spaces occasionally turn into single spaces at the beginning of a line; and the essay would benefit a lot from headings and a TOC.