This is a problem for me, a few days after starting to (try) doing this kind of research. Any directions?
The main reason for me is that I want feedback on my ideas, to push me away from directions that are totally useless, which I’m afraid to fall into since I’m not an experienced researcher.
I recommend discussing in the original comment as opposed to splitting up the comments between places, if you have something to ask/say
Poll: Agree/Disagree:
Working for a company that advances AI capabilities is a good idea for advancing-safety because you can speak up if you disagree with something, and this outweighs the downside of how you’d help them advance capabilities
Poll: Agree/disagree:
Working for companies that advance AI capabilities is generally a good idea for people worried about AI risk
Could you help me imagine an AGI that “took over” well enough to modify it’s own code or variables—but chooses not to “wire head” it’s utility variable but rather prefers to do something in the outside world?
This looks like a guide for [working in a company that already has a research agenda, and doing engineering work for them based on what they ask for] and not for [trying to come up with a new research direction that is better than what everyone else is doing], right?
Ah,
I thought it was “I’m going to sacrifice sleep time to get a few extra hours of work”
My bad
I try to both [be useful] and [have a good life / don’t burn out]
I started thinking a few days ago about investments. Initial thoughts:
Given we’re not all dead, what happens and how to get rich?
Guess 1: There was a world war or something similar that got to all AI labs worldwide
My brain: OMG that sounds really bad. Can I somehow avoid the cross fire?
Guess 2: One specific org can generate 100x more tech and science progress than the entire rest of the world combined
My brain: I hope they will be publicly tradable, still respect the stock market, and I can buy their stock in advance?
Problem: Everyone wants to invest in AI companies already. Do I have an advantage?
If there will be a few years of vast-strangeness before we’ll probably all die, can I get very rich beforehand and maybe use that for something?
(Similar to Guess 2 above, and also doesn’t seem promising)
This is just initial, I’m happy in anyone joining the brainstorm, it’s easier together
I disagree with “sleeping less well at night”.
I think if you’re able to sleep well (if you can handle the logistics/motivation around it, or perhaps if sleeping well is a null action with no cost), it will be a win after a few days (or at most, weeks)
When I ask this question, my formulation is “50% of the AI capabilities researchers [however you define those] stop [however you define that] for 6 months or more”.
I think that your definition, of “making people change their mind” misses the point that they might, for example, “change their mind and work full time on making the AGI first since they know how to solve that very specific failure mode” or whatever
Epistemic Status: Trying to form my own views, excuse me if I’m asking a silly question.
TL;DR: Maybe this is over fitted to 2020 information?
My Data Science friends tell me that to train a model, we take ~80% of the data, and then we test our model on the last 20%.
Regarding your post: I wonder how you’d form your model based on only 2018 information. Would your model nicely predict the 2020 information or would it need an update (hinting that it is over fitted)? I’m asking this because it seems like the model here depends very much on cutting edge results, which I would guess makes it very sensitive to new information.
May I ask what you are calling “general alignment sympathy”? Could you say it in other words or give some examples?
I don’t think “infinite space” is enough to have infinite copies of me. You’d also need infinite matter, no?
[putting aside “many worlds” for a moment]
Anonymous question (ask here) :
Why do so many Rationalists assign a negligible probability to unaligned AI wiping itself out before it wipes humanity out?
What if it becomes incredibly powerful before it becomes intelligent enough to not make existential mistakes? (The obvious analogy being: If we’re so certain that human wisdom can’t keep up with human power, why is AI any different? Or even: If we’re so certain that humans will wipe themselves out before they wipe out monkeys, why is AI any different?)
I’m imagining something like: In a bid to gain a decisive strategic advantage over humans and aligned AIs, an unaligned AI amasses an astonishing amount of power, then messes up somewhere (like AlphaGo making a weird, self-destructive move, or humans failing at coordination and nearly nuking each other), and ends up permanently destroying its code and backups and maybe even melting all GPUs and probably taking half the planet with it, but enough humans survive to continue/rebuild civilisation. And maybe it’s even the case that hundreds of years later, we’ve made AI again, and an unaligned AI messes up again, and the cycle repeats itself potentially many, many times because in practice it turns out humans always put up a good fight and it’s really hard to kill them all off without AI killing itself first.
Or this scenario considered doom? (Because we need superintelligent AI in order to spread to the stars?)
(Inspired by Paul’s reasoning here: “Most importantly, it seems like AI systems have huge structural advantages (like their high speed and low cost) that suggest they will have a transformative impact on the world (and obsolete human contributions to alignment retracted) well before they need to develop superhuman understanding of much of the world or tricks about how to think, and so even if they have a very different profile of abilities to humans they may still be subhuman in many important ways.” and similar to his thoughts here: “One way of looking at this is that Eliezer is appropriately open-minded about existential quantifiers applied to future AI systems thinking about how to cause trouble, but seems to treat existential quantifiers applied to future humans in a qualitatively rather than quantitatively different way (and as described throughout this list I think he overestimates the quantitative difference).”)
If this question becomes important, there are people in our community who are.. domain experts. We can ask
Hey,
TL;DR I know a researcher who’s going to start studying C. elegans worms in a way that seems interesting as far as I can tell. Should I do something about that?
I’m trying to understand if this is interesting for our community, specifically as a path to brain emulation, which I wonder if could be used to (A) prevent people from dying, and/or (B) creating a relatively-aligned AGI.
This is the most relevant post I found on LW/EA (so far).
I’m hoping someone with more domain expertise can say something like:
“OMG we should totally extra fund this researcher and send developers to help with the software and data science and everything!”
“This sounds pretty close to something useful but there are changes I’d really like to see in that research”
“Whole brain emulation is science fiction, we’ll obviously destroy the world or something before we can implement it”
“There is a debate on whether this is useful, the main positions are [link] and [link], also totally talk to [person]”
Any chance someone can give me direction?
Thx!
(My background is in software, not biology or neurology)
I heard “kiwi” is a company with a good reputation, but I didn’t try their head strap myself. I have their controller-straps which I really like
(I’m not sure but why would this be important? Sorry for the silly answer, feel free to reply in the anonymous form again)
I think a good baseline for comparison would be
Training large ML models (expensive)
Running trained ML models (much cheaper)
I think comparing to blockchain is wrong, because
it was explicitly designed to be resource intensive on purpose (this adds to the security of proof-of-work blockchains)
there is a financial incentive to use a specific (very high) amount of resources on blockchain mining (because what you get is literally a currency, and this currency has a certain value, so it’s worthwhile to spend any money lower than that value on the mining process)
None of these are true for ML/AI, where your incentive is more something like “do useful things”
+1 for the Abusive Relationships section.
I think there’s a lot of expected value in a project that raises awareness to “these are good reason to break up” and/or “here are common-but-very-bad reasons to stay in an abusive relationship”, perhaps with support for people who choose to break up. It’s a project I sometimes think of opening but I’m not sure where I’d start
[I also just got funded (FTX) to work on this for realsies 😸🙀 ]
I’m still in “learn the field” mode, I didn’t pick any direction to dive into, but I am asking myself questions like “how would someone armed with a pretty strong AI take over the world?”.
Regarding commitment from the mentor: My current format is “live blogging” in a Slack channel. A mentor could look whenever they want, and comment only on whatever they want to. wdyt?
(But I don’t know who to add to such a channel which would also contain the potentially harmful ideas)