Oh I support increasing the karma cutoff.
I do think that running such an exercise is valuable, if only because it allows us to learn things about our community.
sudo -i
Karma: 107
Opted out. Also, IQ?
Edit: ah I get it now
It’s an exercise in collective adequacy.
I like this post. We should be able to practice collective adequacy without public shaming. In the real world, policies that rely on public shaming are quite imprecise, often cruel, and frequently ineffective.
It seems that it would actually be quite hard for M to hack D. M would then have to emulate P to figure out which sensor states are likely to be produced given its actions. It would then have to figure out how the consequent worldstate is likely to be perceived. However, this doesn’t just require emulating a human, since even humans can‘t go directly from sensor states to favorability. M would probably need to also emulate D so that it can output text descriptions, and then select on those text descriptions with an emulated human.
The ELK Framing I’ve Used
A corollary to this is that it should be possible to find mistakes in the works of almost everyone.
It is probably good practice to do this.
It takes a certain degree of maturity and thought to see that a lot of advice from high profile advice-givers are bad.
It could be valuable to do point-by-point critiques of popular advice by high profile technologists.
The Value of Not Being an Imposter
I’m generally pretty pessimistic about research-automation proposals analogous to “train a net to mimic a human sticking their finger in some stuff and reporting hotness”.
I haven’t read your whole post yet, but I predict that this is because you wouldn’t have reason to believe that you can extrapolate out of the distribution that is measurable by humans.
I think LW folks often underestimate the importance of serendipity, especially for pre-paradigmatic fields like AI Alignment.
You want people learning functional programming and compiler design and writing kernels and playing around with new things, instead of just learning the innards of ML models and reading other people’s alignment research.
You even want people to go very deep into tangential things, and become expert kernel designers or embedded systems engineers. This is how people become capable.
Does anyone know if UpToDate includes dental information?
Lies on the internet
[Question] Would you ask a genie to give you the solution to alignment?
It seems like your model is that we should be working in one of two modes:
Developing better alignment ideas
Implementing our current best alignment idea
However, in my model, there are a lot of alignment ideas which are only worth developing given certain timelines. [edit: Therefore, “you should be developing better alignment ideas anyway” is a very vague and questionably actionable strategy.]
Do you believe this is the crux?
To identify the crux here, would you care about timelines if it took five years to bring our best alignment idea to production?
Ah, I see. Thanks.
In my view, there are alignment strategies that are unlikely to pay off without significant time investment, but which have large expected payoffs. For example, work on defining agency seems to fit this category.
There are also alignment strategies that have incremental payoffs, but still seem unsatisfactory. For example, we could focus on developing better AI boxing techniques that just might buy us a few weeks. Or we could discover likely takeover scenarios, and build warnings for them.
There’s an analogy for this in self driving cars. If you want to ship an impressive demo right away, you might rely on a lot of messy case handling, special road markings, mapping, and sensor arrays. If you want to solve self driving in the general case, you’d probably be developing really good end to end ML models.
I do not believe that the chain of reasoning would be “3.5 years ago I predicted 5 years, and I also predicted 1.5 years to implement the best current idea, so it is time to implement the best current idea now.”
Why not?
Most people in the Bay Area rationalist community are not financially limited on classes, books, and intelligence-enhancing drugs.
I’ve never met someone who told me they didn’t take modafinil because they couldn’t afford it.