Rudi C(Luna Rimar)

Karma: 507

Rudi C 14 Jan 2022 0:33 UTC
3 points
in reply to: Daniel Kokotajlo’s comment on: How to think about and deal with OpenAI
Epistemic status: I am not an expert on this debate, I have not thought very deeply about it, etc.
1. I am fairly certain that as long as we don’t fail miserably (i.e., a loose misaligned super AI that collapses our civilization), FOSS AI is extremely preferable to proprietary software. The reasons are common to other software projects, though the usefulness and blackboxy-ness of AGI make this particularly important.
2. I am skeptical of “conspiracies.” I think a publicly auditable, transparent process with frequent peer feedback on a global scale is much more likely to result in trustable results with fewer unforeseen consequences/edge cases.
3. I am extremely skeptical of the human incentives that a monopoly on AGI encourages. E.g., when was the single time atomic bombs were used? Exactly when there was a monopoly on them.
4. I don’t see the current DL approaches as at all near achieving efficient AGI that would be dangerous. AI alignment probably needs more concrete capability research IMO. (At least, more capability research is likely to contribute to safety research as well.) I like the world to enjoy having better narrow AI sooner, and I am not convinced delaying things is buying all that much. (Full disclosure: I am weighing the lives of my social bubble and contemporaries more than random future lives. Though if I were to not do this, then it’s likely that intelligence will evolve again in the universe anyhow, and so humanity failing is not that big of a deal? None of our civilization is built that long-termist, too, so it’s pretty out of distribution for me to think about. Related point: I have an unverified impression that the people who advocate slowing capability research are already well off and healthy, so they don’t particularly need technological progress. Perhaps this is an unfair/false intuition I have, but I do have it, and disabusing me of it will change my opinion a bit.)
5. In a slow takeoff scenario, my intuition is that multiple competing super intelligences will leave us more leverage. (I assume in a fast takeoff scenario the first such intelligence will crush the others in their infancy.)
6. Safety research seems to be more aligned with academic incentives than business incentives. Proprietary research is less suited to academia though.

Rudi C 13 Jan 2022 10:43 UTC
1 point
in reply to: Zack_M_Davis’s comment on: Blood Is Thicker Than Water 🐬
My point is more about prioritization. English, math, programming and computer literacy, economics, basic home skills (cooking, trivial repairs, etc.), and possibly rationality (though the existence of “The Dark Valley of Rationality” makes me a bit hesitant on this one) are much better subjects for a “general info” curriculum.

PS: Knowing about elementary particles (without a mathematical model of them) is trivial. You can fit all such facts into a single year’s science curriculum. The things that take time to learn are calculations, e.g., finding the mass of some reagent after some chemical reaction.

Rudi C 13 Jan 2022 10:31 UTC
3 points
in reply to: Daniel Kokotajlo’s comment on: How to think about and deal with OpenAI
Ironically, I am a believer in FOSS AI models, and I find OpenAI’s influence anything but encouraging in this regard. The only thing they are publicly releasing is marketing nowadays.

Rudi C 11 Jan 2022 16:33 UTC
3 points
in reply to: Daniel Kokotajlo’s comment on: How to think about and deal with OpenAI
Taboo “racing”? I don’t understand what concrete actions were thought to have been skipped.

Rudi C 11 Jan 2022 15:10 UTC
1 point
on: Blood Is Thicker Than Water 🐬
I find that understanding the ways in which dolphins are mammalian, is very much an informational challenge; I need to know a lot more biology to be able to use the category of “genetically mammal” than just plain old “fish.” Obviously, knowing more is better, but it is not obvious to me that forcing everybody to learn the more informed categories is societally optimal. I doubt anyone but specialists will ever find the information instrumentally useful, so we are just wasting some limited bandwidth to teach people stuff they don’t need to know. (On the other hand, people mostly waste their time with, e.g., learning about the differences between ten different Robin characters, so perhaps the endeavor is justified after all.)

Rudi C 11 Jan 2022 13:56 UTC
1 point
on: Launched: Friendship is Optimal
I just finished the story. I think the main idea is obviously good, but I find the execution too simplistic. The princess’s intelligence comes off as human in her social interactions (not superhuman), and humans seem dumbed down. I don’t think the AI could have forced everyone to “upload” with the level of coercion shown in the novel.

I myself would not have uploaded unless I was already dying. I see no benign reason that the process of uploading needs to be destructive.

I also doubt people with real power would be easily swayed to upload, e.g., the original programmer. In the end, people’s values are in conflict, and there could be alien powers, etc. There is every reason to believe that whoever still controls some resources in the “real” world will still be able to “satisfy” their values more than the powerless ponies.

Rudi C 8 Jan 2022 1:30 UTC
0 points
on: Signaling isn’t about signaling, it’s about Goodhart
This post is simplistic and vague. E.g., does the OP think dressing in dirty, shabby clothes (which, from a nonsignalling perspective, aren’t a negative in our environment) is not an obvious failure of marketing that leads to lost opportunities?

Anyhow, caring about signaling is a priority in most cultures that I have glimpsed, almost all businesses, and human nature. Against such a strong prior, the OP fails to provide any strong updates to the contrary.

Rudi C 15 Dec 2021 20:03 UTC
1 point
in reply to: Valentine’s comment on: Kenshō
An interesting fact came to my mind; music that affects one’s mental state is forbidden in Islam.

Perhaps the reason this theme of “sinful pleasure” keeps repeating is the observation that pleasure is a reward signal that does not quite match the utility functions of the conscious mind. At least, that has always been the key motivator of this idea subspace to me.

Rudi C 13 Dec 2021 20:44 UTC
3 points
in reply to: Valentine’s comment on: Kenshō

Can you imagine what it would be like to try to convey to pre-music folk even that music is real and that it might be worth learning how to listen to it?

This analogy is capturing my current understanding of this post and its various comments pretty well: Looking is like music, in that it is a difficult, voluntaryish act of observing and manipulating hidden mental states. This will result in wireheading, among other things, but it might be sometimes useful. (Note that music is also wireheading, but it can still be useful in narrow contexts.)

Rudi C 13 Dec 2021 19:32 UTC
2 points
in reply to: Alexander’s comment on: Is “gears-level” just a synonym for “mechanistic”?
But mechanistic world models do suggest that meaning in a traditional (mystical? I can’t really define it, as I find the concept itself incoherent) sense does not (and cannot) exist; so I think the “negative” connotations are pretty fair, it’s just that they aren’t that negative or important in the first place. (“Everything adds up to normalcy.”) Rebranding is still a sound marketing move, of course.

Rudi C 13 Dec 2021 19:13 UTC
1 point
in reply to: gwern’s comment on: Teaser: Hard-coding Transformer Models
(Unrelated.) Have you considered putting an RSS field of your Twitter account on its bio? This way people can follow you without you needing to approve them, and since it’s read-only, your burden won’t increase.

(Not to mention that RSS is a much better medium than Twitter in the first place.)

Rudi C 13 Dec 2021 18:16 UTC
1 point
in reply to: xuan’s comment on: Writing Causal Models Like We Write Programs
What useful problems do PPLs solve? Ideally some applications that are interesting for us non-corporate people. Can it be used for medical statistics (e.g., in nutrition)? (Any examples?) Is the reason it is not used the illiteracy of the scientists, or are the mainstream methods better?

Rudi C 1 Dec 2021 10:49 UTC
1 point
on: EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised
Expertise status: I am just starting with RL.

Will using a hardcoded model of the environment improve these models, or do the models need the representations they learn?

Using EfficientZero’s architecture, how many hours does it take on a single TPUv2 for the agent to reach amateur human level? In general, is EfficientZero being sample efficient or compute efficient?

What is the currently most compute efficient algorithm for simple, two-player deterministic games with a lot of states (e.g., go)?

PS: The reason I am asking is that I learn stuff by coding much better than merely reading, and I have been trying to write the simplest non-search-based, self-supervised algorithm I can for games like chess.

One of the main challenges I have faced is that cloning the current state, doing an action on it, and then using the neural net (a simple 4-layered MLP) to infer the new state’s value is quite slow (on the order of a second), so running MCTS seems to require a lot more compute. (Done using Colab Pro’s TPUv2, but the game state is managed by OpenSpiel on the CPU.)

Rudi C 28 Nov 2021 23:06 UTC
3 points
in reply to: tkpwaeub’s comment on: Omicron Variant Post #1: We’re F***ed, It’s Never Over
Sources?

Rudi C 21 Nov 2021 19:07 UTC
4 points
on: Sasha Chapin on bad social norms in rationality/EA
I myself sometimes feel bad when I engage in, say, writing fiction. (Reading fiction is pretty obviously useless, so I know I am just doing it for “fun.” It doesn’t confuse me the way producing fiction does.) I was like this before I even knew there was a Rationality subculture. I don’t try to justify these behaviors at all; I am just not sure if they are aligned with my values, or not, and in what quantities they are healthy.

So while I agree with the gist of this post, I believe the core issue to be more of a tradeoff rather than an obvious evil.

Rudi C 2 Nov 2021 15:47 UTC
1 point
in reply to: Slider’s comment on: We Live in a Post-Scarcity Society
The stock’s value declines (as supply has gone up). So the “frozen” money declines, too.

Rudi C 2 Nov 2021 15:40 UTC
3 points
in reply to: Liron’s comment on: The Best Software For Every Need
(I simply use unused books to raise the monitor. Works like a charm.)

Rudi C 2 Nov 2021 15:39 UTC
1 point
in reply to: Tom’s comment on: The Best Software For Every Need
I doubt Google can “add closed source components” to Chrome with any success. MS will simply recreate the extensions in open-source, getting a lot of mindshare and PR in the process. Android became what it is because Google was ahead of the curve and other companies did not know how useful mobile OSes were going to be.

Rudi C 2 Nov 2021 15:24 UTC
1 point
in reply to: Hans R’s comment on: The Best Software For Every Need
See also Nyxt. I haven’t tried it myself yet, as its macOS support seems to be immature, but it is one of those projects I have an eye on. It could one day be the emacs of web browsers.

There is also https://github.com/emacs-eaf/emacs-application-framework, but the security might be sketchy. I am not holding my breath for performance either.

Rudi C 2 Nov 2021 14:58 UTC
2 points
in reply to: madasario’s comment on: The Best Software For Every Need
Why do you want to switch to macOS? The only thing going for it is having Adobe and Office software. On the other hand, it is likely to do explicit on-device scanning, it doesn’t support Docker well, it is generally slow and can hang when the internet connection has problems. Hell, even its API for changing the background wallpaper doesn’t work reliably for me.