Amalthea

Karma: 462

Amalthea 28 Nov 2022 20:43 UTC
1 point
0
in reply to: So8res’s comment on: Open technical problem: A Quinean proof of Löb’s theorem, for an easier cartoon guide
I was interpreting things somewhat differently:
1. “this” should be a reference to the Gödel code of the current proof
2. the OP would like to have what is essentially an application of the diagonal lemma to the whole proof instead of a proposition
3. “zip”/”unzip” is simply taking the Gödel numbering, and doesn’t hide any extra complexity
In this case the problem here is making precise what one hopes for in step 2. (I don’t really see how one would do it), and then to prove that one can make it work.

Amalthea 29 Nov 2022 16:50 UTC
1 point
0
on: Open technical problem: A Quinean proof of Löb’s theorem, for an easier cartoon guide
I think I really don’t see what the intuition behind the suggested strategy is.
The standard proof of Löb’s theorem as I understand it can be obtained by trying to formalize the paradoxical sentence “If this sentence is true, then C”. Intuitively, this sentence is clearly true, therefore C. (This of course doesn’t work formally, what is missing is exactly the assumption that you can conclude C from a proof of C).
With your suggested proof, I just wouldn’t believe it’s a proof in the first place, so (for me) there’s no reason to believe that it should work. If you can explain in more detail what exactly you would like to formalize, I would be curious.

Amalthea 30 Jan 2023 8:35 UTC
1 point
0
on: What is the ground reality of countries taking steps to recalibrate AI development towards Alignment first?
Probably the most relevant post I’ve seen on here in a while. I do think the politics around AI (especially global politics) are the most pressing issue at this point in time.

Amalthea 11 Feb 2023 0:42 UTC
1 point
0
AF
in reply to: James Payor’s comment on: Modal Fixpoint Cooperation without Löb’s Theorem
So, ideally you would like to assume only
1. □A→B
2. □B→A
and conclude A and B ?

Amalthea 17 Feb 2023 21:47 UTC
2 points
0
in reply to: Jozdien’s comment on: Bing chat is the AI fire alarm
I think it might be a dangerous assumption that training the model better makes it in any way less problematic to connect to the internet. If there is an underlying existential danger, then it is likely from capabilities that we don’t expect and understand before letting the model loose. In some sense, you would expect a model with obvious flaws to be strictly less dangerous (in the global sense that matters) than a more refined one.

Amalthea 19 Feb 2023 11:31 UTC
8 points
2
on: AGI in sight: our look at the game board
Good article! I share some skepticism on the details with other comments. Let me take this opportunity to point out that the government would be in a good position to slow down AI capabilites research.

Amalthea 19 Feb 2023 14:10 UTC
8 points
5
in reply to: Risk+Progress’s comment on: AGI in sight: our look at the game board
I found this a very lucid write-up of the case for slowing down and how realistic/unrealistic it is:
https://www.lesswrong.com/posts/uFNgRumrDTpBfQGrs/let-s-think-about-slowing-down-ai
If, say, the US government were to regulate OpenAI and Big Tech in general to slow them down significantly, this might buy a few years. In the longer term you’d need to get China etc. on board—but that is not completely unfathomable and should be significantly easier, if you’re not running ahead full steam yourself.

Amalthea 19 Feb 2023 16:51 UTC
3 points
0
AF
in reply to: Vladimir_Nesov’s comment on: AGI in sight: our look at the game board
Are there any reasons to believe that LLMs are in any way more alignable than other approaches?

Amalthea 19 Feb 2023 21:46 UTC
2 points
0
on: A Way To Be Okay
Is there actually a discussion somewhere of what we can do? I basically only ever see discussion of AI alignment, which seems to me more of a long-term strategy and not the best place to spend resources on right now (although I’m confident that it is going to have payoffs if we weather the next several years). I kind of conceptualize this as a war—its normal to be devastated and the stakes are incredibly high, but in the end you have to pull together and try to figure out the most strategic thing to do without giving in to panicked thinking. And we’re all in this together.

Amalthea 19 Feb 2023 22:01 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: AGI in sight: our look at the game board
Aah okay, that makes some sense. It still sounds like a vague hope for me, but it’s at least conceivable. I tend to visualize it like an alien civilization developing around trying to decipher some oracle (after seeing Eliezer’s stories), which would run counter to what you suggest, but it’s seems like anyone’s guess at the moment.

Amalthea 20 Feb 2023 1:29 UTC
3 points
0
in reply to: Raemon’s comment on: A Way To Be Okay
Thanks! These tags are very relevant and I wasn’t aware—that’s maybe part of the problem (although I’m relatively new in using this forum). I might simply not be aware of the concrete efforts to mobilize/draw public attention and raise public concern about the issue. Certainly many of the leading figures of EA/rationalism appear to be very stuck in their heads from the outside and—I don’t really know how to phrase this—not willing to make a stand? E.g. Eliezer’s famously defeatist attitude probably doesn’t send a good signal to the outside (I do mostly agree with him on his general opinions).
When I say alignment seems mostly relevant long-term, I really mean the technical parts. It is definitely good to recruit and educate people in that direction even right now. I just think the political dimension—public perception, and government involvement are likely much more relevant short term (and could yield a lot of resources towards alignment). Since I haven’t been seeing much discussion of this here, I felt it is underappreciated.
What precisely do you mean by “missing the real problem”?

Amalthea 20 Feb 2023 20:19 UTC
7 points
−28
on: Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky
Why is Eliezer such a downer? We simply don’t know how things are going to turn out—I believe he’s right about how we should approach AI. And regarding any technical points I’d guess he is more right than anyone else. That doesn’t justify, in my opinion, going around and instilling a defeatist attitude in anyone who wants to take him seriously. Seriously, people are looking up to you. That’s not how you treat them.

EDIT: I rewatched part of the podcast. Previously, I had only seen the final snippets, in particular where he talks about the conference involving Elon Musk. The problem I have doesn’t pertain to the rest of the podcast, which makes up a big proportion, so I would weaken the indignant tone above quite a bit.

Still, I have an issue with that snippet, and I believe it is no coincidence that it was isolated for emotional effect. Concretely, Eliezer strikes this hurt tone and starts talking about how we couldn’t even react in a dignified way and so on. This does strike me as unnecessary and even bordering toward offending the viewer/society at large (whether or not that is intended or might be justified aside).

Amalthea 20 Feb 2023 20:55 UTC
1 point
−4
in reply to: Vladimir_Nesov’s comment on: Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky
Well, he might be right—and I align with his views more than with many others—but you still have to realize that you can’t literally predict the future.

I think there’s a difference between not wanting to elicit false hope and taking out your negative emotions on others (even if it’s about a reasonable expectation of the world). Of course, he has a right to experience these emotions—but I believe it would be more considerate to do that in private.

I should say that I have great respect to him and his efforts and insights. This is not a critique of the person, just of a concrete behavior.

Amalthea 20 Feb 2023 21:19 UTC
1 point
0
in reply to: Rafael Harth’s comment on: Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky
Thanks for bringing this up. Yes I think that is very important and is not what I’m trying to criticize. I will update the previous comment to clarify.

Amalthea 9 Mar 2023 23:24 UTC
11 points
9
on: Speed running everyone through the bad alignement bingo. $5k bounty for a LW conversational agent
Personally, I’d much rather have a FAQ or a list of good responses to takes, both for myself and especially to point people towards. I’d expect carefully curated information to still be much more valuable than even a somewhat good chatbot. (But I might be out of touch with the Zeitgeist)

Amalthea 16 Mar 2023 22:07 UTC
1 point
0
on: Is it a bad idea to pay for GPT-4?
I’m gonna pass on the question of whether it’s defensible (like you, the thought of giving money to OpenAI makes me uneasy), but I do like the idea of an “Alignment tax”. By general principles one should expect that there is some ideal proportion of money flowing into alignment/regulation efforts vs. AI development that makes the future maximally safe. So steering towards that seems like the right thing to do.

Amalthea 22 Mar 2023 4:06 UTC
4 points
0
in reply to: Noosphere89’s comment on: AI #4: Introducing GPT-4
What is your evidence/reasoning that we are out of the pessimistic scenario? How do you think the medium difficulty scenario could play out?

Amalthea 22 Mar 2023 16:34 UTC
0 points
−1
in reply to: Noosphere89’s comment on: AI #4: Introducing GPT-4
For context: Are you part of that alignment effort?

I could easily imagine that changes to the pre-training regime can lead to more robust agents with less of the obvious superficial failure modes. Naively, it also does make sense that it moves us into a regime that appears strictly safer, than doing unconstrained pretraining and then doing RLHF. I don’t see how it reliably tells us anything about how things generalize to a system that is capable of coherent reasoning that is not necessarily legible to us.

I.e. I don’t see how to update at all away from the (alignment-technical) pessimistic scenario. I could see how it might help move us away from a maximally socially pessimistic scenario, i.e. one where the techniques that we pursue seem to aggressively optimize for deception and try to fix failure modes only after they have already appeared.

Amalthea 22 Mar 2023 19:05 UTC
0 points
−1
in reply to: Noosphere89’s comment on: AI #4: Introducing GPT-4
Has anyone tried to point out expected failure modes of that approach (beyond the general “we don’t know what happens when capabilities increase” that I was pointing at)?
I’ll admit I don’t understand the details enough right now to say anything, but it seems worth to look at!

I’m not sure I can follow your Meta-reasoning. I agree that news are overly focused on current problems, but I don’t really see how that applies to AI alignment (except maybe as far as bias etc. are concerned). Personally, I try to go by who has the most logically inevitable-seeming chains of reasoning.

Amalthea 24 Mar 2023 15:37 UTC
7 points
5
in reply to: DragonGod’s comment on: continue working on hard alignment! don’t give up!
I don’t see how you arrive at these conclusions at all. I agree that how alignment of the current models works there’s some vague hope that things might keep going like this even when capabilities increase. Is there any specific thing that makes you update more strongly?