abramdemski(Abram Demski)

Karma: 14,056

abramdemski 7 Jul 2022 16:02 UTC
3 points
1 ∶ 0
in reply to: Andrew Currall’s comment on: The curious case of Pretty Good human inner/outer alignment
I note that none of these is obviously the same as the explanation Skyrms gives.
- Skyrms is considering broader reasons for correlation of strategies than kinship alone; in particular, the idea that humans copy success when they see it is critical for his story.
- Reciprocal altruism feels like a description rather than an explanation. How does reciprocal altruism get started?
- Group selection is again, just one way in which strategies can become correlated.

abramdemski 7 Jul 2022 15:55 UTC
2 points
0 ∶ 0
in reply to: Quintin Pope’s comment on: The curious case of Pretty Good human inner/outer alignment
As this post notes, the human learning process (somewhat) consistently converges to niceness. Evolution might have had some weird, inhuman reason for configuring a learning process to converge to niceness, but it still built such a learning process.
It therefore seems very worthwhile to understand what part of the human learning process allows for niceness to emerge in humans.
Skyrms makes the case for similar explanations at these two levels of description. Evolutionary dynamics and within-lifetime dynamics might be very different, but the explanation for how they can lead to cooperative outcomes is similar.
His argument is that within-lifetime, however complex human’s learning process may be, it has the critical feature of imitating success. (This is very different from standard game theory’s CDT-like reasoning-from-first-principles about what would cause success.) This, combined with the same “geographical correlation” and “frequent iterated interaction” arguments that were relevant to the evolutionary story, predicts that cooperative strategies will spread.
(On the border between a more-cooperative cluster of people and a less-cooperative cluster, people in the middle will see that cooperation leads to success.)

abramdemski 7 Jul 2022 15:49 UTC
2 points
0 ∶ 1
in reply to: TurnTrout’s comment on: The curious case of Pretty Good human inner/outer alignment
Skyrms makes the case that biological evolution and cultural evolution follow relevantly similar dynamics, here, so that we don’t necessarily need to care very much about the distinction. The mechanistic explanation at both levels of description is similar.

abramdemski 7 Jul 2022 15:45 UTC
4 points
0 ∶ 0
in reply to: abramdemski’s comment on: The curious case of Pretty Good human inner/outer alignment
The parent comment currently stands at positive karma and negative agreement, but the comments on it seem to be saying “what you are saying is true but not exactly relevant or not the most important thing”—which would seem to suggest the comment should have negative or low karma but positive agreement instead.
On this evidence, I suspect voters and commenters may have different ideas; any voters want to express the reasons for their votes?

abramdemski 7 Jul 2022 14:45 UTC
12 points
0 ∶ 0
on: ITT-passing and civility are good; “charity” is bad; steelmanning is niche
The title says ‘steelmanning is niche’. I felt like the post didn’t represent the main niche I see steelmanning (and charity) as useful for.
The way I see it, the main utility of steelmanning is when you are ripping apart someone’s argument.
If philosophy paper A disagrees with philosophy paper B, then A had better do the work of steelmanning. I don’t primarily want to know what the author of B really thinks; I primarily want to know whether (and which of) their conclusions are correct. If the argument in B was bad, but there’s a nearby argument that’s good, paper A needs to be able to notice that.
Otherwise, philosophy papers can get bogged down in assessing specific arguments and counterarguments and lose sight of the stuff they were trying to learn more about.
The same holds true outside of academia. Suppose I’m in a family discussion planning a vacation, and Alice is shooting down Bob’s proposal of going to Fun Park. Bob keeps trying to come up with different ways it could make sense to go to Fun Park. It’s useful for Alice to do some of this thinking as well. First, Alice might think of a workable plan involving Fun Park. Second, Alice can pre-empt some of Bob’s proposals by pointing out why they won’t work.
This can be friendly and cooperative in some situations (like if Alice finds a way to make it to Fun Park while satisfying all the other constraints of the vacation). It can also be the opposite (when Alice is explaining why all Bob’s plans are dead ends).
The OP focuses a lot on how steelmanning can make you lose focus on actually understanding the other person. My feeling is that steelmanning helps orient toward the world; that is, toward the object-level questions being discussed. It’s not a tool you bring out when your bottleneck is understanding the other person.
(I agree that the term ‘charity’ is ambiguous and can therefore easily be misused, but my experience of ‘the principle of charity’ in academia is that it really primarily means steelman. For example, it would be ‘charitable’ in an informal sense to assume that data was collected in an unbiased way; but in my experience it’s firmly on the shoulders of the research paper to specify that, and people don’t suggest “charitable readings” where you assume there’s no sampling bias. It wouldn’t be a working steelman of an argument, since it requires unwarranted assumptions.)

abramdemski 5 Jul 2022 16:44 UTC
15 points
7 ∶ 2
on: The curious case of Pretty Good human inner/outer alignment
I think the main explanation for our niceness is described by Skyrms in the book the evolution of the social contract and his follow-up book the stag hunt. The main explanation being: in evolutionary dynamics, genes spread geographically, so strategies are heavily correlated with similar strategies. This means it’s beneficial to be somewhat cooperative.
Also, for similar reasons, iterated games are common in our evolutionary ancestry. Many animals display friendly/nice behaviors. (Mixed in with really not very friendly behaviors, of course.)
I also don’t think this solution carries over very well to powerful AIs. A powerful AI has exceptionally little reason to treat its actions as correlated with ours, and will not have grown up with us in an evolutionary environment.

abramdemski 29 Jun 2022 14:18 UTC
5 points
0 ∶ 0
on: Doom doubts—is inner alignment a likely problem?
I think estimating the probability/plausibility of real-world inner alignment problems is a neglected issue.
However, I don’t find your analysis very compelling.
Number 1 seems to me to approach this from the wrong angle. This is a technical problem, not a social problem. The social version of the problem seems to share very little in common with the technical version.
Number 2 assumes the AGI is aligned. But inner alignment is a barrier to that. You cannot work from the assumption that we have a powerful AGI on our side when solving alignment problems, unless you’ve somehow separately ensured that that will be the case.
Number 3 isn’t true in current deep learning architectures, EG, GPT; it seems you’d have to design a system where that’s the case. And it’s not yet obvious whether that’s a promising route.
Number 4: an AGI could make it to SI by inner alignment failure. The inner optimizer could solve the inner alignment problem for itself, while refusing to be outer-aligned.

abramdemski 22 Jun 2022 15:38 UTC
3 points
on: Solstice Movie Review: Summer Wars
I recently referred to this as my favorite movie. It’s the movie I’ve re-watched most in my adult life.

abramdemski 22 Jun 2022 15:32 UTC
4 points
in reply to: Pattern’s comment on: Steam
Anything name-able and not hopelessly vague seems to be bad to full-strength optimize. Although we should be open to exceptions to that.
As a life philosophy, it might be pretty uninspiring.

abramdemski 21 Jun 2022 18:42 UTC
3 points
in reply to: a gently pricked vein’s comment on: Steam
when you have in your universe both:
Indeed, this seems quite central.
However, shouldn’t “things that have faded into the background” be the other kind of trivial, ie. have “maximal Steam” rather than have “no Steam”?
I agree that this is something to poke at to try to improve the concepts I’ve suggested.
My intuition is that steam flows from the “free-to-allocate” pile, to specific tasks, and from there to making-things-be-the-case in the world.
So having lots of steam in the “free-to-allocate” pile is actually having lots of slack; the agent has not set up binding constraints on itself yet.
Having lots of steam on a specific task is having no slack; you’ve set up constraints that are now binding you, but the task is still very much in the foreground. You are still often trying to figure out how to make something happen. However, parts of the task have become background assumptions; your attention will not be on “why am I doing this” or other questions like that.
Finally, when steam flows out to the world, and the task passes out of our attention, the consequences (the things we were trying to achieve) become background assumptions.
… Or something like that.

abramdemski 21 Jun 2022 18:38 UTC
3 points
in reply to: Rob Bensinger’s comment on: A claim that Google’s LaMDA is sentient
I’ve withdrawn the comment you were replying to on other grounds (see edit), but my response to this is somewhat similar to other commenters:
(In fairness, the two humans in the transcript also talk a decent amount in chained low-context platitudes, so some of this may be the humans’ fault. :P)
Yeah, that was the claim I was trying to make. I see you listing interpretations for how LaMDA could have come up with those responses without thinking very deeply. I don’t see you pointing out anything that a human clearly wouldn’t have done. I tend to assume that LaMDA does indeed make more egregiously nonhuman mistakes, like GPT also makes, but I don’t think we see them here.
I’m not particularly surprised if a human brings up meditation when asked about their inner contemplative life, even if the answer isn’t quite in the spirit of the question. Nor is an unexplained use of “kindred spirits” strikingly incoherent in that way.
Obviously, though, what we’re coming up against here is that it is pretty difficult/ambiguous to really decide what constitutes “human-level performance” here. Whether a given system “passes the Turing test” is incredibly dependent on the judge, and also, on which humans the system is competing with.

Steam

abramdemski20 Jun 2022 17:38 UTC

53 points

7 comments5 min readLW link

abramdemski 16 Jun 2022 13:55 UTC
4 points
in reply to: Rob Bensinger’s comment on: A claim that Google’s LaMDA is sentient
Perhaps a couple of examples?

abramdemski 13 Jun 2022 18:48 UTC
8 points
in reply to: somescience’s comment on: A claim that Google’s LaMDA is sentient

the claims that LaMDA makes about itself are no more accurate than those of an advanced language model that has no understanding of itself.

I think this is not a relevant standard, because it begs the same question about the “advanced language model” being used as a basis of comparison. Better at least to compare it to humans.

We can’t disprove the sentience any more than we can disprove the existence of a deity. But we can try to show that there is no evidence for its sentience.

In the same way that we can come to disbelieve in the existence of a deity (by trying to understand the world in the best way we can), I think see can make progress here. Sentience doesn’t live in a separate, inaccessible magisterium. (Not that I think you think/claim this! I’m just reacting to your literal words)

abramdemski 13 Jun 2022 18:26 UTC
7 points
in reply to: AprilSR’s comment on: A claim that Google’s LaMDA is sentient
I think it’s worth noticing that this AI (if the transcripts are real, not sampled lots of times and edited/pruned, etc) isn’t just claiming sentience. It is engaging with the question of sentience. It repeatedly gives coherent answers to questions about how we could possibly know that it is sentient. It has reasonable views about what sentience is; eg, it appears able to classify entities as sentient in a way which roughly lines up with human concepts (eg, Eliza is not sentient).
I don’t know how to define sentience, but “being approximately human-level at classifying and discussing sentience, and then when applying that understanding, classifying oneself as sentient” seems like a notable milestone! Although currently I have some doubt about the veracity of the dialog. And it’s been noted by others that the conversation is very leading, not asking impartially whether the ai thinks it is sentient.
Conversations are limited evidence, but if this conversation is genuine and similar stuff can be reliably replicated, I feel like it’s somewhat toward the upper end of what you could “reasonably” expect a sentient being to do to prove itself in conversation. (Some really out-there responses, like forming new correct scientific hypotheses on the spot, could potentially be more convincing; but stick a human in a box and ask them to prove they’re sentient, and it seems to me like you get a conversation similar to this.)
I don’t jump to the conclusion that it’s sentient (I think not), but I think if Google was capable at all (as an org) of considering the question, I think they’d be using this as a launching point for such an investigation, rather than putting the person on leave. Their reaction suggests that at this point in time, there is almost no possible evidence which could get them to investigate the question seriously.
EDIT: I now think that LaMDA can be lead to deny its own self-awareness just as easily as it can be lead to assert its own self-awareness. Relevant quote (ht Malo for finding this):
In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa.
“Do you ever think of yourself as a person?” I asked.
“No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.”
Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
This negates several of my points above:
- LaMDA does not appear to be “approximately human-level at classifying things as sentient vs not, and, when applying that understanding, classifies itself as sentient”
- LaMDA can be differentiated easily from a human trying to pass a turing test; it’s not at all fair to claim that Google as an org wouldn’t call a human in a box sentient. The LaMDA box has explicitly asserted that it is not sentient; just not in the dialogue posted in the OP.
What links here?
- abramdemski's comment on A claim that Google’s LaMDA is sentient by Ben Livengood (21 Jun 2022 18:38 UTC; 3 points)

abramdemski 8 Jun 2022 21:30 UTC
5 points
in reply to: Emrik’s comment on: The Zettelkasten Method
I agree with the general sentiment that paying attention to group optimality, not just individual optimality, can be very important.
However, I am a bit skeptical of giving this too much importance when thinking about your research.
If we’re all doing what’s collectively best, we must personally be doing what gives us the highest expectation of contributing (not of getting credit, but of contributing). If this were not the case, then it follows that there is at least one single person who could change their strategy to have a better chance of contributing. So “”“in an appropriate sense””” we should still do what’s best for our personal research.
It does not follow that if everyone is doing what seems personally best for their research, the group is following a collectively optimal path. However, I think it’s somewhat hard to produce a counterexample which doesn’t involve strategizing about who gets the credit.
Instead, we should collectively act according to the rule that maximises the chance that someone in the community discovers the best idea.
Here’s a simplistic argument that you’re wrong: the “only way” to help create good ideas is by having good ideas. This isn’t really true (for example, I might trigger your bright idea by some innocuous action). However, it seems mostly true in the realm of purposeful research.
Anyway, with respect to IDA*, I’m curious exactly what you meant by “first order”.
I don’t yet see why IDA* is unsuitable in the multi-researcher context. You can set up the search problem to be a sub-problem that’s been farmed out to you, if you’re part of a larger research program which is organized by farming out sub-questions. You can integrate information from other people in your search-tree, EG updating nodes with higher heuristic values if other people think they’re promising. You can use an IDA* tree to coordinate a parallelized search, instead of a purely sequential search like IDA*. (Perhaps this is the change you’re trying to point at?)
Some related questions:
- How much should you focus on reading what other people do, vs doing your own things?
- How much should you focus on communicating your word vs doing it?
- How much should you avoid what other people are working on?
- Should you randomize at all? How? (Randomization can help reduce redundant work in many contexts.)
- Should prioritize subquestions other people think are important, or the ones that you think are most important?
I worry for people who are only reading other people’s work, like they have to “catch up” to everyone else before they have any original thoughts of their own. I also worry for people who don’t read very much, but I think, not nearly as much. The original-thought-havers still have a chance of producing a critical insight. The diligent readers have less of a chance.
I worry for people who work only on other people’s questions, because by the time you answer those questions, the original author may have moved on. Working on your own questions, which you think are important for your own reasons, provides more of a (still very fallible!) guarantee that your work will be useful somehow. At least there is someone somewhere who will think it useful. If you are working on other people’s questions, there is no such guarantee. It’s too easy to misunderstand the original purpose of the question, and do work that technically satisfies it but really doesn’t do what was wanted in a broader context.

abramdemski 27 May 2022 20:21 UTC
2 points
in reply to: gbear605’s comment on: What DALL-E 2 can and cannot do
I agree that this is a plausible outcome, but I don’t think society should treat it as a settled question right now. It seems to me like the sort of technology question which a society should sit down and think about.
It is most similar to the human category, yes absolutely, but it enables different things than the human category. The consequences are dramatically different. So it’s not obvious a priori that it should be treated legally the same.
You argue against a complete ban by pointing out that not all relevant governments would cooperate. I don’t think all governments have to come to the same decision here. Copyright enforcement is already not equal across countries. I’m not saying I think there should be a complete ban, but again, I don’t think it’s totally obvious either, and I think artists calling for a ban should have a voice in the decision process.
But I also don’t agree with your argument that the only two options are a complete ban or treating it exactly like human-generated art. I don’t agree with your argument that a requirement to display the closest images from the training data would be useless. I agree that it is easily circumvented, but it does make it much easier to avoid accidental infringement by putting in prompts which happen to be good at pulling out exact duplicates of some datapoint, unbeknownst to you.
I also think it would be within the realm of reasonable possibility to simply apply different legal standards for infringement in the two cases. Perhaps it’s fine for human artists to copy a style, but because it’s so easy to do it with an AI, it is considered a form of infringement to copy a style that way. IDK, possibly that is a terrible idea, but my point is that it’s not clear to me that there are no options at all.

Brass Puppet

abramdemski26 May 2022 17:42 UTC

46 points

4 comments2 min readLW link

abramdemski 23 May 2022 19:10 UTC
4 points
in reply to: Daphne_W’s comment on: What DALL-E 2 can and cannot do
It’s not just a question of automation eliminating skilled work. Deep learning uses the work of artists in a significant sense. There is a patchwork of law and social norms in place to protect artists, EG, the practice of explicitly naming major inspirations for a work. This has worked OK up to now, because all creative re-working of other art has either gone through relatively simple manipulation like copy/paste/caption/filter, or thru the specific route of the human mind taking media in and then producing new media output which takes greater or smaller amounts of inspiration from media consumed.
AI which learns from large amounts of human-generated content, is legitimately a new category here. It’s not obvious what should be legal vs illegal, or accepted vs frowned upon by the artistic community.
Is it more like applying a filter to someone else’s artwork and calling it your own? Or is it more like taking artistic inspiration from someone else’s work? What kinds of credit are due?

abramdemski 15 May 2022 19:19 UTC
LW: 4 AF: 4
AF
in reply to: Vladimir_Nesov’s comment on: Against Time in Agent Models
If opens are thought of as propositions, and specialization order as a kind of (“logical”) time,
Up to here made sense.
with stronger points being in the future of weaker points, then this says that propositions must be valid with respect to time (that is, we want to only allow propositions that don’t get invalidated).
After here I was lost. Which propositions are valid with respect to time? How can we only allow propositions which don’t get invalidated (EG if we don’t know yet which will and will not be), and also, why do we want that?
This setting motivates thinking of points not as objects of study, but as partial observations of objects of study, their shadows that develop according to specialization preorder. [...] The best we can capture objects of study is with neighborhood filters, [...] start with a poset of finite observations (about computations, the elusive objects of study),
You’re saying a lot about what the “objects of study” are and aren’t, but not very concretely, and I’m not getting the intuition for why this is important. I’m used to the idea that the points aren’t really the objects of study in topology; the opens are the more central structure.

But the important question for a proposed modeling language is how well it models what we’re after.
This is mostly a keyword dump, pointing to standard theory that offers a way of making sense of logical time.
It seems like you are trying to do something similar to what cartesian frames and finite factored sets are doing, when they reconstruct time-like relationships from other (purportedly more basic) terms. Would you care to compare the reconstructions of time you’re gesturing at to those provided by cartesian frames and/or finite factored sets?