Thane Ruthenis

• Thane Ruthenis 25 Nov 2024 4:25 UTC

  3 points

  1

  in reply to: mako yass’s comment on: Are You More Real If You’re Really For­get­ful?

  But yeah, personally, I think this is all a result of a kind of precious view about experiential continuity that I don’t share

  Yeah, I don’t know that this glyphisation process would give us what we actually want.

  “Consciousness” is a confused term. Taking on a more executable angle, we presumably value some specific kinds of systems/​algorithms corresponding to conscious human minds. We especially value various additional features of these algorithms, such as specific personality traits, memories, et cetera. A system that has the features of a specific human being would presumably be valued extremely highly by that same human being. A system that has fewer of those features would be valued increasingly less (in lockstep with how unlike “you” it becomes), until it’s only as valuable as e. g. a randomly chosen human/​sentient being.

  So if you need to mold yourself into a shape where some or all of the features which you use to define yourself are absent, each loss is still a loss, even if it happens continuously/​gradually.

  So from a global perspective, it’s not much different than acausal aliens resurrecting Schelling-point Glyph Beings without you having warped yourself into a Glyph Being over time. If you value systems that are like Glyph Beings, their creation somewhere in another universe is still positive by your values. If you don’t, if you only value human-like systems, then someone creating Glyph Being bring no joy. Whether you or your friends warped yourself into a Glyph Being in the process doesn’t matter.

• Thane Ruthenis 25 Nov 2024 4:12 UTC

  5 points

  2

  in reply to: James Camacho’s comment on: Are You More Real If You’re Really For­get­ful?

  A dog will change the weather dramatically, which will substantially effect your perceptions.

  In this case, it’s about alt-complexity again. Sure, a dog causes a specific weather-pattern change. But could this specific weather-pattern change have been caused only by this specific dog? Perhaps if we edit the universe to erase this dog, but add a cat and a bird five kilometers away, the chaotic weather dynamic would play out the same way? Then, from your perceptions’ perspective, you wouldn’t be able to distinguish between a dog timeline and a cat-and-bird timeline.

  In some sense, this is common-sensical. The mapping from reality’s low-level state to your perceptions is non-injective: the low-level state contains more information than you perceive on a moment-to-moment basis. Therefore, for any observation-state, there are several low-level states consistent with it. Scaling up: for any observed lifetime, there are several low-level histories consistent with it.

• Thane Ruthenis 25 Nov 2024 1:50 UTC

  LW: 5 AF: 4

  0

  AF

  in reply to: Charlie Steiner’s comment on: Are You More Real If You’re Really For­get­ful?

  Sure. This setup couldn’t really be exploited for optimizing the universe. If we assume that the self-selection assumption is a reasonable assumption to make, inducing amnesia doesn’t actually improve outcomes across possible worlds. One out of 100 prisoners still dies.

  It can’t even be considered “re-rolling the dice” on whether the specific prisoner that you are dies. Under the SSA, there’s no such thing as a “specific prisoner”, “you” are implemented as all 100 prisoners simultaneously, and so regardless of whether you choose to erase your memory or not, ¹⁄₁₀₀ of your measure is still destroyed. Without SSA, on the other hand, if we consider each prisoner’s perspective to be distinct, erasing memory indeed does nothing: it doesn’t return your perspective to the common pool of prisoner-perspectives, so if “you” were going to get shot, “you” are still going to get shot.

  I’m not super interested in that part, though. What I’m interested in is whether there are in fact 100 clones of me: whether, under the SSA, “microscopically different” prisoners could be meaningfully considered a single “high-level” prisoner.

[Question] Are You More Real If You’re Really For­get­ful?

• Thane Ruthenis 22 Nov 2024 15:25 UTC

  8 points

  6

  in reply to: Bogdan Ionut Cirstea’s comment on: Bog­dan Ionut Cirstea’s Shortform

  Agreed. I think a type of “stop AGI research” argument that’s under-deployed is that there’s no process or actor in the world that society would trust with unilateral godlike power. At large, people don’t trust their own governments, don’t trust foreign governments, don’t trust international organizations, and don’t trust corporations or their CEOs. Therefore, preventing anyone from building ASI anywhere is the only thing we can all agree on.

  I expect this would be much more effective messaging with some demographics, compared to even very down-to-earth arguments about loss of control. For one, it doesn’t need to dismiss the very legitimate fear that the AGI would be aligned to values that a given person would consider monstrous. (Unlike “stop thinking about it, we can’t align it to any values!”.)

  And it is, of course, true.

• Thane Ruthenis 18 Nov 2024 17:56 UTC

  8 points

  4

  in reply to: Seth Herd’s comment on: OpenAI Email Archives (from Musk v. Alt­man)

  That’s probably not what Page meant. On consideration, he would probably have clarified that AI that includes what we value about humanity would be a worthy successor. He probably wasn’t even clear on his own philosophy at the time.

  I don’t see reasons to be so confident in this optimism. If I recall correctly, Robin Hanson explicitly believes that putting any constraints on future forms of life, including on its values, is undesirable/​bad/​regressive, even though lack of such constraints would eventually lead to a future with no trace of humanity left. Similar for Beef Jezos and other hardcore e/​acc: they believe that a worthy future involves making a number go up, a number that corresponds to some abstract quantity like “entropy” or “complexity of life” or something, and that if making it go up involves humanity going extinct, too bad for humanity.

  Which is to say: there are existence proofs that people with such beliefs can exist, and can retain these beliefs across many years and in the face of what’s currently happening.

  I can readily believe that Larry Page is also like this.

• Thane Ruthenis 16 Nov 2024 16:08 UTC

  35 points

  14

  in reply to: Akash’s comment on: OpenAI Email Archives (from Musk v. Alt­man)

  Also this:

  From Altman: [...] Admitted that he lost a lot of trust with Greg and Ilya through this process. Felt their messaging was inconsistent and felt childish at times. [...] Sam was bothered by how much Greg and Ilya keep the whole team in the loop with happenings as the process unfolded. Felt like it distracted the team.

  Apparently airing such concerns is “childish” and should only be done behind closed doors, otherwise it “distracts the team”, hm.

• Thane Ruthenis 7 Nov 2024 9:07 UTC

  2 points

  0

  in reply to: Johannes C. Mayer’s comment on: Goal: Un­der­stand Intelligence

  Perhaps if you did have the full solution, but it feels like that there are some things of a solution that you could figure out, such that that part of the solution doesn’t tell you as much about the other parts of the solution.

  I agree with that.

• Thane Ruthenis 5 Nov 2024 0:37 UTC

  4 points

  0

  in reply to: Johannes C. Mayer’s comment on: Goal: Un­der­stand Intelligence

  I’d think you can define a tedrahedron for non-euclidean space

  If you relax the definition of a tetrahedron to cover figures embedded in non-Euclidean spaces, sure. It wouldn’t be the exact same concept, however. In a similar way to how “a number” is different if you define it as a natural number vs. real number.

  Perhaps more intuitively, then: the notion of a geometric figure with specific properties is dependent on the notion of a space in which it is embedded. (You can relax it further – e. g., arguably, you can define a “tetrahedron” for any set with a distance function over it – but the general point stands, I think.)

  Just consider if you take the assumption that the system would not change in arbitrary ways in response to it’s environment. There might be certain constrains. You can think about what the constrains need to be such that e.g. a self modifying agent would never change itself such that it would expect that in the future it would get less utility than if it would not selfmodify.

  Yes, but: those constraints are precisely the principles you’d need to code into your AI to give it general-intelligence capabilities. If your notion of alignment only needs to be robust to certain classes of changes, because you’ve figured out that an efficient generally intelligent system would only change in such-and-such ways, then you’ve figured out a property of how generally intelligent systems ought to work – and therefore, something about how to implement one.

  Speaking abstractly, the “negative image” of the theory of alignment is precisely the theory of generally intelligent embedded agents. A robust alignment scheme would likely be trivial to transform into an AGI recipe.

• Thane Ruthenis 4 Nov 2024 23:54 UTC

  4 points

  1

  in reply to: Johannes C. Mayer’s comment on: Goal: Un­der­stand Intelligence

  I am pretty sure you can figure out alignment in advance as you suggest

  I’m not so sure about that. How do you figure out how to robustly keep a generally intelligent dynamically updating system on-target without having a solid model of how that system is going to change in response to its environment? Which, in turn, would require a model of what that system is?

  I expect the formal definition of “alignment” to be directly dependent on the formal framework of intelligence and embedded agency, the same way a tetrahedron could only be formally defined within the context of Euclidean space.

• Thane Ruthenis 4 Nov 2024 21:59 UTC

  21 points

  8

  on: Ex­plore More: A Bag of Tricks to Keep Your Life on the Rails

  Exploit your natural motivations

  There’s a relevant concept that I keep meaning to write about, which I could summarize as: create gradients towards your long-term aspirations.

  Humans are general intelligences, and one of the core properties of general intelligence is not being a greedy-optimization algorithm:

  • We can pursue long-term goals even when each individual step towards them is not pleasurable-in-itself (such as suffering through university to get a degree in a field jobs in which require it).

  • We can force ourselves out of local maxima (such as quitting a job you hate and changing careers, even though it’d mean a period of life filled with uncertainty and anxieties).

  • We can build world-models, use them to infer the shapes of our value functions, and plot a path towards their global maximum, even if it requires passing through negative-reward regions (such as engaging in self-reflection and exploration, then figuring out which vocation would be most suitable to a person-like-you).

  However, it’s hard. We’re hybrid systems, combining generally-intelligent planning modules with greedy RL circuitry. The greedy RL circuitry holds a lot of sway. If you keep forcing yourself to do something it assigns negative rewards to, it’s going to update your plan-making modules until they stop doing that.

  It is much, much easier to keep doing something if every instance of it is pleasurable in itself. If the reward is instead sparse and infrequent, you’d need a lot of “willpower” to keep going (to counteract the negative updates), and accumulating that is a hard problem in itself.

  So the natural solution is to plot, or create, a path towards the long-term aspiration such that motion along it would involve receiving immediate positive feedback from your learned and innate reward functions.

  A lot of productivity advice reduces to it:

  • Breaking the long-term task into yearly, monthly, and daily subgoals, such that you can feel accomplishment on a frequent basis (instead of only at the end).

  • Using “cross-domain success loops”: simultaneously work on several projects, such that you accomplish something worthwhile along at least one of those tracks frequently, and can then harness the momentum from the success along one track into the motivation for continuing the work along other tracks.

    • I. e., sort of trick your reward system into confusing where exactly the reward is coming from.

    • (I think there was an LW post about this, but I don’t remember how to find it.)

  • Eating something tasty, or going to a party, or otherwise “indulging” yourself, every time you do something that contributes to your long-term aspiration.

  • Finding ways to make the process at least somewhat enjoyable, through e. g. environmental factors, such as working in a pleasant place, putting on music, using tools that feel satisfying to use, or doing small work-related rituals that you find amusing.

  • Creating social rewards and punishments, such as:

    • Joining a community focused on pursuing the same aspiration as you.

    • Finding “workout buddies”.

    • Having friends who’d hold you accountable if you slack off.

    • Having friends who’d cheer you on if you succeed.

  • And, as in Shoshannah’s post: searching for activities that are innately enjoyable and happen to move you in the direction of your aspirations.

  None of the specific examples here are likely to work for you (they didn’t for me). But you might be able to design or find an instance of that general trick that fits you!

  (Or maybe not. Sometimes you have to grit your teeth and go through a rewardless stretch of landscape, if you’re not willing to budge on your goal/​aspiration.)

  ---

  Other relevant posts:

  • Venkatesh Rao’s The Calculus of Grit. It argues for ignoring extrinsic “disciplinary boundaries” (professions, fields) when choosing your long-term aspirations, and instead following an “internal” navigation system when mapping out the shape of the kind-of-thing that someone-like-you is well-suited to doing.

    • Note that this advice goes further than Shoshannah’s: in this case, you don’t exert any (conscious) control even over the direction you’d like to go, much less your “goal”.

    • It’s likely to be easier, but the trade-off should be clear.

  • John Wentworth’s Plans Are Predictions, Not Optimization Targets. This connection is a bit more rough, but: that post can be generalized to note that any explicit life goals you set for yourself should often be treated as predictions about what goal you should pursue. Recognizing that, you might instead choose to “pursue your goal-in-expectation”, which might be similar to Shoshannah’s point about “picking a direction, not a goal”.

• Thane Ruthenis 4 Nov 2024 11:53 UTC

  5 points

  5

  in reply to: johnswentworth’s comment on: The Me­dian Re­searcher Problem

  Your use of “memetic” here did struck me as somewhat idiosyncratic; I had to infer it. I would have used “memetically viral” and derivatives in its place. (E. g., in place of “lots of work in that field will be highly memetic despite trash statistics”, I would’ve said “lots of ideas in that field will be highly viral despite originating from research with trash statistics” or something.)

• Thane Ruthenis 2 Nov 2024 9:56 UTC

  4 points

  2

  in reply to: habryka’s comment on: The Com­pendium, A full ar­gu­ment about ex­tinc­tion risk from AGI

  Sure. But:

  and then they can do a bunch of the work of generalizing

  This is the step which is best made unnecessary if you’re crafting a message for a broad audience, I feel.

  Most people are not going to be motivated to put this work in. Why would they? They get bombarded with a hundred credible-ish messages claiming high-importance content on a weekly basis. They don’t have the time nor stamina to do a deep dive into each of them.

  Which means any given subculture would generate its own “inferential bridge” between itself and your message, artefacts that do this work for the median member (consisting of reviews by any prominent subculture members, the takes that go viral, the entire shape of the discourse around the topic, etc.). The more work is needed, the longer these inferential bridges will be. The longer they are, the bigger the opportunity to willfully or accidentally mistranslate your message.

  Like I said, it doesn’t seem wise or even fair to your potential audience, to act as if those dynamics don’t take place. As if the only people that deserve consideration are those that would put in the work themselves (despite the fact it may be a locally suboptimal way to distribute resources under their current world-model), and everyone else are lost causes.

• Thane Ruthenis 1 Nov 2024 22:02 UTC

  9 points

  7

  in reply to: habryka’s comment on: The Com­pendium, A full ar­gu­ment about ex­tinc­tion risk from AGI

  I agree that inventing new arguments for X that sound kind-of plausible to you on the surface level, and which you imagine would work well on a given demographic, is not a recipe for good communication. Such arguments are “artificial”, they’re not native citizens of someone’s internally consistent world-model, and it’s going to show and lead to unconvincing messages that fall apart under minimal scrutiny.

  That’s not what I’m arguing for. The case for the AGI risk is overdetermined: there are enough true arguments for it that you can remove a subset of them and still end up with an internally consistent world-model in which the AGI risk is real. Arguably, there’s even a set of correct arguments that convinces a Creationist, without making them not-a-Creationist in the process.

  Convincing messaging towards Creationists involves instantiating a world-model in which only the subset of arguments Creationists would believe exist, and then (earnestly) arguing from within that world-model.

  Edit: Like, here’s a sanity-check: suppose you must convince a specific Creationist that the AGI Risk is real. Do you need to argue them out of Creationism in order to do so?

• Thane Ruthenis 1 Nov 2024 21:51 UTC

  7 points

  3

  in reply to: habryka’s comment on: The Com­pendium, A full ar­gu­ment about ex­tinc­tion risk from AGI

  Perhaps. Admittedly, I don’t have a solid model of whether a median American claiming to be a Creationist in surveys would instantly dismiss a message if it starts making arguments from evolution.

  Still, I think the general point applies:

  • A convincing case for the AGI Omnicide Risk doesn’t have to include arguments from human evolution.

  • Arguments from human evolution may trigger some people to instinctively dismiss the entire message.

  • If the fraction of such people is large enough, it makes sense to have public AI-Risk messages that avoid evolution-based arguments when making their case.

• Thane Ruthenis 1 Nov 2024 21:36 UTC

  26 points

  13

  in reply to: habryka’s comment on: The Com­pendium, A full ar­gu­ment about ex­tinc­tion risk from AGI

  I think it’s perfectly sensible to constrain yourself to only make arguments based on true premises, and then optimize your message for convincingness under this constraint. Indeed, I would argue it’s the correct way to do public messaging.

  It’s not even at odds with “aim to explain, not persuade”. When explaining, you should be aiming to make your explanations clear to your audience. If your audience will predictably misunderstand arguments of a certain form, due to e. g. political poison, you should mindfully choose arguments that route around the poison, rather than pretending the issue doesn’t exist. Approaches for generating explanations that don’t involve this are approaches that aren’t optimizing the message for their audiences at all, and which therefore aren’t approaches for generating explanations to begin with. They’re equivalent to just writing out your stream of consciousness. Messaging aimed at people you think your audience ought to be, rather than who they are.

  That said, I don’t think you can optimize any one of your messages to be convincing to all possible audiences, or even to the majority of the people you ought to be trying to convince. IMO, there should be several “compendiums”, optimized to be convincing to different large demographics. As an obvious example: a Democrats-targeting one and a Republicans-targeting one.

  Or perhaps this split in particular is a bad idea. Perhaps an explanation that is deliberately optimized to be bipartisan is needed. But if that is the aim, then writing it would still require actively modeling the biases of both parties, and mindfully routing around them – rather than pretending that they don’t exist.

  I feel this is a significant problem with a lot of EA/​R public messaging. The (correct) idea that we should be optimizing our communication for conveying the truth in an epistemically sound way gets (incorrectly) interpreted as a mindset where thinking about the optics and the framing at all is considered verboten. As if, by acting like we live in a world where Simulacrum Levels 3-4 don’t exist, we can actually step into that nice world – rather than getting torn apart by SL3-4 agents after we naively expose square miles of attack surfaces.

  We should “declaw” ourselves: avoid using rhetorical tricks and other “dark arts”. But that doesn’t mean forgetting that everyone else still has claws they’re eager to use. Or, for that matter, that many messages you intend as tonally neutral and purely informative may have the effect of a rhetorical attack, when deployed in our sociocultural context.

  Constantly keeping the political/​cultural context in mind as you’re phrasing your messages is a vital part of engaging in high-epistemic-standards communication, rather than something that detracts from it.

• Thane Ruthenis 1 Nov 2024 13:28 UTC

  9 points

  3

  in reply to: Meme Marine’s comment on: The Com­pendium, A full ar­gu­ment about ex­tinc­tion risk from AGI

  Strong-upvoted, this is precisely the kind of feedback that seems helpful for making the document better.

Towards the Oper­a­tional­iza­tion of Philos­o­phy & Wisdom

• Thane Ruthenis 28 Oct 2024 3:19 UTC

  13 points

  2

  in reply to: johnswentworth’s comment on: john­swent­worth’s Shortform

  There’s a general-purpose trick I’ve found that should, in theory, be applicable in this context as well, although I haven’t mastered that trick myself yet.

  Essentially: when you find yourself in any given cognitive context, there’s almost surely something “visible” from this context such that understanding/​mastering/​paying attention to that something would be valuable and interesting.

  For example, suppose you’re reading a boring, nonsensical continental-philosophy paper. You can:

  • Ignore the object-level claims and instead try to reverse-engineer what must go wrong in human cognition, in response to what stimuli, to arrive at ontologies that have so little to do with reality.

  • Start actively building/​updating a model of the sociocultural dynamics that incentivize people to engage in this style of philosophy. What can you learn about mechanism design from that? It presumably sheds light on how to align people towards pursuing arbitrary goals, or how to prevent this happening...

  • Pay attention to your own cognition. How exactly are you mapping the semantic content of the paper to an abstract model of what the author means, or to the sociocultural conditions that created this paper? How do these cognitive tricks generalize? If you find a particularly clever way to infer something form the text, check: would your cognitive policy automatically deploy this trick in all context where it’d be useful, or do you need to manually build a TAP for that?

  • Study what passages make the feelings of boredom or frustration spike. What does that tell you about how your intuitions/​heuristics work? Could you extract any generalizable principles out of that? For example, if a given sentence particularly annoys you, perhaps it’s because it features a particularly flawed logical structure, and it’d be valuable to learn to spot subtler instances of such logical flaws “in the wild”.

  The experience of reading the paper’s text almost certainly provides some data uniquely relevant to some valuable questions, data you legitimately can’t source any other way. (In the above examples: sure you can learn more efficiently about the author’s cognition or the sociocultural conditions by reading some biographies or field overviews. But (1) this wouldn’t give you the meta-cognitive data about how you can improve your inference functions for mapping low-level data to high-level properties, (2) those higher-level summaries would necessarily be lossy, and give you a more impoverished picture than what you’d get from boots-on-the-ground observations.)

  Similar applies to:

  • Listening to boring lectures. (For example, you can pay intense attention to the lecturer’s body language, or any tricks or flaws in their presentation.)

  • Doing a physical/​menial task. (Could you build, on the fly, a simple model of the physics (or logistics) governing what you’re doing, and refine it using some simple experiments? Then check afterwards if you got it right. Or: If you were a prehistoric human with no idea what “physics” is, how could you naturally arrive at these ideas from doing such tasks/​making such observations? What does that teach you about inventing new ideas in general?)

  • Doing chores. (Which parts of the process can you optimize/​streamline? What physical/​biological conditions make those chores necessary? Could you find a new useful takeaway from the same chore every day, and if not, why?)

  Et cetera.

  There’s a specific mental motion I associate with using this trick, which involves pausing and “feeling out” the context currently loaded in my working memory, looking at it from multiple angles, trying to see anything interesting or usefully generalizable.

  In theory, this trick should easily apply to small-talk as well. There has to be something you can learn to track in your mind, as you’re doing small-talk, that would be useful or interesting to you.

  One important constraint here is that whatever it is, it has to be such that your outwards demeanour would be that of someone who is enjoying talking to your interlocutor. If the interesting thing you’re getting out of the conversation is so meta/​abstract you end up paying most of the attention to your own cognitive processes, not on what the interlocutor is saying, you’ll have failed at actually doing the small-talk. (Similarly, if, when doing a menial task, you end up nerd-sniped by building a physical model of the task, you’ll have failed at actually doing the task.)

  You also don’t want to come across as sociopathic, so making a “game” of it where you’re challenging yourself to socially engineer the interlocutor into something is, uh, not a great idea.

  The other usual advice for finding ways to enjoy small-talk are mostly specialized instances of the above idea that work for specific people. Steering the small-talk to gradient-descend towards finding emotional common ground, ignoring the object-level words being exchanged and build a social model of the interlocutor, doing a live study of the social construct of “small-talk” by playing around with it, etc.

  You’ll probably need to find an instance of the trick that works for your cognition specifically, and it’s also possible the optimization problem is overconstrained in your case. Still, there might be something workable.

• Thane Ruthenis 24 Oct 2024 20:39 UTC

  17 points

  6

  on: Claude Son­net 3.5.1 and Haiku 3.5

  That seems… good? It seems to be a purely mundane-utility capability improvement. It doesn’t improve on the architecture of the base LLM, and a base LLM that would be omnicide-capable wouldn’t need the AGI labs to hold its hand in order to learn how to use the computer. It seems barely different from AutoGPT, or the integration of Gemini into Android, and is about as existentially dangerous as advances in robotics.

  The only new dangers it presents are mundane ones, which are likely to be satisfactorily handled by mundane mechanisms.

  It’s bad inasmuch as this increases the attention to AI, attracts money to this sector, and increases competitive dynamics. But by itself, it seems fine. If all AI progress from this point on consisted of the labs racing in this direction, increasing the integration and the reliability of LLMs, this would all be perfectly fine and good.

  I say Anthropic did nothing wrong in this one instance.

  Anthropic did note that this advance ‘brings with it safety challenges.’ They focused their attentions on present-day potential harms, on the theory that this does not fundamentally alter the skills of the underlying model, which remains ASL-2 including its computer use. And they propose that introducing this capability now, while the worst case scenarios are not so bad, we can learn what we’re in store for later, and figure out what improvements would make computer use dangerous.

  A safety take from a major AGI lab that actually makes sense? This is unprecedented. Must be a sign of the apocalypse.