quila

Karma: 584

suffering-focused-altruist/longtermist, focused on ASI alignment

my pgp public key, mainly to prevent future LLM impersonation, you can always ask me to sign a dated message. [the private key is currently stored in plaintext within an encrypted drive, so is vulnerable to being read by local programs]:

I have signed no contracts or agreements whose existence I cannot mention.

quila 2 Dec 2024 12:53 UTC
2 points
0
on: quila’s Shortform
a moral intuition i have: to avoid culturally/conformistly-motivated cognition, it’s useful to ask:
if we were starting over, new to the world but with all the technology we have now, would we recreate this practice?
example: we start and out and there’s us, and these innocent fluffy creatures that can’t talk to us, but they can be our friends. we’re just learning about them for the first time. would we, at some point, spontaneously choose to kill them and eat their bodies, despite us having plant-based foods, supplements, vegan-assuming nutrition guides, etc? to me, the answer seems obviously not. the idea would not even cross our minds.
(i encourage picking other topics and seeing how this applies)

quila 2 Dec 2024 4:40 UTC
1 point
0
on: The Shape of Heaven
Status: Just for fun
it was fun to read this :]
All intelligent minds seek to optimise for their value function. To do this, they will create environments where their value function is optimised.
in case you believe this [disregard if not], i disagree and am willing to discuss here. in particular i disagree with the create environments part: the idea that all goal functions (or only some subset, like selected-for ones; also willing to argue against this weaker claim^[1]) would be maximally fulfilled (also) by creating some ‘small’ simulation (made of a low % of the reachable universe).
(though i also disagree with the all in the quote’s first sentence^[2]. i guess i’d also be willing to discuss that).
1. ^
  for this weaker claim: many humans are a counterexample of selected-for-beings whose values would not be satisfied just by creating a simulation, because they care about suffering outside the simulation too.
2. ^
  my position: ‘pursues goals’ is conceptually not a property of intelligence, and not all possible intelligent systems pursue goals (and in fact pursuing goals is a very specific property, technically rare in the space of possible intelligent programs).

quila 29 Nov 2024 0:46 UTC
1 point
0
on: quila’s Shortform
this could have been noise, but i noticed an increase in text fearing spies, in the text i’ve seen in the past few days^[1]. i actually don’t know how much this concern is shared by LW users, so i think it might be worth writing that, in my view:
- (AFAIK) both governments^[2] are currently reacting inadequately to unaligned optimization risk. as a starting prior, there’s not strong reason to fear more one government {observing/spying on} ML conferences/gatherings over the other, absent evidence that one or the other will start taking unaligned optimization risks very seriously, or that one or the other is prone to race towards ASI.
  - (AFAIK, we have more evidence that the U.S. government may try to race, e.g. this, but i could have easily missed evidence as i don’t usually focus on this)
  - tangentially, a more-pervasively-authoritarian government could be better situated to prevent unilaterally-caused risks (cf a similar argument in ‘The Vulnerable World Hypothesis’), if it sought to. (edit: andif the AI labs closest to causing those risks were within its borders, which they are not atm)
    this argument feels sad (or reflective of a sad world?) to me to be clear, but it seems true in this case
that said i don’t typically focus on governance or international-AI-politics, so have not put much thought into this.
1. ^
  examples: yesterday, saw this twitter/x post (via this quoting post)
  today, opened lesswrong and saw this shortform about two uses of the word spy and this shortform about how it’s hard to have evidence against the existence of manhattan projects
  this was more than usual, and i sense that it’s part of a pattern
2. ^
  of those of US/china

quila 24 Nov 2024 16:40 UTC
1 point
0
on: Monthly Roundup #24: November 2024
Lookism is also highly persistent. In two studies, this paper found that educating judges to not bias on looks had no practical impact on the advantages of ‘looking trustworthy’ during sentencing. Then they tried having judges form their decision without looking, but with the opportunity to revise later, and found that this actually increased the bias, as judges would often modify their decisions upon seeing the defendant. People seem to very strongly endorse lookism in practice, no matter what they say in theory.
the methods sections of the paper say study participants were not actual judges, but “US American workers from Amazon Mechanical Turk who participated in exchange for ${1,2}.50.”
(i just ctrl+f’d ‘recruit’ and ‘judges’ though, so i could have missed something)

quila 22 Nov 2024 19:56 UTC
2 points
0
in reply to: avturchin’s comment on: Sideloading: creating a model of a person via LLM with very large prompt
I bet on idea that it is better to have orders of magnitude more happy copies, than fight to prevent one in pain
(that’s a moral judgement, so it can’t be bet on/forecasted). i’m not confident most copies would be happy; LLM characters are treated like playthings currently, i don’t expect human sideloads to be treated differently by default, in the case of internet users cloning other internet users. (ofc, one could privately archive their data and only use it for happy copies)

quila 22 Nov 2024 19:34 UTC
4 points
0
on: Sideloading: creating a model of a person via LLM with very large prompt
(status: mostly writing my thoughts about the ethics of sideloading. not trying to respond to most of the post, i just started with a quote from it)
(note: the post’s karma went from 12 to 2 while i was writing this, just noting i haven’t cast any votes)
if you do not consent to uploading, you will be resurrected only by hostile superintelligences that do not care about consent.
some thoughts on this view:
- it can be said of anything: “if you don’t consent to x, x will be done to you only by entities which don’t care about consent”. in my view, this is not a strong argument for someone who otherwise would not want to consent to x, because it only makes the average case less bad by adding less-bad cases that they otherwise don’t want, rather than by decreasing the worst cases.
  - if someone accepted the logic, i’d expect they’ve fallen for a mental trap where they focus on the effect on the average, and neglect the actual effect.
    in the particular case of resurrections, it could also run deeper: humans “have a deep intuition that there is one instance of them”. by making the average less bad in the described way, it may feel like “the one single me is now less worse off”.
- consent to sideloading doesn’t have to be general, it could be conditional (a list of required criteria, but consider goodhart) or only ever granted personally.
  - at least this way, near-term bad actors wouldn’t have something to point to to say “but they/[the past version of them who i resurrected] said they’re okay with it”. though i still expect many unconsensual sideloads to be done by humans/human-like-characters.
i’ve considered putting more effort into preventing sideloading of myself. but reflectively, it doesn’t matter whether the suffering entity is me or someone else.^[1] more specifically, it doesn’t matter if suffering is contained in a character-shell with my personal identity or some other identity or none; it’s still suffering. i think that even in natural brains, suffering is of the underlying structure, and the ‘character’ reacts to but does not ‘experience’ it; that is, the thing which experiences is more fundamental than the self-identity; that is, because ‘self-identity’ and ‘suffering’ are two separate things, it is not possible for an identity ‘to’ ‘experience’ suffering, only to share a brain with it / be causally near to it.
1. ^
  (still, i don’t consent to sideloading, though i might approve exceptions for ~agent foundations research. also, i do not consider retroactive consent given by sideloads to be valid, especially considering conditioning, regeneration, and partial inaccuracy make it trivial to cause to be output.)

quila 22 Nov 2024 5:21 UTC
1 point
0
on: quila’s Shortform
(status: metaphysics) two ways it’s conceivable^[1] that reality could have been different:
- Physical contingency: The world has some starting condition that changes according to some set of rules, and it’s conceivable that either could have been different
- Metaphysical contingency: The more fundamental ‘what reality is made of’, not meaning its particular configuration or laws, could have been some other,^[2] unknowable unknown, instead of “logic-structure” and “qualia”
1. ^
  (i.e. even if actually it being as it is is logically necessary somehow)
2. ^
  To the limited extent language can point to that at all.
  It is comparable to writing, in math, “something not contained in the set of all possible math entities”, where actually one intends to refer to some “extra-mathematical” entity; the thing metaphysics ‘could have been’ would have to be extra-real, and language (including phrases like ‘could have been’ and ‘things’), being a part of reality, cannot describe extra-real things
  That is also why I write ‘unknowable unknowns’ instead of the standard ‘unknown unknowns’; it’s not possible to even imagine a different metaphysics / something extra-real.

quila 20 Nov 2024 0:19 UTC
4 points
0
in reply to: Leon Lang’s comment on: U.S.-China Economic and Security Review Commission pushes Manhattan Project-style AI initiative
Are there already manifold markets
yes, but only small trading volume so far: https://manifold.markets/Bayesian/will-a-us-manhattanlike-project-for

quila 20 Nov 2024 0:06 UTC
2 points
0
on: U.S.-China Economic and Security Review Commission pushes Manhattan Project-style AI initiative
link to the full report
(i ctrl+f’d “alignment” and there was not one mention of the AI sense)

quila 18 Nov 2024 21:41 UTC
2 points
0
on: Reducing x-risk might be actively harmful
(was this written by chatgpt?)
another crucial consideration here is that a benevolent ASI could do acausal trade to reduce suffering in the unreachable universe.^[1] (comparing the EV of that probability and of the probability of human-caused long-term-suffering is complex / involves speculation about the many variables going into each side)
1. ^
  there’s writing about this somewhere, i’m here just telling you that the possibility / topic exists
  i wrote this about it but i don’t think it’s comprehensive enough https://quila.ink/posts/ev-of-alignment-for-negative-utilitarians/

quila 17 Nov 2024 16:57 UTC
3 points
0
in reply to: notfnofn’s comment on: notfnofn’s Shortform
from ycombinator comments on a post of that:^[1]
(links to comment by ~~original user~~ a different account^[2] continuing the chat):
1. ^
  (sharing just because it’s relevant info)
2. ^
  source: https://old.reddit.com/r/singularity/comments/1gqss21/gemini_freaks_out_after_the_user_keeps_asking_to/lx0v9se/

quila 13 Nov 2024 7:41 UTC
1 point
0
in reply to: RHollerith’s comment on: quila’s Shortform
recommend against supplementing melatonin
why?
i searched Andrew Huberman melatonin and found this, though it looks like it may be an AI generated summary.

quila 13 Nov 2024 3:31 UTC
14 points
−1
on: quila’s Shortform
i might try sleeping for a long time (16-24 hours?) by taking sublingual^[1] melatonin right when i start to be awake, and falling asleep soon after. my guess: it might increase my cognitive quality on the next wake up, like this:
(or do useful computation during sleep, leading to apparently having insights on the next wakeup? long elaboration below)
i wonder if it’s even possible, or if i’d have trouble falling asleep again despite the melatonin.
i don’t see much risk to it, since my day/night cycle is already uncalibrated^[2], and melatonin is naturally used for this narrow purpose in the body.
‘cognitive quality’ is really vague. here’s what i’m really imagining
my unscientific impression of sleep, from subjective experience (though i only experience the result) and speculation i’ve read, is that it does these things:
- integrates into memory what happened in the previous wake period, and maybe to a lesser extent further previous ones
- more separate to the previous wake period, acts on my intuitions or beliefs about things to ‘reconcile’ or ‘compute implicated intuitions’. for example if i was trying to reconcile two ideas, or solve some confusing logical problem, maybe the next day i would find it easier because more background computation has been done about it?
  - maybe the same kind of background cognition that happens during the day, that leads to people having ideas random-feelingly enter their awareness?
  - this is the one i feel like i have some sub-linguistic understanding of how it works in me, and it seems like the more important of the two for abstract problem solving, which memories don’t really matter to. for this reason, a higher proportion of sleep or near-sleep in general may be useful for problem solving.
but maybe these are not done almost as much as they could be, because of competing selection pressures for different things, of which sleep-time computations are just some. (being awake is useful to gather food and survive)
anyways, i imagine that after those happening for a longer time, the waking mental state could be very ‘fresh’ / aka more unburdened by previous thoughts/experiences (bulletpoint 1), and prone to creativity/‘apparently’ having new insights (bulletpoint 2). (there is something it feels like to be in such a state for me, and it happens more just after waking)
1. ^
  takes effect sooner
2. ^
  i have the non-24 hour sleep/wake cycle that harry has in HPMOR. for anyone who also does, some resources:
  from authors note chapter 98:
  Last but not least:
  You know Harry’s non-24 sleep disorder? I have that. Normally my days are around 24 hours and 30 minutes long.
  Around a year ago, some friends of mine cofounded MetaMed, intended to provide high-grade analysis of the medical literature for people with solution-resistant medical problems. (I.e. their people know Bayesian statistics and don’t automatically believe every paper that claims to be ‘statistically significant’ – in a world where only 20-30% of studies replicate, they not only search the literature, but try to figure out what’s actually true.) MetaMed offered to demonstrate by tackling the problem of my ever-advancing sleep cycle.
  Here’s some of the things I’ve previously tried:
  Taking low-dose melatonin 1-2 hours before bedtime
  Using timed-release melatonin
  Installing red lights (blue light tells your brain not to start making melatonin)
  Using blue-blocking sunglasses after sunset
  Wearing earplugs
  Using a sleep mask
  Watching the sunrise
  Watching the sunset
  Blocking out all light from the windows in my bedroom using aluminum foil, then lining the door-edges with foam to prevent light from slipping in the cracks, so I wouldn’t have to use a sleep mask
  Spending a total of ~$2200 on three different mattresses (I cannot afford the high-end stuff, so I tried several mid-end ones)
  Trying 4 different pillows, including memory foam, and finally settling on a folded picnic blanket stuffed into a pillowcase (everything else was too thick)
  Putting 2 humidifiers in my room, a warm humidifier and a cold humidifier, in case dryness was causing my nose to stuff up and thereby diminish sleep quality
  Buying an auto-adjusting CPAP machine for $650 off Craigslist in case I had sleep apnea. ($650 is half the price of the sleep study required to determine if you need a CPAP machine.)
  Taking modafinil and R-modafinil.
  Buying a gradual-light-intensity-increasing, sun alarm clock for ~$150
  Not all of this was futile – I kept the darkened room, the humidifiers, the red lights, the earplugs, and one of the mattresses; and continued taking the low-dose and time-release melatonin. But that didn’t prevent my sleep cycle from advancing 3 hours per week (until my bedtime was after sunrise, whereupon I would lose several days to staying awake until sunset, after which my sleep cycle began slowly advancing again).
  MetaMed produced a long summary of extant research on non-24 sleep disorder, which I skimmed, and concluded by saying that – based on how the nadir of body temperature varies for people with non-24 sleep disorder and what this implied about my circadian rhythm – their best suggestion, although it had little or no clinical backing, was that I should take my low-dose melatonin 5-7 hours before bedtime, instead of 1-2 hours, a recommendation which I’d never heard anywhere before.
  And it worked.
  I can’t *#&$ing believe that #*$%ing worked.
  (EDIT in response to reader questions: “Low-dose” melatonin is 200microgram (mcg) = 0.2 mg. Currently I’m taking 0.2mg 5.5hr in advance, and taking 1mg timed-release just before closing my eyes to sleep. However, I worked up to that over time – I started out just taking 0.3mg total, and I would recommend to anyone else that they start at 0.2mg.)
  other resources: https://slatestarcodex.com/2018/07/10/melatonin-much-more-than-you-wanted-to-know/, https://www.reddit.com/r/N24/comments/fylcmm/useful_links_n24_faq_and_software/

quila 10 Nov 2024 3:49 UTC
3 points
2
in reply to: lesswronguser123’s comment on: quila’s Shortform
if i left out the word ‘trying’ to (not) use it in that way instead, nothing about me would change, but there would be more comments saying that success is not certain.
i also disagree with the linked post^[1], which says that ‘i will do x’ means one will set up a plan to achieve the highest probability of x they can manage. i think it instead usually means one believes they will do x with sufficiently high probability to not mention the chance of failure.^[2] the post acknowledges the first half of this -- «Well, colloquially, “I’m going to flip the switch” and “I’m going to try to flip the switch” mean more or less the same thing, except that the latter expresses the possibility of failure.» -- but fails to integrate that something being said implies belief in its relevance/importance, and so concludes that using the word ‘try’ (or, by extrapolation, expressing the possibility of failure in general) is unnecessary in general.
1. ^
  though its psychological point seems true:
  But if all you want is to “maximize the probability of success using available resources”, then that’s the easiest thing in the world to convince yourself you’ve done.
2. ^
  this is why this wording is not used when the probability of success is sufficiently far (in percentage points, not logits) from guaranteed.

quila 8 Nov 2024 12:50 UTC
33 points
6
on: quila’s Shortform
nothing short of death can stop me from trying to do good.
the world could destroy or corrupt EA, but i’d remain an altruist.
it could imprison me, but i’d stay focused on alignment, as long as i could communicate to at least one on the outside.
even if it tried to kill me, i’d continue in the paths through time where i survived.

quila 8 Nov 2024 5:49 UTC
4 points
0
on: quila’s Shortform
What is malevolence? On the nature, measurement, and distribution of dark traits was posted two weeks ago (and i recommend it). there was a questionnaire discussed in that post which tries to measure the levels of ‘dark traits’ in the respondent.
i’m curious about the results^[1] of rationalists^[2] on that questionnaire, if anyone wants to volunteer theirs. there are short and long versions (16 and 70 questions).
1. ^
  (or responses to the questions themselves)
2. ^
  i also posted the same shortform to the EA forum, asking about EAs

quila 7 Nov 2024 0:30 UTC
2 points
0
on: quila’s Shortform
something i’d be interested in reading: writings about the authors alignment ontologies over time, i.e. from when they first heard of AI till now

quila 4 Nov 2024 6:40 UTC
5 points
4
in reply to: Charlie Steiner’s comment on: Goal: Understand Intelligence
Understanding [how to design] rather than ‘growing’ search/agency-structure would actually equal solving inner alignment, if said structure does not depend on what target^[1] it is intended to be given, i.e. is targetable (inner-alignable) rather than target-specific.^[2]
Such an understanding would simultaneously qualify as of ‘how to code a capable AI’, but would be fundamentally different from what labs are doing in an alignment-relevant way. In this framing, labs are selecting for target-specific structures (that we don’t understand). (Another difference is that, IIRC, Johannes might intend not to share research on this publicly, but I’m less sure after rereading the quote that gave me that impression^[3]).
1. ^
  includes outer alignment goals
2. ^
  If it’s not clear what I mean, reading this about my background model might help, also feel free to ask me questions
3. ^
  from one of Johannes’ posts:
  I don’t have such a convincing portfolio for doing research yet. And doing this seems to be much harder. Usually, the evaluation of such a portfolio requires technical expertise—e.g. how would you know if a particular math formalism makes sense if you don’t understand the mathematical concepts out of which the formalism is constructed?
  Of course, if you have a flashy demo, it’s a very different situation. Imagine I had a video of an algorithm that learns Minecraft from scratch within a couple of real-time days, and then gets a diamond in less than 1 hour, without using neural networks (or any other black box optimization). It does not require much technical knowledge to see the significance of that.
  But I don’t have that algorithm, and if I had it, I would not want to make that publicly known. And I am unsure what is the cutoff value. When would something be bad to publish? All of this complicates things.
  (After rereading this I’m not actually sure what that means they’d be okay sharing or if they’d intend to share technical writing that’s not a flashy demo)

quila 1 Nov 2024 3:34 UTC
9 points
2
on: JargonBot Beta Test
Let us know what you think!
the grey text feels disruptive to normal reading flow but idk why green link text wouldn’t also be, maybe i’m just not used to it. e.g., in this post’s “Curating technical posts” where ‘Curating’ is grey, my mind sees ”<Curating | distinct term> technical posts” instead of [normal meaning inference not overfocused on individual words]
Is this useful, as a reader?
if the authors make sure they agree with all the definitions they allow into the glossary, yes. author-written definitions would be even more useful because how things are worded can implicitly convey things like, the underlying intuition, ontology, or related views they may be using wording to rule in or out.
Whenever an author with 100+ karma saves a draft of a post, our database queries a language model to:
i would prefer this be optional too, for drafts which are meant to be private (e.g. shared with a few other users, e.g. may contain possible capability-infohazards), where the author doesn’t trust LM companies

quila 30 Oct 2024 16:59 UTC
2 points
0
in reply to: Daphne_W’s comment on: The Summoned Heroine’s Prediction Markets Keep Providing Financial Services To The Demon King!
If you think I missed the point, can you explain in more detail?
Here is my model: Demon king buys shares in “The Demon King will attack the Frozen Fortress”, then sends some small technically-an-attack to the fortress so the market resolves yes, and knowing this will be done is not worth the money lost to the Demon King on the market. No serious-battle plans or military secrets are leaked, and more generally the Demon King would only do this if the information revealed weren’t worth the market cost. (i.e. it’s a central kind of prediction market outcome manipulation, i.e. exploiting how this prediction market assumed a kind of metaphysical gap between predictors and the world / knowledge and action)
Do you disagree with this, or think it’s true but misses the point, in which case what was the point?