quila

Karma: 252

Independent researcher trying to directly solve a form of alignment.

If you disagree with me about alignment for reasons you expect I’m not aware of, please tell me.

If you have/find an idea/chain-of-logic that’s genuinely novel/out-of-human-distribution, you’re welcome to send it to me to ‘introduce chaos into my system’.

Contact: {discord: quilalove, matrix: @quilauwu:matrix.org, email: quila1<at>protonmail.com}

(Most of my writing here is optimized for pure understandability rather than other, secondary things, cause of this)

quila 1 Jun 2024 20:20 UTC
2 points
1
on: Scanning your Brain with 100,000,000,000 wires
(epistemic status: same as the post, I don’t know neuroscience)
I mis-heard the song version and had a different interpretation of something, but it actually seems like a good idea to consider so here it is :).
Remove the assumption that the biological version of oneself needs to survive the process. Then, perform very quick scan of all the neurons on a very short timescale without worrying about the brain being destroyed afterwards, only about accurately scanning all the neurons before that happens. Would this data of the whole brain for a very short time (maybe some fraction of a second) be enough to digitally reconstruct the mind and run it?
Maybe it wouldn’t be because of neuroscience reasons I wouldn’t know, or ‘neural spikes over time’ not being predictable from the very-short-timespan data (again for some reason I wouldn’t know). Also, maybe such a fast scan is not physically possible with current technology (I’d guess that something more efficient than inserting 100 billion wires would be needed).
But if it were possible and feasible, I think it would be worth it, the world’s at stake after all. I’d volunteer.

quila 1 Jun 2024 12:10 UTC
3 points
0
in reply to: Kaj_Sotala’s comment on: quila’s Shortform
thanks for sharing. here’s my thoughts on the possibilities in the quote.
Suffering subroutines—maybe 10-20% likely. i don’t think suffering reduces to “pre-determined response patterns for undesirable situations,” because i can think of simple algorithmic examples of that which don’t seem like suffering.
suffering feels like it’s about the sense of aversion/badness (often in response a situation), and not about the policy “in <situation>, steer towards <new situation>”. (maybe humans were instilled with a policy of steering away from ‘suffering’ states generally, and that’s why evolution made us enter those states in some types of situation?). (though i’m confused about what suffering really is)
i would also give the example of positive-feeling emotions sometimes being narrowly directed. for example, someone can feel ‘excitement/joy’ about a gift or event and want to <go to/participate in> it. sexual and romantic subroutines can also be both narrowly-directed and positive-feeling. though these examples lack the element of a situation being steered away from, vs steering (from e.g any neutral situation) towards other ones.
Suffering simulations—seems likely (75%?) for the estimation of universal attributes, such as the distribution of values. my main uncertainty is about whether there’s some other way for the ASIs to compute that information which is simple enough to be suffering free. this also seems lower magnitude than other classes, because (unless it’s being calculated indefinetely for ever-greater precision) this computation terminates at some point, rather than lasting until heat death (or forever if it turns out that’s avoidable).
Blackmail—i don’t feel knowledgeable enough about decision theory to put a probability on this one, but in the case where it works (or is precommitted to under uncertainty in hopes that it works), it’s unfortunately a case where building aligned ASI would incentive unaligned entities to do it.
Flawed realization—again i’m too uncertain about what real-world paths lead to this, but intuitively, it’s worryingly possible if the future contains LLM-based LTPAs (long term planning agents) intelligent enough to solve alignment and implement their own (possibly simulated) ‘values’.

quila 1 Jun 2024 10:15 UTC
3 points
0
in reply to: Tamsin Leake’s comment on: carado’s Shortform
I’ve replied to/written my current beliefs about this subject here

quila 1 Jun 2024 10:14 UTC
13 points
0
on: quila’s Shortform
i currently believe that working on superintelligence-alignment is likely the correct choice from a fully-negative-utilitarian perspective.^[1]
for others, this may be an intuitive statement or unquestioned premise. for me it is not, and i’d like to state my reasons for believing it, partially as a response to this post concerned about negative utilitarians trying to accelerate progress towards an unaligned-ai-takeover.
there was a period during which i was more uncertain about this question, and avoided openly sharing minimally-dual-use alignment research (but did not try to accelerate progress towards a nonaligned-takeover) while resolving that uncertainty.
a few relevant updates since then:
1. decrease on the probability that the values an aligned AI would have would endorse human-caused moral catastrophes such as human-caused animal suffering.
  
  i did not automatically believe humans to be good-by-default, and wanted to take time to seriously consider what i think should be a default hypothesis-for-consideration upon existing in a society that generally accepts an ongoing mass torture event.
2. awareness of vastly worse possible s-risks.
  
  factory farming is a form of physical torture, by which i mean torture of a mind which is done through the indirect route of effecting its input channels (body/senses). it is also a form of psychological torture. it is very bad, but situations which are magnitudes worse seem possible, where a mind is modulated directly (on the neuronal level) and fully.
  
  compared to ‘in-distribution suffering’ (eg animal suffering, human-social conflicts), i find it further less probable that an AI aligned to some human-specified values^[2] would create a future with this.
  
  i think it’s plausible that it exists rarely in other parts of the world, though, and if so would be important to prevent through acausal trade if we can.
i am not free of uncertainty about the topic, though.
in particular, if disvalue of suffering is common across the world, such that the suffering which can be reduced through acausal trade will be reduced through acausal trade regardless of whether we create an AI which disvalues suffering, then it would no longer be the case that working on alignment is the best decision for a purely negative utilitarian.
despite this uncertainty, my current belief is that the possibility of reducing suffering via acausal trade (including possibly such really-extreme forms of suffering) outweighs the probability and magnitude of human-aligned-AI-caused suffering.^[3]
also, to be clear, if it ever seems that an actualized s-risk takeover event is significantly more probable than it seems now^[4] as a result of unknown future developments, i would fully endorse causing a sooner unaligned-but-not-suffering takeover to prevent it.
1. ^
  i find it easier to write this post as explaining my position as “even for a pure negative utilitarian, i think it’s the correct choice”, because it lets us ignore individual differences in how much moral weight is assigned to suffering relative to everything else.
  i think it’s pretty improbable that i would, on ‘idealized reflection’/CEV, endorse total-negative-utilitarianism (which has been classically pointed out as implying, e.g, preferring a universe with nothing to a universe containing a robust utopia plus an instance of light suffering).
  i self-describe as a “suffering-focused altruist” or “negative-leaning-utilitarian.” ie, suffering seems much worse to me than happiness seems good.
2. ^
  (though certainly there are some individual current humans would do this, for example to digital minds, if given the ability to do so. rather, i’m expressing a belief that it’s very probable that an aligned AI which practically results from this situation would not allow that to happen.)
3. ^
  (by ‘human-aligned AI’, I mean one pointed to an actual CEV of one or a group of humans (which could indirectly imply the ‘CEV of everyone’ but without actually-not-being-that and failing in the below way, and without allowing cruel values of some individuals to enter into it).
  I don’t mean an AI aligned to some sort of ‘current institutional process’, like voting, involving all living humans—I think that should be avoided due to politicization risk and potential for present/unreflective(/by which i mean cruel)-values lock-in.)
4. ^
  there’s some way to formalize with bayes equations how likely, from a negative-utilitarian perspective, an s-risk needs to be (relative to a good outcome) to terminate a timeline.
  it would intake probability distributions related to ‘the frequency of suffering-disvalue across the universal distribution of ASIs’ and ‘the frequency of various forms of s-risks that are preventable with acausal trade’. i might create this formalization later.
  if we think there’s pretty certainly more preventable-through-trade-type suffering-events than there is altruistic ASIs to prevent it, a local preventable-type s-risk might actually need to be ‘more likely than the good/suffering-disvaluing outcome’
What links here?
- quila's comment on Tamsin Leake’s Shortform by Tamsin Leake (1 Jun 2024 10:15 UTC; 3 points)

quila 30 May 2024 22:36 UTC
1 point
0
in reply to: quila’s comment on: quila’s Shortform
Maybe someone has advice for finalizing-writing faster (not at the expense of clarity)? I think I can usually end up with something that’s clear, at least if it’s just a basic point that’s compatible with the reader’s ontology, but it still takes a long time.

quila 30 May 2024 22:04 UTC
3 points
4
in reply to: RedMan’s comment on: Non-Disparagement Canaries for OpenAI
I’m a different person but I would support contracts which disallow spread of capabilities insights, but not contracts which disallow criticism of AI orgs (and especially not surprise ones).
IIUC the latter is what what the OAI-NonDisparagement-controversy has been about.
I’m not confident the following is true, but it seems to me that your first question was written under a belief that the controversy was about both of those at once. It seems like it was trying (under that world model) to ‘axiomatically’ elicit a belief in disagreement with an ongoing controversy, which would be non-truthseeking.

quila 30 May 2024 8:11 UTC
21 points
10
on: Value Claims (In Particular) Are Usually Bullshit
By “parasitic memes” I mean memes whose main function is to copy themselves—as opposed to, say, actually provide value to a human in some way (so that the human then passes it on). [...] And I noticed that the papers, blog posts, commenters, etc, who were most full of shit were ~always exactly the ones which answered that question [“what diet/exercise/supplements/lifestyle changes should I make to stay healthier?”]
“x is healthy” is a factual claim. Those papers/blog posts/etc, if true, would “actually provide value to a human in some way,” but are false by your account.
That falseness would also be hard for most to verify, because such claims are supposed to come from a specialized understanding. This gives them low entanglement with other beliefs in the audience’s world model, which is the property you note (in ‘idea 1’) that value claims have.
I point this out to help locate what your heuristic is really approximating^[1]. I.e., two components of something like memetic fitness: (1) a reason to care, (2) low entanglement with other beliefs.
1. ^
  ~~and as wording practice~~

quila 29 May 2024 7:12 UTC
5 points
0
on: quila’s Shortform
From an OpenAI technical staff member who is also a prolific twitter user ‘roon’:
the timelines are now so short that public prediction feels like leaking rather than scifi speculation
An official OpenAI post has also confirmed:
OpenAI has recently begun training its next frontier model and we anticipate the resulting systems to bring us to the next level of capabilities on our path to AGI

quila 27 May 2024 20:06 UTC
3 points
9
on: Real Life Sort by Controversial
These don’t seem to be scissor statements to me, since most of what could make them provocative is the particular wordings used^[1] and not very much the position itself.
Some of the examples are at least about topics that are currently very controversial in normal society, but I still feel this fails to meet the tier of the psychological effect in my memory of the original “Sort By Controversial” fiction:
If you just read a Scissor statement off a list, it’s harmless. It just seems like a trivially true or trivially false thing.^[2] It doesn’t activate until you start discussing it with somebody.
You asked:
Did you find yourself drawn to any of the controversial statements?
I didn’t, though I only skimmed through the list. If others did, and didn’t know this was possible or anticipate it, then the conclusion (“we should epistemically lower our trust in published media from here onwards”) has some truth for them.
If I was really trying to create virality or controversy. I could spend an hour or two on a single topic, encoding the unique polarity as I see it
But would current LLMs be of much help to you in this, such that you do it better than other humans do it?
1. ^
  E.g., ‘recklessly endangering’, ‘intellectually lazy’.
2. ^
  Disagreement about the nature of qualia comes to mind as meeting the first half of the description—seeming trivially true or trivially false to different people—but fails to meet the second half of evoking extreme opposition upon disagreement.

quila 27 May 2024 8:37 UTC
1 point
0
on: quila’s Shortform
a super-coordination story with a critical flaw
part 1. supercoordination story
- select someone you want to coordinate with without any defection risks
- share this idea with them. it only works if they also have the chance to condition their actions on it.
- general note to maybe make reading easier: this is fully symmetric.
- after the acute risk period, in futures where it’s possible: run a simulation of the other person (and you).
- the simulation will start in this current situation, and will be free to terminate when actions are no longer long-term relevant. the simulation will have almost exactly the same starting state and will develop over time in the same way.
- there will be one change to the version of you in the simulation. this change is that the version of you in the simulation will have some qualities replaced with those of your fellow supercoordinator. these qualities will be any of those which could motivate defection from a CDT agent: such as (1) a differing utility function and maybe (2) differing meta-beliefs about whose beliefs are more likely to be correct under disagreement.
- this is to appear to be an ‘isolated’ change. in other words, the rest of the world will seem coherent in the view of the simulated version of you. it’s not truly coherent for this to be isolated, because it would require causally upstream factors. however, it will seem coherent because the belief state of the simulated version of you will be modified to make the simulated world seem coherent even if it’s not really because of this.
- given this, you’re unsure which world you’re acting in.
- if you’re the simulacra and you defect, this logically corresponds to the ‘real’ version of you defecting.
- recall that the real version of you has the utility functions and/or action-relevant beliefs of your (in-simulation) supercoordinator.
- because of that, by defecting, it is ⁵⁰⁄₅₀ which ‘qualities’ (mentioned above) the effects of your actions will be under: ‘yours or theirs.’ therefor, the causal EV of defection will always be at most 0.
- often it will be less than 0, because the average of you two expects positive EV from collaboration, and defecting loses that in both possible worlds (negative EV).
part 2. the critical flaw
one can exploit this policy by engaging in the following reasoning. (it might be fun to see if you notice it before reading on :p)
1. some logical probabilities, specifically those about what a similar agent would do, depend on my actions. ie, i expect a copy of me to act as i do.
2. i can defect and then not simulate them.
3. this logically implies that i would not be simulated.
4. therefor i can do this and narrow down the space of logically-possible realities to those where i am not in this sort of simulation.
when i first wrote this i was hoping to write a part 3. how to avoid the flaw, but i’ve updated towards it being impossible.

quila 22 May 2024 20:10 UTC
1 point
0
on: quila’s Shortform
I wrote this for a discord server. It’s a hopefully very precise argument for unaligned intelligence being possible in principle (which was being debated), which was aimed at aiding early deconfusion about questions like ‘what are values fundamentally, though?’ since there was a lot of that implicitly, including some with moral realist beliefs.
1. There is an algorithm behind intelligent search. Like simpler search processes, this algorithm does not, fundamentally, need to have some specific value about what to search for—for if it did, one’s search process would always search for the same thing when you tried to use it to answer any unrelated question.
2. Imagine such an algorithm which takes as input a specification (2) of what to search for.
3. After that, you can combine these with an algorithm which takes as input the output of the search algorithm (1) and does something with it.
For example, if (2) specifies to search for the string of text that, if displayed on a screen, maximizes the amount of x in (1)’s model of the future of the world that screen is in, then (3) can be an algorithm which displays that selected string of text on the screen, thereby actually maximizing x.
Hopefully this makes the idea of unaligned superintelligence more precise. This would actually be possible even if moral realism was true (except for versions where the universe itself intervenes on this formally possible algorithm).
(2) is what I might call (if I wasn’t writing very precisely) the ‘value function’ of this system.
notes:
- I use ‘algorithm’ in a complexity-neutral way.
- An actual trained neural network would of course be more messy, and need not share something isomorphic to each of these three components at all
- This model implies the possibility of an algorithm which intelligently searches for text which, if displayed on the screen, maximizes x—and then doesn’t display it, or does something else with it, not because that other thing is what it ‘really values’, but simply because that is what the modified algorithm says. This highlights that the property ‘has effects which optimize the world’ is not a necessary property of a(n) (super)intelligent system.

quila 21 May 2024 17:14 UTC
1 point
1
in reply to: mesaoptimizer’s comment on: Fund me please—I Work so Hard that my Feet start Bleeding and I Need to Infiltrate University
I think what quila is pointing at is their belief in the supposed fragility of thoughts at the edge of research questions.
Yes, thanks for noticing and making it explicit. It seems I was modelling Johannes as having a similar cognition type, since it would explain their behavior, which actually had a different cause.
I believe that if your research process / cognition algorithm is fragile enough that you’d be willing to take physical damage^[1] to hold onto an inchoate thought, maybe consider making your cognition algorithm more robust.
My main response to ‘try to change your cognition algorithm if it is fragile’ is to remind that human minds tend to work differently on unexpected dimensions. (Of course, you know this abstractly, and have probably read the same post about the ‘typical mind fallacy’. But the suggestion seems like harmful advice to follow for some of the minds it’s directed at.) (Alternatively, since you wrote ‘maybe’, this comment can be seen as describing a kind of case where it would be harmful)
My highest value mental states are fragile: they are hard to re-enter at will once left, and they take some subconscious effort to preserve/cultivate. They can also feel totally immersing and overwhelming, when I manage to enter them. (I don’t feel confident in my ability to qualitatively write more, as much as I would like to (maybe not here or now)).
This is analogous to Johannes’ situation in a way. They believe the problem they have of working too hard is less bad to have than the standard problem of not feeling motive to work. The specific irrational behavior their problem caused also ‘stands out’ more to onlookers, since it’s not typical. (One wouldn’t expect the top comment here if one described succumbing to akrasia; but if akrasia was rare in humans, such that the distribution over most probable causes included some worrying possibilities, we might)
In the same way, I feel like my cognition-algorithm is in a local optima which is better than the standard one, where one lesser-problem I face is that my highest output mental states are ‘fragile’, and because this is not typical it may (when read of in isolation) seem like a sign of ‘a negative deviation from the normal local optima, which this person would be better off if they corrected’.
From my inside perspective, I don’t want to try to avoid fragile mental states, because I think it would only be a possible change as a more general directional change away from ‘how my cognition works (at its best)’ towards ‘how human cognition typically works’.
(And because the fragility-of-thought feels like a small problem, once I learned to work around it, e.g learning to preserve states and augmenting with external notes. At least when compared to the problem most have of not having a chance at generating insights of a high enough quality as our situation necessitates.)
… although, if you knew of a method to reduce fragility while not reducing other things, then I’d love to try it :)
1. ^
  On ‘willing to take physical damage …’, footnoted because it seems like a minor point—This seems like another case of avoiding the typical-mind-fallacy being important, since different minds have different pain tolerances / levels of experienced pain from a cut.

quila 21 May 2024 0:05 UTC
2 points
0
in reply to: yanni kyriacos’s comment on: yanni’s Shortform
I’m not aware of research on this. LW has a poll feature that at least the mods can use (based on embedded emoji reacts, which could be used for pain levels), maybe ask the LW team to make one and a post for this

quila 20 May 2024 23:54 UTC
2 points
0
on: quila’s Shortform
random (fun-to-me/not practical) observation: probability is not (necessarily) fundamental. we can imagine totally discrete mathematical worlds where it is possible for an entity inside it to observe the entirety of the that world including itself. (let’s say it monopolizes the discrete world and makes everything but itself into 1s so it can be easily compressed and stored in its world model such that the compressed data of both itself and the world can fit inside of the world)
this entity would be able to ‘know’ (prove?) with certainty everything about that mathematical world, except it would remain uncertain whether it’s actually isolated (/simulated) inside some larger world. (possibly depending on what algorithms underly cognition), it might also have to be uncertain about whether its mind is being edited from the outside.
the world we are in may be disanalogous to that one in some way that makes probability actually-fundamental here, and in any case probability is necessary because this one is complex.

quila 20 May 2024 14:09 UTC
2 points
0
on: quila’s Shortform
Platonism
(status: uninterpretable for ²⁄₄ reviewers, the understanding two being friends who are used to my writing style; i’ll aim to write something that makes this concept simple to read)
‘Platonic’ is a categorization I use internally, and my agenda is currently the search for methods to ensure AI/ASI will have this property.
With this word, I mean this category acceptance/rejection:
✅ Has no goals
✅ Has goals about what to do in isolation. Example: “in isolation from any world, (try to) output A”^[1]
❌ Has goals related to physical world states. Example: “(try to) ensure A gets stored in memory on the computer in the physical world that’s computing my output.”^[2]
A can be ‘the true answer to the input question’, ‘a proof of x conjecture’, ‘the most common next symbol in x world prior to my existence in it’, etc.
As written here, this is a class of outer alignment solution. I need to write about why I believe it’s a more reachable target for ‘inner alignment’/‘training stories’, too.
1. ^
  A more human-intuitive transcription may include wording like: “try to be the kind of program/function which would (in isolation from any particular worldstate/physics) output A.”
  I’m leaving this as a footnote because it can also confuse people, leading to questions like “What does it mean to ‘try to be a kind of program’ when it’s already determined what kind of program it is?”
2. ^
  This class of unaligned ‘physical goals’ is dangerous because if the system can’t determine A, its best method to fulfill the goal is through instrumental convergence.

quila 20 May 2024 4:17 UTC
10 points
0
on: quila’s Shortform
A quote from an old Nate Soares post that I really liked:
It is there, while staring the dark world in the face, that I find a deep well of intrinsic drive. It is there that my resolve and determination come to me, rather than me having to go hunting for them.
I find it amusing that “we need lies because we can’t bear the truth” is such a common refrain, given how much of my drive stems from my response to attempting to bear the truth.
I find that it’s common for people to tell themselves that they need the lies in order to bear reality. In fact, I bet that many of you can think of one thing off the top of your heads that you’re intentionally tolerifying, because the truth is too scary to even consider. (I’ve seen at least a dozen failed relationships dragged out for months and months due to this effect.)
I say, if you want the intrinsic drive, drop the illusion. Refuse to tolerify. Face the facts that you feared you would not be able to handle. You are likely correct that they will be hard to bear, and you are likely correct that attempting to bear them will change you. But that change doesn’t need to break you. It can also make you stronger, and fuel your resolve.
So see the dark world. See everything intolerable. Let the urge to tolerify it build, but don’t relent. Just live there in the intolerable world, refusing to tolerate it. See whether you feel that growing, burning desire to make the world be different. Let parts of yourself harden. Let your resolve grow. It is here, in the face of the intolerable, that you will be able to tap into intrinsic motivation.

quila 19 May 2024 17:28 UTC
3 points
0
in reply to: Kaj_Sotala’s comment on: Fund me please—I Work so Hard that my Feet start Bleeding and I Need to Infiltrate University
This sounds like you’re saying that they made a rational prioritization and then, separately from that, forgot that it was there
That implication wasn’t intended. I agree that (for basic reasons) the probability of a small cut was higher given their choice.
Rather, the action itself seems rational to me when considering:
1. That outcome seems unprobable (at least if they were sitting down), but actual in this particular timeline.
2. The effects of a cut on the foot are really low (with I’d guess >99.5% probability, for an otherwise healthy person—on reflection, maybe not cumulatively low enough for the also-small payoff?), and if so ~certain to not significantly curtail progress.
That doesn’t necessarily imply the policy which produced the action is rational, though. But when considering the two hypotheses: (1) OP is mentally unwell, and (2) They have some them-specific reason^[1] for following a policy which outputs actions like this, I considered (2) to be a lot more probable.
Meta: This comment is (genuinely) very hard/overwhelming-feeling for me to try to reply to, for a few reasons specific to my mind, mainly about {unmarked assumptions} and {parts seeming to be for rhetorical effect}. (For that reason I’ll let others discuss this instead of saying much further)
I think it’s also really important for there to be clear, public signals that the community wants people to take their well-being seriously
I agree with this, but I think any ‘community norm reinforcing messages’ should be clearly about norms rather than framed about an individual, in cases like this where there’s just a weak datapoint about the individual.
1. ^
  A simple example would be “Having introspected and tested different policies before determining that they’re not at risk of burnout from the policy which gives this action.”
  A more complex example would be “a particular action can be irrational in isolation but downstream of a (suboptimal but human-attainable) policy which produces irrational behavior less than is typical”, which (now) seems to me to be what OP was trying to show with this example given their comment

quila 19 May 2024 12:04 UTC
3 points
−6
in reply to: Alex_Altair’s comment on: Fund me please—I Work so Hard that my Feet start Bleeding and I Need to Infiltrate University
I think this post is a red flag about your mental health. “I work so hard that I ignore broken glass and then walk on it” is not healthy.
Seems like a rational prioritization to me if they were in an important moment of thought and didn’t want to disrupt it. (Noting of course that ‘walking on it’ was not intentional and was caused by forgetting it was there.)
Also, I would feel pretty bad if someone wrote a comment like this after I posted something. (Maybe it would have been better as a PM)

quila 18 May 2024 11:35 UTC
2 points
0
on: Language Models Model Us
This ability has been observed more prominently in base models. Cyborgs have termed it ‘truesight’:
the ability (esp. exhibited by an LLM) to infer a surprising amount about the data-generation process that produced its prompt, such as a user’s identity, motivations, or context.
Two cases of this are mentioned at the top of this linked post.
---
One of my first experiences with the GPT-4 base model also involved being truesighted by it. Below is a short summary of how that went.
I had spent some hours writing and {refining, optimizing word choices, etc}^[1] a more personal/expressive text. I then chose to format it as a blog post and requested multiple completions via the API, to see how the model would continue it. (It may be important that I wasn’t in a state of mind of ‘writing for the model to continue’ and instead was ‘writing very genuinely’, since the latter probably has more embedded information)
One of those completions happened to be a (simulated) second post titled ideas i endorse. Its contents were very surprising to then-me because some of the included beliefs were all of the following: {ones I’d endorse}, {statistically rare}, and {not ones I thought were indicated by the text}.^[2]
I also tried conditioning the model to continue my text with..
- other kinds of blog posts, about different things—the resulting character didn’t feel quite like me, but possibly like an alternate timeline version of me who I would want to be friends with.
- text that was more directly ‘about the author’, ie an ‘about me’ post, which gave demographic-like info similar to but not quite matching my own (age, trans status).
Also, the most important thing the outputs failed to truesight was my current focus on AI and longtermism. (My text was not about those, but neither was it about the other beliefs mentioned.)
1. ^
  The sum of those choices probably contained a lot of information about my mind, just not information that humans are attuned to detecting. Base models learn to detect information about authors because this is useful to next token prediction.
  Also note that using base models for this kind of experiment avoids the issue of the RLHF-persona being unwilling to speculate or decoupled from the true beliefs of the underlying simulator.
2. ^
  To be clear, it also included {some beliefs that I don’t have}, and {some that I hadn’t considered so far and probably wouldn’t have spent cognition on considering otherwise, but would agree with on reflection. (eg about some common topics with little long-term relevance)}

quila 16 May 2024 22:59 UTC
1 point
0
in reply to: Emrik’s comment on: quila’s Shortform
Record yourself typing?

quila

Platonism