Lucian Hardy

Karma: 6

Lucian Hardy 22 Apr 2026 7:00 UTC
2 points
0
in reply to: Breck Yunits’s comment on: Annoyingly Principled People, and what befalls them
The problems with “Claude Schemed …” or “ChatGPT Thought …” style wording in Academic and Media writing and conversation. I am actually stopping myself from going on “the rant” here, just from your question. Pretty good indicator that it’s a fixation causing Alice/Alex behavior.

Lucian Hardy 22 Apr 2026 4:40 UTC
1 point
0
on: Annoyingly Principled People, and what befalls them
Ha, I feel a little bit called out in this post. Behavior I’ve been displaying matches the profile, perhaps I should look a bit inward.
To add to the discussion: The most reliable solution to this problem from my point of view in dealing with others is to establish the exact subject and expected engagement points the annoyingly principled person is attempting to use. Understand that they may not actually know what those are, and direct a meta conversation that leads to defining these things. Finally, engage with them in the manner they attempted to initiate, and end with your fully formed stance. More often than not, this will resolve the source of annoyance. Though you might end up turning yourself into an annoyingly principled person in the process.
To use the closest example already in comments, the initiator of the discussion on Catan^[1] was most likely attempting to engage in a conversation about ensuring fun was not lost in the game, while discussing what victory conditions they held and validating their frustration at being targeted. However, they initiated the conversation in a way that was interpreted as “explain the strategy you were employing when you targeted me”. To which, the other player responded in the manner I’d expect. If instead, they’d responded by starting a meta conversation about what they were talking about (i.e., “Are you looking at game strategy, or is it concerning you because of something else?”), then perhaps the two in the example could have walked away with a better understanding of both sides of the issue.
1. ^
  https://www.lesswrong.com/posts/xG9Y2Mct7uZyt98yb/annoyingly-principled-people-and-what-befalls-them?commentId=iPtoPovNbBDmdbsep

Lucian Hardy 20 Apr 2026 2:41 UTC
4 points
0
in reply to: Brendan Long’s comment on: Lucian Hardy ’s Shortform
Completely fair stance, I’m a bit… aggressive with where I fall on the discussion. Statistical pattern matching meets the requirement to explain the anthropomorphized behavior to me, and nothing else has been fully proven. That’ll have to remain a disagreement, and a healthy one if you ask me. This is probably a bias I am having issues breaking, but it has brought value to me and the research I’ve done/am doing.
To answer your question though, I was running the paper in the following manner:
1. I tasked ChatGPT to give an explanation on why I disagreed with a post on LW, and pasted the markup of the post in the same message.
2. I asked Claude to disprove any and every point of a post on LW, and pasted the markup of the post in the same message.
3. I directly took those outputs, read them, and then started mirror conversations, where I asked Claude to explain why I agreed with a post, and chatGPT to defend the post from someone very against the post. Push output of the attacker and defender until the actual points against the article are fully formed thoughts, fully explored, or completely defeated.
4. I took those formed thoughts and all prior messages, and reasoned through their arguments, impact, and potential solutions. Edited the post with my final reasonings, and returned to step 1.

Lucian Hardy 20 Apr 2026 2:30 UTC
1 point
0
in reply to: Brendan Long’s comment on: Not a Goal. A Goal-like behavior.
It feels dismissive, but I’d like to state that I was asked to provide an ontological exit state (which while useful, is not the intention), and thus provided one. This was done in spite of the language being more of a discipline towards looking shallower before deeper, and writing in a manner that leads others to follow suit. The exact boundary would be better described as “intentionally fuzzy and hard to exceed”.
I would state that the exit point is when the correct behavior is to diagnose and prevent mesa-optimization/behaviors, instead of analyze how the model could have statistically arrived at an output, if it’s reward hacking, if there is priors that lead to an output deterministically, or any number of other simpler diagnosis… but that’s putting the cart before the horse, and indirectly saying “use it when you feel like it”.
For condition 1: The exact paper referenced^[1] helps detail further meaning. The CoT models are outputting thought-like behavior, yes. Is it true thought? Does the model know anything? I see no proof of either being true in the paper. Instead I see support of the potential proof. Which is below the bar I have set intentionally high. In reality, I would explain thought-like and reasoning-like behavior as statistical story-telling that affects the remainder of the statistical pattern matching. While the effect is still the same, it invokes a different causal state, and thus invokes a different set of actions from myself.
To detail why, Consider the following from the papers conclusion:
1a: “Key findings include the emergence of goal-directed deception without explicit instruction, suggesting it’s a byproduct of advanced reasoning. Representation engineering successfully quantified deception via high-accuracy steering vectors, establishing it as a measurable property. The developed framework allows for precise induction or suppression of deception, offering a pathway for balancing capability and safety in AI deployments. These results highlight the dual-use potential of CoT models and underscore the necessity of rigorous monitoring and control through methods like representation engineering for AI safety.”
1b: “Key findings include the emergence of output that is like goal-directed deception without explicit instruction, suggesting it’s a byproduct of advanced reasoning-like outputs. Representation engineering successfully quantified deception-like bias^[2] via high-accuracy steering vectors, establishing these biases as a measurable property. The developed framework allows for precise induction or suppression of deception-like bias, offering a pathway for balancing capability and safety in AI deployments. These results highlight the dual-use potential of CoT models and underscore the necessity of rigorous monitoring and control through methods like representation engineering for AI safety.”
Specifically, the next paragraph leads to different insights depending on which is read prior.
“Despite demonstrating significant insights, the study has limitations. The influence of contextual framing on deception tendencies, as seen in performance disparities between paradigms, was not fully disentangled. Furthermore, while representation engineering showed layer correlations, it didn’t pinpoint precise architectural components encoding deception and task semantics, limiting understanding of mechanistic drivers. Future work should systematically investigate how contextual framing modulates deception and employ mechanistic interpretability to identify specific architectural elements responsible, enabling more targeted detection and mitigation strategies.”
I would like to know if the change in language sparked any different ideas on what the further research would look like. I don’t any evidence this language has the intended effect on others that it does on me.
Regarding Point 2: to meet the requirement, the information storage system in question must be proven opaque, retrievable, persistent, and mutable simultaneously. Let’s test:
- weights :
- - Opaque: Not fully proven. some weights are entangled, but weights in general can be inspected and investigated. Basically, while the window is proven black for an end user, it’s not proven black for a ML researcher.
  - Retrievable: Proven true, if abstractly. all weights fire, attention and activation determines intensity, potentially leading to selective retrieval. Which is probably the word choice I should have used.
  - Persistent: Proven True.
  - Mutable: Proven False. Weights never change once frozen.
- context:
- - Opaque: Not yet proven. If the information in question is “hidden bias or feature”… Then this is actually something I’m researching.
  - Retrievable: Proven false in most cases. The model does not have the ability to retrieve context, unless tool usage enables this, and even then, the “hidden bias or feature” is definitely not proven retrievable.
  - Persistent: Proven true.
  - Mutable: Proven… actually, compaction of context kind of proves this true, but in a very messy, unpredictable way. The model doesn’t have the ability to modify information in a controlled manner. Especially if the information is opaque, the compaction is highly likely to do more damage than good in the models case. Disregarding compaction, context is add only, which is by definition immutable storage.
Even if placed together, as weights + context, the condition holds. If we wanted to abstract significantly, with several assumptions… one could claim that the user is the opaque, retrievable, persistent, mutable storage system for the model. Honestly, that’s a bit too meta for me to truly engage.
I can definitely see how the definition of the two conditions feels very much like a fuzzy thing that may currently be possible to meet. I am admittedly not perfect with word choice, and the point is not a perfect definition… but instead a potential change in discipline of thought. As such… I’m open to a better set of conditions.
1. ^
  https://arxiv.org/pdf/2506.04909
2. ^
  Bias could also be described as a feature in this context.
3. ^
  Discounting temperature

Lucian Hardy 19 Apr 2026 9:25 UTC
1 point
0
in reply to: Linch’s comment on: Linch’s Shortform
To engage in speculation with speculation, I’ve thought for a while now that the behavior is most likely presenting, is that it is a larger and larger portion of the training corpus. But I’d also connect to the supposition that the discussion of training, along with the model engaging the pattern “I need to think”, start to push the model further and further into fictional work representation. As in storytelling. If not that, then representation of anthropomorphic works involving LLMs. The logic for both is statistical distribution of the text present in the training data. How many posts on LW involve the phrase “I need to think”? How many fictional books and shortform reddit posts have that phrase present? I haven’t checked the facts on this, but something tells me that the books have the majority, merely by volume. The chain from recognition-like behavior of the model being an LLM, to it assuming eval is as you stated. Following that, if the model continues the pattern generation… statistically the next tokens are AI spooky storytelling, or research that presents as such. While I wouldn’t put it as stated fact, I would say it’s sharp intuition.

Lucian Hardy 19 Apr 2026 9:04 UTC
1 point
0
on: Lucian Hardy ’s Shortform
Spooky thought tonight, A cognitive issue the presents as a shared hazard:

Assuming the following: An LLM that doesn’t have mesa-optimization, and features are not cleanly separated. Training data that contains an opinion that is harmful to society.

The model is trained, harmful output is noted and then modern alignment practices are applied. Ablation, RLHF, the usual. Any alignment that buries the harmful output, but does not make it impossible. The models output is still changed from what it would have been if never exposed to that harmful opinion. It’s possible (and for the case of this hazard, assumed) that ALL output from that model now shifts slightly towards the harmful opinion when compared to the same model if the opinion had never been present during initial training. This may not present as a directly testable condition, it could be as subtle as framing, writing style, or even a single benign word being output over another. When a human performs this action, it is commonly referred to as subliminal messaging.

A user interacts with the LLM for a significant time, to the point where the models output starts affecting the user’s knowledge or opinions. Because the user was influenced by the biased output, they now carry a slight bias towards the harmful opinion. This manifests in their communication with others… even if it’s a subtle as choosing the same word as the LLM. Given a sufficient number of users all experiencing the same slight bias… the dangerous opinion now has a higher chance of manifesting, without ever being output by the model.

To give this as my personal experience, I’ve got exactly one LW post under my belt. I did ask both ChatGPT and Claude to review my work, and both informed me that I was writing at a level significantly lower than LW expects, and would not even truly engage with the ideas until they were framed in a more complex fashion. Now, I’ve read multiple posts (not all of the sequences. It’s so far been just a refresher from college, and I’m working through them. I am remembering and learning still), so I should have known that I was overcomplicating my thoughts. Somehow though, that bias was absorbed still. And now, I have a overcomplicated post sitting in public domain that would support the framing and stylistic choice that Claude and ChatGPT venomously argue is required for LW. During the change of framing and style… my viewpoint has shifted. Was that a result of more thought and effort, or was that a result of a hidden bias taking hold? And am I spreading it by leaving a LLM reviewed post on my profile?

Lucian Hardy 18 Apr 2026 12:23 UTC
2 points
0
in reply to: Richard_Kennaway’s comment on: Not a Goal. A Goal-like behavior.
I agree that it’s a bit of a hard distinction to be made, especially when applied outside of LLM space; I hadn’t considered its use outside the scope of my conversation. The examples you gave are excellent, the calculator example in particular is fairly interesting to explore and has led me to crystallize this fairly cleanly. To be clear, the article’s position is that “-like” can be dropped when the term is better explained by the object performing the term, rather than by the object producing it via statistical probability.… which is admittedly too conceptual to be workable, as you correctly called out. oops. I’ve had to rethink this entire thing from the ground up.

To save some time (I know this is a book, I think long when it matters to me), the answer to your final question is that -like applies when:

1: The object lacks the proven capacity for the mechanism that the unqualified term defines. i.e., “deception is the knowledge of two distinct states, one of which is intentionally false. the model does not have proven capacity for the action of deception, lacking the ability to represent multiple outputs and select a less truthful output intentionally, therefore the model performed deception-like behavior”.

2: The object lacks proven information storage that is opaque, retrievable, persistent, and mutable. i.e., “the model has no proven way of opaquely storing and retrieving information”.

The intention is that the qualifier would be dropped when any of the following are met:
The object has been proven to have gained the capacity for the action that the unqualified term defines.
The object has with high probability gained that capacity due to opaque state structures supporting the capacity for the action.

This has been specifically split into two conditions because 2 is an early exit that would then warrant the intentional stance’s vocabulary instead. The “chatbot potentially deceived” is plausible and fine in condition 2′s case. In specific, a chatbot would honestly either require mesa-optimization proven to say it’s deceiving, OR it would require proof of opaque persistent mutable information storage. The presence of opaque storage makes disproving capacity too difficult, and the risk of the behavior being genuine too high to use the qualifier. I’m not claiming these capabilities are impossible for the chatbot, but I am saying the burden of proof is on the individual dropping -like from the conversation. An intentionally high bar to promote rational thought.

To go a bit deeper into the reasoning I have now:

The intent of the vocabulary is not to be a long-term ontologically precise way of discussing everything, despite that being a very apt way of distinguishing its definition. It is meant to have immediate methodological effect in the scope of LLM studies, and help enforce epistemic discipline. In the post, I made the distinction that it should be an authorship question. i.e., “where did the behavioral pattern originate”? However if it is an authorship question, then the calculator is exhibiting calculation-like behavior, as you noted. If I change the metric to “when authorship is in question”, then mesa-optimization is unfairly dismissed by the nature of the statement. Additionally, the definition of “in question” becomes… something in question. I will admit that my conclusion on the subject of mesa-optimization is that the area of study is currently a distraction from ongoing problems that desperately need attention, which has significant bias in my framing that I’m fighting against. However I do not suppose mesa-optimization should be dismissed outright, just that currently understood mechanisms must be explored prior to deferring to an unproven theory.

To address your examples directly:

1/2/3: the bird, bomb, and calculator are resolved by capacity. Notably, the calculator is not resolved with my prior modeling based on authorship, prompting me to analyze several other things to properly define my intention.
4: Children have capacity for conversation. the radical behaviorist phrasing correctly showed that I was doing the opposite of what I intended, I’m saying to attribute what has been proven, and use -like when it hasn’t.
5: This splits asymmetrically. Even under authorship, the user was simply deceived, and the chatbot exhibited deception-like behavior. However it’s much more clear cut when capacity is utilized.

Interestingly, 4a actually hits the main intention of this language shift perfectly. Was the parallel intentional? By having anthropomorphized toys, children conversed with them. When that wasn’t the case, they didn’t. That exact behavioral shift is what I’m attempting to state matters in alignment research.

This distinction could still be an error on my part, or the bias of my personal beliefs on the field… but it could also be the actual definition I have been circling. Current public discourse is happening with inherent assumptions that are unproven. The stance I am taking is that there is still plenty of research to be done before limiting ourselves to a “spooky action at a distance” equivalent. This is what I’m planning on discussing in my posts, using this language. Solidifying my vocabulary first is very important to me so I don’t have to redefine goal-like when I’d rather be redefining the actual statements in future posts.

Lucian Hardy 13 Apr 2026 3:18 UTC
1 point
0
on: Rationality: An Introduction
I got stuck while reading this article on the premise that you’re sure the letter came from one of the six classmates. The assumption is stated, but it is presented as a given rather than flagged as a simplification, with the footnote marker acknowledging that simplification appearing later in the post. I found myself unable to think about anything else without questioning that premise for the remainder of the post until I located the footnote. By that point I’d already spent the entire example questioning the premise instead of engaging with the math. It feels like a missed opportunity to demonstrate the skill being taught. I will concede however that getting stuck was itself a useful exercise in flagging unstated simplifications.