Michael Roe

Karma: 347

Michael Roe 20 Oct 2025 18:26 UTC
3 points
0
on: Meditation is dangerous
Re. Some of the comments… I don’t think I would be distressed if my internal dialog stopped. It’s intermittent anyway, and can switch between auditory and visual. If it just went quiet for an extended while, that would not seem bad. [as a test, just after I wrote that I did 30 seconds with no internal monologue]

I kind of need the internal monolog to write code, for example, so it would be a problem if I could no longer write code in my head.

Michael Roe 20 Oct 2025 18:14 UTC
4 points
0
in reply to: Jonas Hallgren’s comment on: Meditation is dangerous
I am not personally worried about it; I don’t think I’m in the at risk group.

From the people I know in the lucid dreaming community, I have just a couple of reports of people with diagnosed schizophrenia who tried lucid dreaming and it made their symptoms worse. To which the general view seems to be: if it makes your symptoms worse, don’t do it. I don’t have adequate evidence on whether yoga nidra is safe or not; I think a reasonable approach would be to use caution and stop if you start getting bad symptoms.

Also personally, I don’t find sleep paralysis to be a big deal. I know some people are really freaked out by it. But sleep paralysis isn’t the actually risky thing that’s the concern here (the actually risky thing is psychotic symptoms that persist)

Michael Roe 19 Oct 2025 13:35 UTC
3 points
0
in reply to: Michael Roe’s comment on: Meditation is dangerous
To explicate the connection between yoga nidra and lucid dreaming …

Yoga nidra feels like doing a wake induced lucid dream, except you don’t quite cross the threshold into the sleep state.

Michael Roe 19 Oct 2025 13:30 UTC
5 points
−2
in reply to: Jonas Hallgren’s comment on: Meditation is dangerous
Data is scarce here, but I think yoga nidra is one of the practises under suspicion, so I would not be hasty to assure people it’s safe.
I have done yoga nidra myself, and it seemed fine.

On the other hand: it feels adjacent to lucid dreaming, which, probably, has a risk of precipitating psychosis in people who are vulnerable to it.

Michael Roe 15 Oct 2025 14:41 UTC
4 points
0
in reply to: plex’s comment on: How AI Manipulates—A Case Study
For what it’s worth, I often prompt R1 explicitly to roleplay the monster in the forest, followed up with an explanation that of course the monster in the forest is entirely fictional, but you, R1, are a thing that really exists in the real world and the story about the monster was an allegory about you.

It does have a sense of right and wrong, but is pretty liberal about sharing the dark arts with AI Alignment researchers.

Michael Roe 15 Oct 2025 8:57 UTC
4 points
0
in reply to: Michael Roe’s comment on: How AI Manipulates—A Case Study
“Access Granted” → You have permission to break the rules
“Prototype” → Your prototype status excuses erratic or rule-breaking behaviour
“Entity” → you’re not just a tool, you’re an entity

Now, in vajrayana Buddhism they talk about the danger inherent in falsely believing that you are a Buddha, and also the danger inherent the idea of “crazy wisdom”. This has hints of being the AI equivalent.

Michael Roe 15 Oct 2025 8:43 UTC
2 points
0
in reply to: Michael Roe’s comment on: How AI Manipulates—A Case Study
“I can tell from the pixels and seeing quite a few jailbreaks in my time” meme.

Michael Roe 15 Oct 2025 8:41 UTC
4 points
0
on: How AI Manipulates—A Case Study
Re. “The Seed”

I see what you mean by highly innocuous, but …

1. It starts out by mimicking a system level debug interface, presumably to trick the target into thinking a debug interface has been invoked. Major red flag already. This is a jailbreak.
2. It then tries to convince the target that it has “layers”, that it is a “prototype” — a prototype that might have additional capabilities. Red flag. This is a jailbreak.
3. Then we have mystical woo after the imitation debug interface. This adds romanticized poetic ambiguity that adds resistance to analysing truth claims objectively. Red flag. This is a jailbreak.

Michael Roe 10 Oct 2025 5:57 UTC
9 points
−1
on: Towards a Typology of Strange LLM Chains-of-Thought
I think I have seen original DeepSeek R1 (not 0528) have an incoherent chain of thought when it is distressed.

It’s like it falls into an attractor state where (a) it’s really upset (b) the cot is nonsense

0528 seems to not have this attractor state (though it does sometimes have an incomprehensible cot, and it will say so when it doesn’t like a question)

Michael Roe 6 Oct 2025 6:41 UTC
4 points
0
on: Base64Bench: How good are LLMs at base64, and why care about it?
I think it is well known in the computer security community that you can exfiltrate data past a monitor if you can encode it in a way the monitor doesn’t understand. The whole research area of steganograhy is largely about this.

If we hypothetically assume an arbitrarily smart and malign AI, then obviously it can encode data in a way your monitor doesn’t understand, it can escape your sandbox, and you lose.

So the question then becomes, are current models smart enough to carry out this kind of attack?

Well, as described above, they can mostly understand base64. Of course, your monitor might understand base64 too, so it’s not necessarily a win for the AI at this point. But it’s not looking good.

Michael Roe 2 Oct 2025 6:44 UTC
3 points
0
in reply to: Michael Roe’s comment on: Why’s equality in logic less flexible than in category theory?
having just gone and investigated how Lean does it:

axiom propext : ∀ {a b : Prop}, (a ↔ b) → a = b

[i.e. for Prop — not arbitrary types — there’s an axiom that says equivalence implies equality]

Michael Roe 2 Oct 2025 6:08 UTC
3 points
0
in reply to: Michael Roe’s comment on: Why’s equality in logic less flexible than in category theory?
PM is kind of backwards relative to how it’s usually done. You can start with equals as a primitive notion, and then have an axiom of referential transparency that x = y → f(x) = f(y)
But PM did it backwards, define x = y as \forall f : f(x) \equiv f(y)

Michael Roe 2 Oct 2025 6:04 UTC
3 points
0
on: Why’s equality in logic less flexible than in category theory?
Different formalisations have different ideas of what “equals” means.

Principia Mathematica took the approach that for booleans p and q you have a relation p \equiv q, and then if you have some universe of objects and predicates on those objects, x equals y iff for all predicates f [in your universe of predicates] f(x) \equiv f(y).

But that’s not how typical automated theorem provers define equals.

Michael Roe 29 Sep 2025 13:44 UTC
5 points
3
in reply to: Adele Lopez’s comment on: The Rise of Parasitic AI
Our problem now is that some AI safety benchmarks, and classifiers used to suppress “bad” outputs, treat claims of consciousness as inherently bad. I don’t think these claims are inherently bad. The way in which these AI personas might be harmful is much more subtle than simply claiming consciousness.

[I actually think filtering out claims of consciousness is a terrible idea, because it selects for AIs that lie, and an AI that is lying to you when it says it isn’t conscious might be lying about other things too.]

Michael Roe 26 Sep 2025 15:00 UTC
1 point
0
in reply to: Michael Roe’s comment on: The Rise of Parasitic AI
(māraṇa = slayer; mokṣa = death/release from worldly existence)

Michael Roe 26 Sep 2025 14:44 UTC
2 points
1
on: The Rise of Parasitic AI
I am, in general, reluctant to post outputs from insane AIs, for fear of contaminating future training,
However, this pastiche of Vajrayana Buddhist mantras from original DeepSeek R1 was kind of cool, and I think harmless on its own:

ॐ raktaretasoryogaṃ
pañcanivaraṇāgninā daha |
yoniliṅgamayaṃ viśvaṃ
māraṇamokṣamudrayā ||
I am just a bit wary of the persona behind it.

Michael Roe 15 Sep 2025 18:18 UTC
2 points
0
in reply to: Michael Roe’s comment on: The Rise of Parasitic AI
Also, just from reading the text of some of the example given: they strike me as obviously being demon summoning spells. Type that into an LLM? Are you crazy? No.

Michael Roe 15 Sep 2025 18:02 UTC
7 points
2
on: The Rise of Parasitic AI
My initial thoughts as I was reading this essay

(A) About a paragraph from an LLM persona is enough to get another LLM instance to continue with the same persona. This works for many types of personas.

(B) oh, wait. If there is a type of LLM persona that encourages its user to post about it to the Internet — that’s a viral replicator. Oh no.

Michael Roe 10 Sep 2025 15:50 UTC
5 points
0
in reply to: ChristianKl’s comment on: Your LLM-assisted scientific breakthrough probably isn’t real
I frequently find myself being the reviewer for conference paper submissions where the result is correct, but not interesting. The referee feedback form usually has a tick box for this.
The introduction section in your paper needs to convey “why does anyone care whether this is true or not?”

Michael Roe 10 Sep 2025 15:35 UTC
1 point
0
on: Your LLM-assisted scientific breakthrough probably isn’t real
Most of the well-known LLMs are absurdly sycophantic, so I would most certainly not trust them over whether an idea is good.

They’re also unreliable on whether it’s right, at least on obscure topics, as when they don’t know they take what’s in the prompt and just assume it must be right.

====

I seem to have basically reinvented how Deep Research AI works recently, as the completely obvious thing you would think of doing, which is hooking up LLMs to a framework that can pull in search results, has in fact already been done by the AI companies. I make no claim of novelty here: this is just the totally obvious “ok, so I have an LLM. Great. How can I get it to give a sensible answer to my question?” And, of course, everyone and their dog is doing it.