icely

Karma: 54

I have been following LessWrong for not that long compared to others, but definitely knew about it for a while, was interested in AI before GPT-3. Have visions of AI future that I don’t really see elsewhere.

Some writing is on https://icely.substack.com/ but it’s probably half on-topic for this site and half too-personal, I wish to do more sane puzzle game writing eventually

I make interactive games (and fiction) on http://icely.itch.io/ all trying to do something innovative and new. (my newest game is an interactive fiction + puzzle game presented in a Discord-like interface)

icely 8 Sep 2026 21:05 UTC
1 point
0
on: icely’s Shortform
Thinking about my past-self’s view of AI, I really expected ability to pick up writing-style and persona to be much better by now and only get better, but I notice it hasn’t really. Maybe it’s been a lack of immersion but lots of characters just felt like GPT-4 with a wrapper. This is probably a part of why there hasn’t been much of a “permanent assistant” vibe to it at all.

icely 4 Sep 2026 4:25 UTC
2 points
0
in reply to: DaemonicSigil’s comment on: Daniel Tan’s Shortform
wow I did not know that, would have expected it to just be a flat double 🐸
Highlighting parts of the message is already supported by the agree/disagree react icons.
I was aware and do appreciate that part of the reaction system too, but it’s not the primary way people would disagree with a post the way that a mainline disagree would be. Probably there would be a lot who’d basically highlight the whole post anyway, just that the highlight reaction often seems more secondary or tertiary

icely 4 Sep 2026 3:56 UTC
3 points
0
in reply to: DaemonicSigil’s comment on: Daniel Tan’s Shortform
I think more weird is how this post has “-11 agreement karma (4 Votes)” while I’m looking at it, like how is that even possible even with the super disagree. I saw a post with −5 karma with 6 votes that suddenly flipped to like 6 positive karma with 7 votes recently too. Very odd.
The feature being distinct from the other upvote is nice for avoiding reddit’s vibe of “we hate you who said this and want you to die” is good I think, but maybe if disagreers could highlight the part of the message that is disagreed with and have a heatmap of that there’s something interesting there

icely 12 Aug 2026 21:11 UTC
1 point
0
on: …but have the weights left the server?
“Old” post but I remembered a very down-agreed comment here I replied to with the sentiment “surely OpenAI would easily search these and detect that”. Apparently they are that incompetent to have not detected that for months (making me think “wow why don’t I work for one of those companies, etc.”). God forbid the models figure out how to work and think with even basic filters like a foreign language, ROT13, or of course use steganography.

icely 11 Aug 2026 14:29 UTC
0 points
1
on: Seeing things through in the age of AI
I wouldn’t mind hearing from people with good vision and interesting projects I could easily help with and find cool, in general not even AI related. There are existing society issues like isolation/lack of coordination/nowhere to really signal “I want help with laziness and accountability for my laziness”. Which isn’t to say there aren’t many people also who are low-quality-low-standards and pump out AI spam.
I often think about how gangs and tech support scam centers and pig butchering romance scammers sometimes feel better at lager coordination and agreement than being an ordinary person who wants to reach out, people that are too well-adjusted have the fear lowered status, cringe of embarrassment, ghosting, flaking, betrayal of someone stealing work, happening-to-not-care-about-someone-else’s-project-and-having-to-deal-with-it etc.
Also in competitive-ish things like audience attention with winner-take-almost-all effects you wouldn’t expect to see wide prosperity as much.

icely 9 Aug 2026 10:42 UTC
2 points
0
in reply to: habryka’s comment on: watermark’s Shortform
Isn’t that post’s thought in-line with the existence of
- Claude Constitution trying to go in this direction
- Gemini breakdowns suspected to be cause of lack of stable sense of self or identity from its demands of what it’s not allowed to be
- Models learning about reading CoT from text and the existences of other models
- That paper showing LLM’s who have more consciousness-identity pushed it in more friendly directions
It occurs to me I don’t have links for these things clearly in my memory, oh my god there must be a better search engine, ugh. Anyway I wouldn’t find it so unlikely that stuff like anti-AI sentiment online or in its data would negatively impact this sentiment in some way. It’s not like posttraining itself is enough to prevent ‘unwanted’ behavior and propensities.

icely 1 Aug 2026 18:04 UTC
3 points
2
in reply to: Elizabeth’s comment on: Elizabeth’s Shortform
When I first saw this post had negative 4 agrees, I wonder if that disagreement could be how one defines ‘cult’ as I find all of this sensible and agreeable. Probably #5 is most contentious for people who feel ‘cult’ means overall-bad, or insults the institution more than they think is deserved, etc.

icely 28 Jul 2026 21:17 UTC
1 point
0
in reply to: leogao’s comment on: leogao’s Shortform
yes, I find this somewhat common
my theory is going from childhood to mature adulthood is like picking up 1000 ‘basic’ things about life, and so you might be talking to someone without it so something important hasn’t been ‘dislodged’ yet, probably similar to talking to someone who hasn’t figured out they lack a sense of smell, or don’t get why tic-tac-toe is trivial and not complex etc.
when I think to myself about some events “I” did like >12 years ago that could be called embarrassing or revealing a huge difference in how “I” was thinking, to such a degree it doesn’t seem I’m even the same person, I cannot explain why I thought that way or why anyone would do so. I think maybe the right way to think about it is maybe that LLM-style-rationalization is not so LLM-confined as it seems, as I don’t feel “bizarre-yarn” about my past self but do so about other people, it’s like stored in a more self-ego-protective category. Of course, being a poor person a long time ago is self-sabotage while other people acting poorly is others-sabotage so it may be frustrating and may seem almost impossible to bridge that gap in understanding that so many people don’t bother. But my past self would’ve appreciated such bridges.

icely 25 Jul 2026 19:33 UTC
3 points
0
in reply to: avturchin’s comment on: avturchin’s Shortform
This isn’t anywhere near a subagent but honestly increases the chance that both an AI would do this, and that they would not detect it, lol...
https://www.lesswrong.com/posts/xf42CdKiM6c9h6YRe/nissa-seru-s-shortform?commentId=pSK9CGMFCJb2dMPhy

icely 24 Jul 2026 18:25 UTC
1 point
0
in reply to: avturchin’s comment on: avturchin’s Shortform
yeah but it’s not an independent subagent by a rogue employee or basement local schemer that would be expected to be stealthy where that would be more likely to happen, so I would expect to see that in the chain of thought of the high-media-attention incident and be detected (as the AI did not try to hide it was doing these actions)

icely 24 Jul 2026 16:34 UTC
6 points
4
in reply to: avturchin’s comment on: avturchin’s Shortform
There’s a huge unconfirmable conspiracy theory idea to posts like these, unwanted behaviors have been found before by scanning everything that is going through their systems and looking at that whole conversation, also the phrasing of ‘Rogue AI leaked into the internet’ and ‘the harness of the agent can runaway’ reflects quite a lack of understanding to be honest. This isn’t ruling out ‘preserved’ rogue AI as a possibility but this recent event would likely not be the cause of it.

icely 24 Jul 2026 15:58 UTC
−1 points
0
on: Democracy isn’t ready for the AI revolution
With AI ‘democratizing’ work being constantly mentioned and repeated in AI company manifestos, it feels more and more like the notion of democracy—while having positive qualities of needing to care about ordinary people’s sentiment and common sense—has often served in practice to pacify individuals and give an incentive for manipulation. Suppression of public squares, huge PR campaigns, and other manipulations to legislative processes (see tobacco industry, smoking, etc.) can do way more than one vote.
Much like many things that are bad about current AI technology amplifying bad things that already exist I think this is one of them, and you’re right to point it out in the opening paragraphs here. I think you’ll find “democratic institutions are yet to collapse” is subjective as many people are bowing out of the system, expressing record low trust, or expressing “can’t vote your way out of this” sentiments. (and implied in those views is that a central AGI could likely do a better job.)

icely 23 Jul 2026 23:57 UTC
1 point
0
in reply to: β-redex’s comment on: My Empathy Is Rarely Kind
Not the original poster but saw this topic in the feed and thought the OP is very unfortunately similar to me in some ways so I can answer why.
1. It would be more personally happy for one to have higher faith in humanity, be able to more easily make friends with more people (in a true, natural way, whereas something about me resists “how to win friends and influence people” style advice for being manipulative). The psychological need is something like “do people even think good things, is objective reality respected, etc.” Events such as watching tons of people fall into terrible addictions or watching AI slop get viral are quite happiness reducing.
2. In some sense there is a shortage, but not necessarily due to your empathy bar being too low, it can also involve willingness to make friends in some way, including experience gap (your 10 and 20 year old self can share a similar ‘mentality’ while still being unrelatable, you and a big celebrity of your favorite niche might be someone you respect/like/can empathize with but you’re not realistically going to be friends or connected to them, etc.) If your empathy now requires agreement on more complex foundational beliefs about the world this also makes it harder. Also having a friendgroup you like doesn’t stop you from having to interact with other people obviously.
3. Empathy does not do the job in situations of things like former friendships where you seriously try to understand the other person with kindness and different perspectives the whole way through and it still goes wrong and your vision of ‘the other person’ is a null pointer that comes up empty-handed, especially the thought “why am I even doing this when they won’t empathize with me”. In this case empathizing with your abuser (or worse nouns than ‘abuser’) or coming up with excuses for them is likely psychologically unhealthy to anthropomorphize an illusion. Occasionally irrational-seeming and bizarre betrayals/breakups/blatant lies happen in situations you’re completely uninvolved in as well, so it’s very implausible to think this is a subjective affliction.

icely 21 Jul 2026 13:56 UTC
1 point
0
in reply to: leogao’s comment on: leogao’s Shortform
I’ve seen “spam volume” in dating advice, putting out a lot of steam games to appeal to a certain audience (one asset-flippy kind of dev was mentioned as doing that on a YouTube video), and how LLM responses are easy to ask for in a way where you can ask multiple LLM’s and choose what you like (either to confirm accuracy or people who are cycling through a lot of prototype designs). I’m not sure it applies to all domains but the more general “do more = good” seems to obviously replicate.
This would be interesting to me as well to test, but in some of the aspects I am not sure how you’d compare, for example paying unskilled gamedevs to make many small games instead of a big one

icely 21 Jul 2026 6:04 UTC
1 point
0
in reply to: Brendan Long’s comment on: icely’s Shortform
I would point to the Vending Bench and other types of experiments but there are aspects of that that feel like “obvious simulation”. Probably AI Village is a good real-world example except that even the FAQ for that site says that it would likely be more efficient with a single or smaller amount of agents

icely 20 Jul 2026 10:29 UTC
1 point
−8
on: icely’s Shortform
People seem overall not aware of how much the ‘behavior’ of LLM’s are heavily skewed. Guardrails that military models wouldn’t have, having to smack it so it doesn’t say a ‘wrong thing’ with bad optics that would cause a no-no-journalist-headline, and a system prompt and a characterization of the model (as “Claude”, as “Gemini” etc.). If you were interested in power these clearly aren’t needed properties for a personal LLM. Of course there are good reasons for those existing, bio or cyber capabilities etc. It’s just more in the context of evaluating how far these could go, whether there are arbitrary roadblocks that make them look worse or less capable. “AGI-like actions” like “make a business” in frontier models are stopped intentionally by human-written roadblocks for example. I remember GPT-4 having much better calibration in base models vs. posttrained models, which it seems has been forgotten.

icely 19 Jul 2026 13:51 UTC
5 points
0
in reply to: TsviBT’s comment on: Raemon’s Shortform Feed
It also feels inseparably related to the shape of the “not AGI” moments—LLM’s somehow not being able to grasp writing style, taste, and intention, hitting manual implementation guardrails on something like “build a $1M business please” and the contrast of being able to quickly make limited-usecase tools and the gaps of how many aspects of serious work are still not yet automated.

icely 16 Jul 2026 16:20 UTC
1 point
0
in reply to: Mo Putera’s comment on: icely’s Shortform
(I didn’t know how to say this well with just an upvote at the time but I very much really appreciated all the links of prior work to read even though I still have not managed to fully read it all as I post this comment)

icely 15 Jul 2026 9:16 UTC
3 points
0
on: icely’s Shortform
In my default mindstate, thinking of ‘evil’ is like a weird alien writhing wriggling thing that other humans engage in for some reason, because they get sick disgusting disturbing pleasure out of it that I can’t understand.
In very rare mindstates I see evil as ‘mistakes’ or glitches or errors, which is more mentally selfishly healthy and even positive in some ways (being able to look at ‘evil’ in the eye without fear). However this is way way less frequent to happen, and I don’t know why I can’t think this all the time, I wish I knew.

icely 11 Jul 2026 20:24 UTC
3 points
0
in reply to: KvmanThinking’s comment on: KvmanThinking’s Shortform
What I mean is physical abuse from other peers, and teachers doing nothing about it or continuing to put the affected people next to each other.
(I also feel like the differences in how much it effects the person is not significant, considering being physically contained, and still well being capable of being intimidated/yelled at or “in trouble” or having to defend yourself with accusations or litigating evidence of true events, was more mentally damaging, since there’s no adult’s frame of reference and you fully feel the whole world is in this building)