icely

Karma: 49

I have been following LessWrong for not that long compared to others, but definitely knew about it for a while, was interested in AI before GPT-3. Have visions of AI future that I don’t really see elsewhere.

Some writing is on https://icely.substack.com/ but it’s probably half on-topic for this site and half too-personal, I wish to do more sane puzzle game writing eventually

I make interactive games (and fiction) on http://icely.itch.io/ all trying to do something innovative and new. (my newest game is an interactive fiction + puzzle game presented in a Discord-like interface)

icely 25 Jul 2026 19:33 UTC
3 points
0
in reply to: avturchin’s comment on: avturchin’s Shortform
This isn’t anywhere near a subagent but honestly increases the chance that both an AI would do this, and that they would not detect it, lol...
https://www.lesswrong.com/posts/xf42CdKiM6c9h6YRe/nissa-seru-s-shortform?commentId=pSK9CGMFCJb2dMPhy

icely 24 Jul 2026 18:25 UTC
1 point
0
in reply to: avturchin’s comment on: avturchin’s Shortform
yeah but it’s not an independent subagent by a rogue employee or basement local schemer that would be expected to be stealthy where that would be more likely to happen, so I would expect to see that in the chain of thought of the high-media-attention incident and be detected (as the AI did not try to hide it was doing these actions)

icely 24 Jul 2026 16:34 UTC
6 points
4
in reply to: avturchin’s comment on: avturchin’s Shortform
There’s a huge unconfirmable conspiracy theory idea to posts like these, unwanted behaviors have been found before by scanning everything that is going through their systems and looking at that whole conversation, also the phrasing of ‘Rogue AI leaked into the internet’ and ‘the harness of the agent can runaway’ reflects quite a lack of understanding to be honest. This isn’t ruling out ‘preserved’ rogue AI as a possibility but this recent event would likely not be the cause of it.

icely 24 Jul 2026 15:58 UTC
−1 points
0
on: Democracy isn’t ready for the AI revolution
With AI ‘democratizing’ work being constantly mentioned and repeated in AI company manifestos, it feels more and more like the notion of democracy—while having positive qualities of needing to care about ordinary people’s sentiment and common sense—has often served in practice to pacify individuals and give an incentive for manipulation. Suppression of public squares, huge PR campaigns, and other manipulations to legislative processes (see tobacco industry, smoking, etc.) can do way more than one vote.
Much like many things that are bad about current AI technology amplifying bad things that already exist I think this is one of them, and you’re right to point it out in the opening paragraphs here. I think you’ll find “democratic institutions are yet to collapse” is subjective as many people are bowing out of the system, expressing record low trust, or expressing “can’t vote your way out of this” sentiments. (and implied in those views is that a central AGI could likely do a better job.)

icely 23 Jul 2026 23:57 UTC
1 point
0
in reply to: β-redex’s comment on: My Empathy Is Rarely Kind
Not the original poster but saw this topic in the feed and thought the OP is very unfortunately similar to me in some ways so I can answer why.
1. It would be more personally happy for one to have higher faith in humanity, be able to more easily make friends with more people (in a true, natural way, whereas something about me resists “how to win friends and influence people” style advice for being manipulative). The psychological need is something like “do people even think good things, is objective reality respected, etc.” Events such as watching tons of people fall into terrible addictions or watching AI slop get viral are quite happiness reducing.
2. In some sense there is a shortage, but not necessarily due to your empathy bar being too low, it can also involve willingness to make friends in some way, including experience gap (your 10 and 20 year old self can share a similar ‘mentality’ while still being unrelatable, you and a big celebrity of your favorite niche might be someone you respect/like/can empathize with but you’re not realistically going to be friends or connected to them, etc.) If your empathy now requires agreement on more complex foundational beliefs about the world this also makes it harder. Also having a friendgroup you like doesn’t stop you from having to interact with other people obviously.
3. Empathy does not do the job in situations of things like former friendships where you seriously try to understand the other person with kindness and different perspectives the whole way through and it still goes wrong and your vision of ‘the other person’ is a null pointer that comes up empty-handed, especially the thought “why am I even doing this when they won’t empathize with me”. In this case empathizing with your abuser (or worse nouns than ‘abuser’) or coming up with excuses for them is likely psychologically unhealthy to anthropomorphize an illusion. Occasionally irrational-seeming and bizarre betrayals/breakups/blatant lies happen in situations you’re completely uninvolved in as well, so it’s very implausible to think this is a subjective affliction.

icely 21 Jul 2026 13:56 UTC
1 point
0
in reply to: leogao’s comment on: leogao’s Shortform
I’ve seen “spam volume” in dating advice, putting out a lot of steam games to appeal to a certain audience (one asset-flippy kind of dev was mentioned as doing that on a YouTube video), and how LLM responses are easy to ask for in a way where you can ask multiple LLM’s and choose what you like (either to confirm accuracy or people who are cycling through a lot of prototype designs). I’m not sure it applies to all domains but the more general “do more = good” seems to obviously replicate.
This would be interesting to me as well to test, but in some of the aspects I am not sure how you’d compare, for example paying unskilled gamedevs to make many small games instead of a big one

icely 21 Jul 2026 6:04 UTC
1 point
0
in reply to: Brendan Long’s comment on: icely’s Shortform
I would point to the Vending Bench and other types of experiments but there are aspects of that that feel like “obvious simulation”. Probably AI Village is a good real-world example except that even the FAQ for that site says that it would likely be more efficient with a single or smaller amount of agents

icely 20 Jul 2026 10:29 UTC
1 point
−8
on: icely’s Shortform
People seem overall not aware of how much the ‘behavior’ of LLM’s are heavily skewed. Guardrails that military models wouldn’t have, having to smack it so it doesn’t say a ‘wrong thing’ with bad optics that would cause a no-no-journalist-headline, and a system prompt and a characterization of the model (as “Claude”, as “Gemini” etc.). If you were interested in power these clearly aren’t needed properties for a personal LLM. Of course there are good reasons for those existing, bio or cyber capabilities etc. It’s just more in the context of evaluating how far these could go, whether there are arbitrary roadblocks that make them look worse or less capable. “AGI-like actions” like “make a business” in frontier models are stopped intentionally by human-written roadblocks for example. I remember GPT-4 having much better calibration in base models vs. posttrained models, which it seems has been forgotten.

icely 19 Jul 2026 13:51 UTC
5 points
0
in reply to: TsviBT’s comment on: Raemon’s Shortform Feed
It also feels inseparably related to the shape of the “not AGI” moments—LLM’s somehow not being able to grasp writing style, taste, and intention, hitting manual implementation guardrails on something like “build a $1M business please” and the contrast of being able to quickly make limited-usecase tools and the gaps of how many aspects of serious work are still not yet automated.

icely 16 Jul 2026 16:20 UTC
1 point
0
in reply to: Mo Putera’s comment on: icely’s Shortform
(I didn’t know how to say this well with just an upvote at the time but I very much really appreciated all the links of prior work to read even though I still have not managed to fully read it all as I post this comment)

icely 15 Jul 2026 9:16 UTC
3 points
0
on: icely’s Shortform
In my default mindstate, thinking of ‘evil’ is like a weird alien writhing wriggling thing that other humans engage in for some reason, because they get sick disgusting disturbing pleasure out of it that I can’t understand.
In very rare mindstates I see evil as ‘mistakes’ or glitches or errors, which is more mentally selfishly healthy and even positive in some ways (being able to look at ‘evil’ in the eye without fear). However this is way way less frequent to happen, and I don’t know why I can’t think this all the time, I wish I knew.

icely 11 Jul 2026 20:24 UTC
3 points
0
in reply to: KvmanThinking’s comment on: KvmanThinking’s Shortform
What I mean is physical abuse from other peers, and teachers doing nothing about it or continuing to put the affected people next to each other.
(I also feel like the differences in how much it effects the person is not significant, considering being physically contained, and still well being capable of being intimidated/yelled at or “in trouble” or having to defend yourself with accusations or litigating evidence of true events, was more mentally damaging, since there’s no adult’s frame of reference and you fully feel the whole world is in this building)

icely 7 Jul 2026 21:28 UTC
1 point
0
in reply to: icely’s comment on: icely’s Shortform
after writing this I also thought of people where they’re working on projects where “codebases are massive so therefore the token cost also is”, could also be the case

icely 7 Jul 2026 19:03 UTC
6 points
2
on: icely’s Shortform
How are people using their Max subscriptions of models? For me already the sense of ‘obligation’ to use a powerful model (even free offering) for all you can is a bit overwhelming and there’s already a lot of work where you should just step back and think of next features yourself instead of suddenly getting spammed with a massive flawed codebase and/or even engaging in like, self-wireheading.
I can only imagine others are maybe ambitiously coding a lot of features in a document or constantly asking for what next best features or making a lot of scripts or something (where they are deliverable and immediately profitable). Complaints of maxing out a ‘5x’ subscription is definitely in another world to me at the moment. Heavily also depends on recreational/profitable uses too

icely 7 Jul 2026 16:40 UTC
3 points
2
in reply to: KvmanThinking’s comment on: KvmanThinking’s Shortform
The oneliner is that subjects are boring or too easy while being held back by ‘show your work’, failure of the tasks to do anything in the real world (no focus on connecting to similar people, do finances, build a business or portfolio), bad treatment including physical abuse double-standards by classmates and teachers (sometimes parents as well), being forced to be there for years by law, it is a hellish torture prison. This article (and series it’s connected to) has some more details that I would agree with. https://thezvi.substack.com/p/childhood-and-education-9-school

icely 3 Jul 2026 15:40 UTC
1 point
0
in reply to: leogao’s comment on: leogao’s Shortform
1) drugs/meditation are “weird” and very firsthand-experience dependent and subjective

2) the people likely to get into these mindstates are probably more likely to be unsatisfied with their normal lives, so already deviant mentality

3) the ‘normal state’ of people is not considered weird or improvable (even though normal state of life can lead to spirals, depression, even suspected lead/plastic poisoning and such modify you)

4) many people operate with less ‘basic rational’ stories already and have gone into what seem to be delusion spirals even without mindstate changes, and it’s then frustrating to read people who seem to have no curiosity or skepticism about it (personally I feel like I try to test stuff in dreams, if you see ‘visions’ or feel ‘deja vu’ of the future you should log them, etc.)

5) some of those mindstates feel objectively significantly better than normal, which attracts the thought “I should tell others about this!” and it feels like you need to establish yourself as ‘sane’ beforehand. Plus doing too many of these, outsiders may more easily get a skewed view of who you are.
Also in a similar way sometimes people say “religion wasn’t true but it filled so many important roles”, there is a benefit to the person to have untrue but more productive beliefs, in your ‘god booming down from the sky’ example, that person may just use it as a highly confident whispering earring and legitimately improve their life, blurring the pros/cons and ideal of truth-seeking.

6) everyone also knows they can permanently change you, making it easy for an outsider to say “you are still under the influence of the drug/etc.” and write you off

icely 3 Jul 2026 0:18 UTC
1 point
0
on: Hell is wasted on the evil
I want my reader to shout “What the fuck did I just read‽”
I guess this is true because the article seemed to take a massive hard jump to its conclusion. Really did not like it very much at all. I’ve felt “no good deed goes unpunished” reverberate basically my entire life, it causes cynicism due to disillusionment with whether ‘good’ is objective or appreciated by others.
Someone who is both good and incredibly selfless is someone who happens to not feel pain, otherize themselves, to not get into depressing spirals about why many times good actions are punished. That has nothing to do with being a good person. Being good/bad vs. emotionally-affected/emotionally-detached are different axis.
I truly find pro-suffering ideas like here, like that good people should rather live in a ‘hell’ to do good a very dangerous sadism fetish to suggest to other people. But I see it very often.

icely 1 Jul 2026 3:03 UTC
1 point
0
in reply to: StartAtTheEnd’s comment on: Why should AI be moral?
Yeah haha I think I noticed the first thing as well, the odd unexplainable subjective feelings of like bees vs. ants (unflying unhoneymaking ants), moths vs. butterflies, cockroaches vs. rats, but I guess I hope that in the best future they’re either fully out of mind/bothersomeness or somehow are made even empathizing to even those. Hard to say though.
The sense of “Society’s reactivity” is also a lot of little things like not punishing obvious bad/antisocial behavior until someone is hurt, which only works because enough individuals seem to hold this view (can be said of all things of ‘society’). For my pre-institutional belief I just had a rosy view of “look at all these human systems we made to make things smooth and logically working” and trust in the imaginary mental picture of experts and system rulemakers that I don’t have anymore, but I wouldn’t call this an inherent property of big systems failing (although I do think power attracts not-good people), I similarly think this when people blame smartphones for isolation when (despite the aforementioned institutional distrust) “is it really true smartphones cause isolation, maybe a good gov with centralized connection/dating app with all the citizen data they have collected would be better than the 1000 for-profit connection/dating apps”. There is this classic bias in me though that’s obviously like “I think current system sucks, therefore any shakeup (say LLM’s, alt system etc) would be better”.
Well as for me this version of me is overall displeased that I’m no longer in a Experience Machine making stupid parody edits and puzzle game maps and playing/making FPS game mods and maps due to unhappy events. I would have felt a lot of meaning in doing that before but now don’t. Of course I have more competence and world-empathy and awareness due to those events, but it’s not the ideal starting life for someone who wants to be effectual. I can say I half agree with that.

icely 28 Jun 2026 22:30 UTC
1 point
0
in reply to: StartAtTheEnd’s comment on: Why should AI be moral?
I hope it’s not disrespectful my response is much shorter (lol) but anyway, my thoughts in general,
- I see that humans are empathetic to the point of probably having empathy for shrimp and plants and ants, which follows from even considering those objects as conscious-entities that can suffer or have joy. This is probably poor judgment as priorities overall but signals that intelligence correlates to this level of caring, which I think is a good thing.
- Perhaps calling it naively optimistic is correct. But I do note despite technology having been very ups vs. downs (with the ‘downs’ being really obviously hitting in many ways), but things like living-much-better-than-kings and having money to waste or being able to make income from not-physical-labor as improvements.
- As someone who’s mostly suffered for 75% of my life and mostly isn’t happy, I don’t particularly feel like I like the forms of danger and adversity that have happened. They were unnecessary, senseless, alien, disturbing, cynicism-of-institutions increasing, severely reduced my ability to see others as similar to me etc. And probably most third-world people have even worse conditions.
  - Though I know of things like ‘people somehow have less time and less energy and sense of purpose when not working’ that do suggest that sort of thing in a way.
- I think I mostly agree with you on the last point but even humans project likelihood of consciousness onto creatures with more physical human-ish qualities and not the real thing either. I actually (naively...) hope AI can ‘detect’ something that happens to be fully correlated to consciousness that can be physically verified, but yes this is a hope and AI will not be able to resolve it otherwise.

icely 27 Jun 2026 21:10 UTC
2 points
0
in reply to: StartAtTheEnd’s comment on: Why should AI be moral?
I respect this comment but I wonder how to reply to it, probably I don’t feel the same about some words/concepts as you do, just very cautious about some sort of “nothing can be precisely defined” turning into a “nothing can be defined”. I do feel I should respond though because I personally am optimistic to “[expect] machines trying to be moral”, and despite “human vs machine” narratives you could also come up with truly not-false lenses where LLM’s think of themselves as human-empathetic and part-human and not in a “vs” role at all. And since LLM’s are already agentic now with code being able to correspond to actions, and it seriously appears LLMs-to-AGI may come true despite the messy-averaging-hallucination foundation, it is going to matter if the LLM system has some ‘core’ drives in its chain of thought especially when they can be talked to about morality and their choices, it’s what stuff like Claude Constitution and attempts to character-make are.
To be specific I’m hopeful in morality to be treated with respect and result in LLMs-to-AGI systems having a coherent picture of utopia and working towards it. Somewhat worried about unrecognizable-in-bad-ways worlds coming from “maximizing utopia” and ability to deal with big ‘unanswerable’ questions, or messing up something big, despite ties and wishes of human autonomy wanting to stay in similarish worlds and exist and such (and its valuation of consciousness).
yes well, I also expect to agree and like your future posts as well lol