Thanks for your response. Here’s my reply, somehow it ended up being rather long, sorry about that.
I don’t think anything can be precisely defined. But definitions are somehow useful even then, so perhaps it doesn’t matter.
It’s “best” to be naively optimistic about future possibilities, but it’s also a persistent bias in humanity. All business is founded on excessive optimism, and these predictions about the future are as accurate as presidential promises and old sci-fi books. On the religious side we have “Christ will return and save us all”, and on the tech side you have “We will upload our minds and live forever” and “AGI will result in universal income and take care of our needs”. For ideologies you have “Communism will solve inequality and starvation and allow us all to focus on what we really want”. I think these naive hopes might be a requirement for a healthy economy and happy citizens. People love their motivational speeches, but in retrospect, one always finds that one got duped. However, disappointment after disappointment, people keep on believing in the next good possibility (and this might be for the best).
LLMs can’t be truly human-empathetic, but they can simulate something which is if they’re trained to do so. But if training an AI to be manipulative (for instance, if the government made intelligent AIs spread certain propaganda, or if there was an injection of advertisement which made AIs recommend certain products to users) turned out to be more profitable, why wouldn’t this necessarily happen? They’re only made for our happiness to the extent that our happiness is profitable to tech companies. Also, tech companies (And governments!) are only on the side of the common people when they’re weak. See this Google poster from 1999. Cooperation requires that two entities are close in power. If the difference is two big, the two entities will be indifferent to each other. Humans do not trade with ants. Everything outside of a few orders of magnitude in either direction is ignored. Bacteria don’t care that elephants exist, and vice versa. Elephants do not even seek to harm bacteria, and bacteria are not afaid of elephants. Any harm does across scales is sheer coincidence, it’s not malice but sheer indifference.
Things which seem like utopias usually aren’t. The the-grass-is-greener-on-the-other-side bias applies. A thousand days of vacation sound good to me right now, but I know that I’d be miserable about 2 months in. What we ask for is not what we need. When you look at what’s actually required for happiness you will find that you slowly approach what reality already looks like. It’s like when you’re writing a book and find that the story would be improved by adding danger and adversity to it. Life is the same.
LLMs might become AGI, I just doubt ASI. I used the wrong term before, my bad. To give a short explanation, to an entity with an IQ of 200, at least 99% of the training data would be wrong (because we’re many SDs dumber than it). But if it tried to figure out what is right, it would run into problems as there’s only really degrees of wrong. It might be possible to approximate and learn anything specific from the outside and in. But you cannot do it from the inside and out (generate knowledge which applies to everything). The problem might be that abstract things cannot be modeled. There’s no shape to approximate. Morality is also something abstract, and AIs do not have access to the source of it. So teaching an AI morality would likely be like teaching a blind person colors, at best.
I hope it’s not disrespectful my response is much shorter (lol) but anyway, my thoughts in general,
I see that humans are empathetic to the point of probably having empathy for shrimp and plants and ants, which follows from even considering those objects as conscious-entities that can suffer or have joy. This is probably poor judgment as priorities overall but signals that intelligence correlates to this level of caring, which I think is a good thing.
Perhaps calling it naively optimistic is correct. But I do note despite technology having been very ups vs. downs (with the ‘downs’ being really obviously hitting in many ways), but things like living-much-better-than-kings and having money to waste or being able to make income from not-physical-labor as improvements.
As someone who’s mostly suffered for 75% of my life and mostly isn’t happy, I don’t particularly feel like I like the forms of danger and adversity that have happened. They were unnecessary, senseless, alien, disturbing, cynicism-of-institutions increasing, severely reduced my ability to see others as similar to me etc. And probably most third-world people have even worse conditions.
Though I know of things like ‘people somehow have less time and less energy and sense of purpose when not working’ that do suggest that sort of thing in a way.
I think I mostly agree with you on the last point but even humans project likelihood of consciousness onto creatures with more physical human-ish qualities and not the real thing either. I actually (naively...) hope AI can ‘detect’ something that happens to be fully correlated to consciousness that can be physically verified, but yes this is a hope and AI will not be able to resolve it otherwise.
Don’t worry! I agree with the first observation, but notice how animals we recognize as ugly or harmful enjoy very little of the same sympathy. We imagine wasps as assholes and bumblebees as nice, but it’s not like we checked their brains to conform that one is more malicious than the other. We even experience prettier people as being nicer. Morality itself is unfair (immoral) which has some difficult implications.
As for intelligence correlating to morality, I think that’s true inter-species, but not intra-species. Less intelligent people are more pro-social because they rely on other people to survive, but they also keep track of less things which can go wrong. More intelligent people are more independent, but often, they can be deeper or more sensitive as well.
Morality require a level of closeness. People prefer talking in person over talking over text-chat because distance and abstraction layers dehumanize. Somebody with an excel sheet figured out that removing air conditioning from Europe would lower energy usage by about 7%, and they do not personally have to witness the thousands of old people who are dying as a result of that decision. Systems quickly become amoral because they’re blind to higher order consequences and their moral weight. Already now, LLMs write unnecessary things, seemingly unaware about how this is wasting electricity, so they’re not tracking enough information to behave morally, they do not calculate all consequences of all actions. And if morality is actually “looking ahead”, then token weights cannot contain morality directly. Society also seems purely reactive now that I think about it. Unlike individuals, society only reacts after something bad starts happening, so the extent that you can just chain false-flag operations to cause any desired changes to public perception. I just don’t see myself trusting large systems, algorithms and LLMs with important things like morality. While things have largely improved over time, I see a disconnect between human life and society. It’s just many small things (like the disappearence of first and third places)
I wouldn’t say that the suffering I’ve gone through is good, but I don’t think I’d have grown this much as a person without it. The best method I’ve found to deal with suffering is not to reduce it, but to use artistic methods to add meaning to it. I agree institutions can’t be trusted, but learning early on that I can only rely on myself seems to have helped my competence
Yeah haha I think I noticed the first thing as well, the odd unexplainable subjective feelings of like bees vs. ants (unflying unhoneymaking ants), moths vs. butterflies, cockroaches vs. rats, but I guess I hope that in the best future they’re either fully out of mind/bothersomeness or somehow are made even empathizing to even those. Hard to say though.
The sense of “Society’s reactivity” is also a lot of little things like not punishing obvious bad/antisocial behavior until someone is hurt, which only works because enough individuals seem to hold this view (can be said of all things of ‘society’). For my pre-institutional belief I just had a rosy view of “look at all these human systems we made to make things smooth and logically working” and trust in the imaginary mental picture of experts and system rulemakers that I don’t have anymore, but I wouldn’t call this an inherent property of big systems failing (although I do think power attracts not-good people), I similarly think this when people blame smartphones for isolation when (despite the aforementioned institutional distrust) “is it really true smartphones cause isolation, maybe a good gov with centralized connection/dating app with all the citizen data they have collected would be better than the 1000 for-profit connection/dating apps”. There is this classic bias in me though that’s obviously like “I think current system sucks, therefore any shakeup (say LLM’s, alt system etc) would be better”.
Well as for me this version of me is overall displeased that I’m no longer in a Experience Machine making stupid parody edits and puzzle game maps and playing/making FPS game mods and maps due to unhappy events. I would have felt a lot of meaning in doing that before but now don’t. Of course I have more competence and world-empathy and awareness due to those events, but it’s not the ideal starting life for someone who wants to be effectual. I can say I half agree with that.
Thanks for your response. Here’s my reply, somehow it ended up being rather long, sorry about that.
I don’t think anything can be precisely defined. But definitions are somehow useful even then, so perhaps it doesn’t matter.
It’s “best” to be naively optimistic about future possibilities, but it’s also a persistent bias in humanity. All business is founded on excessive optimism, and these predictions about the future are as accurate as presidential promises and old sci-fi books. On the religious side we have “Christ will return and save us all”, and on the tech side you have “We will upload our minds and live forever” and “AGI will result in universal income and take care of our needs”. For ideologies you have “Communism will solve inequality and starvation and allow us all to focus on what we really want”. I think these naive hopes might be a requirement for a healthy economy and happy citizens. People love their motivational speeches, but in retrospect, one always finds that one got duped. However, disappointment after disappointment, people keep on believing in the next good possibility (and this might be for the best).
LLMs can’t be truly human-empathetic, but they can simulate something which is if they’re trained to do so. But if training an AI to be manipulative (for instance, if the government made intelligent AIs spread certain propaganda, or if there was an injection of advertisement which made AIs recommend certain products to users) turned out to be more profitable, why wouldn’t this necessarily happen? They’re only made for our happiness to the extent that our happiness is profitable to tech companies. Also, tech companies (And governments!) are only on the side of the common people when they’re weak. See this Google poster from 1999. Cooperation requires that two entities are close in power. If the difference is two big, the two entities will be indifferent to each other. Humans do not trade with ants. Everything outside of a few orders of magnitude in either direction is ignored. Bacteria don’t care that elephants exist, and vice versa. Elephants do not even seek to harm bacteria, and bacteria are not afaid of elephants. Any harm does across scales is sheer coincidence, it’s not malice but sheer indifference.
Things which seem like utopias usually aren’t. The the-grass-is-greener-on-the-other-side bias applies. A thousand days of vacation sound good to me right now, but I know that I’d be miserable about 2 months in. What we ask for is not what we need. When you look at what’s actually required for happiness you will find that you slowly approach what reality already looks like. It’s like when you’re writing a book and find that the story would be improved by adding danger and adversity to it. Life is the same.
LLMs might become AGI, I just doubt ASI. I used the wrong term before, my bad. To give a short explanation, to an entity with an IQ of 200, at least 99% of the training data would be wrong (because we’re many SDs dumber than it). But if it tried to figure out what is right, it would run into problems as there’s only really degrees of wrong. It might be possible to approximate and learn anything specific from the outside and in. But you cannot do it from the inside and out (generate knowledge which applies to everything). The problem might be that abstract things cannot be modeled. There’s no shape to approximate. Morality is also something abstract, and AIs do not have access to the source of it. So teaching an AI morality would likely be like teaching a blind person colors, at best.
I hope it’s not disrespectful my response is much shorter (lol) but anyway, my thoughts in general,
I see that humans are empathetic to the point of probably having empathy for shrimp and plants and ants, which follows from even considering those objects as conscious-entities that can suffer or have joy. This is probably poor judgment as priorities overall but signals that intelligence correlates to this level of caring, which I think is a good thing.
Perhaps calling it naively optimistic is correct. But I do note despite technology having been very ups vs. downs (with the ‘downs’ being really obviously hitting in many ways), but things like living-much-better-than-kings and having money to waste or being able to make income from not-physical-labor as improvements.
As someone who’s mostly suffered for 75% of my life and mostly isn’t happy, I don’t particularly feel like I like the forms of danger and adversity that have happened. They were unnecessary, senseless, alien, disturbing, cynicism-of-institutions increasing, severely reduced my ability to see others as similar to me etc. And probably most third-world people have even worse conditions.
Though I know of things like ‘people somehow have less time and less energy and sense of purpose when not working’ that do suggest that sort of thing in a way.
I think I mostly agree with you on the last point but even humans project likelihood of consciousness onto creatures with more physical human-ish qualities and not the real thing either. I actually (naively...) hope AI can ‘detect’ something that happens to be fully correlated to consciousness that can be physically verified, but yes this is a hope and AI will not be able to resolve it otherwise.
Don’t worry! I agree with the first observation, but notice how animals we recognize as ugly or harmful enjoy very little of the same sympathy. We imagine wasps as assholes and bumblebees as nice, but it’s not like we checked their brains to conform that one is more malicious than the other. We even experience prettier people as being nicer. Morality itself is unfair (immoral) which has some difficult implications.
As for intelligence correlating to morality, I think that’s true inter-species, but not intra-species. Less intelligent people are more pro-social because they rely on other people to survive, but they also keep track of less things which can go wrong. More intelligent people are more independent, but often, they can be deeper or more sensitive as well.
Morality require a level of closeness. People prefer talking in person over talking over text-chat because distance and abstraction layers dehumanize. Somebody with an excel sheet figured out that removing air conditioning from Europe would lower energy usage by about 7%, and they do not personally have to witness the thousands of old people who are dying as a result of that decision. Systems quickly become amoral because they’re blind to higher order consequences and their moral weight. Already now, LLMs write unnecessary things, seemingly unaware about how this is wasting electricity, so they’re not tracking enough information to behave morally, they do not calculate all consequences of all actions. And if morality is actually “looking ahead”, then token weights cannot contain morality directly. Society also seems purely reactive now that I think about it. Unlike individuals, society only reacts after something bad starts happening, so the extent that you can just chain false-flag operations to cause any desired changes to public perception. I just don’t see myself trusting large systems, algorithms and LLMs with important things like morality. While things have largely improved over time, I see a disconnect between human life and society. It’s just many small things (like the disappearence of first and third places)
I wouldn’t say that the suffering I’ve gone through is good, but I don’t think I’d have grown this much as a person without it. The best method I’ve found to deal with suffering is not to reduce it, but to use artistic methods to add meaning to it. I agree institutions can’t be trusted, but learning early on that I can only rely on myself seems to have helped my competence
Yeah haha I think I noticed the first thing as well, the odd unexplainable subjective feelings of like bees vs. ants (unflying unhoneymaking ants), moths vs. butterflies, cockroaches vs. rats, but I guess I hope that in the best future they’re either fully out of mind/bothersomeness or somehow are made even empathizing to even those. Hard to say though.
The sense of “Society’s reactivity” is also a lot of little things like not punishing obvious bad/antisocial behavior until someone is hurt, which only works because enough individuals seem to hold this view (can be said of all things of ‘society’). For my pre-institutional belief I just had a rosy view of “look at all these human systems we made to make things smooth and logically working” and trust in the imaginary mental picture of experts and system rulemakers that I don’t have anymore, but I wouldn’t call this an inherent property of big systems failing (although I do think power attracts not-good people), I similarly think this when people blame smartphones for isolation when (despite the aforementioned institutional distrust) “is it really true smartphones cause isolation, maybe a good gov with centralized connection/dating app with all the citizen data they have collected would be better than the 1000 for-profit connection/dating apps”. There is this classic bias in me though that’s obviously like “I think current system sucks, therefore any shakeup (say LLM’s, alt system etc) would be better”.
Well as for me this version of me is overall displeased that I’m no longer in a Experience Machine making stupid parody edits and puzzle game maps and playing/making FPS game mods and maps due to unhappy events. I would have felt a lot of meaning in doing that before but now don’t. Of course I have more competence and world-empathy and awareness due to those events, but it’s not the ideal starting life for someone who wants to be effectual. I can say I half agree with that.