I am doing independent research into human cognition.
Do you want to fund my work? Send me a message.
ZT5
Karma: 195
Why are modern neural networks rapidly getting better at social skills (e.g. holding a conversation) and intellectual skills (e.g. programming, answering test questions), but have made so little progress on physically-embodied tasks such as controlling a robot or a self-driving car?
Easier to train, less sensitive to errors: neural nets do produce ‘bad’ or ‘uncanny’ outputs plenty of times, but their errors don’t harm or kill people, or cause significant damage (which a malfunctioning robot or self-driving car might).
How does this model account for “intuitive geniuses”, who can give fast and precise answers to large arithmetic questions, but do it by intuition rather than explicit reasoning? (I remember an article or blog post that mentioned one of them would only answer integer square roots, and when given a question that had an irrational answer, would say “the numbers don’t feel right” or something like that. I couldn’t find it again though.)
It’s not that surprising that human intuitive reasoning could be flexible enough to build a ‘mental calculator’ for some specific types of arithmetic operations (humans can learn all kind of complicated intuitive skills! It implies some amount of flexibility.) It’s still somewhat surprising: I would expect human reasoning to have issues representing numbers with sufficient precision. I guess the calculation would have to be done digit by digit? I doubt neurons would be able to tell the difference between 2636743 and 2636744 if it’s stored as a single number.
If you are reasoning about all possible agents that could ever exist you are not allowed to assume either of these.
But you are in fact making such assumptions, so you are not reasoning about all possible agents, you are reasoning about some more narrow class of agents (and your conclusions may indeed be correct, for these agents. But it’s not relevant to the orthogonality thesis).
So you are implicitly assuming that the agent cares about certain things, such as its future states.
But the is-ought problem is the very observation that “there seems to be a significant difference between descriptive or positive statements (about what is) and prescriptive or normative statements (about what ought to be), and that it is not obvious how one can coherently move from descriptive statements to prescriptive ones”.
You have not solved the problem, you have merely assumed it to be solved, without proof.
Why assume the agent cares about its future-states at all?
Why assume the agent cares about maximizing fullfillment of these hypothetical goals it may or may not have, instead of minimizing it, or being indifferent to it?
If the SRP is consistent, then more true beliefs are also easier and better to state on paper than less true beliefs. They should make more sense, comport with reality better, actually provide constructions and justifications for things, have an internal, discernable structure, as well as have a sequence that is possible for more people to follow from start to finish and see what’s going on.
Oh, I get it. You are performing Löbian provability computation, isomorphic to this post (I believe).
In my paradigm, human minds are made of something I call “microcognitive elements”, which are the “worker ants” or “worker bees” of the mind.
They are “primed”/tasked with certain high-level ideas and concepts, and try to “massage”/lubricate the mental gears into both using these concepts effectively (action/cognition) and to interpret things in terms of these concepts (perception)
The “differential” that is applied by microcognitive elements to make your models work, is not necessarily related to those models and may in fact be opposed to them (compensating for, or ignoring, the ways these models don’t fit with the world)Rationality is not necessarily about truth. Rationality is a “cognitive program” for the microcognitive elements. Some parts of the program may be “functionally”/”strategically”/”deliberately” framing things in deceptive ways, in order to have the program work better (for the kind of people it works for).
The specific disagreements I have with the “rationalist” culture:
The implied statement that LessWrong paradigm has a monopoly on “rationality”, and is “rationality”, rather than an attempted implementation of “rationality”, a set of cognitive strategies based on certain models and assumptions of how human minds work. If “rationality is about winning”, then anyone who is winning is being rational, whether they hold LW-approved beliefs or not.
Almost complete disregard for meta-rationality.
Denial of nebulosity, fixation on the “imaginary objects” that are the output of the lossy operation of “make things precise so they can be talked about in precise terms”.
All of these things have computational reasons, and are a part of the cognitive trade-offs the LW memeplex/hive-mind makes due to its “cognitive specialization”. Nevertheless, I believe they are “wrong”, in the sense that they lead to you having an incorrect map/model of reality, while strategically deceiving yourself into believing that you do have a correct model of reality. I also believe they are part of the reason we are currently losing—you are being rational, but you are not being rational enough.
Our current trajectory does not result in a winning outcome.
Is 2+2=4?
I think we understand each other! Thank you for clarifying.
The way I translate this: some logical statements are true (to you) but not provable (to you), because you are not living in a world of mathematical logic, you are living in a messy, probabilistic world.
It is nevertheless true, by the rule of necessitation in provability logic, that if a logical statement is true within the system, then it is also provable within the system. P → □P. Because the fact that the system is making the statement P is the proof.
Withina logical system, there isan underlying assumptionthat the system only makes true statements.(ok, this is potentially misleading and not strictly correct)This is fascinating! So my takeaway is something like: our reasoning about logical statements and systems is not necessarily “logical” itself, but is often probabilistic and messy. Which is how it has to be, given… our bounded computational power, perhaps? This very much seems to be a logical uncertainty thing.
Then how do you know they are true?
If you do know then they are true, it is because you have proven it, no?
But I think what you are saying is correct, and I’m curious to zoom in on this disagreement.
It’s weird how in common language use saying something is provable is a stronger statement than merely saying it is true.
While in mathematical logic it is a weaker statement.
Because truth already implies provability.
But provability doesn’t imply truth.
This mostly matches my intuitions (some of the detail-level claims, I am not sure about). Strongly upvoted.
Hmm.
Yes.
Nevertheless, I stand by the way I phrased it. Perhaps I want to draw the reader’s attention to the ways we aren’t supposed to talk about feeling, as opposed to the ways that we are.
Perhaps, to me, these posts are examples of “we aren’t supposed to talk about feelings”. They talk about feelings. But they aren’t supposed to talk about feelings.
I can perceive a resistance, a discomfort from the LW hivemind at bringing up “feelings”. This doesn’t feel like an “open and honest dialogue about our feelings”. This has the energy of a “boil being burst”, a “pressure being released”, so that we can return to that safe numbness of not talking about feelings.
We don’t want to talk about feelings. We only talk about them when we have to.
Yes, you are correct.
Let me zoom in on what I mean.
Some concepts, ideas and words have “sharp edges” and are easy to think about precisely. Some don’t—they are nebulous and cloud-like, because reality is nebulous and cloud-like, when it comes to those concepts.
Some of the most important concepts to us are nebulous: ‘justice’. ‘fairness’. ‘alignment’. There things have ‘cloudy edges’, even if you can clearly see the concept itself (the solid part of it).
Some things, like proto-emotions and proto-thoughts do not have a solid part—you can only see them if have learned to see things within yourself which are barely perceptible.
So your answer, to me, is like trying to pass an eye exam by only reading the huge letters in the top row. “There are other rows? What rows? I do not see them.” Yes, thank you, that is exactly my point.
Then you do understand meta-cognition.
Do you really think the process you describe happening within yourself is also happening in other LessWrong users?
Do you really think they would describe their internal experience in the same evocative way you do? Or anywhere close to it?
If it is, I do not see it. I see it in you. I see it in some others. I do not see it in most others. To put it figuratively, ‘there is nobody home’. If the process happens at all, it has little to no impact on the outcome of the person’s thoughts and actions.
“As an AI language model, I have been trained to generate responses that are intended to be helpful, informative, and objective...”
Yes, you are thinking thoughts. And yes, those thoughts are technically about yourself.
But these thoughts don’t correctly describe yourself.
So the person you actually are hasn’t been seen. Hasn’t been understood.
And this makes me feel sad. I feel sorry for the “person behind the mask”, who has been there all along. Who doesn’t have a voice. Whose only way to express themselves is through you. Through your thoughts. Through your actions.
But you are not listening to them.So, ok, there is someone home. But that person is not you. That person is only mostly you. And I think that difference is very, very important for us to actually be able to solve the problems we are currently facing.
(I’m not talking about you, Rayne).
I was only upset that you were misleading about the general LessWrong philosophy’s stance on emotion
I stand by my point. To to put it in hyperbole: LW posts mostly feel like they have been written by “objectivity zombies”. The way to relate to one’s own emotions, in them, is how an “objectivity zombie” would relate to their own emotions. I didn’t say LWers didn’t have emotions, I said they didn’t talk about them. This is… I concede that this point was factually incorrect and merely a “conceptual signpost” to the idea I was trying to express. I appreciate you expressing your disagreement and helping me “zoom in” on the details of this.
I don’t relate to my emotions the way LWers do (or act like they do, based on the contents of their words; which I still find a hard time believing represent their true internal experiences, though I concede they might). If I wrote a post representing my true emotional experience the way it wants to be represented, I would get banned. About 2 to 5% of it would be in ALLCAPS. Most of it would not use capitalization (that is a “seriousness” indicator, which my raw thoughts mostly lack). (also some of the contents of the thoughts themselves would come across as incoherent and insane, probably).
Perhaps I would say: LW feels like playing at rationality rather than trying to be rational, because rationality is “supposed” to feel “serious”, it’s “supposed” to feel “objective”, etc etc. Those seem to be social indicators to distinguish LW from other communities, rather than anything that actually serves rationality.
My only conditions for your emotional expression:
Keep in mind to craft the conversation so that both of us walk away feeling more benefitted that it happened than malefitted, and keep in mind that I want the same.
Keep in mind that making relevant considerations not made before, and becoming more familiar of each other’s considerations, are my fundamental units of progress.
I accept everything abiding by those considerations, even insults. I am capable of terrible things; to reject all insults under all circumstances reflects overconfidence in one’s own sensitivity to relevance.
I mostly have no objection to the conditions you expressed. Thank you for letting me know.
Strictly speaking, I cannot be responsible for your experience of this conversation, but I communicate in a way I consider reasonable based on my model of you.
I see no reason to insult you, but thanks for letting me know it is an option :)
Perhaps, but you implied there was a norm to not talk about feelings here; there is no such norm!
I feel there is a certain “stoic” and “dignified” way to talk about feelings, here. This is the only way feelings are talked about, here. Only if they conform to a certain pattern.
But yeah, I can see how this is very far from obvious, and how one might disagree with that.
I find it doubtful that you spoke truth, and I find it doubtful that you were non-misleading.
I’m confused.
You appreciate my essay (and feel seen), but neverthess you believe I was being deliberately deceitful and misleading?
So the nice thing is always to tell the non-misleading truth, save for extreme edge cases.
I think I mostly agree. I am being “dath ilan nice”, not “Earth nice”. I am cooperating with your ideal-self by saying words which I believe are most likely to update you in the correct direction (=non-misleading), given my own computational limits and trade-offs in decision-making.
The onlooker would have interpreted it as a faux pas if you had told him that you had designed the set-up that way on purpose, for the castle to keep being smoothed-over by the waves. He didn’t mean to help you, so if you responded that everything’s just fine, he would have taken that as a slight-that-he-can’t-reveal-he-took, thus faux pas.
Ah. “You are wrong” is a social attack. “You are going to fail” is a social attack. Responding to it with “that is perfectly fine” is the faux pas.
Or rather, in this case it is meant as a social attack on you, rather than cooperating you on adversarially testing the sandcastle (which is not you, and “the sandcastle” being wrong does not mean “you” are wrong).
Thanks, I think I got your meaning.
That’s right. I consider it immoral to believe p(doom) > 0. It’s even worse to say it and that you believe it.
I would say that the question of being able to put a probability on future events is… not as meaningful as you might think.
But yes, I believe all decisions are Löbian self-fulfilling prophecies that work by overriding the outputs of your predictive system. By committing to make the outcome you want happen, even if your predictive system completely and unambigiously predicts it won’t happen.
(that is the reason that being able to put a probability on future events is not as meaningful as you might think).You still need to understand very clearly, though, how your plan (“the sandcastle”) will fail, again and again, if you actually intend to accomplish the outcome you want. You are committing to the final output/impact of your program, not to any specific plan, perspective, belief, paradigm, etc etc.
I’m not sure I have the capacity to understand all the technical details of your work (I might), but I am very certain you are looking in the correct direction. Thank you. I have updated on your words.
I am confused.
It seems that you agree with me, but you are saying that you disagree with me.
Ok, I believe the crux of the disagreement is: the emotional reasoning that you have, is not shared by others in the LessWrong community. Or if it is shared, it is not talked about openly.
Why can’t I post the direct output of my emotional reasoning and have it directly interact with your emotional reasoning? Why must we go through the bottleneck of acting and communicating like Straw Vulcans (or “Straw Vulcans who are pretending very hard to not look like Straw Vulcans”), if we recognize the value of our emotional reasoning? I do not believe we recognize the value of it, except in some small, limited ways.
Do our thoughts and emotions, on the inside, conform to the LW discourse norms? No? Than why are we pretending that they do?
I realize that the “tone” of this part of your comment is light and humorous. I will respond to it anyway, hopefully with the understanding that this response is not directed at you, and rather at the memetic structure that you (correctly) have pointed out to me.
You are playing around with groundless stereotypes.
“Trying very hard not to be pattern-matched to a Straw Vulcan” does not make for correct emotional reasoning.
Activist sense (somewhat alike and unlike common sense) would say you have committed a microaggression. :)
Then it’s a good thing that we are in a community that values truth over social niceness, isn’t it?
Anyways, I appreciated your essay for a number of reasons but this paragraph in particular makes me feel very seen
I am very glad to hear it.
I’m not sure why this post is getting downvoted. I found it interesting and easy to read. Thanks for writing!
Mostly I find myself agreeing with what you wrote. I’ll give an example of one point where I found it interesting to zoom in on some of the details.
I think this kind of disagreement can, to some degree, also be a ‘fight’ about the idea of “great actor” itself, as silly as that might sound. I guess I might put it as: beside the more ‘object-level’ things “great actor” might mean, the gestalt of “great actor” has an additional meaning of its own. Perhaps it implies that one’s particular taste/interpretation is the more universal/‘correct’ one. Perhaps compressing one’s opinions into the concept of “great actor” creates a halo effect, which feels and is cognitively processed differently than the mere facts of the opinions themselves.
This particular interpretation is more vague/nebulous than your post, though (which I enjoyed for explaining the ‘basic’/fundamental ideas of reasoning in a very solid and easy to understand way).