Software engineering, parenting, cognition, meditation, other
Linkedin, Facebook, Admonymous (anonymous feedback)
Gunnar_Zarncke
Karma: 8,395
I have discussed the ChatGPT responses in some depth with a friend and shed some light on the behavior:
ChatGPT does know that Tequila is associated with sugar—via the inulin in the Tequila plant (it does bring this up in the dialog). That the sugar is completely gone via distillation is a complex logical inference that it might come up with via step-by-step reasoning but that it may not have seen in text (or memorized).
Taste is affected by many things. While it is logical in a mechanistic sense that sweetness depends on sugar being present, that’s not all there is about taste. Ingredients might alter taste perception, e.g., flavor enhancers or think miracle berries. Sweetness might also result from interactions between the ingredients, like freeing sugar from other ingredients.
There are probably a lot of texts out there where people claim that stuff X has property Y that it doesn’t, in fact, have—but ChatGPT has no way to figure this out.
I’m not saying that this is the case with ChatGPT here. I’m saying the answer is more complicated than “Tequila has no sugar and thus can’t make things sweet, and ChatGPT is inconsistent about it.”
Part of the answer is, again, that ChatGPT can give an excellent impression of someone who knows a lot (like the detail about inulin) and seems to be able to reason but is not actually doing this on top of a world model. It may seem like it has a systematic understanding of what sweetness is, or taste, but it only draws on text. It is amazing what it does, but its answers do not result from reasoning thru a world model but from what other people have written after they used their world model. Maybe future GPTs will get there, but right now, you have to take each answer it gives as a combination of existing texts.
Reminding again of Paul Graham on Twitter:
For me one of the biggest surprises about current generative AI research is that it yields artificial pseudo-intellectuals: programs that, given sufficient examples to copy, can do a plausible imitation of talking about something they understand.
ADDED: And how much people are fooled by this, i.e., seem to assume that reasoning—of misdirection is going on that is not.
It is interesting from an alignment perspective to try to align yourself. What is aligning what with what? Reflection is a bit like (recursive) self-improvement in AIs, only on limited to the “software” level of the brain.
Personality traits are highly heritable and not very malleable/depend on the early environment. Indeed more experience reduces personality:
Decades of research have shown that about half of individual differences in personality traits is heritable. Recent studies have reported that heritability is not fixed, but instead decreases across the life span. [...] For most traits, findings provided evidence for an increasing relative importance of life
experiences contributing to personality differences across the life span.How Genetic and Environmental Variance in Personality Traits Shift Across the Life Span: Evidence From a Cross-National Twin Study (just add “gwern” to your heritability Google search)
I don’t think this disproves shard theory. I think that differences in small children’s attention or emotional regulation levels lead to these differences. Shards will form around things that happen reliably in contexts created by the emotional behaviors or the objects of attention. Later on, with more context and abstraction, some of these shards may coalesce or be outbid by more generally adaptive shards.
ADDED: Hm, it seems you have seen The heritability of human values: A behavior genetic critique of Shard Theory which has much more of this.
Please elaborate.
GPT also has problems with the Linda problem for the same reason:
https://twitter.com/dggoldst/status/1598317411698089984
Hm, I used the inline comment function but somehow this comment doesn’t show up inline.
if I have a bias that a field is hard to break into
Where does your bias come from?
See the studies listed above.
I found two studies that seem relevant:
Naturalistic Stimuli in Affective Neuroimaging: A Review
Naturalistic stimuli such as movies, music, and spoken and written stories elicit strong emotions and allow brain imaging of emotions in close-to-real-life conditions. Emotions are multi-component phenomena: relevant stimuli lead to automatic changes in multiple functional components including perception, physiology, behavior, and conscious experiences. Brain activity during naturalistic stimuli reflects all these changes, suggesting that parsing emotion-related processing during such complex stimulation is not a straightforward task. Here, I review affective neuroimaging studies that have employed naturalistic stimuli to study emotional processing, focusing especially on experienced emotions. I argue that to investigate emotions with naturalistic stimuli, we need to define and extract emotion features from both the stimulus and the observer.
https://www.frontiersin.org/articles/10.3389/fnhum.2021.675068/full
An Integrative Way for Studying Neural Basis of Basic Emotions With fMRI
How emotions are represented in the nervous system is a crucial unsolved problem in the affective neuroscience. Many studies are striving to find the localization of basic emotions in the brain but failed. Thus, many psychologists suspect the specific neural loci for basic emotions, but instead, some proposed that there are specific neural structures for the core affects, such as arousal and hedonic value. The reason for this widespread difference might be that basic emotions used previously can be further divided into more “basic” emotions. Here we review brain imaging data and neuropsychological data, and try to address this question with an integrative model. In this model, we argue that basic emotions are not contrary to the dimensional studies of emotions (core affects). We propose that basic emotion should locate on the axis in the dimensions of emotion, and only represent one typical core affect (arousal or valence). Therefore, we propose four basic emotions: joy-on positive axis of hedonic dimension, sadness-on negative axis of hedonic dimension, fear, and anger-on the top of vertical dimensions. This new model about basic emotions and construction model of emotions is promising to improve and reformulate neurobiological models of basic emotions.
https://www.frontiersin.org/articles/10.3389/fnins.2019.00628/full
I also put higher probability on AGI also using fast serial coprocessors to unlock algorithmic possibilities that brains don’t have access to, both for early AGI and in the distant future. (Think of how “a human with a pocket calculator” can do things that a human can’t. Then think much bigger than that!)
Does anybody know of research in this direction?
And this is an example of our more general dispositions where I tend to think “10% of evolutionary psychology is true important things that we need to explain, let’s get to work explaining them properly” and Jacob tends to think “90% of evolutionary psychology is crap, let’s get to work throwing it out”. These are not inconsistent! But they’re different emphases.
Top highlight. Nice reflection.
You may want to have a look at the Reply to Eliezer on Biological Anchors post, which itself refers to Forecasting transformative AI timelines using biological anchors. I think your writeup falls into this wider category, and you may see which of the discussed estimates (which get weighted together in the post) is closest to your approach or whether it is systematically different.
I agree that heuristics would have been a better word choice, but its intended purpose was clear to me. Bias and heuristic look at the same thing through a different lens.
I had a look at The Plan and noticed something I didn’t notice before: You do not talk about people and organization in the plan. I probably wouldn’t have noticed if I hadn’t started a project too, and needed to think about it. Google seems to think that people and team function play a big role. Maybe your focus in that post wasn’t on people, but I would be interested in your thoughts on that too: What role did people and organization play in the plan and its implementation? What worked, and what should be done better next time?
To me, ChatGPT reads like people would explain their reasoning missteps. That’s because most people don’t systematically reason all the time—or have a comprehensive world model.
Most people seem to go through life on rote, seemingly not recognizing when something doesn’t make sense because they don’t expect anything to make sense.
-- Aiyen
And the same applies to most text ChatGPT has seen.
ChatGPT can’t concentrate and reason systematically at all, though the “let’s think step by step” is maybe a step (sic) in that direction). Humans Who Are Not Concentrating Are Not General Intelligences and ChatGPT is quite a lot like that. If you expect to discuss with ChatGPT like with a rationalist, you are up for disappointment. Quite an understandable disappointment. Paul Graham on Twitter today:
For me one of the biggest surprises about current generative AI research is that it yields artificial pseudo-intellectuals: programs that, given sufficient examples to copy, can do a plausible imitation of talking about something they understand.
I don’t mean this as an attack on this form of AI. The imitations continue to improve. If they get good enough, we’re splitting hairs talking about whether they “actually” understand what they’re saying. I just didn’t expect this to be the way in.
This approach arguably takes the Turing Test too literally. If it peters out, that will be its epitaph. If it succeeds, Turing will seem to have been transcendently wise.
In which way is sharing some common social knowledge relevantly different from sharing the same physical universe?
By the same argument, religion, or at least some of its arguments, like Pascal’s wager, should probably also be scrubbed.
I think I know at least some of the examples you refer to. I think the causality in these cases is a shared past of the agents making the trade. But I’m not sure that breaks the argument in cases where the agents involved are not aware of that, for example but not limited to, having forgotten about it or intentionally removed the memory.
Science starts with observations and then summarizes them into theories. Math starts with axioms and then generates theorems via a series of proofs.
Nice. And close to how I defined these for myself a long time ago:
Math is everything valid (in the strict sense if being derived by a consistency-preserving process).
Physics is everything useful (in the strict sense of tracing back observable reality)
Hm, translating to English this doesn’t come out too well. But anyway, I agree that it is possible to carve truth in different ways. One with a focus on consistency and one on observability.
I don’t feel like I learned anything from Socrates and Aristotle.
My guess would be because you already knew all of the facts. We don’t learn philosophy to learn true facts. You have to read philosophy backwards: How did people arrive at truth?
Rationalism defines truth as that which is observable, legible and tautologically consistent. Science is useful for the “observable” part. Math is useful for the “consistency” part. Both are necessarily legible, since without well-defined terms you end up back in the quagmire of meaningless philosophy.
I think you are misreading what philosophy does (or at least did fir most of the time): Struggling to build words that describe difficult to describe things. Creating legibility out of nothing.
Yes, you need legibility, but where does it come from? Once you have words like “observation” or “particle” (aka atom) or force you can do stuff with it. But reality is not legibile enough to provide these for free. Reality has joints but they are obscursed by a lot of messy flesh that is in the way. You have to throw words around that you gave an intuition that may or may not be right and see what sticks. And you don’t do this alone because it is too easy to spot fake patterns. Other people will try to catch yiu. But you can’t exchange intuitions directly. You have only words with all their imprecision. You can try definitions—but they must be circular because nothing is grounded—yet. That’s philosophy: Creating legibility out of nothing. At least that’s some idealized view of philosophy. Reals philosophy is not like that. But ideal math is also not like real math.
Anyway, I think you can observe this process of creating legibility about something in real time with consciousness. I think progress is being made with recent posts by Scott Alexander and others as well as real studies—putting a meditator into a CT and measuring what goes on.
It depends. Can you give some indication of what coalition with this intent you assume?
A strong majority of the government is in favor of introducing the subject (say, >60%).
A group of experts advising the government has strong evidence that the subject provides large societal value (say, equal to an expected increase of GDP by 1%).
The general population has a consensus that the new subject is valuable (in some clear but not necessarily precise sense, e.g., “good for future job prospects”).
The teachers that will teach the subject are generally in favor of it (either enough of the existing teachers who will have to do this, or the experts who will have to teach the future teachers).
The administrative organs of the government are given the order to implement the new subject, e.g., by passing a suitable law (assuming the administrative organs are sufficiently capable of doing so).
Additionally, I guess we assume that adequate monetary means are available for this endeavor, right? It is an investment, the new subject will start to pay off after 20 years earliest (30 if you have to teach the teachers), right?