Kaj_Sotala

Karma: 48,382

Kaj_Sotala 19 Jan 2025 18:26 UTC
5 points
0
in reply to: Raemon’s comment on: Don’t ignore bad vibes you get from people
(I read it as humor rather than criticism.)

Kaj_Sotala 19 Jan 2025 18:15 UTC
5 points
0
in reply to: LVSN’s comment on: Don’t ignore bad vibes you get from people
If we have two kinds of people and two kinds of effects that this advice might have:
1. Dogmatics who would use this as an excuse for censorship, with the world getting worse as they hear this message
2. Genuinely thoughtful people inclined toward self-doubt who are encouraged to listen to their gut, with the world getting better as it helped them avoid substantial damage
Then I acknowledge that the first effect probably exists, but I expect the second effect to dominate. The kinds of people who would ignore everything underneath the title and were just looking for an excuse for dogmatism would likely find some other excuse anyway, so the size of the effect seems small. While the kinds of people who are thoughtful but heavy on self-doubt seem to me much more substantially affected by public messaging giving them permission to listen to their intuition. (Source: I’m one, and I think I would have benefited from this kind of advice.)

Kaj_Sotala 19 Jan 2025 16:30 UTC
14 points
0
in reply to: Tamay’s comment on: meemi’s Shortform
We acknowledge that OpenAI does have access to a large fraction of FrontierMath problems and solutions, with the exception of a unseen-by-OpenAI hold-out set that enables us to independently verify model capabilities.
Can you say exactly how large of a fraction is the set that OpenAI has access to, and how much is the hold-out set?

Kaj_Sotala 19 Jan 2025 15:16 UTC
4 points
0
in reply to: Kaj_Sotala’s comment on: Heritability: Five Battles
And here’s an excerpt from her book:
In this final chapter, then, I hope to start the conversation about what it means for science and policy to be actively anti-eugenicist, by offering five general principles:
1. Stop wasting time, money, talent, and tools that could be used to improve people’s lives.
2. Use genetic information to improve opportunity, not classify people.
3. Use genetic information for equity, not exclusion.
4. Don’t mistake being lucky for being good.
4. Consider what you would do if you didn’t know who you would be.
For each of these principles, I will contrast three positions. First, the eugenic position positions genetic influence as a naturalizer of inequality. If social inequalities have genetic causes, then those inequalities are portrayed as the inevitable manifestations of a “natural” order. Genetic information about people can be used to slot them more effectively into that order. Second, the genome-blind position position sees genetic data as the enemy of social equality and so objects to any use of genetic information in social science and policy. Whenever possible, the genome-blind position seeks not to know: scientists ought not to study genetic differences or how they are linked to social inequalities, and other people in society ought not to use any scientific information that is generated for any practical purposes. These two positions can be contrasted with what I am proposing is an anti-eugenic position that does not discourage genetic knowledge but deliberately aims to use genetic science in ways that reduce inequalities in the distribution of freedoms, resources, and welfare. [...]
Stop Wasting Time, Money, Talent, and Tools
- EUGENIC: Point to the existence of genetic influence to deny the possibility of intervening to improve people’s lives.
- GENOME-BLIND: Ignore genetic differences even if it wastes resources and slows down science.
- ANTI-EUGENIC: Use genetic data to accelerate the search for effective interventions that improve people’s lives and reduce inequality of outcome. [...]
Use Genetic Information to Improve Opportunity, Not Classify People
- EUGENIC: Classify people into social roles or positions based on their genetics.
- GENOME-BLIND: Pretend that all people have an equal likelihood of achieving all social roles or positions after taking into account their environment.
- ANTI-EUGENIC: Use genetic data to maximize the real capabilities of people to achieve social roles and positions. [...]
Let’s go back to a specific example that I told you about in chapter 7, about the relationship between the educational attainment polygenic index and mathematics course-taking in high school. Students who had a higher polygenic index were more likely to be enrolled in geometry (versus algebra 1) in the ninth grade, which put them on track to complete calculus by the end of high school. Students who had a higher polygenic index were also less likely to drop out of math once it became optional. What can and should be done with that information?
The eugenic proposal would be to test students’ DNA and use it to assign them to mathematics tracks, such that students with low polygenic indices are excluded from opportunities to learn advanced mathematics. The gene-blind proposal would be to insist that the research connecting genetics and mathematics course taking shouldn’t have been done in the first place. The anti-eugenic proposal is to apply that genetic knowledge toward (a) understanding how teachers and schools can maximize the mathematics learning of their students, and (b) spotlighting how academic tracking entrenches inequalities between students.
Regarding the first goal, consider that one of the greatest challenges to understanding which teachers and schools are best serving the needs of students is that students with different learning needs are not randomly distributed across teachers and schools. A trenchant criticism of using standardized test scores as a metric for teacher and school “accountability”—that is, for identifying poorly performing teachers and schools—is that student test scores are highly correlated with student characteristics, such as family socioeconomic status, that precede the child’s entry to school and that are non-randomly clustered across schools. “Good” schools, defined as schools with high average test scores, are, in actuality, often better described as rich schools with high concentrations of affluent students. (A similar problem besieges identifying the best doctors and hospitals: the best doctor is not the one who avoids treating the sickest patients.)
Researchers have long recognized that estimating school effects on student academic outcomes is a tricky problem, and one can begin to make fair, “apples-to-apples” comparisons among schools only if one incorporates measures of student characteristics such as family background, previous levels of academic knowledge, etc. The appropriate question is not “How do students in school X fare differently than students in school Y?” because the students in school X could be already different from the students in school Y in ways other than the school they attend. The appropriate question is, “How would a particular student have fared differently if he had attended school X rather than school Y?” (Again, we see the importance of counterfactual reasoning for causal inference, as I explained in chapter 5).
In attempting to identify school effects, it is commonplace for researchers, educators, and policymakers to consider information about one accident of birth: a student’s socioeconomic status. But I and others have observed in our research that information from a student’s DNA, in the form of a polygenic index, also predicts academic outcomes, above and beyond information on family socioeconomic status. As I described above, this does not mean that we should use polygenic indices to classify students and restrict their opportunities to learn. It does mean, however, that we can evaluate how students who have equivalent polygenic indices fare differently in their outcomes when they attend different schools.
In one study of US high school students, we found that students with low education-related polygenic indices were, on average, less likely to continue in their mathematics education in high school. But their dropout rates differ substantially across school contexts. In schools that primarily serve students whose parents have high school diplomas, even students with low polygenic indices take a few years of math after the ninth grade. In fact, students with low polygenic indices in high-status schools fare about as well, in terms of their persistence in math, as students with average polygenic indices who attend low-status schools.
This finding is just barely scratching the surface. What, specifically, is happening in higher-status schools that keeps even students who are statistically likely to drop out of math from actually dropping out? How do you make the practices of such schools more widely available to all students? The path from basic research like this study to educational policy reform is long and tortuous.
But even though it is just a first step, this study is revealing a basic and important truth: given a certain fixed starting point in life—inheriting a certain combination of DNA variants—some people get much further in developing their capability to solve mathematical problems. These mathematical skills have lifelong benefits for an individual in terms of future education, participation in the labor force, and ease with navigating problems of everyday living. In fact, math literacy is so important for a student’s future that the opportunity to learn math has been called a civil right. Genetic data has thus revealed an inequality of environmental opportunity, one that calls out for redress.
Other environmental inequalities could be similarly diagnosed using genetic data. Which health interventions reach people who are currently most genetically at risk for poor outcomes? Which schools have the lowest rates of disciplinary problems among youth who are currently at most genetic risk for aggression, delinquency, or substance use problems? Which areas of the country are “opportunity zones,” where opportunity is defined not solely in terms of how children from low-income families fare, but also in terms of how children who are genetically at risk for school problems or mental health problems fare? If researchers embrace principle #1, and start embracing the possibilities of genetic data, we will have a wealth of new information to address these questions.
Use Genetic Information for Equity, Not Exclusion
- EUGENIC: Use genetic information to exclude people from health care systems, insurance markets, etc.
- GENOME-BLIND: Prohibit the use of genetic information per se but otherwise keep markets and systems the same.
- ANTI-EUGENIC: Create health care, educational, housing, lending, and insurance systems where everyone is included, regardless of the outcome of the genetic lottery. [...]
Don’t Mistake Being Lucky for Being Good
- EUGENIC: Point to genetic effects on intelligence as proof that some people naturally have more merit than others.
- GENOME-BLIND: Accept the logic of meritocracy while ignoring the role of genetic luck in developing skills and behaviors that are perceived as meritorious.
- ANTI-EUGENIC: Recognize genetics as a type of luck in life outcomes, undermining the meritocratic logic that people deserve their successes and failures on the basis of succeeding in school. [...]
Consider What You Would Do, If You Didn’t Know Who You Would Be
- EUGENIC: The biologically superior are entitled to greater freedoms and resources.
- GENOME-BLIND: Society should be structured as if everyone is exactly the same in their biology.
- ANTI-EUGENIC: Society should be structured to work to the advantage of people who were least advantaged in the genetic lottery.

Kaj_Sotala 19 Jan 2025 14:58 UTC
4 points
0
on: Heritability: Five Battles
But interestingly, Turkheimer also says some nice things about the book The Genetic Lottery: Why DNA Matters for Social Equality, by his former student Kathryn Paige Harden. Unlike Turkheimer, Harden is much more willing to ascribe a straightforward causal role of genetics in intelligence and other traits. But Harden reaches essentially the same sociopolitical conclusions as Turkheimer (according to Turkheimer). I didn’t read Harden’s book, but I presume that those conclusions are things like “racism is bad”, “we should help the downtrodden”, “don’t be an asshole”, and so on—conclusions that I myself enthusiastically endorse as well.
This New Yorker profile about Harden gets a bit into her philosophy. She wants society to use genetic data to design more effective social interventions for making people better off, and for an improved understanding of the effect of genetics to make people more receptive to programs designed to increase equality of outcome:
The first thing that social-science genomics can do is help researchers control for confounding genetic variables that are almost universally overlooked. As Harden puts it in her book, “Genetic data gets one source of human differences out of the way, so that the environment is easier to see.” For example, beginning in 2002, the federal government spent almost a billion dollars on something called the Healthy Marriage Initiative, which sought to reduce marital conflict as a way of combatting poverty and juvenile crime. Harden was not surprised to hear that the policy had no discernible effect. Her own research showed that, when identical-twin sisters have marriages with different levels of conflict, their children have equal risk for delinquency. The point was not to estimate the effects of DNA per se, but to provide an additional counterfactual for analysis: would an observed result continue to hold up if the people involved had different genes? Harden can identify studies on a vast array of topics—Will coaching underresourced parents to speak more to their children reduce educational gaps? Does having dinner earlier improve familial relationships?—whose conclusions she considers dubious because the researchers controlled for everything except the fact that parents pass along to their children both a home environment and a genome.
She acknowledged that gwas techniques are too new, and the anxieties about behavior genetics too deeply entrenched, to have produced many immediately instrumental examples so far. But she pointed to a study from last year as proof of concept. A team of researchers led by Jasmin Wertz, at Duke, used GWAS results to examine four different “aspects of parenting that have previously been shown to predict children’s educational attainment: cognitive stimulation; warmth and sensitivity; household chaos (reverse-coded to indicate low household chaos); and the safety and tidiness of the family home.” They found that one of them—cognitive stimulation—was linked to children’s academic achievement and their mothers’ genes, even when the children did not inherit the relevant variants. Parental choices to read books, do puzzles, and visit museums might be conditioned by their own genes, but they nevertheless produced significant environmental effects.
Even the discovery that a particular outcome is largely genetic doesn’t mean that its effects will invariably persist. In 1972, the U.K. government raised the age at which students could leave school, from fifteen to sixteen. In 2018, a research group studied the effects of the extra year on the students as adults, and found that their health outcomes for measures like body-mass index, for whatever reason, improved slightly on average. But those with a high genetic propensity for obesity benefitted dramatically—a differential impact that might easily have gone unnoticed.
Some of Harden’s most recent research has looked at curricular tracking for mathematics, an intuitive instance of how gene-environment interactions can create feedback loops. Poor schools, Harden has found, tend to let down all their students: those with innate math ability are rarely encouraged to pursue advanced classes, and those who struggle are allowed to drop the subject entirely—a situation that often forecloses the possibility of college. The most well-off schools are able to initiate virtuous cycles in the most gifted math students, and break vicious cycles in the less gifted, raising the ceiling and the floor for achievement. [...]
Harden is not alone in her drive to fulfill Turkheimer’s dream of a “psychometric left.” Dalton Conley and Jason Fletcher’s book, “The Genome Factor,” from 2017, outlines similar arguments, as does the sociologist Jeremy Freese. Last year, Fredrik deBoer published “The Cult of Smart,” which argues that the education-reform movement has been trammelled by its willful ignorance of genetic variation. Views associated with the “hereditarian left” have also been articulated by the psychiatrist and essayist Scott Alexander and the philosopher Peter Singer. Singer told me, of Harden, “Her ethical arguments are ones that I have held for quite a long time. If you ignore these things that contribute to inequality, or pretend they don’t exist, you make it more difficult to achieve the kind of society that you value.” He added, “There’s a politically correct left that’s still not open to these things.” [...]
The ultimate claim of “The Genetic Lottery” is an extraordinarily ambitious act of moral entrepreneurialism. Harden argues that an appreciation of the role of simple genetic luck—alongside all the other arbitrary lotteries of birth—will make us, as a society, more inclined to ensure that everyone has the opportunity to enjoy lives of dignity and comfort. She writes, “I think we must dismantle the false distinction between ‘inequalities that society is responsible for addressing’ and ‘inequalities that are caused by differences in biology.’ ” She cites research showing that most people are much more willing to support redistributive policies if differences in opportunity are seen as arbitrarily unfair—and deeply pervasive.
As she put it to me in an e-mail, “Even if we eliminated all inequalities in educational outcomes between sexes, all inequalities by family socioeconomic status, all inequalities between different schools (which as you know are very confounded with inequalities by race), we’ve only eliminated a bit more than a quarter of the inequalities in educational outcomes.” She directed me to a comprehensive World Bank data set, released in 2020, which showed that seventy-two per cent of inequality at the primary-school level in the U.S. is within demographic groups rather than between them. “Common intuitions about the scale of inequality in our society, and our imaginations about how much progress we would make if we eliminated the visible inequalities by race and class, are profoundly wrong,” she wrote. “The science confronts us with a form of inequality that would otherwise be easy to ignore.”
The perspective of “gene blindness,” she believes, “perpetuates the myth that those of us who have ‘succeeded’ in twenty-first century capitalism have done so primarily because of our own hard work and effort, and not because we happened to be the beneficiaries of accidents of birth—both environmental and genetic.” She invokes the writing of the philosophers John Rawls and Elizabeth Anderson to argue that we need to reject “the idea that America is or could ever be the sort of ‘meritocracy’ where social goods are divided up according to what people deserve.” Her rhetoric is grand, though the practical implications, insofar as she discusses them, are not far removed from the mid-century social-democratic consensus—the priorities of, say, Hubert Humphrey. If genes play a significant role in educational attainment, then perhaps we ought to design our society such that you don’t need a college degree to secure health care.

Don’t ignore bad vibes you get from people

Kaj_Sotala18 Jan 2025 9:20 UTC

89 points

15 comments2 min readLW link

(kajsotala.fi)

Kaj_Sotala 16 Jan 2025 10:05 UTC
2 points
0
in reply to: Steven Byrnes’s comment on: Heritability: Five Battles
So I think we have two separate questions here:
- Do psychological issues involve reactivation of an earlier memory such that the reactivation plays a causal role in the issue?
- Can you address an issue without explicitly working with an earlier memory?
For the first question, I’d say “it depends”. On one end, we have something like PTSD flashbacks—here a reactivation of a memory is clearly in a causal role, since “a memory getting reactivated to such an extent that the person experiences themselves as literally reliving it” is what a flashback is.
Slightly less strong but still strongly suggestive of a causal role is something where a person imagines themselves doing something, but then—maybe just at the back of their mind—recalls a painful memory and flinches away from doing that. E.g. they consider speaking up, but then a flicker of a memory comes up about a time when they spoke up and somebody ridiculed them, and they quickly close their mouth. Here there seems to at least be a causal path from the memory to the issue, in that the memory is charged with negative affect and that the memory coming up causes the person to reorient to something that makes the memory recede in intensity.
Then we have cases where there’s no obvious memory at first, but directing attention to the issue and asking questions about it brings up a memory, even though none of the questions ever ask about memories directly. For example, someone might feel like they have to act in a certain way in a particular social situation despite finding it unpleasant. Now a therapist might ask them something like “what would be bad about acting differently” and have them focus on what feels emotionally or intuitively bad about it (rather than what logical justifications their mind would offer). Then there might be a line of questioning that went something like:
- “I have difficulty getting a turn to speak because I tend to wait extra long to make sure others have finished speaking before I speak up. And then I wait so long that someone else always starts talking before I can.”
- “Okay, so what would be bad about speaking up before you’re certain that others have finished speaking?”
- “Then I might interrupt them before they’re finished.”
- “Okay, what would be bad about interrupting them before they’re finished?”
- “That’d feel unfair toward them.”
- “In what way does it feel unfair?”
- “Hmm, I’m getting a memory of a time when I was trying to speak up but my father interrupted me, and then I tried talking anyway and then he acted like I had interrupted him and that I should let him talk first. That felt really bad and unfair. I guess I want to make sure to act better than he did, and make sure that I never interrupt someone else when it’s their turn to speak.”
Is the memory in a causal role here? Probably depends on how exactly we define “causal”. But at least it seems like there is some kind of a model about how the person wants to act or not act (“interrupting other people is unfair toward them, and should be avoided”) that was formed due to an earlier experience. When one tries to elicit details about how exactly the model works, the model seems to structurally incorporate the original experience as a reference point for what exactly the core bad thing is. And working with the memory often seems to help with one’s issues.
Given that this kind of a memory seems to have a similar character as the PTSD and the “I can hear the people mocking me” memories, just buried slightly deeper, to me the simplest and most plausible explanation would be that it has a causal role in the same way as the less-buried ones do.
Then on the other hand, it’s not always the case that this kind of questioning leads to any clear-cut memory. Sometimes what comes up feels more like a general model that has been formed out of multiple different life experiences, with none of original instances having been stored. Or there might be an issue that seems to go back to an age young enough that the person doesn’t have any explicit memories of it, and it has only left a general emotional imprint. In those cases the memory doesn’t seem to have any causal role, because there doesn’t seem to be any memory around to begin with.
Or at least not one that would be easily accessible. I’ve heard of claims from people who got into states of deep meditation or strong doses of psychedelics that they managed to access very early painful memories that wouldn’t have been available in a normal therapeutic context, and then got independent confirmation for the truthfulness of those memories afterward. I’ve not looked into these in detail but I’m inclined to suspect they’re true. In part due to my personal experiences of old memories spontaneously coming up in altered states of consciousness (and this sometimes shifting behaviors), in part because “all behaviors involve an original memory trace being stored somewhere and that trace then driving behavior, with some of those traces just being buried deep or in not normally forward-compatible storage formats” would again seem like the most parsimonious model.
As for the second question, I’d again say it depends. If someone is suffering from a PTSD flashback, it’s going to be hard to do anything about that without working with the traumatic memory in some way! But for the ones where the problem isn’t so directly driven by an explicit memory reactivation, there are definitely a lot of approaches that work by changing other parts of the model. E.g. if the model makes a particular prediction about the world in general (“people will always find it unfair if I speak up before being absolutely certain that they’re finished”), then it’s often possible to disprove that prediction without going into the details of the original memory. And while some therapies focus on the episodic memory component of the learned model, others work on different components.

Kaj_Sotala 15 Jan 2025 18:19 UTC
4 points
0
in reply to: Steven Byrnes’s comment on: Heritability: Five Battles
Cool, thanks!
(B’) “…Gee, I guess this reprocessing must have been a kind of ‘training / practice / exercise’ during which I could forge new better subconscious habits and associations related to ‘the feeling of anxiety’ in general. And these new subconscious habits and associations are now serving me well in a wide variety of adult contexts.”
I think that there are definitely techniques that work on one’s reaction to the feeling of anxiety in general, but the specific ones that I had in mind don’t feel like they’d be doing that. Rather they seem much more localized, in that they eliminate some particular anxiety trigger or specific kind of anxiety from getting triggered in the first place. But then if something else happens to trigger the same or a similar anxiety, the person isn’t necessarily any better at dealing with that.
So if someone feels the same kind of anxious around both spiders and snakes, then this kind of an intervention might eliminate the fear of spiders entirely, while leaving the reaction to snakes entirely unaffected (or vice versa).
Also, AFAICT, people achieve great therapeutic success by methods that involve bringing up childhood memories, but other people also achieve great therapeutic success by methods that don’t. :)
Oh yeah definitely, didn’t mean to imply that working with memories would be the only approach that worked.

Kaj_Sotala 15 Jan 2025 15:11 UTC
23 points
19
on: We probably won’t just play status games with each other after AGI
I think that getting too fixated on status games is usually due to some kind of insecurity, e.g. feeling that you need to accumulate status in order to be accepted, respected or something like that. (One can certainly play status games for the fun of it without having such an insecurity, but that seems unlikely to lead to the level of fixation where status game would become the primary activity that humans engage in.)
If every human can close friends or even lovers with AI systems as you suggest, then I would expect that to provide the kind of deep unconditional feeling of security that makes the need for playing status games fall away. If you feel deeply safe, loved, respected, etc., then status can certainly still feel nice to have, but it’s unlikely to feel that important for most people. In the same way that e.g. a loving parent focused on taking care of and spending time with their family may find themselves becoming much less interested in spending their time playing status games.

Kaj_Sotala 14 Jan 2025 21:22 UTC
14 points
0
on: Heritability: Five Battles
Fantastically detailed post, thank you for taking the time to write up all this!
I’d be curious to hear your thoughts about the following. There seems to be an obvious conflict between:
- Behavioral genetics, which has all these findings about the childhood environment only having a limited effect
- Therapy, where memory reconsolidation can achieve significant chances in people’s feelings and behavior by changing subconscious beliefs, and many of those beliefs seem to be related to childhood events and experiences
One hypothesis I’ve had for reconciling those is based on what evolutionary psychologists have learned about fear of snakes. At one point, it was thought that humans might be hardwired with a fear of spiders and snakes in particular. But later work then suggested that this is wrong—instead, humans are evolutionarily biased towards paying extra attention to things like spiders and snakes.
Then because we pay more attention to things that look like that, it’s more likely that we notice something scary about them. Or if we’ve been told that they’re dangerous, then just repeatedly noticing them increases the chance that we develop a mild phobia around them (as it’s increasing the prior of “this dangerous thing might be around and you should notice it”). And that seems to explain why things like spider and snake phobias are much more common than things like electricity phobias:
Seligman’s account suggested that specialised, central mechanisms of fear learning more readily connect aversive events, such as electric shock, with fear-relevant stimuli, such as snakes – which presented genuine threats to our evolutionary ancestors – than with ‘fear-irrelevant’ stimuli such as geometric shapes or flowers. This account predicts that fear of fear-relevant objects should be learned faster, and be extinguished more slowly when shock no longer occurs, as well as being resistant to topdown modification, for example, by instructions indicating that shocks will not occur.
The results of early experiments were consistent with some of these predictions (e.g., [50,51]), but none has withstood extended experimental investigation. Faster or better conditioning with fear-relevant stimuli has rarely been observed, and there is ample evidence that, like most associative learning (e.g., [52]), it can be modified by instruction (reviewed in [53,54]). Initially it seemed that responses to fear-relevant stimuli might extinguish more slowly. However, a recent systematic review [55] found that most positive findings came from a single laboratory, and a large majority of the full set of studies had failed to find differences between fear-relevant and fear-irrelevant stimuli in the rate of extinction.
These results suggest that fear of snakes and other fear-relevant stimuli is learned via the same central mechanisms as fear of arbitrary stimuli. Nevertheless, if that is correct, why do phobias so often relate to objects encountered by our ancestors, such as snakes and spiders, rather than to objects such as guns and electrical sockets that are dangerous now [10]? Because peripheral, attentional mechanisms are tuned to fear-relevant stimuli, all threat stimuli attract attention, but fear-relevant stimuli do so without learning (e.g., [56]). This answer is supported by evidence from conditioning experiments demonstrating enhanced attention to fear-relevant stimuli regardless of learning (Box 2), studies of visual search [57–59], and developmental psychology [60,61]. For example, infants aged 6–9 months show a greater drop in heart rate – indicative of heightened attention rather than fear – when they watch snakes than when they watch elephants [62].
Now suppose that some people carried genes that made them pay extra attention to snakes and/or spiders, and other people didn’t. In that situation, you might observe both that:
- The probability of having a snake or spider phobia was strongly heritable—people with those genes were likely to develop that phobia, with parenting style having little effect
- Therapists employing memory reconsolidation-based methods for treating the phobia could often find some specific childhood experiences that had to do with spiders, that seemed to be at the root of the phobia (and doing reconsolidation on these experiences reliably helped with the phobia)
We could then suppose that a lot of other psychological traits are similar: if you have a certain set of genes, it will make you much more likely to have a particular kind of psychological reaction in response to external events. While a person with another set of genes would react differently. And while people in different kinds of environments will differ in exactly what kinds of events they are exposed, then assuming that they belong to a roughly similar social class within the same country, they will probably still have some experiences that are roughly similar.
For instance, when I was little, some older children in our neighborhood made up a story about a man going around the neighborhood and kidnapping children. I expect that a lot of kids who weren’t particularly inclined toward high neuroticism soon forgot about the whole thing. Meanwhile I got really scared about it and asked my parents if we couldn’t move somewhere else, and then much later as an adult found myself having minor anxiety that seemed to have its roots in this childhood experience.
Now if we hadn’t lived in that particular middle-class neighborhood, I wouldn’t have encountered that particular rumor and it wouldn’t have left a mark on me. But given that I had high-neuroticism genes that made me seriously freaked out by some older children deciding to scare the younger ones a bit, probably something else would have happened in any other middle-class neighborhood that would have felt equally scary and made me somewhat more inclined to anxiety in the future.
That would again lead to the pattern where most major differences seem to come from genetic differences, and at the same time many people with psychological problems can consistently trace the source of their problems to childhood experiences.

Kaj_Sotala 7 Jan 2025 3:40 UTC
12 points
3
on: Recent Sam Altman statements on AGI and ASI
Worth keeping in mind that OpenAI is burning through crazy amounts of money and is constantly in need of more:

OpenAI raised $6.6 billion [in October], the largest venture capital round in history. But it plans to lose $5 billion [2024] alone. And by 2026, it could be losing $14 billion per year. That’s head exploding territory. If OpenAI keeps burning money at this rate, it will have to raise another round soon. Perhaps as early as 2025.

As a result, Altman has a significant financial incentive to believe/say that OpenAI is on the verge of a breakthrough and that it’s worth it for their investors to continue giving them money.

Kaj_Sotala 5 Jan 2025 17:16 UTC
2 points
0
in reply to: quetzal_rainbow’s comment on: quetzal_rainbow’s Shortform
True!

Kaj_Sotala 5 Jan 2025 7:29 UTC
5 points
2
in reply to: quetzal_rainbow’s comment on: quetzal_rainbow’s Shortform
I think even “a detailed picture of the topic of the lesson” can be too high of an expectation for many topics early on. (Ideally it wouldn’t be, if things were taught well, but they often aren’t.) A better goal would be to have just something you understand well enough that you can grab on to, that you can start building out from.

Like if the topic was a puzzle, it’s fine if you don’t have a rough sense of where every puzzle piece goes right away. It can be enough that you have a few corner pieces in place, that you then start building out from.

Kaj_Sotala 2 Jan 2025 20:13 UTC
18 points
0
in reply to: Ben Pace’s comment on: Ben Pace’s Shortform Feed
I recall it heard claimed that a reason why financial crimes sometimes seem to have disproportionately harsh punishments relative to violent crimes is that financial crimes are more likely to actually be the result of a cost-benefit analysis.

Kaj_Sotala 31 Dec 2024 10:27 UTC
LW: 4 AF: 2
0
AF
on: A Three-Layer Model of LLM Psychology
Fantastic post. This has been frequently on my mind after reading it, and especially the surface/character layer split feels very distinct now that I have an explicit concept for it. And then at one point I asked it to profile me based on some fiction I co-wrote with it and it managed to guess that I was Finnish from something I didn’t think had any clues in that direction, which gave me a novel feeling of getting a glimpse into that vast alien ground layer.
The analogy to the character and player distinction in humans also feels very apt.

Kaj_Sotala 29 Dec 2024 21:03 UTC
7 points
0
in reply to: Joseph Miller’s comment on: Joseph Miller’s Shortform
Don’t most browsers come with spellcheck built in? At least Chrome automatically flags my typos.

Kaj_Sotala 29 Dec 2024 20:48 UTC
4 points
0
in reply to: Seth Herd’s comment on: What are the strongest arguments for very short timelines?
Thanks. Still not convinced, but it will take me a full post to explain why exactly. :)
Though possibly some of this is due to a difference in definitions. When you say this:
what I consider AGI—which importantly is fully general in that it can learn new things, but will not meet the bar of doing 95% of remote jobs because it’s not likely to be human-level in all areas right away
Do you have a sense of how long you expect it will take for it to go from “can learn new things” to “doing 95% of remote jobs”? If you e.g. expect that it might still take several years for AGI to master most jobs once it has been created, then that might be more compatible with my model.

Kaj_Sotala 29 Dec 2024 20:44 UTC
6 points
0
in reply to: Noosphere89’s comment on: What are the strongest arguments for very short timelines?
Hmm, some years back I was hearing the claim that self-driving cars work badly in winter conditions, so are currently limited to the kinds of warmer climates where Waymo is operating. I haven’t checked whether that’s still entirely accurate, but at least I haven’t heard any news of this having made progress.

Kaj_Sotala 27 Dec 2024 20:38 UTC
6 points
2
in reply to: Seth Herd’s comment on: What are the strongest arguments for very short timelines?
Thanks, this is the kind of comment that tries to break down things by missing capabilities that I was hoping to see.
Episodic memory is less trivial, but still relatively easy to improve from current near-zero-effort systems
I agree that it’s likely to be relatively easy to improve from current systems, but just improving it is a much lower bar than getting episodic memory to actually be practically useful. So I’m not sure why this alone would imply a very short timeline. Getting things from “there are papers about this in the literature” to “actually sufficient for real-world problems” often takes a significant time, e.g.:
- I believe that chain-of-thought prompting was introduced in a NeurIPS 2022 paper. Going from there to a model that systematically and rigorously made use of it (o1) took about two years, even though the idea was quite straightforward in principle.
- After the 2007 DARPA Grand Challenge there was a lot of hype about how self-driving cars were just around the corner, but almost two decades later, they’re basically still in beta.
My general prior is that this kind of work—from conceptual prototype to robust real-world application—can in general easily take between years to decades, especially once we move out of domains like games/math/programming and into ones that are significantly harder to formalize and test. Also, the more interacting components you have, the trickier it gets to test and train.

Kaj_Sotala 27 Dec 2024 20:22 UTC
7 points
0
in reply to: elifland’s comment on: What are the strongest arguments for very short timelines?
Thanks. I think this argument assumes that the main bottleneck to AI progress is something like research engineering speed, such that accelerating research engineering speed would drastically increase AI progress?
I think that that makes sense as long as we are talking about domains like games / math / programming where you can automatically verify the results, but that something like speed of real-world interaction becomes the bottleneck once shifting to more open domains.
Consider an AI being trained on a task such as “acting as the CEO for a startup”. There may not be a way to do this training other than to have it actually run a real startup, and then wait for several years to see how the results turn out. Even after several years, it will be hard to say exactly which parts of the decision process contributed, and how much of the startup’s success or failure was due to random factors. Furthermore, during this process the AI will need to be closely monitored in order to make sure that it does not do anything illegal or grossly immoral, slowing down its decision process and thus whole the training. And I haven’t even mentioned the expense of a training run where running just a single trial requires a startup-level investment (assuming that the startup won’t pay back its investment, of course).
Of course, humans do not learn to be CEOs by running a million companies and then getting a reward signal at the end. Human CEOs come in with a number of skills that they have already learned from somewhere else that they then apply to the context of running a company, shifting between their existing skills and applying them as needed. However, the question of what kind of approach and skill to apply in what situation, and how to prioritize between different approaches, is by itself a skillset that needs to be learned… quite possibly through a lot of real-world feedback.

Kaj_Sotala

Don’t ig­nore bad vibes you get from people

Don’t ignore bad vibes you get from people