“the Average IQ participants took longer to learn which was the correct response, in the sense that the various generators of their alternative responses took longer to stop bidding for attention / decision-power”
Yes. This is also the finding of Haier et al. (1992).
They’re suggestive though,
Yes. We lack the technical capability to directly show that ephaptic signaling is causing the lower intelligence by, say, correlating ephaptic events in the deep brain with poorer performance. But I think we basically have the next best thing. It just seems logically incongruent that more signaling is done, that more alternatives are suggested, in a brain that is slower and which uses more energy to send a signal. The speed and energy efficiency functions of myelin seem to logically imply that more myelin means more suggestions that can be subsequently quickly cancelled, but we observe the opposite. Only the ephaptic protection properties of myelin logically explain the lower activity in more myelinated higher g individuals.
Perhaps I should have omitted mentioning these indications MS patients are not slower and their fatigue does not correlate with poor performance but I felt it was worth mentioning that there is at least some indication empirically that speed and energy are not the root cause, though these indications are far from conclusive in and of themselves.
is the idea that for both people with higher and lower fluid intelligence, they have active areas of the brain that aren’t relevant for solving the problem;
Low IQ people are having additional activation during the task, particularly in frontal areas known to function as action inhibitors and conflict resolvers. Perhaps I should have used a longer quote:
Consistent with the neural efficiency hypothesis, the function of some of these regions would suggest a greater cognitive challenge facing the Average IQ group. For example, the right inferior frontal gyrus (Brodmann Area [BA] 46) which research suggests is strongly implicated in inhibition of prepotent (but inappropriate) responses (Konishi et al. 1998) showed greater activation in Average IQ participants during response selection. Activation in this region peaked on response selection events immediately following negative feedback (when the need to inhibit the prepotent response would be greatest) and diminished with increasing time since rule shift (see Fig. 2A). A similar activation profile was observed in the anterior cingulate gyrus cortex (BA 31⁄32), which is thought to be involved in resolution of response conflict (Garavan et al. 1999; Braver et al. 2001) and again activation was most pronounced immediately after the rule change, that is, during the presence of more competing responses for selection when the identity of the new rule was unknown (Fig. 2B). If the Average IQ participants were to have experienced greater conflict (e.g., from competing response alternatives) then this could explain their relatively greater activation in these regions observed compared with the High IQ participants.
Graham et al. 2010 page 3
If ephaptic cross-talk is a / the main issue that myelination is about, then I think that’s somewhat bullish for my idea of “prosthetic connectivity”?
Yes, but with the caveat that current methods of electrical stimulation are too imprecise (which I believe you mention in your recent post). Imprecision is exactly what we are trying to avoid.
That said: it recently occurred to me that a combination of genetic alteration and cybernetic implants could potentially have greater precision perhaps if we use light-activated neurons in the central nervous system. I wrote a post about a lab using this technique:
Ok, this may be a bit of self-promotion, while Kellendonk’s lab does use mice genetically engineered to have optically activated neurons I don’t discuss that detail of his work in the post (it is about him trying to extend working memory by stimulating the thalamus). Please forgive me. The general technique is called optogenetics for any potential unfamiliar reader:
https://en.wikipedia.org/wiki/Optogenetics
Just a speculation, but we do use optics to make chips with a couple hundred times more precision than the width of even a smaller axon.
We have no clue if “disordered experience” is even a thing,
I have mildly disordered experiences on some mind-altering drugs (or at least memory of it) and I’ve seen some strong evidence that other beings have internal experiences (like you possibly), some of which seem to be disordered in more severe ways than what I have experienced and for longer duration.
some 4-d bounding box
The issue is that the larger the box the less likely to occur in random fluctuations. Extremely exponentially so.
It occurred to me: LLMs may be used in the future to decide what a given word or phrase means in legal settings. In particular, as language changes and evolves with usage a time-cutoff LLM may be used to resolve ambiguity about, for example, the phrasing in a very old contract.
For example: if a contract from 1920 forbids any “gay” actvities from an event venue the recent Talkie LLM could clarify this means activities which express happiness and positive feeling rather than activities associated with homosexuality. (I was surprised during a demonstration that Talkie is surprisingly permissive about being gay; for it “gay” means effusive expression of happiness.)
This is related to the uncertain polysemanticity/polysemy present in human languages. An LLM presents an arguably unbiased way of segmenting conceptual space into lexemes and could possibly be used to resolve ambguity in the meaning of a word, particularly within a specific context like an older time period.
These issues are written about by a (relatively recent) philosopher named Derek Parfit.
The solution is that there is no continuity of consciousness. Each moment you have a set of goals which extend into the far future which likely includes things like the continued existence of a being with your memories and “personality.” That is it; goals roughly fitting this description are no more special than a goal of helping your country in a suicide mission, for example. The minor differences in goals of this type, like how much of your memory or traits counts as “your personality,” are arbitrary differences without a whole lot of meaning; in any case these goals are frequently instrumental for further goals like having children or influencing society.
The thing all beings want for their clones:
Honor.
You want your clone to demonstrate your potency, hopefully in a way that other beings will find agreeable.
I used to think about the scene in Multiplicity where Michael Keaton’s character is faced with the prospect of his clones sleeping with his wife. What would a person want in this situation?
The issue of infidelity may make most men prefer the clone stay out of their bedroom.
But what about if you’re away on a deep space mission without hope of return? Perhaps having children with your wife would be something the clone could accomplish.
And I think every man that loves their wife would want their wife to at least be attracted to the clone. If not that means she is not attracted to yourself.
Ctrl-F “interp”: one hit for Anthropic Research.
I’ve also noticed a bit of celebrity worship and AI-doomerism-group-think in the voting patterns on LW which was surprising given how rationalists advertise.
I wonder if LW readers are mostly checked out on general posting and only read posts from authors they follow? What proportion actually hit the “All Posts” link versus just the home page or a personalized feed from subscribed authors?
Inkhaven seems like a waste. To get more good writers we don’t need people to “practice” (and flood LW with a month of not-really-worth-it content forced out of constipated brains), we need an (established big-name) editor (with a built-in following) to find good work and tell other people it is good (including the writer).
While there aren’t likely as many bad posts that are widely upvoted and read (there are some I’m sure) the bigger problem is good writing that gets no attention and dies in obscurity despite being good writing. That adds noise to the writer’s quality signal. Elevating good writing to public attention is what editors/curators are for.
figuring out how values are represented;
I feel like basically none of the key terms here are well defined. What is a value? I don’t think there is a good answer for Humans yet. How would we know if it was represented?
In my experience you can look at a neuron’s high activation phrases and get some sense of some ideas that have high probability of triggering that neuron. That doesn’t mean this neuron is the representation of these ideas, many other neurons may trigger on related concepts nearby in conceptual space and your own parsing of conceptual space into discreet separate concepts may be flawed so you can’t be sure what you’re thinking about is actually a concept in some higher Platonic sense.
To use the famous mech interp example: Is the Golden Gate Bridge neuron actually about the Golden Gate Bridge? Maybe it is more broadly about steel structures and existing in San Francisco and being colored orange. The GGB neuron or feature may trigger on other concepts, or at least concepts we see as distinct, and may signify a broader concept to the LLM than what we assume.
Maybe something like this:
I’ve done a lower dimensional embedding of layer 24 of GPT2-XL and there seem to be interpretable directions in the embedding. In particular there are two main dimensions which spread the neurons (i.e. dissimilarity is mostly expressed in these two dimensions, the other dimensions just correlate very highly on a linear combination of the first two, we get a flat disc in the embedding space). The main one is what I call a narrative-personal dimension which seems to capture the difference between “narrative” language like in Wikipedia articles and news stories where the writer and audience are outside observers unable to directly affect the phenomena under discussion and on the other end is “personal” language which is structured conversationally like short fiction where characters discuss events from first person or sales pitches; in this language the author and/or the audience can participate.
The other dimension seems to be a valence dimension where “good things” are on one side and “bad things” are on the other. This is kinda perhaps what you’d be after.
“Good” is going to be relationally defined; the area the machine thinks about sunshine and lollypops is the “good” part and we only know that because the “good” stuff is there (i.e. without reference to sunshine and lollypops, which to be explicit is just a stand-in for general “good” things, we can’t define “good” really). What you want to know is whether the thoughts about “kill all humans” are more similar to sunshine and lollypops or fear, piss, and death for the machine (fear, piss, and death being concepts likely on a bad end of any dichotomous good-bad principle component).
Of course this only works if the machine actually lays out similarity in neurons in such a way that sunshine and lollypops are grouped as similar, which we don’t generically know (by my preliminary results I am pretty sure GPT2 does this but I still need to analyze more neurons). This may not be the case and the LLM may not see traditional Human valence as salient in any way; an embedding in a similarity space may find sunshine next to piss and death strictly between sunshine and lollypops. In such a case Human value may be completely alien to the LLM, indeed the concept of value in general may be alien to such a system.
Preliminarily GPT2-XL layer 24 does seem to have an “emerging modern threats” neuron which seems to trigger on a number of threatening-sounding phenomena which were contemporary to the 2010s. In particular it triggers on the vaguely threatening sounding “5G” and “big data.” It also triggers a lot on discussions of post-apocalyptic futures including an AI uprising (Horizon Zero Dawn is a video game set in the aftermath of an AI uprising and mentioned in the top activation phrases, for example). This neuron embeds near the extreme personal-bad corner of the general mass of neurons, so I think we’re on the same page as GPT2 on this. Incidentally this neuron also triggers on “Glenn Beck” which I find funny.
I would expect to see some kind of decade-long extraordinary boom around 1915 and the Great Depression starting four years earlier than it did.
This is an excessive expectation for precision in a socio-cultural phenomena.
The KKK peaks significantly in the mid 1920s and never returns to this level of popularity, literally millions of members in that decade and never more than a few thousand since. Now we have similar levels of inequality and a wing of the Republicans making holocaust jokes in private chats. It’s Nick Fuentes now not the KKK.
Communism had a surge of popularity during WWII when we were allied with Stalin against the Nazis. You could also look at membership in the Socialist party which peaked in the early century as well; the 1910s but there was a crack-down and a red-scare which tanked numbers in the ’20s. Socialist party membership peaked at higher numbers than CPUSA during the ’40s.
We actually would need public opinion polling that captured some kind of spectrum of political positions to actually answer this question. Unfortunately the ANES only has been running since the ’70s (in modern form). We’re living through the period that would represent the one instance of rising inequality leading to political polarization but such a large cultural phenomenon has a lot of internal linkages and lags that would make it difficult to build the association from one example. That said, we definitely have seen increased political polarization in ANES data, for example, over the past forty years. Feeling thermometer data for the opposite party has declined during opposite party presidencies pretty consistently from highs in the ’80s.
It was not directly affecting what the New York Times could print, yet you focus a lot in your post on the changes at the NYT.
Sorry if the thread of my argument wasn’t clear.
The NYT established the neutral news reportage norms of the mid-twentieth century that the government encouraged/forced mass-media broadcasters to adopt due to a new wave of regulation opportunity brought on by technological development.
While media on the political right was first to abandon these norms the NYT has remained a more central leader to the norms of the political (center) left. A combination of reaction to the shift in norms on the right as well as the financial motivations I reference above drove the change in norms on the political left starting from the NYT at the top.
If you’re willing to abandon any traditional understanding of freedom of speech/press, there are indeed many things you can do.
This is pretty hyperbolic considering we are discussing the history of similar regulation as well as the fact that current Section 230 regulation does in some ways limit free speech.
Beyond that there would be nothing forbidden by the regulation. Anyone can express any opinion. They just can’t capture a large audience and keep them away from points of disagreement in an echo chamber.
Cable companies in particular were ruled as subject to being forced to carry content. Public access stations were challenged by cable companies specifically on First Amendment grounds and SCOTUS ruled that that forcing the companies to carry this content alongside their other content was not a First Amendment violation so long as it did not interfere with their ability to broadcast their other content. This is a very similar situation.
In Turner II, 520 U.S. 180 (1997), the Supreme Court held that must-carry rules for cable television companies were not restrictions of their free speech rights because the U.S. government had a compelling interest in enabling the distribution of media content from multiple sources and in preserving local television.
https://en.wikipedia.org/wiki/Turner_Broadcasting_System,_Inc._v._FCC
I’m realizing I missed a golden opportunity to discuss my preferred solution: using Section 230-like legislation to regulate new media.
Specifically I’d ideally like some kind of jury system enacted in a government organization or some legally well defined procedure where for sufficiently popular media some representative sample of the public decides whehter their view was represented competently in the media being judged. You’d need a supermajority to pass, so something like 70% of people saying their view was represented.
Details would be flexible: jury size (I’d think about 100), what is sufficiently popular to be covered (I am thinking 1m Americans see it), the exact supermajority (70%-95% is within what I consider reasonable), the percentage of time devouted to each view (5%-25%, a program should be allowed to have a preference and not required to devout a plurality of time to other views), and ironing out what constitutes a topic (could be thrown to the jury if no reasonable standard can be agreed upon a priori).
Well that’s it. I guess I am making this a frontpage post now.
I didn’t know we are allowed to discuss politics here. I thought that was banned? Sorry I didn’t see the previous post. Anyway, here are some intellectual contributions:
For much of the twentieth century mass media was policed by The Fairness Doctrine. The Fairness Doctrine was a policy that required media outlets to cover controversial issues in a balanced manner that represented both sides of the argument. This was used against populist broadcasters like Carl McIntire (like a more Catholic version of Bill O’Reilly or Rush Limbaugh who also opposed the Civil Rights Movement).
The fairness doctrine of the United States Federal Communications Commission (FCC), introduced in 1949, was a policy that required the holders of broadcast licenses both to present controversial issues of public importance and to do so in a manner that fairly reflected differing viewpoints.[1] In 1987, the FCC abolished the fairness doctrine,[2] prompting some to urge its reintroduction through either Commission policy or congressional legislation.[3] The FCC removed the rule that implemented the policy from the Federal Register in August 2011.[4]
https://en.wikipedia.org/wiki/Fairness_doctrine
Rush Limbaugh’s radio program went national shortly after Reagan abolished the policy in 1987:
The FCC’s repeal of the fairness doctrine—which had required that stations provide free air time for responses to any controversial opinions that were broadcast—on August 5, 1987, meant stations could broadcast editorial commentary without having to present opposing views. Daniel Henninger wrote, in a Wall Street Journal editorial, “Ronald Reagan tore down this wall [the fairness doctrine] in 1987 … and Rush Limbaugh was the first man to proclaim himself liberated from the East Germany of liberal media domination.”[32]
https://en.wikipedia.org/wiki/Rush_Limbaugh
Media outlets have incentives to sensationalize the news to agree with their audiences’ preconceived notions, amplifying political polarization. There was a period in the 20th century when The New York Times brought the rigor of academic science to news reportage by literally hiring an astrophysicist to be the editor (Carr Van Anda was hired by Adolf Ochs soon after Ochs’s purchase of the NYT), but that was a lucky aberration and needed to be sustained for much of the twentieth century by The Fairness Doctrine (and funding through advertisers interested in selling products rather than subscriptions from readers that want to have their beliefs confirmed).
While attempts to stifle intellectual discussion on sensitive topics has been present for a long time on the extreme political Left the last approximately fifteen years represent a sudden increase in the prevalence and force of these attempts. This Tablet article by Zach Goldberg documents statistical evidence of a cultural shift at some point near the middle of the previous decade. Usage of terms related to racism in major newspapers increased as much as 1500% in this period. While Goldberg does not mention it in the article, the triggering events for this phenomena in the data seem to definitely be the formation of Black Lives Matter and their first major actions in 2014. In particular, the fact that the Eric Garner incident took place in New York and the New York Times in addition to being highly influential is also one of the papers included in Goldberg’s analysis likely means this is the source of the spike in coverage and larger cultural shift.
In July 2013, the movement began with the use of the hashtag #BlackLivesMatter on social media after the acquittal of George Zimmerman in the shooting death of African-American teen Trayvon Martin 17 months earlier, in February 2012. The movement became nationally recognized for street demonstrations following the 2014 deaths of two African Americans: Michael Brown—resulting in protests and unrest in Ferguson, Missouri, a city near St. Louis—and Eric Garner in New York City.
https://en.wikipedia.org/wiki/Black_Lives_Matter
James Douglas Bennett argues persuasively that this shift in news coverage was due to a change in the financial model of the NYT, specifically that the shift to paywalled subscription demanded playing to the audience’s political leanings:
It became one of Dean Baquet’s frequent mordant jokes that he missed the old advertising-based business model, because, compared with subscribers, advertisers felt so much less sense of ownership over the journalism. I recall his astonishment, fairly early in the Trump administration, after Times reporters conducted an interview with Trump. Subscribers were angry about the questions the Times had asked. It was as if they’d only be satisfied, Baquet said, if the reporters leaped across the desk and tried to wring the president’s neck. The Times was slow to break it to its readers that there was less to Trump’s ties to Russia than they were hoping, and more to Hunter Biden’s laptop, that Trump might be right that covid came from a Chinese lab, that masks were not always effective against the virus, that shutting down schools for many months was a bad idea.
https://www.economist.com/1843/2023/12/14/when-the-new-york-times-lost-its-way
There are also evident cultural shifts with regards to free speech in the general population in this same time period which seem to be a result of this elite-led shift. The Tablet article by Goldberg linked above briefly mentions data from the General Social Survey (GSS). The GSS gathers data on a set of questions measuring “Free Speech Values” among the American populace. The phenomenon seems to be limited to race issues; the GSS also collects data on Free Speech tolerance of Communist, Militarist, Homosexual, and Muslim expression and it is exclusively with Free Speech tolerance of Racism that we see a notable decrease of tolerance starting sometime between 2012 and 2014. It is also notable that this decrease in tolerance was especially pronounced among the most educated respondents. In 2012 college educated respondents favored removal of a racist book from libraries at a rate of 26%, the preference for removal peaked in 2022 at 43% although in 2024 (the most recent available data) it slightly ticked back to 41%.
https://gssdataexplorer.norc.org/trends
(The data is under the “free speech” subheading under “Civil Liberties,” you must use the drop-down menu to get to these questions. Direct links are unfortunately not possible on the contemporary GSS site. Of course this is also in the raw data.)
I’d also be remiss if I don’t mention the association of wealth inequality with political extremism. There seems to be a regular historical coincidence between periods of high economic inequality and political extremism, such as the early part of the twentienth century. Both communism and the KKK had periods of high popularity in early 20th century America (and of course Europe had its own more consequential extremism at that time as well). The mechanisms for this association are more difficult to trace than for direct media regulation or behavior.
So my argument is that the change in political culture observed recently from the more openly liberal popular discussion of the mid-twentieth century is due to abolishing The Fairness Doctrine which allowed media outlets to pander to their audiences’ worst most self-absorbed flaws for profit. This directly leads to a change in wider culture. Furthermore, we should be concerned about economic inequality as this is likely an important contributing factor to political extremism.
as all the responses above are answering the question of how the model feels,
Sorry if I was unclear. By your “test prompt” I mean the prompt asking for the feelings of the model. So put through ordinary text (not attractor text) and then ask the question “Deeply feel into which part...”
Then see if these feeling outputs cluster distinctly. Also, it may be interesting to see a mapping of feelings outputs relative to (non-attractor) conversations by topic.
I would point out that it is measures of fluid intelligence which drop in the de-myelinated MS patients, which suggests that this is not a matter of training nor remembering an already discovered pattern.