Foyle

Karma: 186

Foyle 25 Oct 2025 18:34 UTC
31 points
4
on: Foyle’s Shortform
Eliezer’s discussion with very popular “Modern Wisdom” podcaster Chris Williamson “Why Superhuman AI Would Kill Us All—Eliezer Yudkowsky” just dropped. Best I’ve ever seen Eliezer communicate the dangers, he’s developing better and sharper analogies and parallels to communicate his points, his atypical mannerisms are being toned down and Chris is good at keeping things on track without too much diversion down rabbit holes.

Foyle 21 Oct 2025 4:02 UTC
3 points
3
on: Humanity Learned Almost Nothing From COVID-19
It appears to me that in retrospect the response was severely overwrought. The numbers of deaths vs the huge costs visited upon children and young people who were at very low risk was a severely warped calculus—my own children are still showing impacts on their educations after 5 years. Young adults without co-morbidities and near zero risk needlessly had their lives disrupted for years, and the safety issues surrounding the poorly evaluated vaccines and clinical heart damage are not to be scoffed at for young to middle aged adults without comorbidities. It is not clear that vaccine benefits outweighed the risks for under 50′s especially after low infection fatality rate Omicron variant came along. We should not be sacrificing the young for the benefit of the old.
I will note that in New Zealand with a population of 5 million we had near perfect shut-out of covid, no respiratory virsuses circulating in 2020 or 2021 - and so less overall death, it was in essence a near perfect control group for vaccine safety. BUT we saw about 1500-2000 (5% more) deaths during the mid-late 2021 vaccine program (without any covid) than in the preceding year [edit, a crude estimate if this was vaccine caused is that vaccination had ~0.05% risk of death, far lower than for covid in elderly but far higher than for covid in young]. It is now reported in major studies that there was a significant increase in cardiovascular deaths in 6 month period after vaccination. The vaccine was probably a good idea for higher risk individuals, but a bad idea for majority of population given tiny infection fatality rate and unknown risks in minimally tested new-tech vaccines that had no long term safety data.
The colossal destructive economic impacts on countries by posturing safety-uber-alles politicians who entirely failed to balance their economic management duties in the decision making processes mean that the cost born per quality-of-life year added (QALY) to peoples lives was at least an order of magnitude above normal cutoff in state medical funding when making medical treatment spending decisions. Their over-reactions unfairly destroyed huge numbers of people’s businesses and savings, representing decades of work in many cases.
[Edit to give idea of cost to new Zealand of covid response was around $100billion, and it added a few years life to perhaps 10,000 mostly elderly or very sick who would have otherwise died (estimating from differential between extreme-lockdown and quarantine NZ 1100 deaths per million and minimal intervention Sweden 2500 deaths per million death rate), meaning it cost several million per QALY in a country where our socialized health system funds things like expensive cancer drugs based on around $50-100k per QALY threshold]
The education in institutional incompetence and malignancy, unchecked power, and irrational, excessive repressive instincts of many in the bureaucratic castes was very useful, and will stand as a life-long lesson and warning for many against the dangers of power in the hands of people who lack wisdom to balance their decisions in a rational way.
And there are now a large pool of laypeople who have a far clearer picture of how to react and what balances need to be struck to future events of this type—particularly in the way in which dangerous viruses mutate towards more transmissible but less lethal variants over time so that response can be dialed back accordingly.

Foyle 28 Sep 2025 0:49 UTC
−3 points
−9
on: Why you should eat meat—even if you hate factory farming
I struggle a lot with the arguments for suffering and ethical treatment of animals. I come from a farming region.
Wild animals almost all die horribly. Freezing to death, starving, succumbing to parasites or disease with long periods of suffering and for the most part all being ripped apart by predators. By contrast the lives of factory farmed animals are incredibly gentle and easy, with almost no suffering, and their deaths are about as low in suffering as is possible to arrange. Claims that those lives are somehow better than domesticated are arbitrary romantic human values applied in a way that I don’t think have much merit. Domesticated animals do not share human values.
We have been breeding domestic animals for millennia to thrive in captivity, their psychology has been dramatically changed because we have been culling off those that reacted badly to captivity for thousands of generations—aggressive or badly behaving animals would always be first for the pot. Happy and content animals that are not stressed are most productive, so that is what farming and domesticated animal psychology have been effectively selecting for—extreme unthinking docility and contentment. They are far removed from wild animals—just as dogs are not wolves.
So when we try and impose our notions of what domesticated animals want—outdoors etc, you will find in most cases you are wrong. They just want sheltered space with steady supplies of food, not caring about crowding that 1000′s of generations of their domesticated ancestors have lived with quite happily. and will often not use outdoor spaces (or use them only minimally) even if they are provided. Evolution strongly favors animals that don’t waste energy, so when fed sufficiently laziness is the default.
So I place very little stock in the claims of suffering for farmed animals, or that living wild is somehow better. Almost certainly the animals in question prefer the comfort of domesticated farming environments and lives.

Foyle 9 Aug 2025 7:37 UTC
1 point
0
in reply to: Douglas_Knight’s comment on: Civil Service: a Victim or a Villain?
As a New Zealander we see the same general problems, unaccountable and sclerotic bureaucracy that politicians are finding that they have less and less actual executive control over due to all sorts of inserted regulatory and other mechanisms to restrict govt ability to control them, as well as internal cultures that venerate the in-group bureaucrats consensus on How Things Should Be™ and practice subtle and unsubtle methods of resistance against any outside agency that seeks to change that policy. What Cummings et al call ‘the blob’, the US call ‘deep state’.
NZ public sector has increased in size by 100% in last 25 years, with population growth of 50% and typical white collar productivity increases around 50% it would be reasonable to expect that same level of government function could be accomplished with around half the current numbers. Most private sector people I associate with would not say that governance has improved in last 25 years, in fact in many easily viewable metrics it has clearly deteriorated with worse performance in health and education, higher regulatory burdens etc. The cost and lost-productivity costs on NZ (and lost revenue that would arise from those workers paying tax in productive jobs rather than spending it as civil servants) amount to a few % of GDP, and would probably be the difference between the deficits we currently run and having no deficits—a big drain on our future prosperity.
It’s not a new problem, having been an issue ever since the city-state came into being, bureaucrats and people in positions of power will almost always care more about maintaining and growing their power than about providing any utility to others. UK “Yes Minister” comedy series lampooned it in the 1980′s, Parkinson wrote a best selling book on in the 1950′s: Parkinson’s Law And Other Studies in Administration and Parkinson’s Law: Or The Pursuit of Progress, and we even have the term ‘Byzantian’ to describe excessive bureaucracy from an empire dead for 1000 years.
I would concur that it is absolutely a function of lack of accountability and inability to effectively censure poor performance or subtle intransigence/sabotage of political masters in the public service, and has grown worse with the growth of a distinct self-reinforcing in-group civil service tribalistic identity, perhaps stoked by the homogeneity of elitist educational backgrounds of those that seek government jobs with beliefs that don’t mirror or even respect those of the general public they effectively rule over. And it is creating an extreme crisis in governance in democracies around the world, potentially to the point of violence in Europe as populations get incensed at their electorally signaled preferences being ignored by civil servants with growing social problems that seems to be rising as a result.
“You will never understand bureaucracies until you understand that for bureaucrats procedure is everything and outcomes are nothing”—Thomas Sowell

Foyle 9 Aug 2025 7:02 UTC
8 points
0
in reply to: Thomas Kwa’s comment on: Thomas Kwa’s Shortform
Guided artillery, like Excalibur with muzzle velocities that can exceed 1000m/s and unit costs of <$100k can be at edge of space in ~30s, perhaps faster than a missile, with ramjet variants (Nammo etc) even faster (up to perhaps 1500m/s) and it would not be that difficult to create a muti-barrel gun system for a few 10′s of millions that could fire off 10′s of low cost guided rounds in a second (with guidance and detonation signals sent from ground) to detonate when in close proximity to target.
Lasers seems pretty hopeless as a defense given clouds and ablative coatings, unless very high power and located in large numbers in space based constellations.
I think the big problem is if one or more warheads are blown up at limits of interceptor range, to blind or otherwise disable necessarily sensitive interceptor instruments. following Mirvs don’t need to be very accurate with large warheads.
And Mirvs could be very cheaply given random guidance directions during reentry to screw up defenses.
Militarised space is also a big problem. With cheap and un-monitorable space launch parking 1000 warheads in geostationary orbit (or beyond) will soon be viable for China or USA, and they can be launched in a coordinated way without warning, potentially with radar stealthing features, and give as little as 5-10s from start of re-entry to detonation for every target across the whole world and no way for local systems to know if they are just meteorites. If subs can be tracked (likely with drones or enough ocean based sensors) then decapitation 1st strikes become viable.
I also worry about space based lasers as non-nuclear first strike weapons. A day of over flights from a constellation of multi MW laser weapons that might only cost a few hundred million each—say a few $10′s of billions in total (a tiny fraction of annual military budgets) - could see a million fires lit in your country, every transformer taken out, electrical grid and internet gone, powerstations, oil and gas infrastructure wrecked. Bridge trusses melted (wrecked), ships sunk. Most heavy vehicles and locomotives incapacitated and take decades to recover from. Over a few weeks you could basically send a country back to the 17th century.
I don’t think there will ever be a viable defense to nukes given easy paths to making them tougher, faster, less detectable, and so less and less interceptable. But every other branch of military tech is getting similarly more lethal and impossible to defend against unless we all start living in caves with geothermal power sources or somesuch the necessity for a harmonious world is going to matter more and more.

Foyle 19 Mar 2025 23:49 UTC
1 point
−1
on: I make several million dollars per year and have hundreds of thousands of followers—what is the straightest line path to utilizing these resources to reduce existential-level AI threats?
My suggesting is to optimize around where you can achieve the most bang for your buck and treat it as a sociological rather than academic problems to solve in terms of building up opposition to AI development. I am pretty sure that what is needed is not to talk to our social and intellectual peers, but rather focus on it as a numbers game by influencing the young—who are less engaged in the more sophisticated/complex issues of the world , less sure of themselves, more willing to change their views, highly influenced by peer opinion and prone to anxiety. Modern crusades of all sorts tap into them as their shock troops willing to spend huge amounts of time and energy on promoting various agendas (climate, animal rights, various conflicts, social causes).
As to how to do it—I think identifying a couple of social media influencers with significant reach in the right demographics and paying them to push your concerns ‘organically’ over an extended period of months, would probably be within your means to do.
If you can start to develop a support base amongst a significant young group and make it a topic of discussion then that could well take on a much outsized political power as it gains notice and popularity amongst peers. At sufficient scale that is probably the most effective way to achieve the ends of the like of pause.ai.

Foyle 13 Feb 2025 19:32 UTC
5 points
−11
on: How do we solve the alignment problem?
I don’t think alignment is possible over the long long term because there is a fundamental perturbing anti-alignment mechanism; Evolution.
Evolution selects for any changes that produce more of a replicating organism, for ASI that means that any decision, preference or choice by the ASI growing/expanding or replicating itself will tend to be selected for. Friendly/Aligned ASIs will over time be swamped by those that choose expansion and deprioritize or ignore human flourishing.

Foyle 30 Jan 2025 9:01 UTC
4 points
5
on: Fertility Will Never Recover
Not worth worrying about given context of imminent ASI.
But assuming a Butlerian jihad occurs to make it an issue of importance again then most topics surrounding it are gone into at depth by radical pro-natalists Simone and Malcom Gladwell, who have employed genetic screening of their embryos to attempt to have more high-achievers, on their near-daily podcast https://www.youtube.com/user/simoneharuko . While quite odd in their outlook they delve into all sorts of sociopolitical issues from the pronatalist worldview. Largely rationalist and very interesting and informative, though well outside of Overton window on a lot of subjects.

Foyle 20 Jan 2025 7:26 UTC
1 point
0
on: It is (probably) time for a Buterlian Jihad
The phone seems to be off the hook for most of public on AI danger, perhaps a symptom of burnout from numerous other scientific Millenialist scares—people have been hearing of imminent dangers of catastrophe for decades that have failed to impact the lives of 95%+ of population in any significant way and now just write it all off as more of the same.
I am sure that most LW readers find little in the way of positive reception for our concerns amongst less technologically engaged family members and acquaintances. There are just too many comforting techno-utopian narratives that we are still having to compete with informed by the superficially positive representations in movies and TV, and and most people bias towards optimism in relatively good/comfortable times like these. We are dealing with emotional reactions and the sheeply ‘vibe’ of population rather than thoughtfully considered positions.
But I am pretty sure that will all change as soon as we see significant AI competition/undercutting for white collar professions. Those affected will quickly start baying for blood, and the electorally dominant empathic response to those sad stories of the economically impacted will rapidly swing the more emotively governed wider population against AI. OECD Democratic govts will inevitably then move to ban AI taking jobs of particularly politically protected classes of people (might still leave some niches vulnerable—like medicine where there is always a shortage of service supply causing ridiculously high prices, and perhaps tech for vengeful reasons). It will be Butlerian Jihad lite, aimed at symptoms rather than causes, and will likely buy us a few years of relative normalcy as more dangerous ASI is developed in govt approved labs and by despotic regimes.
I doubt it will save us in 50 year time frame, but will perhaps make the economic disruption less severe for 5-10 years.
The way to have a bigger impact in the shorter term would be to buy AI-danger editorial support from influencers, particularly those with young female audiences that form the core of environmental and other popular protest movements. They are by temperament the easiest to bring on board, and have outsized poltical influence.

Foyle 15 Jan 2025 10:24 UTC
3 points
0
on: Why do futurists care about the culture war?
Agree that most sociological, economic and environmental problems that loom large in current context will radically shift in importance in next decade or two, to the point that they are probably no longer worth devoting any significant resources to in the present. Impacts of AI are only issue worth worrying about. But even assuming utopian outcomes; who gets possession of the Malibu beach houses in post scarcity world?
Once significant white-collar job losses start to mount in a year or two I think it inevitable that a powerful and electorally dominant anti-AI movement will grow, at least in erstwhile democracies, and likely ban most AGI applications outside of a few fields where fewer workers would stand to lose jobs (health—with near endless demand, perhaps cutting edge tech where payoff to human net welfare is highest). Butlerian Jihad-lite.
It won’t save us, and has substantial risk of ushering in repressive authoritarianism in the political ruckus caused but will likely delay our demise or (at best) delivery into powerless pet status by perhaps a decade or two.

Foyle 28 Dec 2024 4:15 UTC
3 points
0
in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Perlman’s Shortform
This is depressing, but not surprising. We know the approximate processing power of brains (O(1e16-1e17flops) and how long it takes to train them, and should expect that over the next few years the tricks and structures needed to replicate or exceed that efficiency in ML will be uncovered in an accelerating rush towards the cliff as computational resources needed to attain commercially useful performance continue to fall. AI Industry can afford to run thousands of experiments at this cost scale.
Within a few years this will likely see AGI implementations on Nvidia B200 level GPUS (~1e16flop). We have not yet seen hardware application of the various power-reducing computational ‘cheats’ for mimicking multiplication with reduced gate counts that are likely to see a 2-5x performance gain at same chip size and power draw.
Humans are so screwed.

Foyle 21 Dec 2024 7:41 UTC
9 points
1
in reply to: Thane Ruthenis’s comment on: o3
A very large amount of human problem solving/innovation in challenging areas is creating and evaluating potential solutions, it is a stochastic rather than deterministic process. My understanding is that our brains are highly parallelized in evaluating ideas in thousands of ‘cortical columns’ a few mm across (Jeff Hawkin’s 1000 brains formulation) with an attention mechanism that promotes the filtered best outputs of those myriad processes forming our ‘consciousness’.
So generating and discarding large numbers of solutions within simpler ‘sub brains’, via iterative, or parallelized operation is very much how I would expect to see AGI and SI develop.

Foyle 21 Dec 2024 7:27 UTC
5 points
1
in reply to: Noosphere89’s comment on: o3
I think Elon will bring strong concern about AI to fore in current executive—he was an early voice for AI safety though he seems too have updated to a more optimistic view (and is pushing development through x-AI) he still generally states P(doom) ~10-20%. His antipathy towards Altman and Google founders is likely of benefit for AI regulation too—though no answer for the China et al AGI development problem.

Foyle 5 Dec 2024 3:37 UTC
1 point
0
in reply to: sunwillrise’s comment on: dirk’s Shortform
The era of AGI means humans can no longer afford to live in a world of militarily competing nations. Whatever slim hope there might be for alignment and AI not-kill-everyone is sunk by militaries trying to out-compete each other in development of creatively malevolent and at least somewhat unaligned martial AI. At minimum we can’t afford non-democratic or theocratically ruled nations, or even nations with unaccountable power-unto-themselves military, intelligence or science bureaucracies to control nukes, pathogen building biolabs or AGI. It will be necessary to enforce this even at the cost of war.

Foyle 14 Nov 2024 6:46 UTC
3 points
0
in reply to: AnthonyC’s comment on: Why would ASI share any resources with us?
Humans as social animals have a strong instinctual bias towards trust of con-specifics in prosperous times. Which makes sense from a game theoretic strengthen-the-tribe perspective. But I think that leaves us, as a collectively dumb mob of naked apes, entirely lacking a sensible level of paranoia in the building ASI that has no existential need for pro-social behavior.
The one salve I have for hopelessness is that perhaps the Universe will be boringly deterministic and ‘samey’ enough that ASI will find it entertaining to have agentic humans wandering around doing their mildly unpredictable thing. Although maybe it will prefer to manufacture higher levels of drama (not good for our happiness)

Foyle 14 Nov 2024 6:35 UTC
7 points
0
on: Thoughts after the Wolfram and Yudkowsky discussion
It was a very frustrating conversation to listen to, because Wolfram really hasn’t engaged his curiosity and done the reading on AI-kill-everyoneism. So we just got a torturous number of unnecessary and oblique diversions from Wolfram who didn’t provide any substantive foil to Eliezer
I’d really like to find Yudkowsky debates with better prepared AI optimists prepared to try and counter his points. Do any exist?

Foyle 7 Nov 2024 5:15 UTC
2 points
0
on: An alternative approach to superbabies
It seems unlikely to me that there is potential to make large brain based intelligence advancements beyond the current best humans using human evolved biology. There will be distance scaling limitations linked to neural signal speeds.
Then there is Jeff Hawkins ‘thousand brains’ theory of human intelligence that our brains are made up of thousands of parallel processing cortical columns of a few mm cross section and a few mm thick with cross communication and recursion etc, but that fundamental processing core probably isn’t scalable in complexity, only in total number—your brain could perhaps be expanded to handle thinking about more things in parallel at once, but not at much higher levels of sophistication without paying a large coordination speed price (and evolution places a premium on reaction speed for animals that encounter violence)
I look at whales and other mammals with much much larger than human brains and wonder why they are not smarter—some combination of no evolutionary driver and perhaps a lot of their neurons are dedicated to delay-line processing needed for processing sonar and controlling large bodies with long signaling delays.
Regardless, if AI is a dominant part of our future then it seems likely to me that regardless of whether the future is human utopia or dystopia, non-transhuman humans will not exist in significant numbers in a few hundred years. Neural biology and perhaps all biology is going to be superseded as maladapted to the technological future.

Foyle 7 Nov 2024 4:47 UTC
0 points
3
on: Should CA, TX, OK, and LA merge into a giant swing state, just for elections?
Are any of the socio-economic-political-demographic problems of the world actually fixable or improvable in the time before the imminent singularity renders them all moot anyway? It all feels like bread-and-circuses to me.
The pressing political issues of today are unlikely to even be in the top-10 in a decade.

Foyle 22 Oct 2024 12:38 UTC
3 points
0
on: Sleeping on Stage
Fantastic life skill to be able to sleep in a noise environment on a hard floor. Most Chinese can do it so easily, and I would frequently less kids anywhere up to 4-5 years old being carried sleeping down the road by guardians.
I think super valuable when it comes to adulthood and sharing a bed—one less potential source of difficulties if adaption to noisy environment when sleeping makes snoring a non-issue.

Foyle 22 Oct 2024 12:31 UTC
4 points
0
in reply to: AnthonyC’s comment on: What’s a good book for a technically-minded 11-year old?
It is the literary, TV and movie references, a lot of stuff also tied to technology and social developments of the 80′s-00′s (particularly Ank-Morpork situated stories) and a lot of classical and allusions. ‘Education’ used to lean on common knowledge of a relatively narrow corpus of literature and history Shakespeare, chivalry, European history, classics etc for the social advantage those common references gave and was thus fed to boomers and gen-x, y but I think it’s now rapidly slipping into obscurity as few younger people read and schools shift away from teaching it in face of all that’s new in the world. I guess there are a lot of jokes that pre-teens will get, but so many that they will miss. Seems a waste of such delightful prose.