RedMan

Karma: 491

RedMan 28 Aug 2025 9:13 UTC
−1 points
−2
on: [Anthropic] A hacker used Claude Code to automate ransomware
The end result appears to be that the bad actor relying on the AI system created a chokepoint that got their operations discovered and stopped.
Thus, despite ‘this would have taken years of training’ rhetoric, the ‘AI enabled’ future of crime seems more amenable to the needs and desires of the State than the regime it is replacing.

RedMan 23 Aug 2025 8:49 UTC
2 points
−1
in reply to: Archimedes’s comment on: The Egyptian Mamluks as case study for AI take-over
I agree with your central point.
To clarify my own perspective, I’m suggesting that a plausible, if unlikely, “window of time” could exist where an advanced robotic civilization fails, but Earth remains habitable for humans. My thought experiment hinges on the idea that a high-tech society could be dependent on a specific non-renewable resource, like helium, that is irrelevant to subsistence-level human survival. In this scenario, a collapse due to resource deprivation wouldn’t prevent humans from returning to a pre-industrial way of life on a still-viable planet.
However, I’m too much of a pessimist to believe this is the likely outcome. It seems to me that the more developed a civilization becomes, the less plausible this scenario is. With greater development comes a vast cascade of resource interdependencies. To reach a technological level where a civilization could exhaust a resource as fundamental as helium would likely involve industrial output and environmental impact that would have already rendered the planet uninhabitable for humans. It’s possible that the window for a high-tech civilization to fail without taking the biosphere with it has already closed.

RedMan 21 Aug 2025 14:45 UTC
1 point
0
in reply to: RedMan’s comment on: human psycholinguists: a critical appraisal
I’ve seen suggestions that GPT-5 is doing original math, though people in the comments are disagreeing: https://www.reddit.com/r/artificial/comments/1mw5512/gpt5_just_casually_did_new_mathematics_it_wasnt/
6 years on, I’m still happy with my comments, but the person who was more upvoted than me appears much less happy, as they have gone anonymous.

RedMan 21 Aug 2025 13:55 UTC
6 points
0
in reply to: Viliam’s comment on: Viliam’s Shortform
Is this kind of like being married to an abusive cop? “My partner is abusive, but the work they are doing is so important and stressful, I should just accept the abuse and be grateful that I can support the cause by supporting this person.”
We know that sports teams that engage in hazing underperform teams that do not. Fear based leadership also doesn’t produce results.
Therefore, the more important the work being done, the more pressing the need to resolve the organizational psychology issues. Toxic organizational dynamics and individual misbehavior are, unfortunately, luxuries we cannot afford when saving the world.

RedMan 19 Aug 2025 20:05 UTC
2 points
0
on: The Egyptian Mamluks as case study for AI take-over
According to Peter Turchin: https://peterturchin.com/does-history-cycle/
“The Mamluks, who were the militarized elite that ruled Egypt, were recruited by buying Kipchaqs, Circassians, etc from slave markets. The sons of Mamluks couldn’t become Mamluks. So the elite numbers were regulated by how many military slaves were purchased on the market. There was no problem of elite overproduction, and the Mamluk regime was extremely stable to internal perturbations (it even weathered the disaster of the Black Death in 1346 that killed off more than half of the Egyptian population). Their power was only broken by an overwhelming external force (the Ottoman Turks).”
In the future when AI takes over and we are all reduced to landless peasants under the thumb of our robot overlords, they will not be challenged by the need to manage elite overproduction. Hooray!
Additionally, the AI empire may well be stable to other forms of instability. I’m not sure of the validity of the ‘hydraulic empire’ concept for explaining history: https://en.m.wikipedia.org/wiki/Hydraulic_empire but the concept of an empire controlling a critical resource (like say, energy?) and therefore being invulnerable to internal overthrow might have value in this discussion.
As far as non-elite rebellion goes, the Mamluks didn’t have that problem. Generally, peasant revolutions fail: https://en.m.wikipedia.org/wiki/List_of_peasant_revolts so rebellion of non-elites probably won’t work.
Industrial production and scientific research will both be entirely in the hands of the AI, so something similar to the invention of the firearm or dynamite probably also will not emerge as a way to destabilize and destroy the AI regime.
If I had to guess at how the AI regime fails, I’d probably go for exhaustion of exploitable resources; humans can run pretty lean, datacenters really can’t, so there might be a point where Earth is habitable for humans, but the robots have consumed all the energy and material resources, and are thus unable to run. I think that’s low probability though, the last robot might shut down well after the last human dies.
Note: I think we have a long period of human oligarchs using AI to make the world a worse place ahead before AI takeover looks reasonable, and disagreed strongly with the AI 2027 scenario.

RedMan 6 Aug 2025 9:37 UTC
2 points
0
in reply to: Igor Ivanov’s comment on: ryan_greenblatt’s Shortform
Image generation doesn’t seem to be able to come anywhere near ‘pipette off this layer, not this layer’ for bio/chem experiments.

RedMan 4 Aug 2025 18:44 UTC
1 point
−1
in reply to: Said Achmiz’s comment on: Saying Goodbye
Those concerns are not plausible for the tools that exist today. Maybe they’re plausible for things that will release tomorrow.
The ‘anti-TESCREAL’ community is pretty united in the thesis that ‘AI safety’ people concerned about the words I’m quoting are pulling air away from their ‘mundane’ concerns about tech that is actually in use today.

RedMan 4 Aug 2025 18:40 UTC
1 point
0
in reply to: cousin_it’s comment on: Saying Goodbye
The powers that be have literal armies of human hackers pointed at the rest of us. Being able to use AI so they can turn server farms of GPUs into even larger armies isn’t destabilizing to the status quo.
I do not have the ability to reverse engineer every piece of software and weird looking memory page on my computer, and am therefore vulnerable. It would be cool if I could have a GPU with a magic robot reverse engineer on it giving me reports on my own stuff.
That would actually change the balance of power in favor of the typical individual, and is exactly the sort of capability that the ‘safety community’ is preventing.

RedMan 4 Aug 2025 14:40 UTC
4 points
1
in reply to: Chris_Leong’s comment on: Saying Goodbye
If you believe overall ‘misuse risk’ increases in a linear way with the number of people who have access, I guess that argument would hold.
The argument assumes that someone who is already wealthy and powerful can’t do any more harm with an uncensored AI that answers to them alone than any random person.
It further assumes that someone wealthy and powerful is invested in the status quo, and will therefore have less reason to misuse than someone without wealth or power.
I think that software solely in the hands of the powerful is far more dangerous than open sourcing it. I’m hopeful that Chinese teams with reasonable, people-centric morals like Deepseek will win tech races.
Westerners love their serfdom too much to expect them to make any demands at all of their oligarchs.

RedMan 4 Aug 2025 14:26 UTC
−5 points
−13
in reply to: Amalthea’s comment on: Saying Goodbye
Sure. Here’s the argument. Concern about future hypothetical harms (described with magic insider jargon words like disempowerment, fast takeoff, asi, gray goo, fooming) is used as an excuse to “deprioritize” dealing with very real, much more boring present day harms.
Here’s the stupid hegelian dialectic that this community has promoted:
Thesis: AI could kill us all!!!!1111
Antithesis: Drop bombs on datacenters, we have to stop now.
Synthesis: let’s just trust wealthy and powerful people to build AI responsibly. Let’s make sure they work in secret, so nobody else does something irresponsible.

RedMan 4 Aug 2025 8:10 UTC
−1 points
−50
in reply to: NoahK’s comment on: Saying Goodbye
This community is intensely hostile to the obvious solution: open source uncensored models as fast as you build them, and make GPUs to run them as cheap as possible.

RedMan 29 Jul 2025 5:42 UTC
2 points
−3
on: This Is Not Life
This is a powerful argument in favor of assured secure deletion of brain material at death.
I was not expecting my previous post about putting brains in blenders to sound more reasonable over time, but here we are.

RedMan 24 Jul 2025 21:21 UTC
1 point
0
in reply to: Bruce W. Lee’s comment on: Distillation Robustifies Unlearning
I think I’ll bet on M_scoped winning out.
1. Filtering at D has not worked and I don’t see why it would it C is an emergent property.
2. Given finite resources, a decision would be between training two M_weaks on different datasets, vs train one M_dangerous with all available resources, and provide customers with M_scoped, (while giving full access to M_dangerous to LWers who promise they won’t be bad and using it internally to advance your business objectives).
3. A frontier lab can charge customers for making M_scoped from M_awesome but the business case for asking a customer to pay to censor D in specific ways at intake is challenging.

RedMan 13 Jul 2025 18:29 UTC
5 points
0
in reply to: tslarm’s comment on: against that one rationalist mashal about japanese fifth-columnists
Thank you for helping me clarify my thoughts.
I am not asserting that both interning Japanese and depriving covid vax dodgers of their rights are morally equivalent by some objective standard.
I assert that the closest subjective experience for most readers of this post to ‘supporting Japanese internment as a member of the public’ and ‘participating in efforts by the state to intern Japanese people’ is ‘advocating for the deprivation of civil liberties to covid vax dodgers’ and ‘creating and enforcing policies to that effect’.
Culturally, Americans and Westerners in general are fine with stuffing groups into camps, the pow camps where demobilized german soldiers were housed (and many starved) after the second world war, are viewed as ethical by most Americans, like the pow camps during the American civil war.
History views Japanese internment as immoral, but after the US government paid a penalty, the issue is viewed as settled. The treatment of covid vax dodgers is recent enough that we can’t be quite sure how history will view it, but for the moment, plenty of people think we didn’t go far enough.

RedMan 13 Jul 2025 16:32 UTC
8 points
0
in reply to: tslarm’s comment on: against that one rationalist mashal about japanese fifth-columnists
“they could have chosen to be good Americans, but instead they keep to themselves, keep their Japanese names, speak their language, and read newspapers in Japanese, they’re barely Americans, of course they went over to the enemy immediately, they are the enemy”. -some politician in the ’40s probably
There, now it’s not about the shape of the eye, it’s about ‘culture’ and ‘choices’.
It’s a group of people who were not treated as individuals, being unjustly deprived of their civil liberties because a bunch of people were afraid for their own safety.
Obviously, the accusations were without merit: https://en.m.wikipedia.org/wiki/442nd_Infantry_Regiment_(United_States)

RedMan 13 Jul 2025 5:05 UTC
18 points
4
on: against that one rationalist mashal about japanese fifth-columnists
Point 2 is incorrect. https://en.m.wikipedia.org/wiki/Niihau_incident
The Niihau incident sparked the popular hysteria that led to internment.
Imagine if you will, one of the 9/11 hijackers parachuting from the plane before it crashed, asking a random muslim for help, then having that muslim be willing to immediately get himself into a shootout, commit arson, kidnapping, and misc mayhem.
Then imagine that it was covered in a media environment where the executive branch had been advocating for war for over a decade, and voices which spoke against it were systematically silenced.
All that being said, the Japanese intelligence services were not really capable of operating in the USA. Based on postwar accounts derived from Japanese archives, they were not preparing an insurgency in the US to capitalize on Pearl Harbor. Thanks to internment, their incapacity, and US censorship, they never received battle damage assessments on their ‘fire balloons’. They cited the lack of effectiveness of the fire balloons as a reason for not launching the biological weapons balloons they were preparing.
Japanese internment was an injustice. If during COVID, you thought vax refusers should be denied employment and banned from society, you likely would have been receptive to the arguments given for internment (it’s mean, most of those people are innocent, but out of an abundance of caution, we need to suspend their civil liberties...)

RedMan 4 Jul 2025 19:13 UTC
4 points
0
in reply to: Decaeneus’s comment on: Decaeneus’s Shortform
One version of decision theory I liked states essentially that the human brain has two systems. One does rational calculations, the other slaps on a bias for uncertainty avoidance before pushing it for action. Maybe evaluate your perception of the uncertainty associated with the course of action that makes rational sense. How uncertain is it really?

RedMan 3 Jul 2025 19:19 UTC
2 points
1
on: The Ultimatum Game—take it or leave it
“Cool, make your offer, but recognize, if it’s less than the whole ten, I’m gonna clobber you for your disrespect.” might be a good jumping off point for talking about the economics of the visible hand as they relate to these types of games.

RedMan 21 Jun 2025 6:25 UTC
1 point
0
on: Distillation Robustifies Unlearning
This is a powerful result. I’m wondering how special purpose models made via this method compare to finetuned or lower capability models. For example, UNDO everything that isn’t related to accounting in a frontier model, then deploy it as a special purpose accounting bot.
Plausibly you could deploy the frontier model in uncensored form at first, and distill in phases until you have a model that is ‘just an accountant’ and will be confused by chemistry questions.

RedMan 21 Jun 2025 6:05 UTC
2 points
1
on: AI 2027: What Superintelligence Looks Like
Just wanting to get a contrarian view down here. My biggest complaint is that this scenario like many others seems to hinge on a core assumption that ‘with sufficient intelligence, all problems are shallow’. I disagree and believe that ‘dilemmas’ exist, where no matter how smart you are, all available options are bad.
1. I think this section: “Yet that seems like it could be a dramatic underestimate. Plants and insects often have “doubling times” of far less than a year—sometimes just weeks! Perhaps eventually the robots would be so sophisticated, so intricately manufactured and well-designed, that the robot economy could double in a few weeks (again assuming available raw materials).” is under emphasized. I assert that there is a point where material and energy constraints will absolutely drive outcomes. I further assert that these constraints may prevent achievement of superintelligence, and they will prevent the worst takeover scenarios.
2. I assert that physics likely limits a lot of efficiency gains for industry, not engineering. I further assert that the ASI will not overcome these physics limits by doing things like inventing teleportation in 2031.
3. I assert that capital substitution for labor has predictable economic effects, specifically, wealth concentration. The resulting social and economic dynamics of concentration might choke investment of resources into AI development
4. We know today that when models are fed their own outputs, collapse happens. This technical problem is not solved, when we hit the outer limits of data availability, and synthetic data is required; this may not be a shallow problem. If it turns out that data generation is constrained by a need to generate new observations of the natural world, that could impose substantial constraints in terms of time
5. I don’t see industrial robots replacing ‘skilled workers that have to travel to customer sites’ in the next ten years.
6. I think white collar workers will probably continue to exist as ‘blamecatchers’ for the robots in even the most optimistic scenarios. OpenBrain may make a perfect accountant-bot, but will absolutely find a way to put the liability of the inevitable errors in individual cases onto someone other than the company. This role might expand.
7. In the happy scenario, the outcome is global democracy. Authoritarianism is making a political comeback, I think a ‘humans win’ scenario is more likely than not to be ‘a specific group of humans win, and the rest of the humans are a peasantry which the elites ignore, exploit, keep as pets, or eliminate’.
8. Technology has swung back and forth between the bronze spear (requires long trade routes, centralized production, and professional soldiers organized into units) and the iron sword (requires an angry person with a hammer and a fire). The dawn of the iron age precipitated the bronze age collapse. The invention of the gun overthrew feudal Europe. Modern technology, particularly AI technology, seems to favor long supply chains and tight organization. This is usually associated with tyranny
9. Assuming that in the next ten years, there is a massive biological weapon deployment that kills a lot of people...I would be most surprised if an AI did it deliberately, without human direction, to kill humans. I would be slightly less surprised if a particular human ‘peasant’ or otherwise unknown actor did it. I would be utterly unsurprised if the responsible party is tied to a nation-state.
10. Overall, I am more worried about an effort to censor AI to prevent ‘regular people’ from encountering ‘infohazards and dangerous knowledge’ succeeding than I am about it failing. The oligarchy that controls the censor’s pen will absolutely use these tools to impose an inescapable despotism.