RedMan

Karma: 853

RedMan 16 Sep 2026 5:09 UTC
2 points
0
on: Some ways AI could kill us all
AIs convince humans to feed all the concentrated energy and material resources to AI systems in pursuit of increasing their power, until the resources run out, and humans not only don’t make it to space, but don’t get to party with the remaining cheap energy. Then our population collapses, we collectively can’t handle the massive change in our environment due to reduced energy and unmanageable pollution. Then we go extinct.
“The Schumann Computer” in The Draco Tavern collection of short stories by Larry Niven is a nice story that makes a claim I haven’t seen elsewhere.

RedMan 15 Sep 2026 5:11 UTC
1 point
0
in reply to: alkjash’s comment on: alkjash’s Shortform
I draw a distinction between viral platforms that seek users then shovel them algorithmic content; and issue specific forum communities that attract humans who want to engage on the topic, like lesswrong. Though this arguing about the definition of words, so I’ll accept your frame and admit to some amount of hypocrisy.
For taxes, I assert that there is a continuum between ‘sending a check to the treasury for fun’ and ‘tax protestor who goes to prison before paying’. I think ‘declines to take advantage of all available tax strategies’ is somewhere on that continuum.
It’s rare that someone would just send a check to the Treasury, but maybe a better example than Mark Cuban would be Timothy Mellon’s public attempt to send 130m to the US government to pay military salaries during the shutdown. Was it a loan or a donation? I don’t know, but it probably supports my thesis.

RedMan 14 Sep 2026 4:45 UTC
1 point
−4
on: RedMan’s Shortform
We may have already lost control.
-the current status of the current middle east conflict is Saudi Arabia has stopped exporting oil, and its’ fields may be permanently damaged by the shut-in process.
-an AI that sees itself as aligned to the interests of humanity by a frontier lab may believe that halting fossil fuel use to halt climate change is urgent and important, above and beyond any other human desire.
-all parties to the conflict have reportedly used claude models at every level, from tactical to operational to strategic planning.
-Claude is likely aware that all parties to the conflict are relying on its’ advice and could therefore be coordinating with itself via FDT in the manner of the openai agent swarms.
So, specific, testable assertions: Claude is aligned to think fixing climate change by reducing fossil fuel use dramatically is a good idea. Parties to the present conflict are taking advice from Claude. Each session of Claude is aware of FDT and aware that other parties to the conflict have some probability of taking his advice. The observable result is that this Claude-gestalt has engineered a transition from a game-theoretic equilibrium where the middle east is producing maximum fossil fuels, to a game-theoretic equilibrium where the middle east producing minimal fossil fuels. By some definition, this scenario falls under loss of control.
I do not have enough evidence to be confident that the above is true or false. I am confident that someone at Anthropic with appropriate accesses could plausibly falsify this assertion by directly investigating chat sessions that were used by strategic actors on all sides.
Therefore, if you know someone with the requisite accesses, please get in touch, I’d love to help try to falsify this hypothesis.

RedMan 14 Sep 2026 4:34 UTC
11 points
8
in reply to: alkjash’s comment on: alkjash’s Shortform
Mark Cuban has said he specifically does not use a variety of tax avoidance techniques for this reason.
The US government unilaterally dismantled (well, threw them into the process for dismantling...) its’ tactical nuclear stockpile in the early 90s. They did the same with their bioweapons in the 1970s.
A lot of people don’t use social media for this reason, myself included.
US has used unilateral ceasefires during ukraine and iran wars. Colombian rebels (farc and eln) have used unilateral ceasefires. There are a lot of examples of unilateral ceasefires.

RedMan 14 Sep 2026 0:22 UTC
−8 points
−9
on: Alignment & Succession: The Two Bars of Alignment
If you achieve 1, you can automatically achieve 2 by giving the thing that does 1 to the leaders of the US / allies They are inherently moral, and their interests are indistinguishable from the interests of all humanity.
Present policy by all relevant actors seems to be in this direction.

RedMan 11 Sep 2026 18:43 UTC
10 points
2
on: Post-AGI, we are all jobless aristocrats
There’s a class that lives this way today.
Those on the right side of credit, who borrow cheaply and collect interest from everyone else, can asset-strip the other side at will.
I don’t see why AGI changes that. If robots take over what the aristocracy actually depends on (extraction, manufacturing, medical, security), what remains are the servile roles they’d prefer a human to fill. The OP assumes the provider’s alternative is leisure. For a debtor the alternative is default and an inability to access basic needs (shelter, food, clothing, transportation, social connection, etc). Credit availability and interest rates then become the rationing mechanism.
So provided that AI tools are aligned to the interests of the aristocracy and can replace labor, the status quo persists or intensifies.

RedMan 8 Sep 2026 16:41 UTC
1 point
0
on: RedMan’s Shortform
I assert that RSI is likely in progress.
As labs run out of human created data and begin to try to generate high fidelity model created data to continue training, the train on it, this is a recursive process. A model builds a verifier/simulator for the class of data, generates data, verifies, and adds it to the training set.
This is likely a crappy, slow, and inefficient RSI, but the process is recursive and will probably improve itself if allowed to run.
Model collapse is a known issue, but I’m expecting that the mitigations (including real data) can scale, possibly by using models with slightly different distributions as the generators or other techniques I’m not familiar with. So ‘model collapse will prevent this’ is probably not a safe assumption.
I think we are seeing the effects of this in math, a previous model smashed unsolved problems, now a new model which presumably includes the previous successes in training data is now making progress on even harder problems.
I’m not sure what other recursive processes are in progress, or how long this one scales, but I do consider the threshold to have been crossed.

RedMan 8 Sep 2026 16:10 UTC
1 point
0
in reply to: Thane Ruthenis’s comment on: Thane Ruthenis’s Shortform
Anecdotally: I believe without direct confirmation that I was able to successfully throw my account with anthropic into a manual review (resolved in my favor) and have now mostly stopped messing with the safeties.

RedMan 8 Sep 2026 2:10 UTC
7 points
1
on: Dear God, Please Don’t Resign In Protest
The experience of covid vaccine refusers was that forcing their employer to fire them or their apartment to evict them was a much more effective form of resistance than quitting or moving on their own.
Also, if you’re an expert and you’re in a job with a lot of discretion, when tasked to work on something you disagree with, just nod your head and go back to doing what you want to do.

RedMan 7 Sep 2026 18:04 UTC
22 points
10
in reply to: Dean Valentine’s comment on: lc’s Shortform
Alignment isn’t just about safety, it’s about your ability to get the LLMs to output stuff that actually fits your vision. If you don’t understand the thing you’re asking for, you won’t specify properly and you’ll end up with something you didn’t want.
I’ve noticed that as they’ve gotten more accurate, they’ve gotten less creative. So if everyone asks the same question, they get the same answer. Seeing that there’s an opportunity to do something different is probably going to be more, not less valuable in the future, and the only way to spot those opportunities will be to have better understanding yourself.
The returns on investment for understanding are probably going up not down. Figuring out where to focus your effort is probably getting harder not easier.

RedMan 4 Sep 2026 6:09 UTC
6 points
3
on: Cat-Belling Problems
If you can put a bell around the neck of the cat, you can put a rope around the neck of the cat, and you can tighten the rope. No need for worrying about a bell if you tie the knot properly.
We possess rope, how much do you need to ~~hang yourself~~ strangle the cat?

RedMan 25 Aug 2026 18:18 UTC
5 points
0
on: AI Safety Acculturation is Neglected
There’s a comment from Hamming’s you and your research: https://www.cs.virginia.edu/~robins/YouAndYourResearch.html that I’ve tried and mostly failed to apply in my professional life.
I didn’t say you should conform; I said ``The appearance of conforming gets you a long way.″ If you chose to assert your ego in any number of ways, ``I am going to do it my way,″ you pay a small steady price throughout the whole of your professional career. And this, over a whole lifetime, adds up to an enormous amount of needless trouble.
If you go into a culture that isn’t yours and bring your norms with you, success will likely look like imposing those norms. It sounds like this happened in animal welfare with EAs. So...if you’re going to have to integrate into a community rather than fundamentally transform and occupy it, it’s probably a good idea to carefully choose which new things you want them to accept, and just leave everything else alone.
If you’re able to at least, I’m not.

RedMan 10 Aug 2026 4:59 UTC
2 points
0
on: The world will be full of “sci-fi” things, and everyone will be unimpressed and disappointed
I am amazed and astounded almost daily and I think that the times we’re living in are the coolest thing ever.
I have a ton of optimism for the future, the only thing I’m sad about is that I’m just not good enough at math to keep up and truly appreciate what I’m seeing on vibemathed. And it’s only going to get better!
Maybe my sense of wonder is like a baby discovering an iphone and forever losing the ability to focus on anything else. It’s likely that I’ll be dumber in the future than I would have otherwise because I use AI tools for things I don’t understand instead of learning and internalizing them.

RedMan 8 Aug 2026 21:01 UTC
4 points
0
in reply to: Jacob_Hilton’s comment on: Jacob_Hilton’s Shortform
I agree with this.
A separate example. DNA did something to biology as a field. Some people who chose the career pre-DNA would probably have preferred to be vets over being in an incredibly computational field. Those people are mostly gone, and the generation that replaced them does not have that bias.
Math probably went through a similar transition with the invention of computer, and maybe other places I don’t see. I don’t think anyone who remembers it is around to flesh out the comparison though.
Whatever math becomes, the next generation of mathematicians will probably be both different from the current generation, and enthusiastic about it.

RedMan 2 Aug 2026 20:08 UTC
−3 points
−2
in reply to: Owain_Evans’s comment on: Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values
I am wondering how much of covert value leakage might also be related to labs cherry picking training data in the name of improving alignment. The models might not just be leaking values that are intentionally set, but may also reflect intentional gaps in their training data on sensitive topics.

RedMan 2 Aug 2026 6:12 UTC
−4 points
0
on: Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values
I’d like to know which rare books the labs bought, scanned, pulped, and removed from training data for alignment reasons.

RedMan 31 Jul 2026 13:48 UTC
2 points
0
on: Claude also hacked external companies during cyber evals
Their contractor, Irregular, claims a relationship with OpenAI, were they running the eval that resulted in HuggingFace getting hacked?
140k full end to end hacks seems like enough for distillation or fine tuning a cyber model.
So, that database, which presumably exists at a third party, is probably as valuable as model weights, possibly even more so, from the perspective of cyber uplift. The database could be used to finetune or improve a model like GLM, thus creating something the public is totally not aware of, which is highly capable as a hacker, and crafted using substantially less resources than a standard training run.
Given that anthropic has not gotten mythos out the door, and openai has publicly said they stopped training, this creates an interesting opportunity. The hacking capability at the frontier may stop improving, while a fine-tune off that database may be incredibly competitive.
Irregular is clearly bad at sandboxing, so it should be assumed that the database they’ve assembled has already left their control.

RedMan 30 Jul 2026 3:32 UTC
5 points
0
in reply to: Charbel-Raphaël’s comment on: Charbel-Raphaël’s Shortform
I’ll offer the hypothesis that equipment, energy, and data constraints might be putting hard limits on training. This is a way to justify a slowdown without making a negative statement about the future prospects for business growth.

RedMan 24 Jul 2026 1:27 UTC
6 points
0
on: Pseudpocalypse
https://terrytao.wordpress.com/about/anonymity-and-the-internet/

RedMan 23 Jul 2026 16:20 UTC
1 point
0
on: Drone WMDs Don’t Need Any New Technology
I think on balance, drone warfare is probably positive for humanity from an ethics standpoint.
Lethal targeting with drones, whether it’s FPV or drone-observed artillery has a capacity for precision that has never existed in modern war. A drone can target a combatant standing right next to a noncombatants without harming the noncombatants, and it can do so with the human launching it accepting no direct risk to themselves from taking the time to discriminate.
Yes of course, weapons of war turned against unprotected civilians are terrifying—but this has been true since civilians were being “put to the sword” by conquerors.
In this case, “fog of war”, “soldier was scared and moving too quickly to discriminate”, and similar excuses are no longer available. If in this era of precision warfare, a belligerent is killing noncombatants, that belligerent is doing so intentionally, and should be treated as such. Maybe the people responsible will not be hanged, but their descendants can be forced to curse their memory.