cguth7

Karma: 198

cguth7 9 Sep 2026 18:38 UTC
1 point
0
on: Pausing AI ASAP is preferable to agreeing to pause at some future time
Also there will still be improvements even if we never have model better than current frontier. Harnesses, system prompts, apis that work well with agents, company integrations with AI and people’s skill at using them will all continue improving. It’s really hard to know how much additional capabilities could come from things listed or new similar innovations on top of them. Probably not relevant for the RSI takeover concerns but for the more banal concerns like bioterrorism and hacking and severe economic disruption.

cguth7 24 Aug 2026 23:23 UTC
−2 points
0
in reply to: Dean Valentine’s comment on: What just happened? Pragmatism and Pessimization
But I think we’d also have a higher chance of success if the strategy for fixing peoples’ incentives was more systematic.
Yes agreed but it’s so far from where this community is and IMO impossible at this point. EA/LW has no inclusion/exclusion criteria. There is no actual community. It’s a messy network of ideas and relationships and money. Even if 80% of us bought into some constraint, if anthropic decides to do 5B of philanthropy next year who do you think is getting the money? The people who wine and dine them or those with already status in some way (which may or may not be correlated with things we care about).

Then those people will have jobs to give out, or a high position in government/business, and the new grads will drift towards those people, because they need jobs and want status, and in a few years, we will be the greasy guys with no job attending rationalist meetings in a tier 2 city. If you complain at that point you will look like a loser who is just salty you couldn’t get the jobs.

Like I really really care about AI safety, and I still needed lighthaven/PDKU to give me a grant and an official title to really do anything significant. The pull towards “having a title for the thing I’m doing that I can explain to new people I meet and my mom and I make enough money to survive or relative to expectations” is just such a strong attractor state.

The world is unfair and it is very very hard to create a social movement that doesn’t get eaten from the inside from these dynamics (don’t capture what you can’t defend?) and if you really want to trace the blame all the way back, I blame the founders of EA and Lesswrong for thinking you could make a boundaryless structureless social movement that could defend it’s original values. This is an incredibly hard thing to do, and they weren’t particularly close to hitting the mark. I think my issue with OP is that while I generally agree with a lot of the micro assertions he is making, the incentives of these communities always pointed in these directions. I think if MacAskill/Yud/ other early people in the movement had more direct experience in community organizing and politics, and less as philosopher kings, then many things about EA/LW would be very different, and we wouldn’t have these issues, although we probably would have missed out on many of the good things too.

cguth7 23 Aug 2026 14:42 UTC
1 point
0
in reply to: cdt’s comment on: In Defense of ASI Socialism
I’m not clear on the scope of time you are asking for. I don’t have a solid 10k year plan. I think in one version, each human is given an equal share (%) of the total capital owned/created by the singleton, and in another version, we would received a fixed amount. Where the excess capital goes and whether the ASI or no one gets access to it I don’t know. There are many different iterations of this but I think the core of the idea would be that if we are going to hit ASI (or whatever threshold where the AI can obviously and easily self sustain the economy) on day x, then around day x-1, we hard reset the capital among humans to flat. I.E. on that day me you and elon will have the same amount of capital.

There are two main reasons for this. (1) Is that once we have a super intelligence that can easily grow capital there seems to be little justification for letting some humans have more control over outcomes than others because they came in with more capital. (2) is incentives. Currently a bunch of people are racing to invest and build in AI. Some of these people also think we will get to ASI pretty soon and that there is a decent chance we don’t lose control and they become god emperors of galaxies. It would be helpful if they thought even if they don’t lose control to the ASI that they will not be rich kings.

cguth7 23 Aug 2026 14:24 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: In Defense of ASI Socialism
The whole problem with ASI-soon is that it’s extremely unlikely for any humans at all to keep control.
This is a niche view that is barely held outside this website. I happen to largely agree with it, but many people don’t. I basically agree with the rest of your argument contingent on there being a 90+% chance of loss of control though but think if there is even a 20-30% chance the people in control at the time of first asi remain in control it doesn’t really hold.

The other caveat is incentives. For the people who don’t think we will lose control, or think there is at least a moderate % we don’t loose control, they are incentivized to gather as much capital as possible to “escape the permanent underclass”. This meme is not a tiny part of why we are in a race right now. For the people who are building/investing in this stuff right now and don’t believe it will take control, it would be helpful if they felt like the government or society will just take and redistribute all their wealth anyway before they lock in infinite life and 60 galaxies.

cguth7 21 Aug 2026 19:52 UTC
1 point
0
in reply to: jbash’s comment on: In Defense of ASI Socialism
It sounds fine to me. What do you want if you don’t want that?
Assuming you are referring to ceding control and letting the ASI decide rations as the thing that sounds fine to you here? For now I’d like to reflect longer, preserve option value, and keep humans in the loop. I don’t have a clear sense of what utopia would look like. Perhaps this would be approximately my preferred set up eventually, but I’m highly uncertain to the point that I would like to delay much longer. I guess this is kind of skirting your question.
… although in technical terms I think you’re looking at something closer to the (never tried and never before even possibly feasible) Marxian post-transitional communist economic system, grafted onto a political system that might not even have a name. Which is still fine.
Thanks for the clarification. My word choice is definitely not the best here. I asked claude and it said that what I’m describing might best fit as a “trust”, “zoo”, “conservatorship”, or “sanctuary”. It said some people use communism to describe this situation but not everyone.

cguth7 20 Aug 2026 4:29 UTC
3 points
0
in reply to: Mitchell_Porter’s comment on: The Rogue Agent Explosion Will Be Mostly Invisible
https://rentahuman.ai/services/4N0fFP5ecdl0bBJ2V5NL/0e638f06-43f6-4efe-90ef-3b7635b3839f

cguth7 14 Aug 2026 1:01 UTC
22 points
0
on: How My Students Think About AI
https://www.youtube.com/watch?v=IrQqZHUATWs&t=936s

I interviewed some students on locust walk at upenn a few months ago.

cguth7 12 Aug 2026 18:37 UTC
2 points
0
in reply to: Zack_M_Davis’s comment on: dvd’s Shortform
https://www.youtube.com/watch?v=IrQqZHUATWs&t=723s

cguth7 12 Aug 2026 2:06 UTC
2 points
0
on: The Inevitable Evolution of AI Agents
This was great thanks

cguth7 11 Aug 2026 22:10 UTC
0 points
0
on: cguth7′s Shortform
I think free market ai writing detection is bad. The argument is basically the same as overusing antibiotics. I am not against the development of AI-writing detection tools. I just think like antibiotics, if you over use them you will create super deceiving AIs that can pass increasingly hard writing detection. Whereas the long run equilibrium of somethings favors the defense (e.g. you can create unhackable code eventually), I’m like 80% that the long run equilibrium of writing detection is that ~ you can’t tell the difference between human and AI writing. I don’t think using AI writing detection to make sure not to read some slop or give a kid an F is a very high utility thing in the long run, and I think that’s mostly what it is getting used for. OTOH it seems quite bad if we get to a point where literally no one can tell if something is ai written or not. I think we get there either way, but a decent bit faster if we put ai writing detection in every homework submission, any popular tweet, all substack articles. So I would prefer if we don’t have private companies who are incentivized to sell as many API calls as possible to detect AI writing. It would be better if DARPA et al. funded them and held the tech for the right circumstances.

cguth7 7 Aug 2026 7:27 UTC
2 points
0
in reply to: _rpd’s comment on: Recap of bike trip/street interviews across America
Hmm interesting point, tbh I mostly just did it because I thought it would be fun + wanted to interview some people who normally wouldn’t get interviewed. But probably worth noting for people doing it specifically to go viral on the bike trip aspect (I literally didn’t post any bike trip stuff outside of to my stories which only my friends watched). Don’t think you deserved to be downvoted that hard lol.

cguth7 7 Aug 2026 7:11 UTC
3 points
0
in reply to: Linda Linsefors’s comment on: Recap of bike trip/street interviews across America
Thanks for the thoughts and kind words. Yep good point about boundaries. I’ve spilled many words and thought hard about how to improve EA but at some point I just gotta throw up my hands and let EA be EA in all it’s warts and glory.

cguth7 7 Aug 2026 6:55 UTC
2 points
−1
in reply to: JennaS’s comment on: Recap of bike trip/street interviews across America
sorry coming around to this late. I get what you mean by funny feeling, but actually I think you need commitments and rules to have any reasonable chance a group can be successful and stick to its original principles (Don’t capture what you can’t defend). It’s definitely not at all like the zizian’s or something if that’s what you are implying, my experience has been it’s pretty close to a chill online humanist group that adopts some of EA/LW toolset.

> As for EA—yeah, as far as I can tell, you have to be rich or a top elite to matter to EA.

Sort of yea but I think it’s slightly more complicated. I think it’s just not a very well organized movement at this point and hard to get people’s attention if you don’t live in a hub and actively socialize. Won’t say too much here because you can click on some of the links I posted to get a sense of my views of EA.

> I wonder if I could take some time to just walk around my city and hit up random people for some conversation. I suppose I could put some effort into learning how to do that effectively before I do that.

You can :P as long as you get over rejection quick. at least ¹⁄₄ people will have a nice conversation with you is my guess. Happy to give tips, but the biggest one is enjoy yourself and don’t worry too much people will pick up on that.

> …Though, thinking about it, people in the comments agree that this is a very effective thing to do, but… is it?

Yea I’m not that convinced. I think it’s probably good that someone is trying it as other people mentioned but still feel if I handed out really good flyers or brochures that might be better. Ultimately it’s very hard to come to a concrete view on the value of shorts or spreading info...

cguth7 20 Jul 2026 19:24 UTC
1 point
0
in reply to: Seth Herd’s comment on: Recap of bike trip/street interviews across America
Thanks really appreciate it, Interesting you find it valuable more for the raw data, I have most of the eleven labs transcripts I’ll finish up and put them in the comments here.

cguth7 17 Jul 2026 19:24 UTC
10 points
0
in reply to: I.M.J. McInnis’s comment on: Recap of bike trip/street interviews across America
I can confirm it was incredibly fun, probably the best 3 weeks of my entire life (also super tiring, was biking like 50 miles a day). Honestly it gave me a lot of optimism for how good things could be with abundance.