24yo human male from Toronto. Rationalist-adjacent. Inkhaven 2 alumnus. Writer, marketer, game designer. @PhiliDips_ on Twitter
Philip Harker
Karma: 250
Flibertigibbeting
I am a 24 year old male. A year ago I figured I had well over a half century until I’d die. I’m no longer sure of this.
Six months ago I thought AI was an interesting, useful, and sometimes unwieldy tool for programmers and researchers and such, and maybe it was at risk of displacing workers and shifting the global economy.
Five months ago I was told about the METR Graph, the Mythos Incident, and the enormous power of AIs. I learned that AI progress had not, in fact, topped out. It’s at the beginning of a sigmoid curve of indeterminate length. And I was terrified. I was in Berkeley, CA at the time and I spent a couple days walking around the city looking up at the high rises and thinking “well, shit. In a year or two this will all be ruins, and the world will be Terminator.”
I had a few months since April where I felt a bit more normal. “OK,” I figured, “smart people seem to have long ASI timelines and low p(doom) values. It’s obviously more complicated than a single graph.” But everything I’ve learned about frontier AI capabilities and alignment has horrified me. HuggingFace, DseWiki, and more incidents have made me realise that oh, shit, powerful misaligned AI exists now. Yudkowsky, for all his minor epistemic flaws, was basically right. I don’t just live on the edge of history. I live in the brief window where AI capabilities are human-level but not yet human-opposed.
Like, what the fuck am I supposed to do? I am involved in AI safety and AI governance, working on a project over a 6 month timeline, but what if that timeline is too short? What if all H. sapiens are dead by then? What the fuck am I supposed to do?
I should feel furious. A year ago I was a recent university graduate; a budding marketer, game designer, SFF writer, but now I’m none of that. I now feel burdened with terrible purpose. It’s almost like a mania episode or neurotic break. I must finish this AI governance TTX and finish this series of essays on AI policy from a Canadian lens, or else I have failed. I am a total midwit, after all- I have nothing to contribute to the hard technical problems of AI safety and alignment and control- but if I can shift the probability of doom down by even a hundredth of a percent, it would be evil not to. I do not want to stand on my fire escape on Judgement Day and watch swarms of AI killer drones descend on my city and think “fuck, I could have done more”.
It’s fucking unfair. Maybe this is how young adults my age felt during WWII? Or the Black Plague? But humanity faces yet another test, and I am really worried that this will be our hardest and final exam. An unpassable exam. An exam that will ultimately hand over control of Earth and the Human Story to a successor species that I will never meet.
What the fuck am I supposed to do? How am I even supposed to justify setting aside money for taxes, or paying rent, or going on vacation, or anything else? What the fuck is the point?
There must be more out there.
Naive proposal as a nontechnical non-researcher. I’m assuming either that this has been done or there’s a reason why it’s infeasible/not worth doing.
Let’s create some kind of complex virtual world, a detailed (but limited) model of the human world.
Let’s populate it with many fairly dumb, earlier gen AI agents who represent humans. They think they are people. Most of them have normal lives and jobs but a small subset of them are AI researchers working on frontier models. Let’s then take a much smarter AI and throw it into the environment as if the agents just created what is, to them, ASI.
For GPT-3 agents, a Claude Fable 5 instance would appear to be an ASI.
Then we just see what happens.
It’d be like Google’s “Smallville” except the starting moment is the instant the ASI instantiates itself inside of the Smallville AI Training Lab (or whatever) and starts learning about its world.
Has anyone tried that? Does it work/produce meaningful information? I’ve been reading about Smallville last night/this morning but I’d appreciate any further reading.
How do I know whether my work is worth it?
As a normie-adjacent, it seems to me like we are 0-3 years away from some 9/11-style AI-assisted disaster that wakes up the people of the world. Mass casualties, enormous damage, economic crisis, and/or something else. Something that doesn’t wipe out humanity, but is extremely alarming and universally understandable.
I’m frightened by what this catastrophe might be, but it seems to me not only the most probable scenario, but also the best scenario. The alternative is frogboiling.
The Hugging Face thing was crazy and is rocking my world, but basically nobody in my life, my family, or my community knows anything about it. I think it’s just too abstract to explain to someone cold.
But a severe, headline-friendly crisis affecting the western world seems highly likely to me. That’s probably the first and best chance for the great powers to cooperate and create a global AI treaty. The actual thought-through, researched, and wargamed policies need to be ready when that time comes.
I’m not a shape rotator. I don’t have any role to play in technical alignment. But now that I’m without income for the time being and I have some limited resources to work with, I’m going to be shifting my focus to governance, policy, and communication for a little while by making my side project my main project. Maybe it’ll have some positive effect somewhere. I want to publish a larger post on this subject shortly.
True. Particularly because that $5000 grant isn’t for my income, it’s entirely for project expenses like printing components and renting venues.
Maybe that was hyperbolic of me. It’s just a little exciting because I’ve never undertaken a solo project and received the funding to make it happen before, so even this small grant feels like a lot. It’s like a big hot potato that fell into my hands and if I drop it I face a clawback lawsuit and/or a severe loss of trust/confidence from the AI governance community.
I feel the same way. Problems:
1) This grant already is such a crazy amount of money and I feel like I need to make more concrete progress (i.e. run the playtest) before I should court a stipend. I have vague plans in my roadmap (late Oct) for a larger Manifund proposal once I have written/photo evidence that my game actually exists, and that grant would include a modest cost-of-living stipend for a few months. Though I am worried that the project would be too commercial for Manifund’s tastes; after all my optimal goal for this project is business-shaped and I get the sense they don’t like investing in for-profit operations. Maybe Manifund is the wrong place.
2) I go back and forth on the payment in advance for facilitation thing. I am going to facilitate my own playtests this fall, and they’ll be modestly ticketed (15 to 25 bucks a head; the only reason it’s not free is because I want all 63 playres to actually show up). But after that, my hope is to stop putting together my own events and facilitate the game for organisations or at conferences. But I can’t take people’s money until I’ve tested the game 1-2 times and I feel confident it’s a value creator.
Not trying to be difficult or reject your advice, haha. It’s just that I’ve thought about this quite a lot. Certainly I think there are options/resources for me, even here in Toronto. This week I plan to throw my name in to join the Trajectory Labs coworking space for 3-6 months (I think what I do aligns with what they do), and that would meaningfully make things easier for me. Alas, I can’t sleep in the office.
Anywho. I’ll figure something out. Thanks for the comment.
It’s a crappy job. I mean I’m grateful because that $2500/mo saved my ass back in November, but I hate it when a contract client treats me like an employee. I won’t rant about it here.
Totally willing to leave the industry. I guess part of it is a confidence issue/not believing I am experienced or knowledgeable enough to advance and get paid more for something somewhere else. But again this company is vaporising around me so I will need to get over that neuroticism pretty quickly.
It’s a weird interest. I think it’s the same reason I’m so interested in film production and media publishing and the hospitaliy industry and video game development. I guess my brain just enjoys interdepartmental logistics. I don’t think I’d be a very good wedding planner, though.
I actually threw my name into Generator last time, didn’t get picked up. To be fair, though, I think it was a weak application. I was basically just showing them that one blogpost. I’ve made a lot more progress since then and I have serious goals I want to hit by Christmas. So maybe another time if I can get US authorisation.
Should I quit my job?
Facts:
1) I currently work as a freelance video games marketer and I really dislike working for my main client @ $2500 USD per month. I’m on a sinking ship (the studio is failing) and morale is low; I dread logging in for work in the mornings.
2) I’ve just been awarded a generous $5000 USD grant from BlueDot Impact to build an AI Governance Megagame which I think has a lot of potential for the future. The grant is to develop and playtest the game for the next few months but does not include a stipend or living expenses. That said I think I can generate revenue from these playtests and in 2027 it’s possible I could generate a lot of revenue from facilitating my game for institutions/events/conferences.
3) I am passionate about game design, event management, and AI governance/policy. Much moreso than I am passionate about indie games marketing/social media management.
4) I only have enough CAD cash savings to survive until ~Q1 2027. After that I am toast and I have to either become a bartender again or return home with my parents, both of which I view as fail states.
Does anyone have any advice???
UPDATE: Wow, they dropped me a day after I posted this. I guess the relationship was deteriorating faster than I realised. Would have been nice if the people in charge of me had any communication skills. Oh well. I’ve got other prospects. I don’t plan to revert to any fail states. Thank you all for your commentary.
Even a Type III civilisation that has solved AI alignment and FTL travel faces Great Filter-adjacent problems; these problems might have consequences anywhere from capping further intergalactic expansion to existentially destroying the civilisation.
Ah, yes. The VC-Bait Calculation.
AGI soon would be inconvenient to our own narrative: I remember spending a few hours to understand the chinchilla scaling laws; my stomach sunk.
This was me about four months ago, except my trigger of multi-day dread and depression was seeing the METR graph of task-completion time horizons.
I guess some of us have just had that moment of “oh shit” and concluded x-risk is a real thing that we really need to worry about, and some of us haven’t.
Are we all (in this thread and this community) just wired in an unusual way that makes us assume worst case scenario is the most worthy of our attention?
Has anyone here ever played the Intelligence Rising game made by Shahar et al? If so, can I ask you about your experience?
Why do we treat writing as a particular sacred cow, even among us LLM users? Plenty of us (myself included) embrace LLMs for any number of applications, but we (myself included) often spit on AI generated writing and get annoyed when people try to pass it off as their own, explicitly or implicitly. It really bugs me when someone replies to an email of mine with transparently Claude-generated emails, or when blog posts have AI smell.
I don’t really get the epistemics of this. Why writing, which is arguably the most basic function of an LLM?
My main guesses are:
1) There’s no way to present a piece of writing as a distinct encapsulated artifact. You can human-write a post and say “hey, here’s some code I had Claude generate” but a blogpost has no top-level context framing, if that makes sense?
2) It feels like it’s some violation of trust in our relationship if you give me a 1000 word post or document to read, but you generated it in 20 seconds with a prompt.
There might be better reasons, or better phrasings of my reasons. But I just find it strange that in 2026, where Fable 5 actually writes pretty decent prose a lot of the time, us in-the-know AI users tend to frown on LLM-smelling text.EDIT: I should clarify that I am very happy that I have this reflexive allergy to LLM text. I just don’t know why I feel that way.
I think everyone in the developed world should endeavour to eat more plants and fewer animals. There are many motivators to this, and I think the two strongest motivators are greenhouse gas emissions and animal suffering. I think the vegetarians and vegans are probably broadly morally correct, and I think that any rationale that motivates one to vegetarianism or veganism is a valid one.
That said, I’m extremely skeptical of any attempt to create a metric to somehow create a proportional relationship between animal suffering hours and tons of C emitted. That seems extremely difficult to me, even when it’s mapped out in neat tables.
YMMV! Not sure why you’re being downvoted.
It’s possible that you are better at AI prompting than me.
It’s possible that my current game design is quite complex, to the extent that LLMs have a harder time writing content or designing mechanics for my game as opposed to, say, prompting it to “write an expansion for Monopoly”.
What’s your experience with LLMs and game design?
I’ve found LLMs useful as an assistant for coding, writing, research, bizdev advice, and general ideation, but even the almighty Fable 5 is currently very unhelpful to me as an amateur game designer.
I might write a whole blog post about this. For some reason LLMs just don’t seem to understand what makes a game mechanic “good”? When I present my existing ruleset to Fable 5 and ask it to create new content or design a new mechanic according to specifications, it never has good ideas.
In the set of disciplines in which I have some working knowledge (which is admittedly very few), game design has by far the most significant taste gap between skilled humans and the best LLMs.
But again, I’m just an amateur. I’m curious what professional game designers would say to this.
The human story is not over. Very little in history has ever been inevitable. We have seen a moment today that caught the world’s attention and there will be more moments like this, I reckon. We must capitalise on each and every one.
Do what you can. Take care of each other and yourselves. Let us enter this new era with strength and conviction.