sounds fascinating, thanks for the recommendation!
Prudhviraj Naidu
Karma: 55
I feel like you are just describing life right now for anyone who moves from a developing country to a developed country.
When I first came from India to US, another Indian told me “to learn to pass time”. Because the many challenges of daily life in India don’t exist in US. So one has to figure out joys of life in US. For many places, it is enough to just be able to survive. If you are stuck in a war-torn place, every new day is an achievement.
Isn’t this just describing a split personality disorder?
In a transcript, the LLM is already modelling next-token prediction for assistant and the user (even if it’s not getting gradient signal from the user tokens). When it does <think> or <tool> call, maybe it comes up with a new personality?
Mech interp’s pitch is: since we have the neural network’s weights, we just need to reverse-engineer them. But having the weights works against mech interp’s utility.
We want mechanistic explanations to predict how a network behaves on certain inputs. With the weights in hand, we can just run the inputs and see. Moreover, we can re-create the conditions perfectly for an input. For causal influence, we can perturb inputs and sample many times. And sampling is getting cheaper — both from algorithmic progress and from the growing supply of model-interaction data.
The natural counter is that mech interp can reach rare behaviours sampling can’t. But a rare behaviour only matters if it causes harm, and meaningful harm from an LLM requires many generations, not one. If the behaviour matters, it will be produced often enough that we can figure out how to sample it. For example: it seems unlikely a LLM will one-shot create viruses / commit mass cyber crimes. It seems that would require multiple generations. Thus seems likely that we would be able to figure out how to generate conditions to elicit that behaviour.
Seems unclear how mechanistic interp utility stacks against just sampling in a principled fashion.
Do you keep track of current tasks in a notes and then keep revisiting them?
I was working on an app that maintains a round-robin queue of tasks so that I can switch around them.
How do people manage multi-task with multiple AI instances like Claude Code / ChatGPT?
Top problems I face:
I forget the fine-grained context so I have to spend time reading the history rather than having summarized note
Forget about a task that I left an agent working on
I get mental fatigue faster than just working on a single task
Which ends up in:
Tasks remain partially complete so no task gets actually complete
Mentally exhausted and just want to curl up on a grassy meadow staring tat the sky
Would be happy for any suggestions
My view is that LLMs can’t do context management well that long tasks require. Training approaches rely on the model doing the whole task within it’s context window length. Long context tasks like reading large documents to answer questions are not the same as coding an entire software library from scratch.
RL training, that I know of, still involves trajectories that fit within a context window length. There is no optimization pressure for models to figure out how to do context management unlike humans who develop strategies like note-taking.
Seems related to “Training LLMs for Honesty via Confessions”: https://arxiv.org/abs/2512.08093 based on a cursory read of this post.
Was this the California International Marathon?
Wouldn’t it crash markets because people took on debt to fund chip production? Since, private players can’t reason when governments might interfere, they would not want to fund AI after this. Effectively making AI research a government project?
Why would any government that is not US / China agree to this? They would be worse off if AI is only a government project as their governments can’t hope to compete. If there are private players, then they can get a stake in the private companies and get some leverage.
Firstly, I want to say thank you for commenting on my shortpost.
The “information war” sounds like politics as usual. Propaganda, censorship, Twitter, TikTok—all have existed long before AGI.
It’s more about the magnitude and its effectiveness. For ex, India-Pakistan fire at each other on a fairly regular basis but that doesn’t count as a reason to go to war. But if one of them drops a nuke, then the security of the other is endagenered to the point they will invade.
An AI / AGI powered “propaganda” machine can spin up millions of bots, argue with every individual in a country. Create personalized realistic images / fake news articles and news organizations that overpower information agencies (like actual newspapers). I don’t think pre-GPT3 countries can spin up news agencies on the go that aren’t obvious to the people that they are state-sponsored.State-sponsored cybernetic attacks on other states’ infrastructure? We pretend for decades that we don’t see them, because… well, they are trivial to deny, you just need to play dumb.
My point is that can we actually see them and act against them, even if we publicly deny them. Imagine a social attack that you don’t detect but changes how people vote over the decade towards unproductive politicians.
If you believe AGI can come up with new attack vectors (maybe on the net, it might be new bio-weapons), then the defending country needs to spend expenditure on how to detect them. If the defending country can’t spin up AGIs of its own or lags behind (imagine North Korea), why wouldn’t they just say we can’t possibly hope to keep up or defend against an AGI as time progresses. We should nuke AGI installations while our nukes would still do damage.
Please let me know if something doesn’t make sense or is just not new information to you.
If military AGI is akin to nuclear bombs, then would it be justified to attack the country trying to militarize AGI? What would the first act of war in future wars be?
If a country A is building a nuke, then the argument for country B to pre-emptively attack it is that the first act of war involving nukes would effectively end country B. In this case, the act of war is still a physical explosion.
In case of AI, what would be the first act of war akin to physical explosion? Would country B be able to even detect if AI is being used against it? If there is an intelligent explosion, wouldn’t the nature of war possibly change rapidly that country B can not even detect they are being attacked until it’s too late. For example, you develop AGI and use that to hinder political process and cause slow down in economic growth rate by 2 percentage points. Over a long time horizon like a century, this dooms country B.
Then isn’t it country B’s interest to consider building military AGI as an act of war since it can not hope to detect future acts of war carried out by the AI.
Past History: Britain’s first act of war against Germany in WW1 was cutting off the telegraph lines carried out by a non-military vessel. The first act of war in future wars does not have to be physical explosion and likely be an information attack.
Former Indian Army Chief speaks on how India is losing the Information War:
The ability to do nothing assumes welfare policy for migrants
How? It might be better to be homeless in US than having a house in Afghanistan. Job visa restrictions don’t allow you to be homeless.
I simply mean allowing people to stay in a country when they are in-between jobs or looking for other jobs. Most countries only give you a fixed period like 30-days to find similar work, otherwise you are asked to leave.
If there were no job-specific restrictions, people can save up money for a time period or work in another job / employer other than the one stated on your visa.
The migrants might be better of than in their home country and the person who’s employing them has a cheap servant.
Not arguing against this. But the migrant can’t for example take out a loan and start a business due to visa restrictions or move to more productive parts of the economy.
See this as an example: https://www.reddit.com/r/h1b/comments/1l1lcwq/moving_to_india_after_15_years/
I think even if you open up Western countries, there are more productive areas where labour can be absorbed rather than household work.
Servants in Singapore are probably are a result of Singapore’s migration policy. Let’s say a developed country had a lax migration policy that allowed people to just come and setup a business, work or study or do nothing.
Then this would allow people to take risks such upskilling themselves and moving into more productive sectors of the economy in a similar fashion to the native population. Current migration policies tend to be restrictive as they tie down individuals to a specific job.
Switching jobs, upskilling yourself or starting a business becomes really hard without breaking immigration laws. Thus, condemning workers to the low productive jobs that they first got when they came into the country.
Speaking from experience in Mumbai, just pretending to throw a stone doesn’t necessarily work. You have to pretend to pick up a stone and then throw it.
Maybe this is covered somehow, but you can focus all your attention on values you converge on with your constituents.
Stay mum on the values your constituents are ambivalent on. Then, once elected use state power to change the values of your constituents over things that they are ambivalent on.