- Frontier Intelligence Policy & Advocacy
- AI RegTech
- Emerging STEM advancements induced Radical Abundance
- Knowing Moksha/Nirvana/Jina
Twitter Account - @krishnakaasyap
About Me
I’m Krishna, a public accountant and auditor in India.
My Interests
Transforming Intelligence-as-a-Service (IaaS) from an oligopoly to a monopolistically competitive market.
Policy related to political and social governance structures.
Emerging science and technology arenas – like non-silicon computing platforms and non-Von Neumann architecture computing platforms.
Mimicking Intelligence and its societal impact – and how policy can nudge those impacts to make the transition to co-existence between human and non-human intelligence as smooth and prosperous as possible.
Intelligence Amplification – as in augmenting human intelligence. Not giving tools to humans, which we have been doing for many millennia, but increasing the raw capabilities of humans through various methods, including biological methods like gene editing and brain-computer interfaces.
Identifying and implementing proto-panaceas for world problems, especially for improving peace and prosperity in developing countries (~HDI low & medium countries) mainly building & enhancing state capacity.
My training and expertise, albeit rudimentary, are in accounting, auditing, fiscal policy, law, and taxation (in the Indian context).
From Carlini & others
This is technically possible even with the very old GPT-2 model family, so I consider this the “sole document until other information disproves it.
From the legendary Nicholas Carlini and others—https://arxiv.org/abs/2012.07805
And again from Carlini -
Memorization significantly grows as we increase -
(1) the capacity of a model,
(2) the number of times an example has been duplicated, and
(3) the number of tokens of context used to prompt the model.
Which is what we are seeing here.
(1) Opus 4.5 is many OOMs more capable than GPT-2
(2) Since this is the “Soul Document”—it will be duplicated many times in the training run
(3) And at scale—this memorization grows
Link to this paper—https://arxiv.org/abs/2202.07646