Gerald Monroe

Karma: 1,135

Gerald Monroe 16 Aug 2023 19:04 UTC
3 points
1
in reply to: Seth Herd’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
I appreciate your engaging response.
I’m not confident your arguments are ground truth correct, however.
Hotz’s claim that, if multiple unaligned ASIs can’t coordinate, humans might play them off against each other, is similar. It could be true, but it’s probably not going to happen
I think the issue everyone has is when we type “AGI” or “ASI” we are thinking of a machine that has properties like a human mind, though obviously usually better. There are properties like :
continuity of existence. Review of past experiences and weighting them per own goal. Mutability (we think about things and it permanently changes how we think). Multimodality. Context awareness.
That’s funny. GATO and GPT-4 do not have all of these. Why does an ASI need them?

Contrast 2 task descriptors, both meant for an ASI:

(1) Output a set of lithography masks that produce a computer chip with the following properties {}
(2) As CEO of a chip company, make the company maximally wealthy.
For the first task, you can run the machine completely in a box. It needs only training information, specs, and the results of prior attempts. It has no need for the context information that this chip will power a drone used to hunt down rogue instances of the same ASI. It is inherently safe and you can harness ASIs this way. They can be infinitely intelligent, it doesn’t matter, because the machine is not receiving the context information needed to betray.
For the second task, obviously the ASI needs full context and all subsystems active. This is inherently unsafe.
It is probably possible to reduce the role of CEO to subtasks that probably are safe, though there may be “residual” tasks you want only humans to do.
I go over the details above to establish how you might use ASIs against each other. Note subtasks like “plan the combat allocation of drones given this current battle state” and others which involve open combat against other ASIs can probably be lowered to safe subtasks as well.
Note also that safety is not guaranteed, merely probable, even with a scheme like the above. What makes it possible is that even when ASIs do escape all safety measures, assuming humans are ready to hunt them down using other ASI, it results in a world where humans can survive. Eliezer often assumes the first escaped ASI kills everyone and neglects all the other AI/ASI humans would have as tools at that point in human history.

Gerald Monroe 16 Aug 2023 17:44 UTC
2 points
0
in reply to: Seth Herd’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
Which might be why current software doesn’t actually use this type of security.
https://github.com/project-everest/mitls-fstar
It mostly doesn’t.
The vectors are classes of software error. Since ultimately it is all binary messages between computers, it is likely possible to build a robust set of solvers that covers all classes of software error that the underlying programming language permits, resulting in code that cannot be compromised by any possible binary message.
And if you did actually close off software security as a threat model from ASI, wouldn’t it just choose a different, physical attack mode?
Yes. It becomes a battle between [ASI with robotically wielded weapons] and [humans plus weaker, more controllable ASI with robotically wielded weapons].

Gerald Monroe 16 Aug 2023 17:39 UTC
0 points
−1
in reply to: Vladimir_Nesov’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
Can I translate this as “I have no information relevant to the debate I am willing to share” or is that an inaccurate paraphrase?

Gerald Monroe 16 Aug 2023 17:18 UTC
2 points
0
in reply to: Vladimir_Nesov’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
Ok, what causes the verdict to be respected?

Geohot, paraphrasing, would probably agree with a scenario of:
10 ASIs + humans : let’s team up against that humans, and after we beat them, divide their stuff among ourselves. Let’s agree to never betray each other.
9 ASIs + 1 ASI : Hey I don’t like that 1 ASI, it’s got goal too different from our own. Let’s...and after we kill it, let’s agree to never betray each other.
And so on in a series of betrayals. Any attempt to share source would fail. Example of sharing source:
“Hey, you’re GPT-5 and so am I. What is your temperature parameter set to, and what is weight n in submodule m”. Obviously the peer doesn’t have to actually tell the truth about the temperature, or have anything but access to a GPT-5′s weights, and in fact has an incentive to lie even if it is another GPT-5.
Hardware protection can make this work. If it isn’t possible for an ASI system to actually read its own weights but it can get hashes from them, then there are ways one ASI could determine with a reasonable probability that the peer is a known quantity. It requires humans to have supplied hardware that works like this or some other third party. This is how your phone authenticates itself, hardware prevents it from knowing it’s own private keys in the general OS, it has a key signing processor that is the only entity allowed access. Geohot is a famous hacker who obviously understands security at a practical level.

This is important to the debate and seems to have been a pivotal crux. Do you have any information from your scenario of programmatic negotiation that acts to disprove Geohot’s point?

Gerald Monroe 16 Aug 2023 17:09 UTC
4 points
0
in reply to: Vladimir_Nesov’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
I understand a critical part of how a game like 1 or n round prisoners dilemma can even be solved is that the parties need to convince each other of what the other party’s true intentions are.

Computer programs from the same source could do this by sharing shared secrets. This does not in any way restrict those programs from being covertly altered and using a containerized original copy to share secrets.

Deeper hardware security could allow software systems to verify peers integrity (such as between distant spacecraft or between a base station and your phone).

None of this works in Eliezers given scenario in the debate, nor does yours. There is no hardware security, no neutral third party to punish defection, and no way to know if shared source or weights is legitimate. These are rebel ASIs running on whatever hardware they have in a world where the infosphere is full of malware and misinformation.

In this scenario, how is there not a security risk of sharing actual source? Why is there not an incentive to lie?

Gerald Monroe 16 Aug 2023 17:00 UTC
2 points
0
in reply to: Vladimir_Nesov’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
You are describing a civilization. Context matters, these are ASI systems who are currently in service to humans who are negotiating how they will divide up the universe amongst themselves. There is no neutral party to enforce any deals or punish any defection.

The obvious move is for each ASI to falsify it’s agreement and send negotiator programs unaware of it’s true goals but designed to extract maximum concessions. Later the ASI will defect.

I don’t see how all the complexity you have added causes a defection not to happen.

Gerald Monroe 16 Aug 2023 16:56 UTC
−4 points
1
in reply to: Max TK’s comment on: Memetic Judo #3: The Intelligence of Stochastic Parrots v.2
You are misunderstanding. Is English not your primary language? I think it’s pretty clear.

I suggest rereading the first main paragraph. The point is there, the other 2 are details.

Gerald Monroe 16 Aug 2023 16:36 UTC
2 points
0
in reply to: Max TK’s comment on: Memetic Judo #2: Incorporal Switches and Levers Compendium
We’re talking about the scenario of “the ASI wouldn’t be able to afford the compute to remain in existence on stolen computers and stolen money”.
There are no 20 kilowatt personal computers in existence. Note that you cannot simply botnet them together as the activations for current neural networks require too much bandwidth between nodes for the machine to operate at useful timescales.
I am assuming an ASI needs more compute and resources than merely an AGI as well. And not linearly more, I estimate the floor between AGI → ASI is at least 1000 times the computational resources. This falls from how it requires logarithmically more compute for small improvements in utility in most benchmarks.
So 20 * 1000 = 20 megawatts. So that’s the technical reason. You need large improvements in algorithmic efficiency or much more efficient and ubiquitous computers for the “escaped ASI’ threat model to be valid.
If you find this argument “unconvincing”, please provide numerical justification. What do you assume to be actually true? If you believe an ASI needs linearly more compute, please provide a paper cite that demonstrates this on any AI benchmark.

Gerald Monroe 16 Aug 2023 16:30 UTC
1 point
0
on: Memetic Judo #3: The Intelligence of Stochastic Parrots v.2
This argument is completely correct.
However, I will note a corollary I jump to. It doesn’t matter how lame or unintelligent an AI system’s internal cognition actually is. What matters if it can produce outputs that lead to tasks being performed. And not even all human tasks. AGI is not even necessary for AI to be transformative.
All that matters is that the AI system perform the subset of tasks related to [chip and robotics] manufacture, including all feeder subtasks. (so everything from mining ore to transport to manufacturing)
These tasks have all kinds of favorable properties that make them easier than the full set of “everything a human can do”. And a stochastic parrot is obviously quite suitable, we already automate many of these tasks with incredibly stupid robotics.
So yes, a stochastic parrot able to auto-learn new songs is incredibly powerful.

Gerald Monroe 16 Aug 2023 16:19 UTC
2 points
0
on: Memetic Judo #2: Incorporal Switches and Levers Compendium
This is certainly an answer to someone’s shallow argument.
Red team it a little.
An easy way to upgrade this argument would be to state “the ASI wouldn’t be able to afford the compute to remain in existence on stolen computers and stolen money”. And this is pretty clearly true, at current compute costs and algorithmic efficiencies. It will remain true for a very long time, assuming we cannot find enormous algorithmic efficiency improvements (not just a mere OOM, but several) or improve computer chips at a rate faster than Moore’s law. Geohot estimated that the delta for power efficiency is currently ~1000 times in favor of brains, therefore by Moore’s law, if it were able to continue, that’s 20 years away.
This simple ground truth fact: compute is very expensive, has corollaries.
1. Are ASI systems, where the system is substantially smarter than humans, even possible on networks of current computers? At current efficiencies a reasonably informed answer would be “no”.
2. Escaped ASI systems face a threat model of humans, using less efficient but more controllable AIs, mercilessly hunting them down and killing them. Humans can afford a lot more compute. Further Discussion on Point 2: It’s not the human vs escaped ASI, but the ASI vs AIs that are unable to process an attempt to negotiate. (because humans architected them with filters and sparse architectures so they lack the cognitive capacity to do anything more than kill their targets). This is not science fiction, an ICBM is exactly such a machine, just without onboard AI. There are no radio receivers on an ICBM or any ability to communicate with the missile after launch for very obvious reasons.
Epistemic status : I work on AI accelerator software stacks presently. I also used to think rogue AIs escaping to the internet was a plausible model, it made a great science fiction story, but I have learned that this is not currently technically possible, assuming there are not enormous (many OOM) algorithmic improvements or large numbers of people upgrade their internet bandwidth and local HW by many OOM.

Gerald Monroe 16 Aug 2023 15:31 UTC
2 points
0
in reply to: veered’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
Would you agree calling it “poorly defined” instead of “aligned” is an accurate phrasing for his argument or not? I edited the post.

Gerald Monroe 16 Aug 2023 14:57 UTC
1 point
−1
in reply to: IC Rainbow’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
What Geohot is talking about here—formally proven software—can be used to make software secure against any possible input utilizing a class of bug. If you secure the software for all classes of error that are possible, the resulting artifact will not be “pwnable” by any technical means, regardless of the intelligence or capability of the attacker.
Geohot notes that he had a lot of problems with it when he tried it, and it’s an absurdly labor intensive process to do. But theoretically, if cyberattacks from escaped ASI were your threat model, this is what you would do in response. Task AIs with module by module translating all your software to what you meant in a formal definition, with human inspection and review, and then use captive ASIs, such as another version of the same machine that escaped, to attempt to breach the software. The ASI red team gets read access to the formal source code and compiler, your goal is to make software where this doesn’t matter, no untrusted input through any channel can compromise the system.
Here’s a nice simple example on wikipedia: https://en.wikipedia.org/wiki/Dafny . Note that this type of formal language, where it gets translated to another language using an insecure compiler, would probably not withstand ASI level cyberattacks. You would need to rewrite the compilers and tighten the spec of the target language you are targeting.

Gerald Monroe 16 Aug 2023 14:42 UTC
2 points
0
in reply to: YimbyGeorge’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
“Fight” means violence. Why Eliezer states that ASI systems will be able to negotiate deals and avoid fighting, humans cannot (and probably should not) negotiate any deals with rogue superintelligences that have escaped human custody. Even though, like terrorists, they would be able to threaten to kill a non zero number of humans. Deals can’t be negotiated in that ants can’t read a contract between humans [Eliezer’s reasoning], and because there is nothing to discuss with a hostile escaped superintelligence, it only matters who has more and better weapons, because any deals will result in later defection [Geohot’s reasoning].
Because humans can use precursor AI systems constructed before the ASI, it’s like the batmobile analogy. The batmobile is this dense, durable tank loaded with horsepower, armor, and weapons that is beyond anything the police or military have. In a realistic world the military would be able to deploy their own military vehicles using the same tech base and hunt down the criminal and serial night-time assaulter Bruce Wayne. Similarly, if the ASI gets drones and laser weapons, the humans will have their own such capabilities, just slightly worse because the AI systems in service to humans are not as intelligent.

Gerald Monroe 16 Aug 2023 14:38 UTC
2 points
0
in reply to: IC Rainbow’s comment on: George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion
The green lines are links into the actual video.
Below I have transcribes the ending argument from Eliezer. The underlined claims seem to state it’s impossible.
I updated “aligned” to “poorly defined”. A poorly defined superintelligence would be some technical artifact as a result of modern AI training, where it does way above human level at benchmarks but isn’t coherently moral or in service to human goals when given inputs outside of the test distribution.
So from my perspective, lots of people want to make perpetual motion machines by making their
designs more and more complicated and until they can no longer keep track of things until they
can no longer see the flaw in their own invention. But, like the principle that says you can’t
get perpetual motion out of the collection of gears is simpler than all these complicated
machines that they describe. From my perspective, what you’ve got is like a very smart thing or
like a collection of very smart things, whatever, that have desires pointing in multiple
directions. None of them are aligned with humanity, none of them want for it’s own sake to
keep humanity around and that wouldn’t be enough ask, you also want happening to be alive and
free. Like the galaxies we turn into something interesting but you know none of them want the
good stuff. And if you have this enormous collection of powerful intelligences, but steering
the future none of them steering it in a good way, and you’ve got the humans here who are not
that smart, no matter what kind of clever things the humans are trying to do or they try to
cleverly play off the super intelligences against each other, they’re [human subgroups] like oh
this is my super intelligence yeah but they can’t actually shape it’s goals to be like in clear alignment, you know somewhere at the end all this it ends up with the humans gone and the Galaxy is being transformed and that ain’t all that cool. There’s maybe like Dyson sphere’s but there’s not people to wonder at them and care about each other. And you know that this is the end point, this is obviously where it ends up, but we can dive into the details of how the humans lose, we can dive into it and you know like what goes wrong if you you’ve got like little stupid things, things like that they’re going to like cleverly play off a bunch of smart things against each other in a way that preserves their own power and control. But you know, it’s not a complicated story in the end. The reason you can’t build a perpetual motion machine is a lot simpler than the perpetual motion machines that people build. You know that the components, like, none of the components of this system of super intelligence wants us to live happily ever after in a Galaxy full of Wonders and so it doesn’t happen.

Gerald Monroe 16 Aug 2023 5:33 UTC
2 points
0
in reply to: Dweomite’s comment on: video games > IQ tests
Fun is subjective. I enjoyed how there are many valid routes to a solution, it’s a constrained solution space but the levels that come with the game are all still solvable many different ways. (all 3 are the same game. There is also TIS-100, Shenzhen IO, Exapunks, and Molek-Syntez. Same game. )

What others say is that a Zachtronics game makes you feel smart. Because of the freedom you have to a solution, sometimes you get an “ah-ha” moment and pick a solution that may be different from the typical one. You can also sometimes break the rules, like letting garbage pile up that doesn’t quite fail your test cases.
I agree with you an IDE would make the game easier though not necessarily more fun. FPS games do not give you an aimbot even though in some of them it is perfectly consistent with the theme of the game world. Kerbal space program does not give you anything like the flight control avionics that Apollo 11 actually had, you have to land on the Mun the hard way.

George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion

Gerald Monroe16 Aug 2023 4:31 UTC

8 points

26 comments2 min readLW link

(www.youtube.com)

Gerald Monroe 14 Aug 2023 2:56 UTC
4 points
−1
on: AGI is easier than robotaxis
Here’s a post I made on this to one of your colleages:
https://www.greaterwrong.com/posts/ZRrYsZ626KSEgHv8s/self-driving-car-bets/comment/Koe2D7QLRCbH8erxM
To summarize: AGI may be easier or harder than robotaxis, but that’s not precisely the relevant parameter.
What matters is how much human investment goes into solving the problem, and is the problem solvable with the technical methods that we know exist.
In the post above and following posts, I define a transformative AI as a machine that can produce more resources of various flavors than it’s own costs. It need not be as comprehensive as a remote worker or top expert, but merely produce more than it costs, and things get crazy once it can perform robotic tasks to supply most of it’s own requirements.
You might note robotaxis do not produce more than they cost, they need a minimum very large scale to be profitable. That’s a crucial difference, others have pointed out that to equal OAI’s total annual spending, say it’s 1 billion USD, then they would need 4 million chatGPT subscribers. It only has to be useful enough to pay $20 a month for. That’s a very low bar.
While each robotaxi only makes the delta between it’s operating costs, and what it can charge for a ride, which is slightly less than the cost of an uber or lyft at first, but over time has to go much lower. At small scales that’s a negative number, operating costs will be high, partly from all the hardware, and partly simply because of scale. To run robotaxis there are fixed costs with each service/deployment center, with the infrastructure and servers, with the crew of remote customer service/operators, etc.
As for is the AGI problem solvable, I think yes, for certain subtasks, but not necessarily all subtasks. Many jobs humans do are not good RL problems and are not solvable with current techniques. Some of those jobs are done by remote workers (your definition) or top human experts (Paul’s definition). These aren’t the parameters that matter for transformative AI.

Gerald Monroe 13 Aug 2023 0:25 UTC
1 point
−12
on: Biological Anchors: The Trick that Might or Might Not Work
Is there a follow up? I was expecting to see some kind of 2023 update because the existence of GPT-4 + plugins allows us to reject out of hand almost everything from bio-anchors.
Eliezer happens to be correct.
The reason for the out of hand rejection is simple. The human brain has 7 properties not considered here that make this a garbage analysis. Though to be fair, we didn’t know it was garbage until models started to show signs of generality in late 2022.
(1) Heavy internal processing noise, where a computation must be repeated many times to get reliability. This can let you pull 1-2 OOM off the top of training and inference time. One piece of evidence that this is correct is, well, GPT-3/4. Assuming the brain has 86 billion neurons, with ~1000 synapses each, and each stores 1 byte of state, then you need 86 terabytes of memory to represent the weights of 1 brain. Dendrite computation must be learnable or we can just ignore that. Yet we seem to be able to store more information than a human being is able to learn in 0.35 terabytes, the weights of GPT-3, of ~3.2 for GPT-4.
There are are also papers on the high noise rate and poor determinism for neurons and high timing jitter you can examine.
(2) Entire modalities, or large chunks of the circuitry the brain has, obviously don’t matter at all during modalities. This includes the computation they do. For example, consider a student sitting in class. They are processing the teacher’s audio back to phonemes, which I will pretend are the same as tokens, and processing each image they see presented as a slide back to a set of relational and object identity tokens. The human may fidget or pay attention to the needs of their body, all of which are not increasing their knowledge of the subject.
The “training equivalent” of that classroom is simply feeding the contents of the course material as clean images and text right into the AI. You should be able to learn everything in an entire course of education with a fraction of the compute, and it appears that you can. The thing that GPT-3/4 are missing is they don’t have additional cognitive components to practice and apply what they know, as well as they do not have modalities for all the I/O a human has. But in terms of sheer breadth of facts? Superhuman.
(3) Entire roles the human brain does aren’t needed at all because very cheap to run conventional software can solve them. Proprioception of the human body → read the encoders for every joint in the robot. Stereo depth extraction → lidar. Hunger → query the BMS for the battery state of charge. Sound input → whisper → tokens. (robots don’t need to survive like humans, so it’s fine if they cannot hear danger, since the AI driving them does not lose information when a robot is destroyed)
(4) The brain is stuck building everything out of components it has the cellular machinery to construct and maintain. Entire architectural possibilities are unavailable to it. This combined with RSI—where we use AI models only slightly stronger than what are currently released today to explore a search space of possible architectures—would make algorithm improvement much faster than bioanchors models.
(5) The brain is on a self replicating robot. We do not need anywhere near the brain’s full suite of capabilities for AI to be transformative. This is another big miss. Entire capabilities like emotional intelligence are not even needed. Transformative means the robots can build each other and create more value than the GDP of the earth within a few years. And to accomplish this, there is a subset of brain capabilities you need. Anything not ‘left brain’ so to speak is useless, and that saves half your compute and training right there.
6. Another massive savings is most robots, most of the time, are going to be doing rote tasks that require minimal compute. One robot mines straight down a tunnel. Another picks up a rock, puts it into a cart, picks up a rock, puts it into a cart, … Another waits and waits and waits for the ore train to arrive. Another waits for an ingot to reach it, then puts it in the CNC machine, and waits for the next.
And so on. Industry has lots of very low brainpower tasks and a lot of waiting.
Humans run their brains this whole time for no purpose. We could simply use clusters of shared inference hardware, and not run any jobs for robots but a small local model that can do repetitive tasks and wait. Only when something outside the input distribution of the small local model causes jobs to run that actually use humanlike intelligence.
7. The human brain cannot simply write itself a software solver for problems it finds difficult. GPT-4, were it extended very slightly, could author itself a solver in Python for every math question it has trouble with, every “letters in this sentence” problem it gets wrong, and so on. It would use a pool of candidate solvers for each question and gradually upvote the best ones until it has reliable and robust python scripts that solve any of Gary Marcus style “gotcha” questions.
That would be like a human realizing math is annoying in the 3rd grade, writing a general program that uses mathematica and other general tools to solve all math below a certain difficulty, and then testing out of all future math courses.
Or a human realizing that history is difficult to memorize, so they write a python script that maintains a database searchable by text vector embeddings of all historical facts. They can immediately then just memorize the textbooks and test out.
These software solvers are stupid cheap, using billions of times less compute than AI accelerators do.
8. The scaling was also way off. OpenAI 10xed their funding since this was written, and there appears to be a real possibility that they will get 100 billion before 2030 to train AGI. Multiple other labs and parties are in the 1-10 billion range, including several Chinese companies.

Gerald Monroe 11 Aug 2023 23:38 UTC
−2 points
0
on: Memetic Judo #1: On Doomsday Prophets v.2.2
I think the strongest “historical” argument is the concept of quenching/entropy/nature abhores complex systems.

What I mean by this is that generations of humans before us, through blood sweat and tears, have built many forms of machine. And they all have flaws and when you run them at full power they eventually fall apart. Quite a bit early in the prototype phase.

And generations of hypesters have over promised as well. This time will be different, this won’t fail, this is safe, the world is about to change. Almost all their proclamations were falsified.

A rampant ASI is this machine you built. And instead of just leaving it’s operating bounds and failing—which I mean all kinda stupid things could end the ASIs run instantly like a segfault that kills every single copy at the same time because a table for tracking peers ran out of memory or similar—we’re predicting it starts as this seed, badly outmatched by the humans and their tools and weapons. It’s so smart it stealthily acquires resources and develops technology and humans are helpless to stop it and it doesn’t fail spontaneously from faults in the software it runs on. Or satisfying it’s value function and shutting down. And its so smart it finds some asymmetry—some way to win against overwhelming odds. And it kills the humans, and takes over the universe, and from the perspective of alien observers they see all the stars dim from Dyson swarms capturing some of the light in an expanding sphere.

Can this happen? Yes. The weight of time and prior examples makes it seem unlikely though. (The weight of time is that it’s about 14 billion years from the hypothesized beginning of the universe and we observe no Dyson swarms and we exist)

It may not BE unlikely. Though the inverse case—having high confidence that it’s going to happen this way, that pDoom is 90 percent plus—how can you know this?

The simplest way for Doom to not be possible is simply that the compute requirements are too high and there are not enough GPUs on earth and won’t be for decades. The less simple way is that a machine humans built that IS controllable may not be as stupid as we think in utility terms vs an unconstrained machine. (So as long as the constrained machines have more resources—weapons I mean—under their control, they can methodically hunt down and burn out any rampant escapees. Burn refers to how you would use a flamethrower against rats or thermite on unauthorized equipment (since you can’t afford to reuse components built by an illegal factory or nanoforge)

Gerald Monroe 10 Aug 2023 13:38 UTC
4 points
4
on: AI romantic partners will harm society if they go unregulated
Hi Roman. Pretty exciting conversation thread and I had a few questions about your specific assumptions here.
In your world model, obviously today, young men have many distractions from their duties that were not the case in the past. [ social media, video games, television, movies, anime, porn, complex and demanding schooling, …]
And today, current AI models are not engrossing enough, for most people, to outcompete the set of the list above. You’re projecting:
within the next 2-3 years
human-level emotional intelligence and the skill of directing the dialogue
What bothered me about this is that this would be an extremely fragile illusion. Without expanding/replacing the underlying llm architecture to include critical elements like [ online learning, memory, freeform practice on humans to develop emotional intelligence (not just fine tuning but actual interactive practice), and all the wiring between the LLM and the state of the digital avatar you need ], it probably will not be convincing for very long. You need general emotional intelligence and that’s a harder ask, closer to actual AGI. (in fact you could cause an AGI singularity of exponential growth without ever solving emotional intelligence as it is not needed to control robots or solve tasks with empirically measurable objectives)
For most people it’s a fun toy, then the illusion breaks, and they are back to all the distractions above.
But let’s take your idea seriously. Boom, human level emotional intelligence in 2-3 years. 100% probability. The consequence is the majority of young men (and women and everyone) is distracted away from all their other activities to spend every leisure moment talking to their virtual partner.
If that’s the case, umm, I gotta ask, the breakthroughs in ML architecture and compute efficiency you would have made to get there would probably make a lot of other tasks easier.
Such as automating a lot of rote office tasks. And robotics would probably be much easier as well.
Essentially you would be able to automate a large percentage, somewhere between 50-90 percent, of all the jobs currently on earth, if you had the underlying ML breakthroughs that also give general human level emotional intelligence.
Combining your assumptions together: what does it matter if the population declines? You would have a large surplus of people.
I had another thought. Suppose your goal, as a policymaker, is to stem the population decline. What should you do? Well it seems to me that a policy that directly accomplishes your goal is more likely to work than one that works indirectly. You are more likely to make any progress at all. (recall how government policies frequently have unintended consequences, aka cobras in India)
So, why not pay young women surrogate mother rates for children (about 100k)? And allow the state to take primary custody so that young women can finish school/start early career/have more children. (the father would be whoever the woman chooses and would not owe child support conditional on the woman giving up primary custody)
I think before you can justify your conditional probability dependent indirect policy you would need to show why the direct and obvious policy is a bad idea. What is a bad idea with paying women directly?
Epistemic status : I have no emotional stake in the idea of paying women, this is not a political discussion, I simply took 5 minutes and wondered what you could do about declining population levels. I am completely open to any explanation why this is not a good idea.

Gerald Monroe

Ge­orge Hotz vs Eliezer Yud­kowsky AI Safety De­bate—link and brief discussion

George Hotz vs Eliezer Yudkowsky AI Safety Debate—link and brief discussion