Lucas Pfeifer

Karma: −41

Lucas Pfeifer 31 Mar 2023 3:14 UTC
1 point
0
in reply to: quanticle’s comment on: Limit intelligent weapons
Social, economic, or environmental changes happen relatively slowly, on the scale of months or years, compared to potent weapons, which can destroy whole cities in a single day. Therefore, conventional weapons would be a much more immediate danger if corrupted by an AI. The other problems are important to solve, yes, but first humanity must survive its more deadly creations. The field of cybersecurity will continue to evolve in the coming decades. Hopefully world militaries can keep up, so so that no rogue intelligence gains control of these weapons.

Lucas Pfeifer 28 Mar 2023 19:23 UTC
1 point
0
in reply to: quanticle’s comment on: Limit intelligent weapons
It is a broad definition, yes, for the purpose of discussing the potential for the tools in question to be used against humans.
My point is this: we should focus first on limiting the most potent vectors of attack: those which involve conventional ‘weapons’. Less potent vectors, (those that are not commonly considered as weapons) such as a ‘stock trading algorithm’, are of lower priority, since they offer more opportunities for detection and mitigation.
An algorithm that amasses wealth should eventually set off red flags (maybe banks need to improve their audits and identification requirements). Additionally, wealth is only useful when spent on a specific purpose. Those purposes could be countered by a government, if the government possesses sufficient ‘weapons’ to eliminate the offending machines.
If this algorithm takes such subtle actions that cannot be detected in time to prevent catastrophe, then we are doomed. However, there is also the likelihood that the algorithm will have weaknesses which allow it to be detected.

Lucas Pfeifer 27 Mar 2023 16:55 UTC
1 point
0
in reply to: quanticle’s comment on: Limit intelligent weapons
Entities compete in various ways, yes. Competition is an attack on another entities’ chances of survival. Let’s define a weapon as any tool which could be used to mount an attack. Of course, every tool could be used as a weapon, in some sense. It’s a question of how much risk our tools pose to us, if they were to be used against us.

Lucas Pfeifer 26 Mar 2023 5:20 UTC
1 point
0
in reply to: quanticle’s comment on: Limit intelligent weapons
These memes have been magnified by the words of politicians and media. We need our leaders to discuss things more reasonably.
That said, restricting social media could also make sense. A requirement for in-person verification and limitation to a single account per site could be helpful.

Lucas Pfeifer 25 Mar 2023 2:22 UTC
1 point
0
in reply to: ChristianKl’s comment on: Limit intelligent weapons
More stringent (in-person) verification of bank account ownership could mitigate this risk.
Anyways, the chance of discovery for any covert operation is proportional to the size of the operation and the time that it takes to execute. The more we pre-limit the tools available to a rogue machine to cause harm immediate harm, the more likely we will catch it in the act.

Lucas Pfeifer 24 Mar 2023 20:28 UTC
1 point
0
in reply to: ChristianKl’s comment on: Limit intelligent weapons
Which kinds of power do you refer to? Most kinds of power require human cooperation. The danger that an AI tricks us into destroying ourselves is small (though a false detection of nuclear weapons could do it). We need much more cooperation between world leaders, a much more positive dialogue between them.

Lucas Pfeifer 24 Mar 2023 20:19 UTC
1 point
−1
in reply to: greatBigDot’s comment on: continue working on hard alignment! don’t give up!
Yes, we need to solve the harder alignment problems as well. I suggested limited intelligent weapons as the first step, because these are the most obviously misanthropic AI being developed, and the clearest vector of attack for any rogue AI. Why don’t we focus on that first, before we focus on the more subtle vectors.
The end of the post you linked said, basically, “we need a plan”. Do you have a better one?

Lucas Pfeifer 24 Mar 2023 19:50 UTC
1 point
on: What is Abstraction?
Abstraction means assigning a symbol to reference a set of other symbols. It saves time and memory: time by allowing retrieval of data based on a set of rules, memory by shrinking the size of the reference.
For example: the words ‘natural’ and ‘artificial’: we sort things into one of these labels based on whether or not they were made by a human. A ‘natural’ thing could be ‘physical’ or ‘biological’. An ‘artificial’ thing could be ‘theory’ or ‘implementation’. If I don’t need to distinguish between physical and biological things, instead of referring to them directly, I can use the more abstract reference of ‘natural’ things, saving space and time in my statement.
The challenge with natural language abstraction is agreeing on definitions. Many would define the terms in the above example differently. The more we can agree on definitions of terms, the better we can reason about their subsets.
In a logically valid system of abstraction, any symbol can be related to every other symbol: either by a common parent reference, or by using one to refer to the other.

Lucas Pfeifer 24 Mar 2023 19:06 UTC
1 point
on: How To Write Quickly While Maintaining Epistemic Rigor
We should sort reasoning into the inductive and deductive types: inductive provides a working model, deductive provides a more consistent (less contradictory) model. Deductive conclusions are guaranteed to be true, as long as their premises are true. Inductive conclusions are held with a degree of confidence, and depend on how well the variables in the study were isolated. For the empire example in the original post, there are many variables other than computing power that affect the rise and fall of empires. Computing power is only one of many technologies, and besides technology, there is finance, military, culture, food, health, education, natural disaster, religion, etc. Adding to the uncertainty is the small sample size, relative to the number of variables.
However, we can more easily isolate the effect of computing power on census taking, as mentioned, just as we can draw a more confident conclusion between the printing press and literacy rates. Everything has its scale. Relate big to big, medium to medium, small to small. Build up a structure of microscopic relations to find macroscopic patterns.

Lucas Pfeifer 24 Mar 2023 18:16 UTC
1 point
0
in reply to: Lone Pine’s comment on: Limit intelligent weapons
Yes, the linked post makes a lot of sense: wet labs should be heavily regulated.
Most of the disagreement here is based on two premises:
A: Other vectors (wet labs, etc.) present a greater threat. Maybe, though intelligent weapons are the most clearly misanthropic variant of AI.
B: AI will become so powerful, so quickly, that limiting its vectors of attack will not be enough.
If B is true, the only solution is a general ban on AI research. However, this would need to be a coordinated effort across the globe. There is far more support for halting intelligent weapons development than for a general ban. A general ban could come as a subsequent agreement.

Lucas Pfeifer 24 Mar 2023 17:59 UTC
1 point
0
in reply to: JBlack’s comment on: Limit intelligent weapons
Superintelligence is inherently dangerous, yes. The rapid increase in capabilities is inherently destabilizing, yes. However, practically speaking, we humans can handle and learn from failure, provided it is not catastrophic. An unexpected superintelligence would be catastrophic. However, it will be hard to convince people to abandon currently benign AI models on the principle that they could spontaneously create a superintelligence. A more feasible approach would start with the most dangerous and misanthropic manifestations of AI: those that are specialized to kill humans.

Lucas Pfeifer 24 Mar 2023 17:32 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Limit intelligent weapons
Best to slow down the development of AI in sensitive fields until we have a clearer understanding of its capabilities.

Lucas Pfeifer 24 Mar 2023 17:28 UTC
1 point
0
in reply to: Raemon’s comment on: Limit intelligent weapons
“Advocacy pushes you down a path of simplifying ideas rather than clearly articulating what’s true, and pushing for consensus for the sake of coordination regardless of whether you’ve actually found the right thing to coordinate on.”
1. Simplifying (abstracting) ideas allows us to use them efficiently.
2. Coordination allows us to combine our talents to achieve a common goal.
3. The right thing is the one which best helps us achieve our cause.
4. Our cause, in terms of alignment, is making intelligent machines that help us.
5. The first step towards helping us is not killing us.
6. Intelligent weapons are machines with built-in intelligence capabilities specialized for the task of killing humans.
7. Yes, a rogue AI could try to kill us in other ways: bioweapons, power grid sabotage, communications sabotage, etc. Limiting the development of new microorganisms, especially with regards to AI, would also be a very good step. However, bioweapons research requires human action, and there are very few humans that are both capable and willing to cause human extinction. Sabotage of civilian infrastructure could cause a lot of damage, especially the power grid, which may be vulnerable to cyberattack. https://www.gao.gov/blog/securing-u.s.-electricity-grid-cyberattacks
8. Human mercenaries causing a societal collapse? That would mean a large number of individuals who are willing to take orders from a machine to actively harm their communities. Very unlikely.
9. The more human action that an AI requires to function, the more likely a human will notice and eliminate a rogue AI. Unfortunately, the development of weapons which require less human action is proceeding rapidly.
10. Suppose an LLM or other reasoning model were to enter a bad loop, maybe as the result of a joke, in which it sought to destroy humanity. Suppose it wrote a program which, when installed by the unsuspecting user, created a much smaller model, and this model used other machines to communicate with autonomous weapons, instructing them to destroy key targets. The damage which arises in this scenario would be proportional to the power and intelligence of the autonomous weapons. Hence, the need to stop developing them immediately.

Lucas Pfeifer 24 Mar 2023 0:54 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Limit intelligent weapons
Yes, in the long term we will need a complete alignment strategy, such as permanent integration with our brains. However, before that happens, it would be prudent to limit the potential for a misaligned AI to cause permanent damage.
And, yes, we are in need of a more concrete plan and commitment from the people involved in the tech, especially with regards to lethal AI.

Lucas Pfeifer 24 Mar 2023 0:48 UTC
0 points
0
in reply to: niplav’s comment on: continue working on hard alignment! don’t give up!
1. Which part of my statement does not make sense, and how so?
2. My statement is relevent to the post. The beginning of the article partially defined hard alignment as preventing AI from destroying everything of value to us. The most likely way a rogue AI would do that is by gaining unauthorized access to weapons with built-in intelligence.

Lucas Pfeifer 24 Mar 2023 0:31 UTC
−26 points
−15
on: continue working on hard alignment! don’t give up!
The best way to limit the impact of a rogue AI is to limit the production of autonomous (intelligent) lethal weapons. More details in this post and its comments:
https://www.lesswrong.com/posts/b2d3yBzzik4hajGni/limit-intelligent-weapons

Lucas Pfeifer 24 Mar 2023 0:25 UTC
0 points
0
on: We have to Upgrade
Given the unpredictable emergent behavior in researchers’ AI models, we will likely see emergent AI behavior with real-world consequences. We can limit these consequences by limiting the potential vectors of malignant behavior, the primary being autonomous lethal weapons. See my post and underlying comments for further details:
https://www.lesswrong.com/posts/b2d3yBzzik4hajGni/limit-intelligent-weapons

Lucas Pfeifer 24 Mar 2023 0:14 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Limit intelligent weapons
1. Focus means spending time or energy on a task. Our time and energy is limited, and the danger of rogue AI is growing by the year. We should focus our energies on by forming an achievable goal, making a reasonable plan, and acting according to the plan.
2. Of course, there is a spectrum to the possible outcomes caused by a hypothetical rogue AI (rAI), ranging from insignificant to catastrophic. Any access the rAI might gain to human-made intelligent weapons would amplify the rAI’s power to cause real-world damage.

Lucas Pfeifer 24 Mar 2023 0:02 UTC
1 point
0
in reply to: Raemon’s comment on: Limit intelligent weapons
1. How is the framing of this post “off”? It provides an invitation for agreement on a thesis. The thesis is very broad, yes, and it would certainly be good to clarify these ideas.
2. What is the purpose of sharing information, if that information does not lead in the direction of a consensus? Would you have us share information simply to disagree on our interpretation of it?
3. The relationship between autonomous weapons and existential risk is this: autonomous weapons have built-in targeting and engagement capabilities. If we could make an analogy to a human warrior, in a rogue AI scenario, any autonomous weapons to which the AI gained access would serve as the ‘sword-arm’ of the rogue AI, while a reasoning model would provide the ‘brains’ to direct and coordinate it. The first step towards regaining control would be to disarm the rogue AI, as one might disarm a human, or remove the stinger on a stingray. The more limited the weaponry that the AI has access to, the easier it would be to disarm.

Lucas Pfeifer 23 Mar 2023 23:20 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Limit intelligent weapons
Existential danger is very much related to weapons. Of course, AI could pose an existential threat without access to weapons. However, weapons provide the most dangerous vector of attack for a rogue, confused, or otherwise misanthropic AI. We should focus more on this immediate and concrete risk before the more abstract theories of alignment.