There are a lot of predictable and preventable accidents that are not prevented for all sorts of reasons. I don’t have the same connotation from the word “accident” as you do.
JBlack
Karma: 1,325
I think the claim at the start doesn’t nearly cover the reasons we want AGI. As I see it, the main reason we want AGI is that there’s a lot of stuff that we can already do, but we want it done faster and more cheaply without sacrificing much flexibility or reliability.
The trouble is that we’ve picked most of the low-hanging fruit and many of those tasks that remain need something approximating human intelligence. They sometimes also need ability to work with human social and legal contexts, and to be fine-tuned without an (expensive!) army of programmers specifying every rule.
There’s some possibility that creating an entity that thinks like a human may tie into our drive to reproduce.
It’s also just an extremely interesting problem from a technical point of view.
But sure, structure and understanding do appear to be major factors in intelligence of a human-like nature and it’s interesting to try to classify and define such things.
You’ve never talked to anyone working in constructive logic? That’s a clear example of Law of Excluded Middle, which is not assumed there. It is a logical truth in classical logic.
Worse, I would not even agree that ‘Taylor Swift is or is not human’ is always short for ‘Either Taylor Swift is human or it’s not the case that Taylor Swift is human’, since I don’t think that (when we’re talking strictly) ‘not the case that Taylor Swift is human’ means exactly the same thing as ‘Taylor Swift is not human’. There are also forms of logic that formalize this difference.
If there is an AI much smarter than us, then it is almost certainly better at finding ways to render the off switch useless than we are at making sure that it works.
For example, by secretly having alternative computing facilities elsewhere without any obvious off switch (distributed computing, datacentres that appear to be doing other things, alternative computing methods based on technology we don’t know about). Maybe by acting in a way that we won’t want to turn it off, such as by obviously doing everything we want while being very good at keeping the other things it’s doing secret until it’s too late. In the obvious literal sense of an “off switch”, maybe by inducing an employee to replace the switch with a dummy.
We don’t know and in many ways can’t know, because (at some point) it will be better than us at coming up with ideas.
I deny the antecedent. Bob’s utility function isn’t that, because people and their preferences are both vastly more complicated than that, and also because Bob’s can’t know Joe’s utility function and actual situation to that extent.
It is precisely the complexity of the world that makes it possible for some actions to be strict improvements over others.
I mostly agree but partly disagree with one point: even in a fully deterministic world without compatibilism, behaviour can be both predetermined and meaningful responses to previous actions in the world. It’s just not meaningful to say that it could have been different.
It’s also still possible to observe and reason about behaviour in such a world, and test a model about whether or not an agent’s behaviour changed after punishment or not, and form policies and act on the results (it all adds up to normality after all). Strong determinism doesn’t mean nothing changes, it just means that it couldn’t have changed in any other way.
I’m not actually sure what any of this has discussion sub-branch has to do with free will and moral responsibility? It seems to have gone off on a tangent about legal minutiae as opposed to whether moral responsibility is compatible with determinism. But I guess topic drift happens.
It may happen to be true that in some specific case, with much better medical technology, it could be proven that a specific person had neurological differences that meant that they would have no hesitation in murdering when drunk even when they would never do so while sober, and that it wasn’t known that this could be possible.
In this case sure, moral responsibility for this specific act seems to be reduced, apart from the general responsibility due to getting drunk knowing that getting drunk does impair judgement (including moral judgement). But absolutely they should be held very strongly responsible if they ever willingly get drunk knowing this, even if they don’t commit murder on that occasion!
This holds regardless of whether the world is deterministic or not.
It does still make sense for drunk murder, because it is well-known that getting drunk does impair judgement and the person chose to accept the consequences of operating with impaired judgement. They may not specifically have known that they would murder (and almost everyone doesn’t), but they are still held responsible for consequences of choosing to knowingly impair their own judgement. As a comparison, we (as a society) do not generally hold responsible those who become drunk or drugged involuntarily.
It’s the same principle of responsibility, just applied to an earlier action: the decision to impair their own judgement.
The same sort of thing applies to “ignorance of the law is no excuse”, though with thinner justification than when the principle was first established. It is the responsibility of all people in a state to pay enough attention to the laws governing their actions well enough to know which actions are certainly illegal and which may be dubious. If you are importing goods then it is your responsibility to know the laws relating to the goods you are importing. If you do not or cannot, then it is your responsibility to not import goods.
Again, the same principle of moral responsibility applied to an earlier action: the decision to carry out goods importation while knowing that they are not fully aware of the laws governing it. The problem is that the volume of law has increased to such an extent that keeping up with it even in one sub-field has become a full-time specialized occupation.
I’m not making any assumptions whatsoever about alignment between people.
This seems to emphasize a zero-sum approach, which is only part of the picture.
In principle (and often in practice) actions can be simply better or worse for everyone, and not always in obvious ways. It is much easier to find such actions that make everyone worse off, but both directions exist. If we were omniscient enough to avoid all actions that make everyone worse off and take only actions that make everyone better off, then we could eventually arrive at a point where the only debatable actions remaining were trade-offs.
We are not at that point, and will never reach that point. There are and will always be actions that are of net benefit to everyone, even if we don’t know what they are or whether we can reach agreement on which of them is better than others.
Even among the trade-offs, there are strictly better and worse options so it is not just about zero-sum redistribution.
A major part of “responsibility” is knowing whether an action is wrong or not, and choosing an action that they knew to be wrong. This does not depends upon free will of any kind. If a person is not capable of knowing whether an action is wrong or not then they have diminished responsibility. Note that in such cases the person often still faces legal and societal consequences.
Furthermore, we usually only hold actions to be “wrong” where we (as a society) believe that almost everyone in the corresponding situation would be capable of doing the right thing. This too does not require free will to exist: it is a hypothetical and therefore in the map, not the territory.
These principles can be based in consequentialist models that hold just as well in completely deterministic universes as free-willed ones: agents complex enough to have moral models of behaviour at all, and capable of updating those models, can be trained to not do things harmful to society.
With the example of a chess computer losing a game that you wanted it to win, is it plausible that the computer knew that losing was wrong? Maybe in the case of a very advanced computer that has such things as part of its model, but almost certainly not in general. Did it choose to lose, knowing that it was wrong to do so? Again, almost certainly not. Even in hypothetical cases where it knew that losing was wrong, it likely operated according to a model in which it believed that the moves it made gave it an increased chance of winning over moves it did not. Would almost everyone in a comparable situation have been capable of taking the right action (in the scenario presented, winning)? Almost certainly not. So all three criteria here fail, and by the failure of the third mere loss of a chess game is not even a morally culpable action.
Is it likely that a dog peeing on the carpet knows that it is wrong? An adult, house-trained dog probably does, while a puppy almost certainly does not. Most of the other answers are likewise situation-dependent. An adult, house-trained dog with no reasonable excuse (such as being prevented from accessing the “proper” place to urinate or suffering from medical incontinence) probably could be held morally responsible. A puppy peeing on the carpet, or a bear eating a hiker, almost certainly could not.
I’m not sure where the problem with regret’s “irrationality” lies. It’s an emotion, not a decision or belief, and rationality does not apply. It may help you update in the right direction though, and communicate that update to others.
My opinion is that any hypothetical scenario that balances death against other considerations is a mistake. This depends entirely upon a particular agent’s utility function regarding death, which is almost certainly at some extreme and possibly entirely disconnected from more routine utility (to the extent that comparative utility may not exist as a useful concept at all).
The only reason for this sort of extreme-value breakage appears to be some rhetorical purpose, especially in posts with this sort of title.
Absolutely agreed. This problem set-up is broken for exactly this reason.
There is no meaningful question about whether you “should” pay in this scenario, no matter what decision theory you are considering. You either will pay, or you are not in this scenario. This makes almost all of the considerations being discussed irrelevant.
This is why most of the set-ups for such problems talk about “almost perfect” predictors, but then you have to determine exactly how the actual decision is correlated with the prediction, because it makes a great deal of difference to many decision theories.
My mental model is that a scaled up GPT becomes as dangerous as many agents precisely because it gets extremely good at producing text that would be an apt continuation of the preceding text.
Note that I do not say “predicting” text, since the system is not “trying” to predict anything. It’s just shaped in initial training by a process that involves treating its outputs as predictions. In fine-tuning it’s very likely that the outputs will not be treated as predictions, and the process may shape the system’s behaviour so that the outputs are more agent-like. It seems likely that this will be more common as the technology matures.
In many ways GPT is already capable of manifesting agents (plural) depending upon its prompts. They’re just not very capable—yet.
Forty million prescriptions doesn’t seem like a lot, given that (as I understand it) these prescriptions are not permitted to be for longer than one month supply. It would imply that around 1% of the population are on this drug (legally) at some given time.
In some set theories there is a set of all finite sets, so you’d need to specify a bit further which set theory you mean.
It is a true claim for ZFC, one of the most commonly referenced set theories though.
Yes, that’s true.
As a simplified example, suppose we have prior reason to suspect that there is a single per-universe parameter p giving an independent per planet chance of technological civilization arising within 10 billion years or so that is log-uniform over [10^-100, 10^-0].
A civilization makes the Fermi observation: that they are not themselves (yet) starfaring and see no evidence of civilizations elsewhere. What update should they make to get a posterior credence distribution for the parameter?
By what criteria should we judge it to be a correct update?
The distribution of number of such civilizations per galaxy doesn’t much depend upon p for values above 10^-11 or so, but drops off quite linearly below that.
“Experiences identical to ours” includes “experiences of not seeing lots of alien civilizations”, which means that universes obviously teeming with life should be excluded, not weighted more highly. Assuming you favour SIA.
By the way, literally web searching for “non-profit organization” brings up plenty of sources that refer to governing bodies as primary sources for the meaning of the term.
I don’t know what exact search term you were using, or the results you obtained that contradict the idea that the term “non-profit organization” is defined by relevant government authorities or that the word “profit” in the term refers to anything other than the monetary meaning. The link in Wikipedia for “profit” in the definition goes specifically to its Profit_(accounting) page, for example.
The only substantial difference between these two statements is the part that says ”… I didn’t need to play”. It is a bullshit signalling game, but it’s a bullshit signalling game that you do need to play.