burmesetheater

Karma: 64

burmesetheater 14 Jun 2022 23:13 UTC
14 points
on: Yes, AI research will be substantially curtailed if a lab causes a major disaster
Well, there’s a significant probability COVID isn’t a “natural” pandemic, although the story behind that is too complicated without an unambiguous single point of failure which hinders uptake among would-be activists.
If there’s an AI failure will things be any different? There may be numerous framings of what went wrong or what might be addressed to fix it, details sufficient to give real predictive power will probably be complicate and it’s a good bet that however interested “the powers that be” are in GOF, they’re probably much MUCH more interested in AI development. So there can be even more resources to spin the story in favor of forestalling any pressure that might build to regulate.
Nuclear regulation also might not be a good example of a disaster forcing meaningful regulation because the real pressure was against military use of nuclear power and that seems to have enjoyed general immunity against real regulation. So it’s more like if an AI incident results in the general public being banned from buying GPUs or something while myriad AI labs still churn toward AGI.

burmesetheater 28 Jun 2022 22:47 UTC
13 points
3
on: It’s Probably Not Lithium
More plausible model for why in present day so many are overweight:
--cheap calories that taste good widely available with very low effort to obtain
--tasty food is, other things equal, an easy exploit to reward / motivation loops, so it tends to get used in exactly this way which results in excess calorie consumption and of course this is habit forming. there is probably also a lower threshold to “get into” food vs something else in this class like drugs since eating is already universal and not taboo or otherwise particularly regulated.
--fewer obligatory opportunities for caloric expenditure to balance intake possibly mostly as a result of modern transport and trend toward less need for physical labor in general
Maybe this is off-base, and it may not apply to the lithium hypothesis, but it seems there are a lot of really implausible ideas for why obesity is common which are motivated by a desire to not blame the obese person. Common perceptions of agency might infer that the above model blames the consumer, but the intention is exactly the opposite; since it’s predictable that humans in the above environment will tend to act this way, who can blame them?

burmesetheater 15 Jun 2022 7:07 UTC
7 points
on: Alignment Risk Doesn’t Require Superintelligence
Destructive alignment issues in our species are more mundane. Several leaders in the 20th century killed outright very large numbers of people for completely banal reasons like political ambition. Actually, your intuition that 9/11 events happen “all the time” is only off in a temporal sense; the number of humans unambiguously killed by the coordinated actions of relatively few other unaligned humans in the last 100 years is so great that it is probably enough to have at least one 9/11 a day during that time. Humans are generally unaligned on several levels from personal to egregoric and the only reason this is lately becoming a problem in a species-risk sense is because only now are we getting some powerful technology. A more probable version of the scenario in this post is a suicidal leader triggering a large scale nuclear war through use of their own arsenal either through deception or after taking steps to reduce the possibility of refusal. Of course it would be a great irony if now that global thermonuclear war is actually tested, the opposing forces are unable to make use of their deterrent.

burmesetheater 12 Jun 2022 18:15 UTC
6 points
on: Let’s not name specific AI labs in an adversarial context
This seems like a case of making a rule to fix a problem that doesn’t exist.
Are people harassing individual AI labs or researchers? The tendency for reasonable people who are worried about AI safety should be to not do so, since it predictably won’t help the cause and can hurt. So far there does not seem to be any such problem of harassment discernible from background noise.
Naming individual labs and / or researchers is interesting, useful, and keeps things “real.”

burmesetheater 10 Jun 2022 21:38 UTC
6 points
on: How dangerous is human-level AI?
“human-level AI” is a confusing term for at least a couple reasons: first, there is a gigantic performance range even if you consider only the top 1% of humanity and second it’s not clear that human-level general learning systems won’t be intrinsically superhuman because of things like scalable substrate and extraordinarily high bandwidth access (compared to eyes, ears, and mouths) to lossless information. That these apparent issues are not more frequently enumerated in the context of early AGI is confusing.
As far as I’m aware all serious attempts to take over the world have been by brute force. Historically there are messaging, travel, logistics etc latencies that make this very difficult within one’s lifetime even if potentially world-owning force is available or capable of being mustered. So the window for a single entity (human-level) to take over the world within its lifetime has probably only opened recently, and the number of externalities and internal abilities needed to line up to have a predictably large shot at success are probably many. Accordingly, even situations like Hitler sitting in control of a very powerful Reich which nominally might appear to enable a chance of world ownership are still too fraught with an unoptimized distribution of enabling factors to have any realistic chance of world ownership. There is also a grey area of whether an individual or some collective is responsible for the attempt. One might argue that trends ongoing for at least a few decades suggest that the USA is in a great position to take over the world if China (or someone else) doesn’t “break out” first. But with the way the USA is structured it may be difficult for any “human-level” individual entity to take credit for, or enjoy a firm grasp of the fruits of this conquest.

burmesetheater 2 Apr 2022 19:48 UTC
5 points
in reply to: RHollerith’s comment on: Russian x-risk newsletter March 2022 update
If the US knew of a way to locate subs, then it would worry that Russia or China would figure it out, too
There are many conceivable ways to track subs and this is only part of the problem because subs still need to be destroyed after being located. Russia and China combined don’t have enough nuclear attack subs to credibly do this to the US. The US does have enough nuclear attack subs to credibly destroy Russia’s deterrent fleet, if they can be tracked, with attack subs left over to defend our own ballistic missile subs. A primary mission for nuclear attack subs is to shadow nuclear ballistic missile subs. That Russia is (allegedly) developing weapons like Status-6 and Burvestnik suggests they are not satisfied with the ongoing deterrent capability they already have.
Also, about 2 thirds of the US’s 1357 “strategic” (capable of incinerating the heart of a major city) nuclear warheads are currently on subs, rather than in missile silos or on bombers
The number of weapons deployed, and where they are deployed simply isn’t verifiable. Keep in mind that it is widely held, and codified in public law, that use of nuclear weapons by the US including for retaliatory purposes must follow the kind of centralized authorization that could be extremely difficult to guarantee under a surprise nuclear attack. This would open us up to surprise decapitation attack so the probability it’s true in practice is very low.
What links here?
- DanArmak's comment on Russian x-risk newsletter March 2022 update by avturchin (3 Apr 2022 16:51 UTC; 4 points)

Passable Puppet

burmesetheater29 May 2022 11:07 UTC

4 points

1 comment3 min readLW link

burmesetheater 28 Jun 2022 18:06 UTC
4 points
0
in reply to: benjamincosman’s comment on: Air Conditioner Repair
First bullet, those are good points. It is an interesting question how one would good data on this sort of thing and how accurate that data would be.
Second, this isn’t the intention, it’s to show that the story sounds bizarre. It’s not a political comment.

burmesetheater 9 Jun 2022 23:48 UTC
4 points
on: Why it’s bad to kill Grandma
There seems to be a deep problem underlying these claims: even if humans have loosely aligned intuition about what’s right and wrong, which isn’t at all clear, why would we trust something just because it feels obvious? We make mistakes on this basis all the time and there are plenty of contradictory notions of what is obviously correct—religion, anyone?
Further, if grandma is in such a poor state that simply nudging her would kill her AND the perpetrator is such a divergent individual that they would then use the recovered funds to improve others’ lives (which might have many positive years still available, if such a concept is possible or meaningful) then one might argue that it seems a poor conclusion to NOT kill grandma if one’s concern is for the welfare of others. One could also simply steal grandma’s money, since this is probably easier to get away with than murder, but then you would be leaving an ethical optimization on the table by not ending grandma’s life, which as hinted at earlier is probably in the negative side of qualitative equity.

burmesetheater 3 Apr 2022 1:27 UTC
4 points
on: Book review: Very Important People
This isn’t really much different from life outside the club. Social forces are often not aligned with majority personal preference and can even be in conflict. For example, people want to make friends or hook up but seeking those goals explicitly tends to be perceived as low-class and / or strange.

burmesetheater 2 Apr 2022 20:23 UTC
4 points
on: Interacting with a Boxed AI
I’m not sure considering how to restrict interaction with super-AI is an effective way to address its potential risks, even if some restrictions might work (and it is not at all clear that such restrictions are possible). Humans tend not to leave capability on the table where there’s competitive advantage to be had so it’s predictable that even in a world that starts with AIs in secure boxes there will be a race toward less security to extract more value.

burmesetheater 15 Jun 2022 5:39 UTC
3 points
in reply to: trevor’s comment on: Yes, AI research will be substantially curtailed if a lab causes a major disaster
and have the best forecasters
With forecasters from both sides given equal amounts of information, these institutions might not even reliably beat the Metaculus community. If one is such a great forecaster then they can forecast that jobs like this might not be, among other things, that fulfilling.
I don’t know if we’ve gotten to the point where they can fool the professionals at not getting fooled
Quite a few professionals (not at not getting fooled) still believe in a roughly 0% probability of a certain bio-related accident a couple three years ago thanks in large part to a spun story. Maybe the forecasters at the above places know better but none of the entities who might act on that information are necessarily incentivized to push for regulation as a result. So it’s not clear it would matter if most forecasters know AI is probably responsible for some murky disaster while the public believes humans are responsible.

burmesetheater 4 Apr 2022 6:23 UTC
3 points
on: Avoiding Moral Fads?
How are we to know that we aren’t making similar errors today?
Based only on priors, the probability we aren’t is very low indeed. A better question is, given an identified issue, how can change happen? One main problem is that contra-orthodox information on moral issues tends not to travel easily.

burmesetheater 20 Mar 2022 0:21 UTC
3 points
on: Wargaming AGI Development
Because the benefits of quantum computing were so massive
Please elaborate. I’m aware of Grover’s algorithm, Shore’s algorithm, and quantum communication, and it’s not clear that any of these pose a significant threat to even current means of military information security / penetration.

burmesetheater 27 Jun 2022 20:58 UTC
2 points
in reply to: mruwnik’s comment on: AI Training Should Allow Opt-Out
once the cat is out of the bag it’s out
Since this was not clear, that’s correct. The intention is not to encourage non-contribution to the open internet, including open source projects.
It is a problem in 2022 when someone seriously proposes opt-out as a solution to anything. Our world does not “do” opt-out. Our concept of “opting out” of the big-data world is some inconsequential cookie selection with a “yes” and a buried “no” to make the user feel good. We are far past the point of starting conversations. It’s not productive, or useful, when it’s predictably the case that one’s publicly accessible data will end up used for AI training by major players anyway, many of whom will have no obligation to follow even token opt-out and data protection measures.
Conversations can be good, but founding one on a predictably dead-end direction does not seem to make much sense.
This isn’t a suggestion to do nothing, it’s a suggestion to look elsewhere. At the margin, “opting out” does not affect anything except the gullible user’s imagination.

burmesetheater 23 Jun 2022 17:45 UTC
2 points
on: AI Training Should Allow Opt-Out
Disagree. Data public (and private) will be used by all kinds of actors under various jurisdictions to train AI models and only a fraction of these predictably will pay any heed to an opt-out (and only a fraction of those who do may actually implement it correctly). So an opt-out is not only a relatively worthless token gesture, the premise of any useful upside appears to be based on the assumption that one can control what happens to information one has publicly shared on the internet. It’s well evidenced that this doesn’t work very well.
Here’s another approach: if you’re worried about what will happen to your data, then maybe do something more effective, like not put it out in public.

burmesetheater 31 May 2022 19:47 UTC
2 points
in reply to: dkirmani’s comment on: What is the state of Chinese AI research?
Probability is high that all nations with strong AI research are keeping secrets, since some AI research will naturally go into projects with high secrecy. A better question is what the proportion of published to secret research is in USA, China, etc. It might actually be similar, which could suggest that China is pretty far behind.

burmesetheater 14 Jun 2022 3:43 UTC
1 point
in reply to: acylhalide’s comment on: Let’s not name specific AI labs in an adversarial context
Anyone can try, this seems way out in a practically invisible part of the tail of obstacles to not being destroyed by AGI, if it’s even an obstacle at all.

burmesetheater 13 Jun 2022 16:43 UTC
1 point
on: Why so little AI risk on rationalist-adjacent blogs?
Most probably just haven’t identified it as salient / don’t understand it / don’t take it seriously, and besides there tend to be severely negative social / audience ramifications associated with doomsday forecasting.

burmesetheater 12 Jun 2022 20:49 UTC
1 point
in reply to: acylhalide’s comment on: Let’s not name specific AI labs in an adversarial context
So, to be clear, you don’t think confidently naming people by first name as destroying the world can be parsed emotionally by them?
Mentions of AI companies / AI personalities on LW will intrinsically tend to be adversarial, even if the author spares a polemic or use of terms like “so and so is working to destroy the world” because misaligned AI destroying the world is clearly THE focus of this community. So it can be argued that to be meaningful, a policy of no names would need to be applied to practically any discussion of AI as even if some AI content is framed positively by the author, the community at large will predictably tend to see it in existential risk terms.
That’s one issue. Personally, the calculus seems pretty simple: this well-behaved community and its concerns are largely not taken seriously by “the powers” who will predictably create AGI, there is little sign that these concerns will be taken seriously before reaching AGI, and there is almost no reason to think that humanity will pause to take a break and think “maybe we should put this on hold since we’ve made no discernible progress toward any alignment solutions” before someone trains and runs an AGI. So a conclusion that could be drawn from this is, we might as well have nice uncensored talks about AI free from petty rules until then.

burmesetheater

Pass­able Puppet

Passable Puppet