The Facebook newsfeed also got a lot of opposition in the beginning. Whether or not a very innovative product is good or bad can not be decided on whether people like to oppose or make fun of it in the beginning.
BlenderBot 3 is likely running on substantially fewer resources than GPT-2/3 and thus should not be expected to have the same performance.
Well, so much for that plan!
An incidence rate of 1 in 2 million might not be enough for Danish (5.831 million) and Western Australian (2.667 million) but might be enough for the United Kingdom (67.733 million).
The United Kingdom does want its data to be analyzed. I know an EA who works as a contractor to do ML to help them with data analysis. I however don’t know the exact rules under which data analysis happens. Message me if you want the contact.
You think literally no part of their brains is tracking that policy X is about the coverup of sexual abuse?
The problem is not that no part of their brain tracks it. It’s just that it’s not the central reason when describing why they do what they do and not the story they tell to themselves.
Step 6: “Great, I’ve now figured out that this DNA sequence corresponds to a deadly pathogen. Just synthesize it and release it into the air. Anyone who could have got cancer or already has cancer will die quickly, curing cancer.”
I don’t think that the problematic actions by AGIs are likely of the nature that they can be described in that fashion. They are more likely to be 4D chess moves where the effects are hard to understand directly.
It might be something like: “In our experiments where doctors are supposed to use the AGI to help them make treatment decisions those doctors regularly overrate their own competency and don’t follow the AGI recommendation and as a result patients die unnecessarily. Here’s an online course that your doctors could take that would make them understand why it’s good to follow AGI recommendations”
Actions like that seem totally reasonable but they increase AGI power in contrast to human power. Economic pressure incentives that power transfer.
I wouldn’t expect that we go directly from AGI with human supervision to AGI that kills all humans via a deadly pathogen. We are more likely going from AGI with human supervision to AGI that effectively operates without human supervision. Then in a further step, AGIs that operate without human supervision centralize societal powers on themselves and after a few years, there are no resources for humans left.
There was no mechanism that seemed like it would have reliably stopped these provisions if they had been an order of magnitude or two worse, and indeed the original BBB bill seemed to have a number of things in that category.
The general mechanism for stopping provisions that are an order of magnitude or two worse is lobbying and in this case, that’s likely exactly what happened. The original BBB bill had a bunch of those things and then lobbyists came and fought the bill.
Lobbyist power is not absolute and there will be policies that damage business interests that lobbyists can’t prevent. On the other hand, at present Washington does to me not look like a place where lobbyists have too little power.
People’s willingness to spend on healthcare changes with the amount they are currently suffering. Immediate suffering is a much stronger motivator for behavior than plausible future suffering and even likely future suffering.
I’m surprised that the phrase moral maze doesn’t appear at all in the post. Rules that punish big corporations but don’t punish smaller ones tend to push the world in a direction with fewer maze levels.
Step 4, might rather be: “There are 10,000 unresolved biological questions that I think need to be answered to make progress, shall I give you a list?”
If you look at the Catholic church covering up sexual abuse of children, no church official would have answered the question “Why is policy X going to do?” with “Policy X exists so that more sexual abuse of children happens” and that’s not because they are lying from their own perspective.
Motivations in enantiodromia dynamics just don’t look like that.
Which companies, and to what extent? My internal model says that this is as simple as telling them they have to contract with somebody to dispose it properly.
Electric utilities. Coal plants produced a lot of mercury pollution and adding filters cost money. Given that burning fossil fuels cause the most mercury pollution it’s a really strange argument to treat that as something separate from mercury pollution.
Also mercury pollution is much more localized with clear, more immediate consequences than dealCO2 pollution. It doesnt suffer from any ‘common good’ problems.
Do you think that lowered childhood IQ isn’t a common good issue? That seems like a pretty a strange argument.
I don’t understand your model of this at all, do you think if CO2 wasn’t a controversial topic, we could just raise gas taxes and people would be fine?
I don’t think “just raise gas taxes” is an effective method to dealing with the issue. As an aside, just because the German public cares very much about CO2 doesn’t mean that our government stops subventioning long commutes to work. It doesn’t stop our government either from shutting down our nuclear power plants.
The Kyoto Protocol was negotiated fine in an environment of little public pressure.
I do agree with the sentiment that it’s important to discuss solutions to reducing carbon emissions sector by sector. If there would have been less public pressure, I do think it’s plausible that expert conference discussions would have been more willing to focus on individual sectors and discuss what’s needed in those.
The kind of elite consensus that brought the Kyoto Protocol could also have had a chance to create cap-and-trade.
I’m very curious about what the actual rules of the various gangs look like. If they exist in written form in an environment where it’s easy to confiscate documents I would expect them to be publically accessible.
It seems to me like, while the AI is still running on compute that humans oversee and can turn off, the AI has to discard a bunch of less effortful plans that would fail because they would reveal that it is misaligned (plans like “ask the humans for more information / resources”) and instead go with more effortful plans that don’t reveal this fact.
If I ask an AGI to create a cancer cure and it tells me that it would need more resources to do so and a bunch of information, it wouldn’t feel to me like a clear sign that the AGI is misaligned.
I would expect that companies that want their AGIs to solve real-world problems would regularly be in a situation where the AGI can clearly explain that it currently doesn’t have the resources to solve the problem and that more resources would clearly help.
Those companies that actually are very willing to give their AGIs the resources that the AGI thinks are needed to solve the problems are going to be rewarded with economic success.
I’m not sure how many bytes per second we see, but I don’t think it’s many orders of magnitudes higher than 2kb.
That depends a lot on how you count. A quick Googling suggest that the optic nerve has 1.7 million nerve fibers.
If you think about a neuron firing rate of 20 hz that gives you 34 MB per second.
From my perspective, there’s an important additional level: Economic pressure.
Given that it takes time for messages to travel through the world, there will likely be multiple AGIs in scenarios where there is no pivotal act that limits the world to a single AGI.
Those AGIs that highly prioritize getting more power and resources are likely going to get more power than AGIs that don’t. Competition between AGIs can be more fierce than competition between humans as a human can’t simply duplicate themselves by eating twice as much and raising new children is a time-consuming process but an AGI can use resources that are used by other AGIs to spin up more copies of itself.
In a world where most of the power is held by AGIs valuing human values is an impediment to prioritizing resource acquisition. As AGIs evolve and likely go through core ontological shifts there’s selection pressure toward deemphasizing human flourishing.
World reserves are estimated to be 8M tons.
That does not count uranium in seawater. While we currently can’t mine it at the same cost from seawater as we can mine it elsewhere, we can mine from seawater.
https://cen.acs.org/materials/Fishing-uranium-ocean-spider-silk/97/web/2019/07 suggests that currently, the price for uranium from seawater is six times as expensive as other sources. The price of uranium is not very important for the price of nuclear energy and paying six times as much for it wouldn’t be a problem.
Do you consider Evidence-based Medicine generally to be an argument by intellectual ordeal? (BANG energy drinks would be a medical intervention in this case)
Telling businesses that they have to make expensive investments to cut down mercury pollution is also negatively affecting a lot of businesses.
Solar and wind power plants can be fully controlled and owned by individuals and small communities, with zero intervention from the gov.
But solar and wind power alone doesn’t give you 365/24/7 electricity. If you would make it a standard that people store the amounts of hydrogen needed to have energy in winter you get problems with proliferating the ability to blow things up as well. Some of that hydrogen storage likely would also blow up accidentally.
But in democratic societies, it may be a better idea to build new solar / wind / energy storage factories instead.
Energy storage facilities alone are not enough. Battery-type storage can store energy economically for interday demand differences but would need two orders of magnitude cheaper to be similarly used for storing energy from the summer to be used in the winter.
If you store hydrogen or synthetic gas, you need a way to burn that for energy as well.
But in the systems that can go horribly wrong unless properly maintained, the past performance is not a good indicator of the future performance.
Out of past problems we learned a lot about how to construct nuclear power plants in a way that’s safer.
It depends completely on how you set up the market. Prediction-based Medicine would allow you to have a free market system that actually aligns the market incentives with health outcomes.
Surely, if we knew that the AI was unfriendly
We are not living in a world of certain knowledge about what is and what isn’t dangerous.
The AI box experiment shows an agent that is constantly begging or persuading its gatekeeper to be released, including threats to torture simulated copies of its creators, and I would think such psychotic behavior is already a pretty big red flag.
There’s a good reason that Eliezer decided not to provide any transcript for the original experiment. It seems that those confused you about what the experiment is about.
It seems you are right. I thought previously that BlenderBot was supposed to be useable outside of the research enviroment.
I read through the description and a key difference seems to be that Blenderbot actually has long-term memory about the user with whom it interacts.