M. Y. Zuo(Michael Y. Zuo)
Thanks for taking the time to write out these reflections.
I’m curious about your estimates for self driving cars in the next 5 years, would you take the same bet at 50:50 odds for a 2028 July date?
Can you lay out step by step, and argument by argument, why that should be the case in a real world legal system like the US?
It seems very far from currently accepted jurisprudence and legal philosophy.
Unless the post was edited afterwards I think the last link:
DeepMind reportedly lost a yearslong bid to win more independence from Google
“One suggestion from DeepMind’s founders was apparently for the company to have the same legal structure as a nonprofit, ‘reasoning that the powerful artificial intelligence they were researching shouldn’t be controlled by a single corporate entity, according to people familiar with those plans.’ But Google wasn’t on board with this, telling DeepMind it didn’t make sense considering how much money the company has poured into DeepMind.”
Is suggesting exactly that.
I hesitate to be the first to respond here but it seems there is a point so strange that someone else must have noticed as well so I hope it can be clarified. That is since the main problem your work is tackling is:
“how can we regulate a very complex and very smart system with unpredictable emergent properties using a very simple and dumb system whose properties once created are inflexible”
There must be then some hard constraint for AI work as well that involves some ‘simple and dumb system whose properties once created are inflexible’. Which does not seem to be inevitable.
Utility and objective functions don’t have to follow that kind of description, it is only assumed in certain projections.
If in the future some world compact decided, for example, that some hard coded objective function must be continuously executed and also be very difficult to change, then it seems plausible, but that is by no means preordained. Since there is no clear consensus that such a system can even be maintained perpetually.
Bingo, the root problem is pretending to have any quasi-judicial structure/authority at all.
People of roughly equal status issuing ‘judgements’ or ‘decisions’ on each other really doesn’t make sense for that reason, at best you can do so within a private club and its property lines.
A federation of private clubs may decide to do so, very rarely, only for the most serious cases, because as mentioned in the OP there’s always the risk of some clubs siding with the accused and then deciding to leave, splitting the federation.
Is there a compiled list of what the LTFF has accomplished and how that compares to past goals and promises, if any, made to previous donors?
I know some potential donors that would be more readily convinced if they could see such a comparison and reach out to past donors.
Yeah this seems a bit of a self-defeating exercise.
Who made the decision to go ahead with this method of collecting signatories?
They didn’t even have a verification hold on submitted names...
It does seem like a straightforward conclusion that Eliezer didn’t really understand what he was writing about then.
If so, publishing a revised version with the necessary changes seems like the sensible choice. Especially since the example is frequently referenced throughout.
One other thing I could never get them to do was to ask questions. Finally, a student explained it to me: “If I ask you a question during the lecture, afterwards everybody will be telling me, `What are you wasting our time for in the class? We’re trying to learn something. And you’re stopping him by asking a question’.” It was a kind of one-upmanship, where nobody knows what’s going on, and they’d put the other one down as if they did know. They all fake that they know, and if one student admits for a moment that something is confusing by asking a question, the others take a high-handed attitude, acting as if it’s not confusing at all, telling him that he’s wasting their time.
I explained how useful it was to work together, to discuss the questions, to talk it over,
but they wouldn’t do that either, because they would be losing face if they had to ask someone else. It was pitiful! All the work they did, intelligent people, but they got themselves into this funny state of mind, this strange kind of self-propagating “education” which is meaningless, utterly meaningless!Feynman
An all too common folly.
Here’s a few, unordered:
As We May Think, by Vannevar Bush
Politics and the English Language, by George Orwell
The Tyranny of Structurelessness, by Jo Freeman
Some Moral and Technical Consequences of Automation, by Norbert Wiener
Can we Survive technology?, by John von Neumann (though I may be a bit biased here as I’ve had personal interaction with one of his family members)
Public assessments of existing generative AI systems. The Administration is announcing an independent commitment from leading AI developers, including Anthropic, Google, Hugging Face, Microsoft, NVIDIA, OpenAI, and Stability AI, to participate in a public evaluation of AI systems, consistent with responsible disclosure principles—on an evaluation platform developed by Scale AI—at the AI Village at DEFCON 31. This will allow these models to be evaluated thoroughly by thousands of community partners and AI experts to explore how the models align with the principles and practices outlined in the Biden-Harris Administration’s Blueprint for an AI Bill of Rights and AI Risk Management Framework.
I don’t know anything about the ‘evaluation platform developed by Scale AI—at the AI Village at DEFCON 31’.
Does anyone know if this is a credible method?
I’d be willing to take a bet that the U.S. will not respond with nuclear retaliation against Russia, regardless of what Russia or any of its governmental actors do, for a 1 year period. If you believe there’s any chance.
Does it matter for Villiam’s point whether 10x more Palestinian civilians are killed then Israeli, or 20x, or 30x?
For example, even if it’s perceived by inflated reports to be 30x, and the real figure is 10x, that’s still 10x more civilians in body bags.
My experience is that humans who aren’t paying full attention aren’t good butterfly nurturers, and group interactions invite people to respond while paying less than full attention, often without realizing they’re doing so.
In my experience this is inversely proportional to the seriousness and focus of the group.
The opposite end would be focused and serious interaction by geniuses on a deadline.
Richard Feynman has a notable anecdote where Manhattan Project physicists were sitting around a table in a hours long meeting, and their ability to pay attention seemed superhuman:
One of the first interesting experiences I had in this project at Princeton was meeting great men. I had never met very many great men before. But there was an evaluation committee that had to try to help us along, and help us ultimately decide which way we were going to separate the uranium.
This committee had men like Compton and Tolman and Smyth and Urey and Rabi and Oppenheimer on it. I would sit in because I understood the theory of how our process of separating isotopes worked, and so they’d ask me questions and talk about it. In these discussions one man would make a point.
Then Compton, for example, would explain a different point of view. He would say it should be this way, and he was perfectly right. Another guy would say, well, maybe, but there’s this other possibility we have to consider against it.
So everybody is disagreeing, all around the table. I am surprised and disturbed that Compton doesn’t repeat and emphasize his point. Finally at the end, Tolman, who’s the chairman, would say, “Well, having heard all these arguments, I guess it’s true that Compton’s argument is the best of all, and now we have to go ahead.”
It was such a shock to me to see that a committee of men could present a whole lot of ideas, each one thinking of a new facet, while remembering what the other fella said, so that, at the end, the decision is made as to which idea was the best—summing it all up—without having to say it three times. These were very great men indeed.
I had eaten gluten just fine for years, no family history of gluten issues, DNA testing negative for celiac, etc.
Methods of growing and processing wheat have changed dramatically in the last few decades. Especially for pre-packaged sliced bread, buns for fast food, and anything else bought by a serious cost cutting purchaser.
i.e. The gluten found in any product you can buy for a few dollars in the U.S. or Canada is almost certainly different then the gluten any of your parents or grandparents had eaten.
This isn’t really talked about outside of industry experts and enthusiasts, and maybe some very agitated celiac adjacent folks, because there is no feasible way to go back to the old way of wheat growing and processing without bread doubling or tripling in price.
There are probably niche growers for luxury markets, and of course ‘gluten free’ bread, if you really want to eat bread regardless.
Ideally I would link to concrete examples but I’m afraid it would come across as me calling out someone else, especially if they believe they put in their best effort in writing a serious essay, so I will have to leave it to your imagination.
The Sequences do contain some very good pieces of writing, but they also contain some that are not so, and perhaps it is an artifact of the time period and/or Eliezer’s personal idiosyncrasies, but I can’t honestly say I perceived all of it as entirely wholesome. For example, some of it comes across as more argumentative than necessary, some of it seems a bit too eager for recognition, and so on. Due to the nature of vibes, I’m not sure if I could provide a more convincing explanation.Then again I may just be an outlier.
The top 1% of writings on LW are definitely better than the top 1% of any subreddit I’ve seen. I think they are probably the largest collection of high quality writing by a pool of many dozens (hundreds?) I’ve seen anywhere online.
Making any accusation whatsoever regarding a real, physical, human being, with a known identity, means that they are bringing ‘real physical life into the conversation’. Or at least that’s how I read the parent.
In which case they can’t credibly expect to be protected from getting counter-accused by someone else.
So pseudonymous accounts can accuse each other, pseudonymously, all day long, with an expectation of privacy. But the moment they link a real world identity means that the counter-party(s) can do so too.
It’s probably more so that that the vast majority of families in China have recent historical memories of being duped by sophisticated folks proclaiming to be altruistic, morally superior, etc., and engaging in activities allegedly to the benefit of the wider population.
I’m inclined to believe in the possibility of a memetic ‘inoculation’ so I can see why there would be a huge difference in how such ideas are viewed, to a lesser extent this applies to all of the former Warsaw Pact countries too.
Plus there’s the natural tendency of an older and longer settled society to be more conscious of social status maneuvering.
Most of the population probably won’t have anything against genuinely altruistic people who don’t try to claim any sort of social status, moral authority, etc.
It’s hard to describe in prosaic terms since it was more of an abstract feeling than a literal change in average word count per comment or similar.
Perhaps the best way to put it is that it was the ideal forum culture that many pre-2010s wanted to see come into fruition except focused primarily on finance, without any visible moderation, and with a much higher bar for entry. Or perhaps an intellectual gentleman’s club culture brought online.
This was back when the cheapest annual subscription could have been a mortgage payment for a shabbier townhouse in London.
I couldn’t afford that but still had access because of my university’s arrangements with the then publisher. I only got commenting privileges after vastly cheaper student-only subscriptions were offered as a trial.
So to be fair to the Nikkei, the old culture was already half gone when they took control as the students were inevitably much less sophisticated, including me.
It may sound ridiculous to 2023 readers but approximately 0% of the commentators, even though the very large majority of them were anonymous or pseudo-anonymous, engaged in any of the negative tendencies we now associate with large amounts of online interactions.
If we expect pareto distribution to apply then the folks who will really move the needle 10x or more will likely need to be significantly smarter and more competent than the current leadership of MIRI. There likely is a fear factor, found in all organizations, of managers being afraid to hire subordinates that are noticeably better than them as they could potentially replace the incumbents, see moral mazes.
This type of mediocrity scenario is usually only avoided if turnover is mandated by some external entity, or if management owns some stake, such as shares, that increases in value from a more competent overall organization.
Or of course if the incumbent management are already the best at what they do. This doesn’t seem likely as Eliezer himself mentioned encountering ’sparkly vampires’ and so on that were noticeably more competent.
The other factor is that now we are looking at a group that at the very least could probably walk into a big tech company or a hedge fund like RenTech, D.E. Shaw, etc., and snag a multi million dollar compensation package without a sweat, or who are currently doing so. Or likewise on the tenure track at a top tier school.