ryan_b
Karma: 1,854 (LW), 0 (AF)
NewTop
There is a discussion at OvercomingBias of this work now.
I see a few criticisms about how this doesn’t really solve the problem, it only delays it because we expect a unified agent to outperform the combined services.
It seems to me on the basis of that criticism that this is worth driving as a commercial template anyway. Every R&D dollar that goes into a bounded service is one that doesn’t drive specifically for an unbounded agent; every PhD doing development an individual service is not doing development on a unified agent.
We’re currently still in the regime where first mover advantage is overwhelming; if CAIS were in place rather than win all the marbles immediately they would win all the marbles eventually and so the incentives are reduced. I expect this approach to extend the runway we have for nailing down the safety questions before a unified agent takes off.
I suppose the delaying action could backfire by reducing funding for safety, and also potentially by simplifying the problem of a unified AGI to bootstrapping from a superintelligent CAIS coordinator. Is there any difference between the superintelligent CAIS coordinator and the AGI in terms of alignment?
Yes, I can do that. I estimate ~1 week or so; could I send you a draft then to see if I’m going in a useful direction?
I am currently getting a Page Not Found response from Putanumonit through that link. It seems that the link includes ” avoiding competition where you can” in the address, but shortening it back to winning-is-for-losers works.
You might be interested in the idea of multivocality, which is saying something which can be interpreted many different ways. The tactical idea is to communicate as little about oneself as possible, while allowing everyone else to communicate things about themselves in response. Here is a blog post that talks about how it was used by Francisco Franco and Cosimo de Medici: Francisco Franco, Robust Action, and the Power of Non-Commitment
I don’t think there’s anything wrong with posting such a thing. As long as you are clear up front about your state of confidence and that you are exploring an argument instead of trying to persuade, I expect few people would object. There are also many who enjoy unconventional arguments or counter-intuitive conclusions on their own merits.
Worst case scenario, it remains a personal blog post. I say post it.
Disclaimer: did not do 5 minutes by the clock. Did do 10-15 minutes of discussion and intermittent thinking since.
Desiderata:
Learn how to work with other people towards a goal
Requires skills which can be improved
Short feedback loops
Clear outcomes
Minimally competitive
The best candidate I have come up with is FIRST, the robotics team. This is still a national competition, but the competition is effectively just a show and the competitive activity is tiny compared to the cooperative activity. The goal is to build a robot as a team, so it lends itself instantly to improvable skills, short feedback loops, and clear metrics. It is cooperative mostly in the division-of-labor sense—you can’t expect one or two kids to be able to do all the work. It also strongly incentivizes skill transfer, because the less skilled kids want to succeed and the more skilled kids need them to succeed for the robot to work.
I first considered things that were not sports, like drama or dance. These turn out to be extremely competitive, but at the front end; you need to win the role or a position on the team before the coordination even begins.
I considered intellectual activities, like Math Olympiad or Chess, but these tend to be highly individual and so entail minimal coordination—even team events are mostly just aggregations of individual performance. They largely consist of people just being measured against one another.
There are explicitly social, group activities like Model U.N, but these are plagued by being unclear about the skills involved, have unclear outcomes and no short feedback loops. Even stuff like the Boy Scouts really only do coordination by teaching that being cooperative is a virtue.
Lastly there are clubs of various kinds, which often relax the competitive aspect but usually also abandon any specific notion of skill development or feedback; they are just people hanging out who all enjoy the same thing.
On the flip side of the coin, this is a really good point:
I’m somewhat skeptical that the coordination skills you learn from those places would transfer to more productive activities.
I noticed while thinking about this that the things I think are the most valuable about sports—apart from the exercise and the concept of the team—were either not emphasized or not articulated at all. Stuff like how to think about working with someone else and how to beat something that is thinking about beating you weren’t really a factor. This makes me wonder if there is an entirely different way to present sports that would improve their transfer-ability. Sports is still about hierarchy; it’s only transferable value is that it shifts the perspective from hierarchy-among-individuals to hierarchy-among-groups.
There seems to be an opportunity to add value here, but it is not clear how.
I agree that marriage and a family are great cooperative endeavors, but I am deeply skeptical it is a good idea to learn coordination after getting married. My marriage is great, and my wife and I both put down the lion’s share of the difference between our experience and that of others to being specific about coordinating and being on the same team about everything. It really helps to have these concepts ready to work with before jumping in.
All I have gotten out of it so far is a morbid entertainment value. It does look like it will spend more time talking about subjects adjacent to the Repugnant Conclusion and the voluntary extinction of Absolute Negative Utilitarianism, but it isn’t rigorous (so far) in the sense that we usually prefer here.
The author is a good writer, so it does a pretty good job of holding interest despite the subject matter. I would say it is unproductive aside from the entertainment, and if you find it persuasive even more so.
Apropos of the negative utilitarianism question posted recently, has anyone read any pessimism? I picked up The Conspiracy Against the Human Race: A Contrivance of Horror relatively recently. It is a survey, written by Thomas Ligotti, who is a horror and weird fiction writer.
It is gloriously grim. I recommend against it if you are in a sensitive place, however.
I read this section completely differently.
He points to thinking about the important problems as causing success. When people change what they are doing, then they don’t continue to have it:
In the first place if you do some good work you will find yourself on all kinds of committees and unable to do any more work.
Carrying on from the end of your section:
When you are famous it is hard to work on small problems. This is what did Shannon in. After information theory, what do you do for an encore? The great scientists often make this error. They fail to continue to plant the little acorns from which the mighty oak trees grow. They try to get the big thing right off. And that isn’t the way things go. So that is another reason why you find that when you get early recognition it seems to sterilize you. In fact I will give you my favorite quotation of many years. The Institute for Advanced Study in Princeton, in my opinion, has ruined more good scientists than any institution has created, judged by what they did before they came and judged by what they did after. Not that they weren’t good afterwards, but they were superb before they got there and were only good afterwards.
The talk is about things that cause people to do great work. When those causal factors change, the work output also changes. He goes on to cover other things which are about professional success:
Working with an open office door, to talk to your coworkers
Changing routine work into more general and important work, which is more satisfying
The importance of self-promotion
Working on presentation skills
How to recruit your boss to fight with outside agencies
How to get your boss to give you more resources
Dressing for success, and getting punished for non-conformity
Lastly, he is pretty specific about his motivations (emphasis mine):
I think it is very definitely worth the struggle to try and do first-class work because the truth is, the value is in the struggle more than it is in the result. The struggle to make something of yourself seems to be worthwhile in itself. The success and fame are sort of dividends, in my opinion.
So he is specifically talking about professional success in science. But—things like the rationality project and EA are good candidates for other fields to which the advice could be applied, especially in light of how important science is to them.
I think that personal success is the correct impression:
I noticed a couple of months later he was made the head of the department. I noticed the other day he was a Member of the National Academy of Engineering. I noticed he has succeeded. I have never heard the names of any of the other fellows at that table mentioned in science and scientific circles.
Notice he doesn’t talk about all the amazing things that were solved; he talks about lab positions and Nobel Prizes and getting equations named after himself.
I expect that Hamming would view having an impact on the world as being a good reason to choose going into science instead of law or finance, but once that choice is made being great at science is the reasonable thing to do.
To be clear, I don’t think he viewed reputations and promotions as the goal, I believe he viewed them as reasonable metrics that he was on the right track for doing great science.
Based on the other comments, I feel like it is worthwhile to point out that Hamming is talking about how to be a successful scientist, as measured by things like promotions, publications, and reputation.
He is not talking about the impact of the problems themselves. From the quoted section, emphasis mine:
It’s not the consequence that makes a problem important, it is that you have a reasonable attack. That is what makes a problem important. When I say that most scientists don’t work on important problems, I mean it in that sense.
So it looks like we’re trying to apply the question one entire step before where Hamming did. For example there weren’t—and if I read Hamming right, still aren’t—reasonable attacks to the alignment problem. The prospective consequences are just so great we had to consider what is reasonable in a relative sense, and try anyway.
It feels like rationality largely boils down to the search for a generative rule for reasonable attacks.
Hickel’s suggestion that in pre-colonial times people in those very poor countries were less poor than GDP-based measures suggest because they had highly-non-financial assets like (communal) access to water, livestock, grazing land, etc. This is “a romantic fairy tale”.
Pinker is wrong here. Pastoralists in general and Steppe peoples in particular are a good example.
Though there was an enormous amount of commerce with the latter, and Chinese records are sufficient to make pretty good GDP estimates in the event anyone were to try, as it happens that commerce dried up at the conclusion of the Dzungar-Qing Wars in 1757. With the genocide of the Dzungars, China and Russia conducted trade by treaty, and the civilizations of Inner Asia entered a period of sharp decline right up until the collapse of the Soviet Union.
How much actual time do you spend when you read a paper deeply?
I am currently reading the Minimum Entropy Production Principle paper, dedicating some time to it every day. I am about 3 hours deep now, and only half finished. I am not going as far as to follow all the derivations in detail, only taking the time to put the arguments in the context of what I already know. I expect it will take more than one reading to internalize.
Sometimes I worry this is so slow I am spending my time poorly, but I haven’t any idea how it goes for more serious people, so I thought I would ask.
Open Thread February 2019
I haven’t read either of these books I am about to mention, but you might find cultural evolution to be an interesting subject. This is largely because evolution is pretty well specified and so while culture isn’t, linking the two provides more clarity for the latter. There are reviews of The Secrets of Our Success by Joseph Heinrich, and Darwin’s Unfinished Symphony by Kevin Laland over at Overcoming Bias.
They’re on my list, but the worst thing about reading lists is that they grow.
I have a counter-question.
Where are we going to learn coordination from? It won’t be from our community; no one has those any more. It won’t be from academics or employment, both of which only serve to drive home how much working in groups really sucks and how much better it would be if we never had to do that.
The list of things which involve working on a team and are not straight misery is very short. At the moment I can come up with nothing that isn’t competitive.
This is bothering me. I’m going to have to give it a full five minutes somewhere over the weekend.
Oops! It appears you are right.
Putting the numbers into a more common format for comparing risks via back of the envelope, this means football players get head injuries of about 20 per 1000, and soccer players about 10 per 1000, making a season of football only twice as dangerous. That is surprising to me.
Moving that one to the bottom of the list for my daughter, I suppose.
I’ve often wondered how far we could get if our training systems consisted less of one-to-many instruction and instead focused on deeply monitored, iterated group performance. The latter is how the most extreme environments operate, like space missions and the military, but the expense is hard to justify.
On the other hand, I don’t know of any middle-ground attempts at doing this for more routine environments. Doing such training for a standard office environment, which relies on standard consumer hardware and has no unusual safety or performance requirements, is doubtless much cheaper. How much cheaper would it have to be, and what kind of performance would it have to deliver, to make it worth considering as an alternative to the standard model of education?
As I write this it occurs to me that most of the distinction is down to the environment, and this model could easily suffer from a lack of emphasis on the value-added tasks that companies are concerned with. Of course, formal education does not provide any focus on those tasks either; the degree just offers some confidence that once provided with them, they can be accomplished.