ryan_b

Karma: 4,177

ryan_b 20 May 2022 16:46 UTC
2 points
on: We have achieved Noob Gains in AI
I suppose the obvious follow up question is: do you think there are any interesting ideas being pursued currently? Even nascent ones?

Two that I (a layperson) find interesting are the interpretability/transparency and neural ODE angles, though both of these are less about capability than about understanding what makes capability work at all.

ryan_b 19 May 2022 20:36 UTC
2 points
in reply to: tailcalled’s comment on: Deepmind’s Gato: Generalist Agent
does not have any integrated world-model that it can loop on to do novel long-term planning
I am interested in more of your thoughts on this part, because I do not grok the necessity of a single world-model or long-term planning (though I’m comfortable granting that they would make it much more effective). Are these independent requirements, or are they linked somehow? Would an explanation look like:
- Because the chunks of the world model are small, foom won’t meaningfully increase capabilities past a certain point.
Or maybe:
- Without long-term planning, the disproportionate investment in increasing capabilities that leads to foom never makes sense.

ryan_b 18 May 2022 12:54 UTC
6 points
on: Optimization at a Distance
I cannot find the reference for this despite repeated attempts, but the rock example reminds of a story I once read in a letter from a student describing a lecture Von Neumann gave.
In this lecture, Von Neumann made a reference to thinking of evolution as a universal principle; the gist of it was that if we replace “have many descendants” with “propagate your information into the future” then atoms are excellent from an evolutionary point of view, because most atoms are very stable and therefore very likely to still exist in the future.
So when asked what the rock is optimizing for, I immediately thought of this story, and that the rock is optimizing for being rock in the future and that minimizing interaction with the environment is probably optimal for this purpose.

ryan_b 17 May 2022 17:53 UTC
2 points
on: ryan_b’s Shortform
Quick question: could we use the Gato trick of multi-task → single-model in reverse, such that we exclude tasks?
The idea is that we would specifically create training on “bad” tasks, like connect to the internet, or write code, and then build a single model which includes the “good” ones but excludes the “bad” ones.
Based on my understanding of how these things work there’s no sense in which the tasks would be rejected exactly; rather what I imagine is a kind of pathological underperformance. An analogy would be giving GPT-3 catastrophic dyslexia on purpose.
The natural downside is that we will deliberately be building components that are pretty good at bad tasks, which is dangerous. But our bigger problem is failing to screen out bad things and so they happen accidentally, and this feels at first blush like an option for making incremental progress, at least.

ryan_b 16 May 2022 15:07 UTC
2 points
in reply to: Alex Vermillion’s comment on: What to do when starting a business in an imminent-AGI world?
This makes sense to me, but it seems like a better fit for describing a post-AGI world. I am asking about the period before AGI has arrived (like now) but when the probability of it arriving within the early lifetime of a business is high enough to merit specific consideration (also like now, I claim).
There has to be a transitionary period before AGI actually is in a position to be doing any of these things even in the fast takeoff scenario; there’s too many atoms to move around for it to be otherwise, it seems to me.

ryan_b 13 May 2022 15:18 UTC
4 points
in reply to: NunoSempere’s comment on: Open & Welcome Thread—May 2022
Well I liked the looking back post—though I have only just now noticed they are in a running sequence. Query—would you prefer to have engagement here, or at substack?
Also, once again note to myself to be what-feels-from-the-inside like gushingly, disgustingly effusive but-in-fact-is just positive feedback at all.

ryan_b 13 May 2022 13:43 UTC
2 points
in reply to: shminux’s comment on: What to do when starting a business in an imminent-AGI world?
“Business” is way downstream of other activities that would be impacted by “an imminent-AGI world”.
I am skeptical of this. What causal path for changing the world at scale doesn’t go through businesses, specifically the hardware ones like manufacturing and construction?

[Question] What to do when starting a business in an imminent-AGI world?

ryan_b12 May 2022 21:07 UTC

25 points

7 comments1 min readLW link

ryan_b 12 May 2022 19:39 UTC
47 points
in reply to: gwern’s comment on: Deepmind’s Gato: Generalist Agent
I share the gut-shock. This is really thumping me with the high dimensional world perspective. The dimensionality problem is all:
They had to exercise unrealistic levels of standardization and control over a dozen different variables. Presumably their results will not generalize to real sleds on real hills in the wild.
But stop for a moment to consider the implications of the result. A consistent sled-speed can be achieved while controlling only a dozen variables. Out of literally billions.
But then the scaling hypothesis be like “Just allow for billions of variables, lol.”
Are we actually any farther from game over than just feeding this thing the Decision Transformer papers and teaching it to play GitHub Copilot?

ryan_b 10 May 2022 20:07 UTC
10 points
on: Innovation, Stagnation, and Paratrooper Operations
I was in the 82nd Airborne Division (which is one of the airborne divisions in the United States Army) so I can provide an inside perspective on this.
The frontline infantry all know this already; it is a running joke. At least once a year a special airborne training operation (colloquially just called “jumps”) is held with members of Congress as guests, for the singular purpose of impressing them and keeping our share of the budget.
I expect the United States will sustain airborne operations for longer than other countries because we are the only country which can reliably compensate for many of the identified weaknesses through airpower, like negating enemy airpower or armored formations in the target jump zone.
That being said, there do seem to be advantages to having an airborne outfit, although this has nothing to do with the strategic value of airborne operations. First and foremost is the twice-a-volunteer phenomenon; in the US Army, you need to additionally volunteer to go to airborne training, separately from volunteering for enlistment. This is a filter that ensures a higher-than-average motivation level for the troops. Units in the 82nd also aggressively propagandize elite status on the basis of having airborne training, which cashes out in higher expectations across a variety of metrics like physical fitness and skill with specialized weapon systems.
Airborne troops are also partially inoculated against a notoriously severe problem that commonly infects military formations of every kind: the inability to think on your feet. The two common symptoms are failure to adapt to the situation at hand (too devoted to doctrine, for example) or refusal to take initiative (just sitting there while the situation gets worse in the absence of orders). This comes from the airborne operation experience directly—a jump is at best bounded chaos, and what has to happen in order to succeed is just to link up with whichever guys you landed near and do your best. This means small group initiative and adaptation are viewed as the building block for unit success.
As a disclaimer, through a series of unusual events I never conducted a jump, and am not formally trained, so these observations are from doing jump setup, cleanup, and other supporting functions (and repeating the indoctrination).

ryan_b 9 May 2022 20:55 UTC
4 points
in reply to: Elizabeth’s comment on: Why hasn’t deep learning generated significant economic value yet?
Just ran into the listicle problem myself; it effectively slew searching Google for anything where I don’t already know most of what I need. It feels weird that in the name of ad revenue the algorithm promotes junk whose sole purpose is also to generate ad revenue. Process seems to be cannibalizing itself somehow.
It would be cool to filter GPT-3-ish things. It seems like we could get most of the diversity without anything very sophisticated; something like negatively weighting hits according to how many other results have similar/very similar content. If all the pages containing some variation of “Top #INT #VERB #NOUN” could get kicked to the bottom of the rankings forever, I’d be a happy camper.

ryan_b 5 May 2022 13:48 UTC
2 points
in reply to: Elizabeth’s comment on: Why hasn’t deep learning generated significant economic value yet?
I wonder how we’d go about designing a reward signal for the long-tailed stuff.

ryan_b 3 May 2022 18:38 UTC
2 points
in reply to: AllAmericanBreakfast’s comment on: What Would It Cost to Build a World-Class Dredging Vessel in America?
The story which is being told about the related Jones Act consistently suggests that American commercial shipbuilding missed a critical industry shift in the 70s, of which I have yet to track down any concrete details. My best guess based on the other problems mentioned at the same time—by which I literally mean shortest word distance between a problem with operation costs and this allusion—is industrial automation, both in the manufacture and operation of the ships.
I think this is plausible because the automotive industry was having the same fight, with the Big Three passing on automated manufacturing until the 1980s due to a combination of short-sighted corporate leadership and union opposition. Since the shipping industry was not under increasing pressure from things like Japanese imports due to the laws, it would at least be consistent for them to have made the transition later, and fail to keep abreast of the newest developments.

ryan_b 29 Apr 2022 17:46 UTC
3 points
on: Do FDT (or similar) recommend reparations?
This seems like it would be a real tough call, as the rules by which such things are decided change over the course of the period under consideration.

ryan_b 25 Apr 2022 17:42 UTC
3 points
on: Report likelihood ratios
Is there any kind of paper where reporting likelihood ratios is common? I don’t recall ever reading one, or seeing one discussed.
My current suspicion on why we don’t see more of it comes down to citations; there’s no relevant analysis for a given researcher to cite, upon which they can build or compare their likelihood ratios against.
It feels to me like a viable strategy to kickstart a conversion to likelihood ratios in a field is to fund papers which re-examine the fundamental findings in the field under a likelihood paradigm, to provide a ready basis of citeable references for when people are conducting current research. I’d really like to see something similar to the kind of analysis done here, but for different versions of the 2nd Law of Thermodynamics, for example. Unfortunately I don’t see analysis of data being touted as an area of focus for any of the “new ways to conduct research” orgs that have popped up in the last few years.

ryan_b 25 Apr 2022 13:29 UTC
2 points
on: Will/Has the Russia-Ukraine war been a tipping point for the shift from oil energy?
It feels to me like the decisive metric is going to be investment in non-oil energy infrastructure. There appears to be a big enough shift that it is expected to impact natural gas investments in Europe, but the oil question is quite a bit more diffuse, which might accidentally weigh in oil’s favor. What I have in mind when I say the diffusion weighs in oil’s favor is that it is trivial to get oil from elsewhere, and I expect most of the EU countries would be perfectly happy with buying oil from a Gulf country while that country buys discounted Russian oil in turn.

ryan_b 19 Apr 2022 17:13 UTC
4 points
in reply to: Not Relevant’s comment on: Clem’s Memo
The latter: nuclear deterrence failed in the sense that our models of the world were completely wrong; but they did not fail catastrophically in the sense that we wound up in a nuclear war. A common description of this is “failed to fail,” which is sort of like the operational version of “not even wrong” in math.
This is different from the plan working, which means our models were good enough that our actions caused nuclear peace.
This conclusion arose based in increased access to records and memoranda of Soviet leadership after the Soviet Union fell; what we discovered is that their leaders did not think or feel at all like we thought they did across multiple events.
Edit for sources you might be interested in:
Book: The Great American Gamble, by Keith B. Payne. This is the best history I know of on the subject, and does a good job of distinguishing between things like why ideas that went into deterrence are wrong vs. how we commonly misinterpreted the ideas we were using even so.
Blog posts: Dominic Cummings, if you are interested in reading about a recent government official trying to shift beliefs within government based on this newer information. He’s a controversial personality, but the only personal perspective on this topic I am aware of.

ryan_b 19 Apr 2022 14:48 UTC
2 points
in reply to: ADifferentAnonymous’s comment on: Clem’s Memo
Small quibble: I note that there is a difference between it worked and it failed to fail catastrophically. The current consensus among people who study the subject is that the latter describes reality.
This does not change Attlee being wrong, though. It just means the direction we went instead was wrong for different reasons.

ryan_b 18 Apr 2022 18:47 UTC
3 points
on: Why Rome?
Good post!
Your conclusions broadly agree with the mechanism described by Ibn Khaldun of one tribe adding another’s asabiyyah to their own via conquest. From his Muqaddimah:
Once asabiyah has established superiority over the people who share in that particular asabiyah, it will, by its very nature, seek superiority over people of other asabiyah unrelated to the first. If the one asabiyah is the equal of the other or is able to stave off its challenge, the competing people are even with and equal to each other. Each asabiyah maintains its own domain and people, as is the case with tribes and nations all over the Earth. However, if the one asabiyah overpowers the other and makes it subservient to itself, the two asabiyah enter into close contact, and the defeated asabiyah gives added power to the victorious one, which, as a result, sets it goal of domination and superiority higher than at first.
Tanner Greer has a good summary of the concept over at the Scholar’s Stage.
A singular example of the tribe-absorbs-another-tribe element is provided by Genghis Khan, who pioneered the practice of killing a tribe’s leaders and then adopting the rest of them wholesale into Mongol households. He did this on a limited scale with Merkid, Tatar and Tayichiud prisoners, and then with the Jurkin tribe entire. For this I recommend Jack Weatherford’s Genghis Khan and the Making of the Modern World.

ryan_b 18 Apr 2022 14:13 UTC
9 points
in reply to: johnswentworth’s comment on: Org announcement: [AC]RC
I urge against abandoning the 0-hose case too quickly! Consider direct radiative cooling of people, which appears to be correctly aligned to the human value of thermal comfort, and highly efficient, while still running afoul of the heuristic that if fails to cool any room at all.
This suggests the multi-hose paradigm is already Goodharted, and designs isomorphic to the 0-hose case are a more fruitful path of investigation.

ryan_b

[Question] What to do when start­ing a busi­ness in an im­mi­nent-AGI world?

[Question] What to do when starting a business in an imminent-AGI world?