As far as I can tell, Sam is saying no to size. That does not mean saying no to compute, data, or scaling.
“Hundreds of complicated things” comment definitely can’t be interpreted to be against transformers, since “simply” scaling transformers fits the description perfectly. “Simply” scaling transformers involves things like writing a new compiler. It is simple in strategy, not in execution.
I would like to comment on Budapest Memorandum technicality. You probably already know this since you conceded Russia has a point, but other readers may not. The following is trying to be a neutral summary.
In 1994, in return for Belarus and Ukraine giving up nuclear weapons and joining NPT, US promised to “refrain from economic coercion designed to subordinate to their own interest the exercise by the signatory of the rights inherent in its sovereignty and thus to secure advantages of any kind”.
In 2013, US sanctioned Belarus. Belarus notified US that US broke Budapest Memorandum. US replied it didn’t (what?), because sanctions are for human rights, and not designed to subordinate etc.
In 2014, Russia annexed Crimea.
I am not sure what US was thinking in 2013. If US thought Budapest Memorandum was at all valuable, they should have paused and thought twice about it. Given their frankly absurd reply, I think they didn’t consider it valuable.
I live in South Korea, so my specialty (I am not at all special in South Korea, but that immediately makes me an expert in the global internet) is North Korea. US and North Korea reached an agreement called Agreed Framework (done in Geneva, so better known as Geneva Agreement in South Korea) in 1994. (See the pattern? 1994 was an important year.)
In return for North Korea dismantling nuclear reactors in Nyongbyon and remaining in NPT, US promised to deliver 500,000 tons of heavy oil every year to North Korea for energy production, until non-proliferating nuclear power plant is completed, since North Korea insisted nuclear reactors were for energy production, and there was even some truth to it. North Korea was severely deficient in electricity.
US didn’t deliver heavy oil because Congress didn’t fund it. What were they thinking?! To North Korea, non-proliferation promise was directly linked to delivery of heavy oil.
In 2002, president Bush declared North Korea an Axis of Evil, and the agreement formally broke down.
I see a pattern here. I consider US-Iran relation to be similar. When US makes a deal with another country to join or to remain in NPT, they are willing to promise a lot. Once the deal is done, US does not keep its promise. It’s almost as if US thinks it does not need to keep its promise, since another country should have remained in NPT anyway! US seems to think, why should I pay for it? I must scream now: US must pay for it, because US agreed to. In faith whereof the undersigned plenipotentiaries etc.
I have a recommendation to US. Keep your words. Do not sanction if you promised not to sanction. Do deliver heavy oil if you promised to deliver heavy oil. It’s not that difficult. Thank you.
CL stands for “change list”. It’s not even a tech jargon, it’s a sort of Google jargon, and Google admits as much, see https://github.com/google/eng-practices.
I interpreted that question as “in your lifetime”.
No doubt the earliest pioneers of computer science, emerging from the (relatively) primitive cave of electrical engineering, stridently believed that all future computer scientists would need to command a deep understanding of semiconductors, binary arithmetic, and microprocessor design to understand software.
This is disappointing, although not unexpected. Computer scientists, in general, are tragically bad at the history of their own field, although all of science is like that and it is not specific to computer science. Compare Alan Turing, who wrote in 1945, before there was any actual computer to program:
This process of constructing instruction tables should be very fascinating. There need be no real danger of it ever becoming a drudge, for any processes that are quite mechanical may be turned over to the machine itself.
That is, the earliest pioneers of computer science thought about automating programming tasks, before any of actual programming. As far as I know zero actual pioneers of computer science believed programmers would need to understand hardware design to understand software. It would be especially unlike for Alan Turing, since the big deal about Turing machine was that universal Turing machine can be built such that underlying computing substrate can be ignored. Misconceptions like this seem specific to people like Matt Welsh who came after the field was born and didn’t bother to study the history of how the field was born.
I am a Stratego player, and the answer is no, not really. In fact, DeepNash won 30⁄30 (100%) against Probe, which won Computer Stratego World Championship three times in the past.
But I think Paul is not wrong. While Stratego is mostly skill not luck (it’s not like you are drawing cards and you need good cards, there is zero randomness, just hidden information), there is a bit of rock-paper-scissors involved. Novices can’t beat experts, but I do think experts can beat God.
“Sanctions must have specific offramps” is an argument against sanctions without them. It is unrelated to whether a war of aggression was initiated. Yes Putin is not stupid and subtext may be obvious, but I still support making subtext manifest.
It is legitimate to worry sanctions will continue. For example, sanctions against Iran in fact had a clear stop condition. IAEA will do verification and sanctions will be lifted. IAEA did verification in 2018. Four years have passed, and sanctions against Iran are continuing.
I kind of disagree. (I was on South Korean IMO team.) I agree IMO problems are in similar category of tasks including research math than high school math, but since IMO problems are intended to be solvable within a time limit, there is (quite low, in absolute sense) upper limit to their difficulty. Basically, intended solution is not longer than a single page. Research math problems have no such limit and can be arbitrarily difficult, or have a solution arbitrarily long.
Edit: Apart from time limit, length limit, and difficulty limit, another important aspect is that IMO problems are already solved, so known to be solvable. IMO problems are “Prove X”. Research math problems, even if they are stated as “Prove X”, is really “Prove or disprove X”, and sometimes this matters.
China also seems to be quite far behind the west in terms of LLM
This doesn’t match my impression. For example, THUDM(Tsing Hua University Data Mining lab) is one of the most impressive group in the world in terms of actually doing large LLM training runs.
“Don’t do anything for 6 months” is a ridiculous exaggeration. The proposal is to stop training for 6 months. You can do research on smaller models without training the large one.
I agree it is debatable whether “beyond a reasonable doubt” standard is appropriate, but it seems entirely sane to pause for 6 months and use that time to, for example, discuss which standard is appropriate.
Other arguments you made seem to say “we shouldn’t cooperate if the other side defects”, and I agree, that’s game theory 101, but that’s not an argument against cooperating? If you are saying anything more, please elaborate.
Very interesting! Recently, US started to regulate export of computing power to China. Do you expect this to speed up AGI timeline in China, or do you expect regulation to be ineffective, or something else?
Reportedly, NVIDIA developed A800, which is just A100, to keep the letter but probably not the spirit of the regulation. I am trying to follow closely how A800 fares, because it seems to be an important data point on feasibility of regulating computing power.
Re: Aristotle. Large part of what Aristotle wrote is science and math. If you felt you didn’t learn anything from Aristotle, that’s because only non-science and non-math part of Aristotle is usually taught, because science and math are usually taught in ahistorical manner.
Barbara, Celarent, Darii, Ferio is good math. It is just that first-order logic and Venn diagrams are better, so we don’t teach Barbara etc. One thing lost by this ahistorical teaching is how much better first-order logic is, and how difficult advance it was when it was first done by Frege.
Re: serial interval. This doesn’t seem published anywhere yet, even in preprint form, but Korean CDC daily briefing 2021-12-13 already announced shorter serial interval. Their estimate is 2.8~3.4 days. Their sample is larger than Norway, n=123.
https://www.kdca.go.kr/board/board.es?mid=a20501010000&bid=0015&list_no=717925&act=view
It’s the fifth section titled Epidemiological Characteristics of Omicron Variant-related Cases.
I took the survey. I did Political Compass for the first time, and I found its questions and results rather baffling. Political Compass themselves admit it is culturally biased and mainly for western democracies.