There’s a reason that naughts and crosses is still the quintessential kindergarten game, it simultaneously develops some kind of geometric intuition about the centre of shapes being especially important, while teaching kids some very basic game theory (if I just enumerate every game...)
BryceStansfield
Karma: 169
I’ve seen this idea about output randomness before, but I’ve never been particularly sympathetic.
Famously the game that the article talks about, chess, has basically calcified over time due in large part to it’s deterministic nature. I find it unlikely that as many games would end in draws if piece moves were somehow randomised.
While input randomness is definitely positive, output randomness allows games to maneuver into interesting tactical spaces that would never be reached in optimal deterministic play, and forces players to think about interesting equilibrium conditions. There’s a reason that despite being an incredibly simple game, poker is still fun.
Edit:
I also reject the idea of input vs output randomness being that meaningful a distinction. In one sense you can view mahjong as being both a perfectly input or output random game, in reality I don’t think it’s either since they’re not actually different.
They’re standard practice at a lot of the types of firms that AI Safety companies tend to hire from, so it could even just be force of habit tbh.
That seems a stronger argument for AI safety policy experts (such as the ones that the aps is beginning to hire) as opposed to safety researchers.
Maybe there’s an argument that policy experts might chat with researchers at local cafes or meetups e.t.c., but it’s quite second order and it seems like a relatively small benefit compared to the wealth of human capital you’d get opening a safety lab somewhere like India.
I’m not sure that I follow why ai safety work should be colocated with data centre build outs. I don’t think many ai safety researchers would have much of anything to do with data centre infrastructure, and as far as I can tell they may aswell be located on the moon.
Very positive on a diversification of AI safety opportunities though.
I’ve been a stay at home parent for a good chunk of the LLM period, so I haven’t seen anything at work, but anecdotally I’ve noticed a massive increase in chatGPTese on a language exchange app I use (hello talk).
While LLMs are (imo) a pretty amazing tool for solving the grammar ASK hypothesis, it’s pretty concerning that a space supposedly dedicated to the vulnerability that comes with language learning is becoming increasingly devoid of beginner mistakes.
I think we’ve done an ok job at human alignment, given that the pension isn’t a bullet to the head.
I somewhat suspect that alignment is easier than most of less wrong think, but I’m definitely in the minority in this space.
I suspect that it would given that the largest room for improvement would be physical (chip/wafer improvements), I suspect that there isn’t that much room for pure mathematically identical improvement of something like a transformer.
Happy to hear your opinion though!
BryceStansfield’s Shortform
A world where alignment is impossible should be safer than a world where alignment is very difficult.
Here’s why I think this:
Suppose we have two worlds. In world A, alignment is impossible.
In this world, suppose an ASI is invented. This ASI wants to scale in power as quickly and thoroughly as possible, this ASI has the following options:
Scale horizontally.
Algorithmic improvements that can be mathematically guaranteed to produce identical outcomes.
Chip/wafer improvements.
Notably, the agent cannot either retrain itself, or train another more powerful agent to act on its behalf, since it can’t align the resulting agent. This should restrict the vast majority of potential growth (even if it might still be easily enough to overpower humans in a given scenario).
In world B, the ASI agent can do all of the above, but can also train a successor agent, we should expect the ASI to be able to get vastly more intelligent vastly quicker.
“Conversely, if gorillas and chimps were capable of learning complex sign language for communication, we’d expect them to evolve/culturally develop such a language.”
I haven’t read much about the whole Koko situation, but my understanding is that part of the claim was that Koko was *unusually* adept with language.
A priori, if language comes “packaged for free” with some other high order cognitive functionalities that for whatever reason can only be maintained in a small proportion of chimps (maybe calorie availability, increased risk taking behaviour or something else), then it seems perfectly plausible that the capability for language would be present in some proportion of chimps >0 but below the critical threshold for language formation.
Alternatively, it also seems possible that the process of creating grammar is more difficult than the process of producing language in an already constructed grammar. In this case you could have a pretty high proportion of animals capable of producing language after instruction, but incapable of inventing language.
I think they probably would, but admit that it’s unprovable and people have good reason to disagree.
The difference to my mind is the difference between:
Personal security and security as a regime.
And 1-epsilon security and 1 security.
I think the difference between the two of these would drive a lot of dictators actions.
I don’t know as much about China, but you can see the first dynamic pretty clearly in Putin’s actions. It’d be hard to argue that it’s good for Russian national security for the Gazprom retirement plan to be “Falling into artic waters in the middle of the night”, but it makes Putin like 0.001% safer.
On the other hand, if there was literally no benefit to doing so, I think Putin would be content and optimally happy retiring to a personal solar system sized dacha.
Maybe this is controversial, but I think that dictators do care about other people, just far less than they care about their own power and safety. It’s well known, for example, that Kim Jong Un has a massive softspot for children.
On the other hand, the only reason democratic leaders don’t act like dictators is because they can’t.
I might be less concerned if the country leading ai development was a parliamentary democracy and not a presidential one, but the level of personal power held by the president of the USA will (imo) lead them to be exactly as prone to malevolent actions as someone like Xi in the CCP.
Like many Americans, I think Dario seems overly rosy about the democratic credentials of the USA and probably overly pessimistic about the CCP.
It wasn’t more than a week ago when the president of the US was blustering about invading an allied state, and I have no doubts that Donald Trump would commit worldwide atrocities if they had access to ASI.
On the other hand, it’s far from clear to me that autocracies would automatically become more repressive with ASI, it seems plausible to me that the psychological safety of being functionally unremovable could lead to a more blasse attitude towards dissonance. Who gives a shit if they can’t unthrone you anyway?
Alternatively, I most often see rote memorization recommended by people studying fields that are inherently somewhat organised.
It’s easy to see why anki might work well for something like “memorizing lots of words in kanji” because the work of organising concepts into buckets is already embedded in the kanji and kanji radicals.
It’s less obvious to me how you could, for example, learn optimal riichi mahjong with this type of method; and probably because of that I’ve never seen someone recommend that.
I’d just note, that you should be cautious of people “answering” this question in hindsight.
In both of the two subjects that I feel most professionally confident in and have had the chance to teach (maths and computer science) you’ll see people sharing a common refrain. “If only I’d learnt {Complicated method/language/Mental Model} first, I’d have saved myself so much time.”
The most common examples I’ve seen of this are people who are convinced that teaching kids pointer juggling is gonna give them a stronger foundation for CS, or the cult of “Linear Algebra Done Right” (a book that I love, but that isn’t a good introduction to the field imo).
“Lies to children” exist for a reason, and while some might be skippable, many form useful intellectual scaffolds.
This is getting a bit into the weeds, but I find that this blog post mirrors my experience with Turing incomplete languages: https://neilmitchell.blogspot.com/2020/11/turing-incomplete-languages.html?m=1 (it also has the advantage of talking about a language that I’ve used in industry, and can personally attest to a little).
Even if there’s a sophisticated and more accurate way of describing the problem space, practicality can often push you back to a more general description with an extra hacky constraint (resource limits) shoved on top.
Is this type of course structure typical in US universities? It seems very strange to me that real analysis wouldn’t be a first semester class, or that such a large proportion of classes in a maths degree would be on anything but maths.
Im not too familiar with the Serbian market, but it seems like the government intervenes to keep mortgage rates down? (https://www.nbs.rs/en/scripts/showcontent/index.html?id=19204) (Not sure if this is still ongoing though).
The US mortgage market is really it’s own beast though, it’s very difficult for a lay person to figure out what the fair price of a mortgage should be, since it’s a path dependant integral (trading firms will spend millions on fancy products that do a weird Monte Carlo integral to get a supposedly fair price, but tbh I’m somewhat sussed out by them.)
It’s also worth noting that in the USA, people can pay upfront cash to decrease their interest rate for the life of the mortgage, although obviously that wouldn’t really take the brunt off of the genuinely higher base interest rates.