I operate by Crocker’s rules.
niplav
Karma: 2,733
The obsessive autists who have spent 10,000 hours researching the topic and writing boring articles in support of the mainstream position are left ignored.
It seems like you’re spanning up three different categories of thinkers: Academics, public intellectuals, and “obsessive autists”.
Notice that the examples you give overlap in those categories: Hanson and Caplan are academics (professors!), while the Natália Mendonça is not an academic, but is approaching being a public intellectual by now(?). Similarly, Scott Alexander strikes me as being in the “public intellectual” bucket much more than any other bucket.
So your conclusion, as far as I read the article, should be “read obsessive autists” instead of “read obsessive autists that support the mainstream view”. This is my current best guess—”obsessive autists” are usually not under much strong pressure to say politically palatable things, very unlike professors.
My best guess is that people in these categories were ones that were high in some other trait, e.g. patience, which allowed them to collect datasets or make careful experiments for quite a while, thus enabling others to make great discoveries.
I’m thinking for example of Tycho Brahe, who is best known for 15 years of careful astronomical observation & data collection, or Gregor Mendel’s 7-year-long experiments on peas. Same for Dmitry Belayev and fox domestication. Of course I don’t know their cognitive scores, but those don’t seem like a bottleneck in their work.
So the recipe to me looks like “find an unexplored data source that requires long-term observation to bear fruit, but would yield a lot of insight if studied closely, then investigate”.
I think the Diesel engine would’ve taken 10 years or 20 years longer to be invented: From the Wikipedia article it sounds like it was fairly unintuitive to the people at the time.
A core value of LessWrong is to be timeless and not news-driven.
I do really like the simplicity and predictability of the Hacker News algorithm. More karma means more visibility, older means less visibility.
Our current goal is to produce a recommendations feed that both makes people feel like they’re keeping up to date with what’s new (something many people care about) and also suggest great reads from across LessWrong’s entire archive.
I hope that we can avoid getting swallowed by Shoggoth for now by putting a lot of thought into our optimization targets
(Emphasis mine.)
Here’s an idea[1] for a straightforward(?) recommendation algorithm: Quantilize over all past LessWrong posts by using inflation-adjusted karma as a metric of quality.
The advantage is that this is dogfooding on some pretty robust theory. I think this isn’t super compute-intensive, since the only thing one has to do is to compute the cumulative distribution function once a day (associating it with the post), and then inverse transform sampling from the CDF.
Recommending this way has the disadvantage of not being recency-favoring (which I personally like), and not personalized (which I also like).
By default, it also excludes posts below a certain karma threshold. That could be solved by exponentially tilting the distribution instead of cutting it off (, otherwise to be determined (experimentally?)). Such a recommendation algorithm wouldn’t be as robust against very strong optimizers, but since we have some idea what high-karma LessWrong posts look like (& we’re not dealing with a superintelligent adversary… yet), that shouldn’t be a problem.
- ↩︎
If I was more virtuous, I’d write a pull request instead of a comment.
- ↩︎
There are several sequences which are visible on the profiles of their authors, but haven’t yet been added to the library. Those are:
Why Everyone (Else) Is a Hypocrite: Evolution and the Modular Mind (Kaj Sotala)
The Sense Of Physical Necessity: A Naturalism Demo (LoganStrohl)
Scheming AIs: Will AIs fake alignment during training in order to get power? (Joe Carlsmith)
I think these are good enough to be moved into the library.
Transfer Learning in Humans
My reason for caring about internal computational states is: In the twin prisoners dilemma[1], I cooperate because we’re the same algorithm. If we modify the twin to have a slightly longer right index-finger-nail, I would still cooperate, even though they’re a different algorithm, but little enough has been changed about the algorithm that the internal states that they’re still similar enough.
But it could be that I’m in a prisoner’s dilemma with some program that, given some inputs, returns the same outputs as I do, but for completely different “reasons”—that is, the internal states are very different, and a slight change in input would cause the output to be radically different. My logical correlation with is pretty small, because, even though it gives the same output, it gives that output for very different reasons, so I don’t have much control over its outputs by controlling my own computations.
At least, that’s how I understand it.
- ↩︎
Is this actually ECL, or just acausal trade?
- ↩︎
I don’t have a concrete usage for it yet.
The strongest logical correlation is −0.5, the lower the better.
For and , the logical correlation would be , assuming that and have the same output. This is a pretty strong logical correlation.
This is because equal output guarantees a logical correlation of at most 0, and one can then improve the logical correlation by also having similar traces. If the outputs have string distance 1, then the smallest logical correlation can be only 0.5.
Whenever people have written/talked about ECL, a common thing I’ve read/heard was that “of course, this depends on us finding some way of saying that one decision algorithm is similar/dissimilar to another one, since we’re not going to encounter the case of perfect copies very often”. This was at least the case when I last asked Oesterheld about this, but I haven’t read Treutlein 2023 closely enough yet to figure out whether he has a satisfying solution.
The fact we didn’t have a characterization of logical correlation bugged me and was in the back of my mind, since it felt like a problem that one could make progress on. Today in the shower I was thinking about this, and the post above is what came of it.
(I also have the suspicion that having a notion of “these two programs produce the same/similar outputs in a similar way” might be handy in general.)
Consider proposing the most naïve formula for logical correlation[1].
Let a program be a tuple of code for a Turing machine, intermediate tape states after each command execution, and output. All in binary.
That is , with and .
Let be the number of steps that takes to halt.
Then a formula for the logical correlation [2] of two halting programs , a tape-state discount factor [3], and a string-distance metric could be
The lower , the higher the logical correlation between and . The minimal value is .
If , then it’s also the case that .
One might also want to be able to deal with the fact that programs have different trace lengths, and penalize that, e.g. amending the formula:
I’m a bit unhappy that the code doesn’t factor in the logical correlation, and ideally one would want to be able to compute the logical correlation without having to run the program.
How does this relate to data=code?
- ↩︎
Actually not explained in detail anywhere, as far as I can tell. I’m going to leave out all motivation here.
- ↩︎
Suggested by GPT-4. Stands for joining, combining, uniting. Also “to suit; to fit”, “to have sexual intercourse”, “to fight, to have a confrontation with”, or “to be equivalent to, to add up”.
- ↩︎
Which is needed because tape states close to the output are more important than tape states early on.
- ↩︎
There’s this intro series by @Alex Lawsen.
That seems an odd motte-and-bailey style explanation (and likely, belief. As you say, misgeneralized).
From my side or theirs?
Huh. Intuitively this doesn’t feel like it rises to the quality needed for a post, but I’ll consider it. (It’s in the rats tail of all the thoughts I have about subagents :-))
(Also: Did you accidentally a word?)
What then prevents humans from being more terrible to each other? Presumably, if the vast majority of people are like this, and they know that the vast majority of others are also like this, up to common knowledge, I don’t see how you’d get a stable society in which people aren’t usually screwing each other a giant amount.
Prompted by this post, I think that now is a very good time to check how easy it is for someone (with access to generative AI) impersonating you to get access to your bank account.
On a twitter lent at the moment, but I remember this thread. There’s also a short section in an interview with David Deutsch:
So all hardware limitations on us boil down to speed and memory capacity. And both of those can be augmented to the level of any other entity that is in the universe. Because if somebody builds a computer that can think faster than the brain, then we can use that very computer or that very technology to make our thinking go just as fast as that. So that’s the hardware.
[…]
So if we take the hardware, we know that our brains are Turing-complete bits of hardware, and therefore can exhibit the functionality of running any computable program and function.and:
So the more memory and time you give it, the more closely it could simulate the whole universe. But it couldn’t ever simulate the whole universe or anything near the whole universe because it is hard for it to simulate itself. Also, the sheer size of the universe is large.
I think this happens when people encounter the Deutsch’s claim that humans are universal explainers, and then misgeneralize the claim to Turing machines.
So the more interesting question is: Is there a computational class somewhere between FSAs and PDAs that is able to, given enough “resources”, execute arbitrary programs? What physical systems do these correspond to?
Related: Are there cognitive realms? (Tsvi Benson-Tilsen, 2022)
Yes, I was interested in the first statement, and not thinking about the second statement.
I realized I hadn’t given feedback on the actual results of the recommendation algorithm. Rating the recommendations I’ve gotten (from −10 to 10, 10 is best):
My experience using financial commitments to overcome akrasia: 3
An Introduction to AI Sandbagging: 3
Improving Dictionary Learning with Gated Sparse Autoencoders: 2
[April Fools’ Day] Introducing Open Asteroid Impact: −6
LLMs seem (relatively) safe: −3
The first future and the best future: −2
Examples of Highly Counterfactual Discoveries?: 5
“Why I Write” by George Orwell (1946): −3
My Clients, The Liars: −4
‘Empiricism!’ as Anti-Epistemology: −2
Toward a Broader Conception of Adverse Selection: 4
Ambitious Altruistic Software Engineering Efforts: Opportunities and Benefits: 6