I am also interested in finding a space to explore ideas which are not well-formed. It isn’t clear to me that this is intended to be such a space. This may simply be due to my ignorance of the mechanics around here.
For not well-formed ideas, you can write a Quick Take (can be found by clicking on your profile name in the top right corner) or starting a dialogue if you want to develop the idea together with someone (can be found in the same corner).
I feel like there should exist a more advanced sequence that explains problems with filtered evidence leading to “confirmation bias”. I think the Luna sequence is already a great step in the right direction. I do feel like there is a lack of the equivalent non-fiction version, that just plainly lays out the issue. Maybe what I am envisioning is just a version of What evidence filtered evidence with more examples of how to practice this skill (applied to search engines, language models, someone’s own thought process, information actively hidden from you, rationality in groups etc.).
adult augmentation 2-3std for the average person seems plausible, but for the few +6std people on earth it might just give +0.2std or +0.3std, which tbc I think is incredibly worthwhile.
Such high diminishing returns in g based on genes seems quite implausible to me, but would be happy if you can point to evidence to the contrary. If it works well for people with average Intelligence, I’d expect it to work at most half as well with +6sd.
I am a bit confused why some of these theories would be so hard to test? It seems like some core pathways that seem like they wouldn’t be reversible even in naive stem cells under any circumstances (like transposons copying themselves successfully), could possibly be tested by checking if clones derived from older cells age faster or something along those lines? The same goes for children from older parents? (Not sure to which extent that test would be made harder by all the mechanisms keeping the germ line immortal)
I don’t know where anger fits into this. Also I should look at how these behaviors manifest in other animals.
Hypothesis based on the fact that status is a strong drive and people who are on the outer ends of that spectrum get classified as having a “personality disorder” and are going to be very resistant to therapy:
weak-status-fear==psychopathy: psychopathy is caused by the loop leading to fear of loosing status, being less strong than average or possibly broken. (psychopathy is Probably on a spectrum. I don’t see a reason why little of this feeling would be less optimal than none.)
strong-status-fear==(?histrionic personality disorder)
weak-status-seeking-loop==(?schizoid personality disorder)
strong-status-seeking-loop==(?narcissism)
Was thinking about Steven Byrnes agenda to figure out social drives and what makes a psychopath a psychopath. One clearly existing social drive that seemed to be a thing was “status-seeking” and “status-fear” (fear of loosing status). Both of these could themselves be made of several drives? The idea that status-seeking and status-fear are different came to me when trying to think of the simplest hypothesis explaining psychopathy and from introspecting that both of these feelings feel very different to me and distinct from other fears. These two could be made more mostly separate loops, but I can’t complicate my fake framework even more just yet.
If someone is interested, I’d write a post how to stress test this fake-framework and what I’d expect in the world where it is true or isn’t (Most interesting would be social drives that are distinct from the above? Or maybe they use some of the same sub-circuitry? Like jealousy seems obviously like it would fit under strong status fear, so histrionic personality would go with being more jealous)
Since I am already on the fancy note-taking train, I’d find examples of your actual note files way more interesting.
On my phone, rotating the screen by 180° quickly reverses the direction and then I rotate it back slowly.
I think from the perspective of a radical probabilist, it is very natural to not only have a word of where your current point estimate is at, but also have some tagging for the words indicating how much computation went into it or if this estimate already tries to take the listeners model into account also?
I misread your whole post by thinking your title implied “your post would question whether the entropy increased=> the post argues it decreases” and then I was reading sloppily and didn’t notice my error.
Also you should halt and reevaluate your intuitions if they lead you to believe there is a perpetual motion machine.
Photosynthesis? Most of the carbon is bound from CO2 by using sun exergy.
Cool post. I agree with the many-shot part in principle. It strikes me that in a few years (hopefully not months?), this will look naive in a similar way that all the thinking on ways a well boxed AI might be controlled look naive now. If I understand correctly, these kinds of simulations would require a certain level of slowing down and doing things that are slightly inconvenient once you hit a certain capability level. I don’t trust labs like OpenAI, Deepmind, (Anthropic maybe?) to execute such a strategy well.
If legibility of expertise is a bottleneck to progress and adequacy of civilization, it seems like creating better benchmarks for knowledge and expertise for humans might be a valuable public good. While that seems difficult for aesthetics, it seems easier for engineering? I’d rather listen to a physics PhD, who gets Thinking Physics questions right (with good calibration), years into their professional career, than one who doesn’t.
One way to do that is to force experts to make forecasts, but this takes a lot of time to hash out and even more time to resolve.
One idea I just had related to this: the same way we use datasets like MMLU and MMMU, etc. to evaluate language models, we use a small dataset like this and then experts are allowed to take the test and performance on the test is always public (and then you make a new test every month or year).
Maybe you also get some participants to do these questions in a quiz show format and put it on YouTube, so the test becomes more popular? I would watch that.
The disadvantage of this method compared to tests people prepare for in academia would be that the data would be quite noisy. On the other hand, this measure could be more robust to goodharting and fraud (although of course this would become a harder problem once someone actually cared about that test). This process would inevitably miss genius hedgehog’s of course, but maybe not their ideas, if the generalists can properly evaluate them.
There are also some obvious issues in choosing what kinds of questions one uses as representative.
It not being linked on Twitter and Facebook seems more like a feature than a bug, given that when I asked Gwern why a page like this doesn’t already exist, he wrote me he doesn’t want people to mock it.
> I really like the importance Tags, but what I would really like is a page
> where I can just go through all the posts ordered by importance. I just
> stumbled over another importance 9 post (iron rules) when I thought I had
> read all of them. Clicking on the importance tag, just leads to a page
> explaining the importance tag.
Yeah, that is a mix of ‘too hard to fix’ and ‘I’m not sure I want to
fix it’. (I don’t know how Hakyll works well enough to do it
’normally’, and while I think I can just treat it as a tag-directory,
like ‘meta/importance/1’, ‘meta/importance/2’ etc, that’s a little
awkward.) Do I *want* people to be able to go through a list of
articles sorted by importance and be able to easily mock it—avoiding
any actual substantive critique?
I went through Gwern’s posts and collected all the posts with importance 8 and higher as of 2024-09-04 in case someone else was searching for something like this.
10
9
8
The recent post on reliability and automation reminded me that my “textexpansion” tool Espanso is not reliable enough on Linux (Ubuntu, Gnome, X11). Anyone here using reliable alternatives?
I’ve been using Espanso for a while now, but its text expansions miss characters too often, which is worse than useless. I fiddled with Espanso’s settings just now and set the backend to Clipboard, which seems to help with that, but it still has bugs like the special characters remaining (“@my_email_shorthand” → “@myemail@gmail.com″).
In particular, I think you might need to catch many escape attempts before you can make a strong case for shutting down. (For concreteness, I mostly imagine situations where we need to catch the model trying to escape 30 times.)
So instead of leaving the race once the models start scheming against you, you keep going to gather more instances of scheming until you can finally convince people? As an outside reader of that story I’d just be screaming at the protagonists that clearly everyone can see where this is going where scheming attempt number 11 is just good enough to be successful. And in the worlds where we catch them 30 times successfully it feels like people would argue: this is clear evidence that the models aren’t “actually dangerous” yet, so let’s keep scaling “responsibly”.
There is probably a lot of variation between people regarding that. In my family meds across the board improved people’s sleep (by making people less sleepy during the day, so more active and less naps). When I reduced my medication from 70mg to 50mg for a month to test whether I still needed the full dose, the thing that was annoying the most was my sleep (waking up at night and not falling asleep again increased. Falling asleep initially was maybe slightly easier). Taking it too late in the afternoon is really bad for my sleep, though.
One confounder: depression/mania. Recently (the last ~two weeks) I have been having bad sleep (waking up 3-7 am and not feeling sleepy anymore (usually I sleep from midnight to 9). My current best guess is that the problem is that my life has been going too well recently, leading to a self-sustaining equilibrium where I have little sleep and mania. Reduced my medication today (~55mg instead of 70mg) which seems to have helped with the mania. I had another day with slight mania 1 month ago when sleeping little in order to travel to a conference, so in the future I’ll probably reduce my medication dose on such days. Took a friend describing his symptoms on too much medication for me to realize what is going on.