learn math or hardware
mesaoptimizer
Karma: 735
What do you think antidepressants would be useful for?
In my experience I’ve gone months through a depressive episode while remaining externally functional and convincing myself (and the people around me) that I’m not going through a depressive episode. Another thing I’ve noticed is that with medication (whether anxiolytics, antidepressants or ADHD medication), I regularly underestimate the level at which I was ‘blocked’ by some mental issue that, after taking the medication, would not exist, and I would only realize it previously existed due to the (positive) changes in my behavior and cognition.
Essentially, I’m positing that you may be in a similar situation.
Have you considered antidepressants? I recommend trying them out to see if they help. In my experience, antidepressants can have non-trivial positive effects that can be hard-to-put-into-words, except you can notice the shift in how you think and behave and relate to things, and this shift is one that you might find beneficial.
I also think that slowing down and taking care of yourself can be good—it can help build a generalized skill of noticing the things you didn’t notice before that led to the breaking point you describe.
Here’s an anecdote that might be interesting to you: There’s a core mental shift I made over the past few months that I haven’t tried to elicit and describe to others until now, but in essence it involves a sort of understanding that the sort of self-sacrifice that usually is involved in working as hard as possible leads to globally unwanted outcomes, not just locally unwanted outcomes. (Of course, we can talk about hypothetical isolated thought experiments and my feelings might change, but I’m talking about a holistic relating to the world here.)
Here’s one argument for this, although I don’t think this captures the entire source of my feelings about this: When parts of someone is in conflict, and they regularly are rejecting a part of them that wants something (creature comforts) to privilege the desires of another part of them that wants another thing (work more), I expect that their effectiveness in navigating and affecting reality is lowered in comparison to one where they take the time to integrate the desires and beliefs of the parts of them that are in conflict. In extreme circumstances, it makes sense for someone to ‘override’ other parts (which is how I model the flight-fight-fawn-freeze response, for example), but this seems unsustainable and potentially detrimental when it comes to navigating a reality where sense-making is extremely important.
This is a very interesting paper, thanks.
What was the requirement? Seems like this was a deliberate effect instead of a side effect.
which I know you object to
Buck, could you (or habryka) elaborate on this? What does Buck call the set of things that ARC theory and METR (formerly known as ARC evals) does, “AI control research”?
My understanding is that while Redwood clearly does control research, METR evals seem more of an attempt to demonstrate dangerous capabilities than help with control. I haven’t wrapped my head around ARC’s research philosophy and output to confidently state anything.
If you haven’t read CEV, I strongly recommend doing so. It resolved some of my confusions about utopia that were unresolved even after reading the Fun Theory sequence.
Specifically, I had an aversion to the idea of being in a utopia because “what’s the point, you’ll have everything you want”. The concrete pictures that Eliezer gestures at in the CEV document do engage with this confusion, and gesture at the idea that we can have a utopia where the AI does not simply make things easy for us, but perhaps just puts guardrails onto our reality, such that we don’t die, for example, but we do have the option to struggle to do things by ourselves.
Yes, the Fun Theory sequence tries to communicate this point, but it didn’t make sense to me until I could conceive of an ASI singleton that could actually simply not help us.
I dropped the book within the first chapter. For one, I found the way Bostrom opened the chapter as very defensive and self-conscious. I imagine that even Yudkowsky wouldn’t start a hypothetical 2025 book with fictional characters caricaturing him. Next, I felt like I didn’t really know what the book was covering in terms of subject matter, and I didn’t feel convinced it was interesting enough to continue the meandering path Nick Bostrom seem to have laid out before me.
Eliezer’s CEV document and the Fun Theory sequence were significantly more pleasant experiences, based on my memory.
we don’t give a shit about morality. Instead, we care about social norms that we can use to shame other people, masquerading under the banner of morality.
I think that basically all of moral cognition, actually.
Caring about others to me seems to be entirely separate from moral cognition. (Note that this may be a controversial statement and it is on my to-do list to make a detailed argument for this claim.)
If you are willing to share, can you say more about what got you into this line of investigation, and what you were hoping to get out of it?
Burnt out after almost an year of focusing on alignment research. I wanted to take a break from alignment-ey stuff and also desired to systematically fix the root causes behind the fact that I hit what I considered burn-out.
I don’t feel like I have many issues/baggage/trauma
I felt similar when I began this, and my motivation was not to ‘fix issues’ in myself but more “hey I have explicitly decided to take a break and have fun and TYCS seems interesting let’s experiment with it for a while, I can afford to do so”.
the former guides your exploration towards the most important domains, but the latter is necessary for a deep understanding of them.
Perhaps you meant vice versa? “Touching reality” seems more about details, while crucial considerations seems more about systematizing and “why?”.
Bonus conversation from the root of the tree that is this Twitter thread:
Eliezer Yudkowsky: Your annual reminder that you don’t need to resolve your issues, you don’t need to deal with your emotional baggage, you don’t need to process your trauma, you don’t need to confront your past, you don’t need to figure yourself out, you can just go ahead and do the thing.
Benquo: By revealed preferences almost no one wants to just go ahead and do the thing, even if they expect that things would go better for them if they did. Seems reasonable to try to figure out why that’s the case and how to change it, starting with oneself.
Benquo: Most of this trying will be fake or counterproductive, for the same reasons people aren’t doing the sensible object-level thing, but we don’t get to assume or pretend our way out of a problem, we just get to investigate and think about it and try out various promising solutions.
Given my experiences with both TYCS-like methods and parts-work methods (which is what Benquo is likely proposing one invest in, instead), I’d recommend people invest more in learning and using parts-work techniques first, before they learn and try to use TYCS-like techniques.
As of writing, I have spent about four months experimenting with the Tune Your Cognitive Strategies (TYCS) method and I haven’t gotten any visible direct benefits out of it.
Some of the indirect benefits I’ve gotten:
I discovered introspective ability and used that to get more insight about what is going on in my mind
I found out about the cluster of integration / parts-work based therapy techniques (such as Internal Family Systems), and have fixed some issues in the way I do things (eg. procrastinating on cleaning up my desk), and have also unraveled some deep issues I noticed (due to better introspective ability)
The biggest thing I’ve learned is that better introspective ability and awareness seems to be the most load-bearing skill underlying TYCS. I’m less enthusiastic about the notion that you can ‘notice your cognitive deltas’ in real-time almost all the time—this seems quite costly.
Note that Eliezer has also described that he does something similar. And more interestingly, it seems like Eliezer prefers to invest in what I would call ‘incremental optimization of thought’ over ‘fundamental debugging’:
EY: Your annual reminder that you don’t need to resolve your issues, you don’t need to deal with your emotional baggage, you don’t need to process your trauma, you don’t need to confront your past, you don’t need to figure yourself out, you can just go ahead and do the thing.
On one hand, you could try to use TYCS or Eliezer’s method to reduce the cognitive work required to think about something. On the other hand, you could try to use integration-based methods to solve what I would consider ‘fundamental issues’ or deeper issues. The latter feels like focusing on the cognitive equivalent of crucial considerations, the the former feels like incremental improvements.
And well, Eliezer has seemed to be depressed for quite a while now, and Maia Pasek killed herself. Both of these things I notice seem like evidence for my hypothesis that investing in incremental optimization of the sort that is involved in TYCS and Eliezer’s method seems less valuable than the fundamental debugging that is involved in integration / parts-work mental techniques, given scarce cognitive resources.
For the near future, I plan to experiment with and use parts-work mental techniques, and will pause my experimentation and exploration of TYCS and TYCS-like techniques. I expect that there may be a point at which one has a sufficiently integrated mind such that they can switch to mainly investing in TYCS-like techniques, which means I’ll resume looking into these techniques in the future.
“How could I have thought that faster?”
This is very interesting. I used to use org-roam and also experimented with other zettelkasten software over the past few years, but eventually it all grew very overwhelming because of the problem of updating notes. The bigger your note pile, the bigger the blocker (it seems to me at least) of updating your notes as you get a better understanding of reality.
Could you elaborate more on your setup, especially your knowledge base and how you use it?
Would a paper notepad have worked for you instead of a second device? What’s better with the device?
Answered here, but TLDR is joy of using the Scribe, aversion to using notepads, and a worry of losing logs of what I wrote if written on paper.
Based on a focusing-style attempt at understanding why, it seems like there’s a certain sense of pleasure and delight associated with using the Kindle Scribe to write (and read) on, in my mind, and a sense of inelegance and awkwardness associated with writing on paper. Whatever experiences, beliefs, and sense of aesthetics underlies this is probably the driving factor.
I did have access to notebooks and pens and whiteboards when I bought the second Kindle Scribe, but hesitated to use any of them for writing down my thoughts. One thing that comes to mind when I imagine such alternatives is that I fear losing a log of what I thought and wrote, and I didn’t imagine doing so if I wrote it on the Scribe.
couldn’t stand the kindle interface for books/notes
This is in comparison to using Emacs. When using Emacs as my interface for reading books and writing notes,
I can use a familiar UNIX file system to store my books as PDFs and EPUBs. I can easily back it up and interact with my collection using other tools I have (which is a real benefit of using a general purpose computing device). With the Kindle, creating and managing collections (an arbitrary category-based way of organizing your documents) is awkward enough (you need to select one book at a time and add it to a collection) given my experience with the last Kindle Scribe firmware that I just relied on the search bar to search for book titles.
When using Emacs, I can do a full text search of my notes file simply by pressing “s” (a keybind) and then typing a string. In contrast, while the notes written on the Scribe can be exported as PDFs, you don’t have the ability to search your notes. This wasn’t a dealbreaker for me, though, to be clear.
It is a bit hard to point at the things that make me want to use Emacs for it, because a load-bearing element is my desire to do everything in Emacs. Emacs has in-built documentation for its internals and almost every part of it is configurable—which means you can optimize your setup to be exactly as you like it. It feels like an extension of you, eventually.
This also somewhat drives my desire to use a simple and (eventually, given enough investment) understandable operating system that doesn’t shift beneath my feet. And given that both the interface and the operating system of the Kindle Scribe are opaque and (eventually) leaky abstractions, I feel less enthusiastic about investing my efforts into adapting myself to it.
The Kindle Scribe, IIRC has a 18 ms latency for rendering whatever you write on it using the Wacom-like pen. I believe that was the lowest latency you could get at the time (the Apple Pencil on iPads supposedly has a latency of 7-10 ms, but they use some sort of software to predict what you’ll do next, so that doesn’t count in my opinion).
I found the experience of writing notes on the Kindle Scribe great! It was about as effortless as writing on paper, with the advantage of being able to easily erase what I wrote with the flip of the Premium Pen. There are tail annoyances, but that didn’t seem to me to be worse than the tail annoyances of using physical pen and paper (whether gel ink, fountain pens, or ball point pens).
Writing on the Scribe does drain your battery faster. The number that comes to mind is that you can write on it continually for about eight hours before you wholly drain the battery, while if you only read on it, you don’t need to charge the Kindle for weeks.
I recommend the My Deep Guide Youtube channel for in-depth information about various e-readers and if you want to get up to speed on the current e-reader zeitgeist.
Yes, I believe that one can learn to entirely stop even considering certain potential actions as actions available to us. I don’t really have a systematic solution for this right now aside from some form of Noticing practice (I believe a more refined version of this practice is called Naturalism but I don’t have much experience with this form of practice).