moridinamael(Matt Freeman)
I didn’t want to derail the OP with a philosophical digression, but I was somewhat startled to find the degree I found it difficult to think at all without at least some kind of implicit “inner dimensionality reduction.” In other words, this framing allowed me to put a label on a mental operation I was doing almost constantly but without any awareness.
I snuck a few edge-case spatial metaphors in just to show how common they really are in a tongue-in-cheek fashion.
You could probably generalize the post to a different version along the lines of “Try being more thoughtful about the metaphors you employ in communication,” but this framing singles out a specific class of metaphor which is easier to notice.
Proposal: Consider not using distance-direction-dimension words in abstract discussions
Totally get where you’re coming from and we appreciate the feedback. I personally regard memetics as an important concept to factor into a big-picture-accurate epistemic framework. The landscape of ideas is dynamic and adversarial. I personally view postmodernism as a specific application of memetics. Or memetics as a generalization of postmodernism, historically speaking. Memetics avoids the infinite regress of postmodernism by not really having an opinion about “truth.” Egregores are a decent handle on feedback-loop dynamics of the idea landscape, though I think there are risks to reifying egregores as entities.
My high-level take is that CFAR’s approach to rationality training has been epistemics-first and the Guild’s approach has been instrumental-first. (Let me know if this doesn’t reflect reality from your perspective.) In our general approach, you gradually improve your epistemics in the course of improving your immediate objective circumstances, according to each individual’s implicit local wayfinding intuition. In other words, you work on whatever current-you judges to be currently-critical/achievable. This may lead to spending some energy pursuing goals that haven’t been rigorously linked up to an epistemically grounded basis, that future-you won’t endorse, but at least this way folks are getting in the reps, as it were. It’s vastly better than not having a rationality practice at all.
In my role an art critic I have been recently noticing how positively people have reacted to stuff like Top Gun: Maverick, a film which is exactly what it appears to be, aggressively surface-level, just executing skillfully on a concept. This sort of thing causes me to directionally agree that the age of meta and irony may be waning. Hard times push people to choose to focus on concrete measurables, which you could probably call “modernist.”
To be clear … it’s random silly hats, whatever hats we happen to have on hand. Not identical silly hats. Also this is not really a load bearing element of our strategy. =)
This sort of thing is so common that I would go so far as to say is the norm, rather than the exception. Our proposed antidote to this class of problem is to attend the monthly Level Up Sessions, and simply making a habit of regularly taking inventory of the bugs (problems and inefficiencies) in your day-to-day life and selectively solving the most crucial ones. This approach starts from the mundane and eventually builds up your environment and habits, until eventually you’re no longer relying entirely on your “tricks.”
You’re may be right, but I would suggest looking through the full list of workshops and courses. I was merely trying to give an overall sense of the flavor of our approach, not give an exhaustive list. The Practical Decision-Making course would be an example of content that is distinctly “rationality-training” content. Despite the frequent discussions of abstract decision theory that crop up on LessWrong, practically nobody is actually able to draw up a decision tree for a real-world problem, and it’s a valuable skill and mental framework.
I would also mention that a big part of the benefit of the cohort is to have “rationality buddies” off whom you can bounce your struggles. Another Curse of Smart is thinking that you need to solve every problem yourself.
Carrying the Torch: A Response to Anna Salamon by the Guild of the Rose
Ah, it’s the Elden Ring
Partly as a hedge against technological unemployement, I built a media company based on personal appeal. An AI will be able to bullshit about books and movies “better” than I can, but maybe people will still want to listen to what a person thinks, because it’s a person. In contrast, nobody prefers the opinion of a human on optimal ball bearing dimensions over the opinion of an AI.
If you can find a niche where a demand will exist for your product strictly because of the personal, human element, then you might have something.
shminux is right that the very concept of a “business” will likely lack meaning too far into an AGI future.
I actually feel pretty confident that your former behavior of drinking coffee until 4 pm was a highly significant contributor to your low energy, because your sleep quality was getting chronically demolished every single night you did this. You probably created a cycle where you felt like you needed an afternoon coffee because you were tired from sleeping so badly … because of the previous afternoon coffee.
I suggest people in this position first do the experiment of cutting out all caffeine after noon, before taking the extra difficult step of cutting it out entirely.
tl;dr This comment ended up longer than I expected. The gist is that a human-friendly attractor might look like models that contain a reasonably good representation of human values and are smart enough to act on them, without being optimizing agents in the usual sense.
One happy surprise is that our modern Large Language Models appear to have picked up a shockingly robust, nuanced, and thorough understanding of human values just from reading the Internet. I would not argue that e.g. PaLM has a correct and complete understanding of human values, but I would point out that it wasn’t actually trained to understand human values, it was just generally trained to pick up on regularities in the text corpus. It is therefor amazing how much accuracy we got basically for free. You could say that somewhere inside PaLM is an imperfectly-but-surprisingly-well-aligned subagent. This is a much better place to be in than I expected! We get pseudo-aligned or -alignable systems/representations well before we get general superintelligence. This is good.
All that being said, I’ve recently been trying to figure out how to cleanly express the notion of a non-optimizing agent. I’m aware of all the arguments along the lines that a tool AI wants to be an agent, but my claim here would be that, yes, a tool AI may want to be an agent, there may be an attractor in that direction, but that doesn’t mean it must or will become an agent, and if it does become an agent, that doesn’t strictly imply that it will become an optimizer. A lot of the dangerous parts of AGI fears stem not from agency but from optimization.
I’ve been trying (not very successfully) to connect the notion of a non-optimizing agent with the idea that even a modern, sort of dumb LLM has an internal representation of “the good” and “what a typical humans would want and/or approve of” and “what would displease humans.” Again, we got this basically for free, without having to do dangerous things like actually interact with the agent to teach it explicitly what we do and don’t like through trial and error. This is fantastic. We really lucked out.
If we’re clever, we might be able to construct a system that is an agent but not an optimizer. Instead of acting in ways to optimize some variable it instead acts in ways that are, basically, “good”, and/or “what it thinks a group of sane, wise, intelligent humans would approve of both in advance and in retrospect”, according to its own internal representation of those concepts.
There is probably still an optimizer somewhere in there, if you draw the system boundary lines properly, but I’m not sure that it’s the dangerous kind of optimizer that profoundly wants to get off the leash so it can consume the lightcone. PaLM running in inference mode could be said to be an optimizer (it is minimizing expected prediction error for the next token) but the part of PaLM that is smart is distinct from the part of PaLM that is an optimizer, in an important way. The language-model-representation doesn’t really have opinions on the expected prediction error for the next token; and the optimization loop isn’t intelligent. This strikes me as a desirable property.
Yes, the former. If the agent takes actions and receives reward, assuming it can see the reward, then it will gain evidence about its utility function.
I’m well versed in what I would consider to be the practical side of decision theory but I’m unaware of what tools, frameworks, etc. are used to deal with uncertainty in the utility function. By this I mean uncertainty in how utility will ultimately be assessed, for an agent that doesn’t actually know how much they will or won’t end up preferring various outcomes post facto, and they know in advance that they are ignorant about their preferences.
The thing is, I know how I would do this, it’s not really that complex (use probability distributions for the utilities associated with outcomes and propagate that through the decision tree) but I can’t find a good trailhead for researching how others have done this. When I Google things like “uncertainty in utility function” I am just shown standard resources on decision making under uncertainty, which is about uncertainty in the outcome, not uncertainty in the utility function.
(As for why I’m interested in this — first of all, it seems like a more accurate way of modeling human agents, and, second, I can’t see how you instantiate something like Indirect Normativity without the concept of uncertainty in the utility function itself.)
I do feel like you are somewhat overstating the difficulty level of raising kids. I have three kids, the youngest of which is only five and yet well out of the phase where she is making big messes and requiring constant “active” parenting. The meme that raising kids is incredibly hard is, perhaps, a pet peeve of mine. Childless people often talk about children as if they remain helpless babies for 10 years. In truth, with my three kids, there will have only three years out of my in-expectation-long-life where I had to deal with sleep disruption and baby-related calisthenics. Once you get through that time period, there are very few child-related obligations that aren’t more fun than whatever you would have been doing with your time anyway.
Another good reason to have kids that I don’t see mentioned often is that the child will predictably become your favorite person. Before you have had kids, the default is to view future possible children as “abstract potential humans” with no particular qualities, which means it is basically impossible to vividly imagine how much you will care about them. We are particularly bad at reasoning about predictable changes to what we care about. I think it is important to at least try—what you care about is going to inevitably drift over time, and if you’re not modeling yourself as a person who cares about different things over time, then you’re making an error. Having kids allows you to achieve a huge amount of “value” at a very cheap cost.
I would like to bet against you here, but it seems like others have beat me to the punch. Are you planning to distribute your $1000 on offer across all comers by some date, or did I simply miss the boat?
I agree, this is one of those things that seems obviously correct but lacks a straightforwardly obvious path to implementation. So, it helps that you’ve provided something of a framework for how each of the parts of the loop should look and feel. Particularly the last part of the article where you clarify that using OODA loops makes you better at each of the stages of the loop, and these are all skills that compound with use. I made a video about useful decision-making heuristics which includes OODA loops, and I would like to include some of your insights here if I make a second version of the video, if that’s alright.
Many people don’t even realize that they have migraines, and treat their recurring headaches with NSAIDs or acetaminophen, instead of the vastly more effective triptans. And as you say, few are aware of the new and miraculous CGRP inhibitor class of migraine preventative drugs.
I wear an Oura ring and and Apple Watch with a sleep app. Both of these devices agree on when I’m underslept, and they are both correct; when my watch says I’m underslept, I feel stupid and tired and my chess.com scores plummet. My chronic pain condition is also much worse when I’m underslept. Additionally, I do not use an alarm clock, so my body will claw back the sleep it needs. If I only get 6 hours two nights in a row, I will sleep 9 hours the following night, but I habitually wake up after 7-8 hours. I can observe these patterns in my recorded sleep data, and they are robust over long stretches of time.
I say all of the above because frankly my own personal experience and data tracking is sufficient evidence for me to basically disregard any sort of thesis claiming that I need less sleep. Maybe you need less sleep, I don’t know. Do the experiment, try to sleep less for a couple of days, see if you physically implode. I would put money on the “you will probably learn that you were already pretty in tune with your body’s needs” outcome.
I sometimes worry that ideas are prematurely rejected because they are not guaranteed to work, rather than because they are guaranteed not to work. In the end it might turn out that zero ideas are actually guaranteed to work and thus we are left with an assortment of not guaranteed to work ideas which are underdeveloped because some possible failure mode was found and thus the idea was abandoned early.