moridinamael(Matt Freeman)
So the joke is that Szilard expects the NSF to slow science down.
My interpretation of the joke is that the Szilard is accusing the NSF of effectively slowing down science, the opposite of their claimed intention. Personally I have found that the types of scientists who end up sitting in grant-giving chairs are not the most productive and energetic minds, who tend to avoid such positions. Still funny though.
Better New Year’s Goals through Aligning the Elephant and the Rider
Thanks for the questions. I should have explained what I meant by successful. The criteria we set out internally included:
Maintaining good attendance and member retention. Member attrition this year was far below the typical rate for similar groups.
Maintaining positive post-workshop feedback indicating members are enjoying the workshops (plus or minus specific critical feedback here and there). Some workshops were more well received than others, some were widely loved, some were less popular, but the average quality remains very positive according to user feedback. (We try to collect user feedback at the end of each workshop.)
Demonstrated improvement over time in the recurring workshops. For example, we observed increased fluency with decision theory in the decision-making workshops month to month.
We are happy with our metrics on all these fronts, above expectations, which is “highly successful” by my lights.
The workshops take the following format: Each Guild member is placed in a cohort group according to schedule compatibility upon joining. Let’s assume for the sake of this explanation that you are in the Wednesday night cohort. The landing page (the pages linked in the OP) for the workshop is posted Monday. You check the landing page, and you have until the following Wednesday (>1 week later) to complete the pre-workshop reading or exercises. You then join the workshop session for your cohort time slot via the Guild of the Rose Discord video chat. A cohort session leader guides the members through the in-session exercises and discussions. In the past the sessions lasted one hour but we have more recently been experimenting with 90 minute sessions to good effect. The typical attendance varies depending on the cohort, since some timezones have far fewer Guild members. Workshops sessions are broken out into smaller discussion groups if too many people show up.
It is funny that I kept the commentary at the start of the post short and refrained from talking too much about Guild goals and policies and details not immediately relevant to the workshop overview so that the whole post didn’t come off as an ad … and I still got accused of posting an ad, so I should have just gone for it and laid out all the results in detail. Oh well, next year.
Reflecting on the 2022 Guild of the Rose Workshops
I sometimes worry that ideas are prematurely rejected because they are not guaranteed to work, rather than because they are guaranteed not to work. In the end it might turn out that zero ideas are actually guaranteed to work and thus we are left with an assortment of not guaranteed to work ideas which are underdeveloped because some possible failure mode was found and thus the idea was abandoned early.
I didn’t want to derail the OP with a philosophical digression, but I was somewhat startled to find the degree I found it difficult to think at all without at least some kind of implicit “inner dimensionality reduction.” In other words, this framing allowed me to put a label on a mental operation I was doing almost constantly but without any awareness.
I snuck a few edge-case spatial metaphors in just to show how common they really are in a tongue-in-cheek fashion.
You could probably generalize the post to a different version along the lines of “Try being more thoughtful about the metaphors you employ in communication,” but this framing singles out a specific class of metaphor which is easier to notice.
Proposal: Consider not using distance-direction-dimension words in abstract discussions
Totally get where you’re coming from and we appreciate the feedback. I personally regard memetics as an important concept to factor into a big-picture-accurate epistemic framework. The landscape of ideas is dynamic and adversarial. I personally view postmodernism as a specific application of memetics. Or memetics as a generalization of postmodernism, historically speaking. Memetics avoids the infinite regress of postmodernism by not really having an opinion about “truth.” Egregores are a decent handle on feedback-loop dynamics of the idea landscape, though I think there are risks to reifying egregores as entities.
My high-level take is that CFAR’s approach to rationality training has been epistemics-first and the Guild’s approach has been instrumental-first. (Let me know if this doesn’t reflect reality from your perspective.) In our general approach, you gradually improve your epistemics in the course of improving your immediate objective circumstances, according to each individual’s implicit local wayfinding intuition. In other words, you work on whatever current-you judges to be currently-critical/achievable. This may lead to spending some energy pursuing goals that haven’t been rigorously linked up to an epistemically grounded basis, that future-you won’t endorse, but at least this way folks are getting in the reps, as it were. It’s vastly better than not having a rationality practice at all.
In my role an art critic I have been recently noticing how positively people have reacted to stuff like Top Gun: Maverick, a film which is exactly what it appears to be, aggressively surface-level, just executing skillfully on a concept. This sort of thing causes me to directionally agree that the age of meta and irony may be waning. Hard times push people to choose to focus on concrete measurables, which you could probably call “modernist.”
To be clear … it’s random silly hats, whatever hats we happen to have on hand. Not identical silly hats. Also this is not really a load bearing element of our strategy. =)
This sort of thing is so common that I would go so far as to say is the norm, rather than the exception. Our proposed antidote to this class of problem is to attend the monthly Level Up Sessions, and simply making a habit of regularly taking inventory of the bugs (problems and inefficiencies) in your day-to-day life and selectively solving the most crucial ones. This approach starts from the mundane and eventually builds up your environment and habits, until eventually you’re no longer relying entirely on your “tricks.”
You’re may be right, but I would suggest looking through the full list of workshops and courses. I was merely trying to give an overall sense of the flavor of our approach, not give an exhaustive list. The Practical Decision-Making course would be an example of content that is distinctly “rationality-training” content. Despite the frequent discussions of abstract decision theory that crop up on LessWrong, practically nobody is actually able to draw up a decision tree for a real-world problem, and it’s a valuable skill and mental framework.
I would also mention that a big part of the benefit of the cohort is to have “rationality buddies” off whom you can bounce your struggles. Another Curse of Smart is thinking that you need to solve every problem yourself.
Carrying the Torch: A Response to Anna Salamon by the Guild of the Rose
Ah, it’s the Elden Ring
Partly as a hedge against technological unemployement, I built a media company based on personal appeal. An AI will be able to bullshit about books and movies “better” than I can, but maybe people will still want to listen to what a person thinks, because it’s a person. In contrast, nobody prefers the opinion of a human on optimal ball bearing dimensions over the opinion of an AI.
If you can find a niche where a demand will exist for your product strictly because of the personal, human element, then you might have something.
shminux is right that the very concept of a “business” will likely lack meaning too far into an AGI future.
I actually feel pretty confident that your former behavior of drinking coffee until 4 pm was a highly significant contributor to your low energy, because your sleep quality was getting chronically demolished every single night you did this. You probably created a cycle where you felt like you needed an afternoon coffee because you were tired from sleeping so badly … because of the previous afternoon coffee.
I suggest people in this position first do the experiment of cutting out all caffeine after noon, before taking the extra difficult step of cutting it out entirely.
tl;dr This comment ended up longer than I expected. The gist is that a human-friendly attractor might look like models that contain a reasonably good representation of human values and are smart enough to act on them, without being optimizing agents in the usual sense.
One happy surprise is that our modern Large Language Models appear to have picked up a shockingly robust, nuanced, and thorough understanding of human values just from reading the Internet. I would not argue that e.g. PaLM has a correct and complete understanding of human values, but I would point out that it wasn’t actually trained to understand human values, it was just generally trained to pick up on regularities in the text corpus. It is therefor amazing how much accuracy we got basically for free. You could say that somewhere inside PaLM is an imperfectly-but-surprisingly-well-aligned subagent. This is a much better place to be in than I expected! We get pseudo-aligned or -alignable systems/representations well before we get general superintelligence. This is good.
All that being said, I’ve recently been trying to figure out how to cleanly express the notion of a non-optimizing agent. I’m aware of all the arguments along the lines that a tool AI wants to be an agent, but my claim here would be that, yes, a tool AI may want to be an agent, there may be an attractor in that direction, but that doesn’t mean it must or will become an agent, and if it does become an agent, that doesn’t strictly imply that it will become an optimizer. A lot of the dangerous parts of AGI fears stem not from agency but from optimization.
I’ve been trying (not very successfully) to connect the notion of a non-optimizing agent with the idea that even a modern, sort of dumb LLM has an internal representation of “the good” and “what a typical humans would want and/or approve of” and “what would displease humans.” Again, we got this basically for free, without having to do dangerous things like actually interact with the agent to teach it explicitly what we do and don’t like through trial and error. This is fantastic. We really lucked out.
If we’re clever, we might be able to construct a system that is an agent but not an optimizer. Instead of acting in ways to optimize some variable it instead acts in ways that are, basically, “good”, and/or “what it thinks a group of sane, wise, intelligent humans would approve of both in advance and in retrospect”, according to its own internal representation of those concepts.
There is probably still an optimizer somewhere in there, if you draw the system boundary lines properly, but I’m not sure that it’s the dangerous kind of optimizer that profoundly wants to get off the leash so it can consume the lightcone. PaLM running in inference mode could be said to be an optimizer (it is minimizing expected prediction error for the next token) but the part of PaLM that is smart is distinct from the part of PaLM that is an optimizer, in an important way. The language-model-representation doesn’t really have opinions on the expected prediction error for the next token; and the optimization loop isn’t intelligent. This strikes me as a desirable property.
Yes, the former. If the agent takes actions and receives reward, assuming it can see the reward, then it will gain evidence about its utility function.
My advice here: doing any amount of editing for ums, ahs and fillers will take, at a minimum, the length of the entire podcast episode, since you have to listen to the whole thing. This is more than a trivial inconvenience. It’s a pretty serious inconvenience! Instead, don’t do any editing of this kind (unless there was a real interruption or sound issue) and just train yourself not to use excessive filler words. People really don’t mind a reasonable amount of filler words, anyway. They are sort of like verbal punctuation, and the idea that you should not use them ever is a weird artifact of the way public speaking is taught.
You should edit your podcast for sound quality. Remove hiss, use compression and loudness-matching to even out the volume.