Assumption 3. A listener is said to have minimally consistent beliefs if each proposition X has a negation X*, and P(X)+P(X*)≤1.
One thing that’s interesting to me is that this is assumption is frequently not satisfied in real life due to underspecification, e.g. P(I’m happy) + P(I’m not happy) ≥ 1 because “happy” may be underspecified. I can’t think of a really strong minimal example, but I feel like this pops up a lot of discussions on complex issues where a dialectic develops because neither thesis nor antithesis capture everything and so both are underspecified in ways that make their naive union exceed the available probability mass.
To say a little more, programming is the thing that resonates for me most strongly with the above description of baking as not ritual. I notice it because there are lots of people who learn to program by ritual. They do well enough to get jobs and become my coworkers, but then they also write code full of cruft because they are programming by ritual rather than through deep understanding.
The classic examples of this that come to my mind are things like people thinking the iterator variable on a for loop must be named i or else it won’t work, or struggling with sed because they don’t realize they can pick the separator character on each invocation, or thinking they need [technology X] because they heard successful businesses use [technology X].
I like it. FWIW, I find this is exactly the same line that must be crossed to become an expert at many things. Some examples that come to mind, there are probably more I just don’t have enough experience with to be able to be sure they are examples:
Things that seem like anti-examples, i.e. things that are not in this category with baking:
rock climbing (although there is a lot of skill involved, you don’t have to understand the gears to succeed, you can learn to intuitively make efficient motions that move you up without understanding too much about why or how that happens)
meditating (although lots of people get into the gears, plenty of people successfully meditate either without a deep understanding of what they’re doing or possibly actively mistaken theories that work anyway)
Again, I’m sure there are more, and I think for all of these there’s plenty of room to argue on the margins.
If I had to guess at what the divider is it’s something like how robust the ontological abstractions around the topic are at levels that matters for the task. The more robust, the less like baking, the more fragile and leaky, the more like baking.
Seems like a type error to me, like saying crying is an emotional state rather than a thing that happens in response to emotions or other things, or like saying finding out my friend lied to me is an emotion rather than a thing that causes emotions. Okayness is an understanding of just that which is, and emotions are an expression of that but are also not okayness itself.
Keep a record of tasks you’ve decided to do.
So I really liked GTD when I read it back in the mid 00s, and it’s stuck with me in deep ways. In particular, this insight, but I wouldn’t describe it this way. I’d say something like
use a system.
I’ve phrased it this way because there’s more to it than remembering tasks to do. To me the whole core of the book is that building systems that support us set us free. When you have a system you can trust to help you do whatever it is that’s import to you, you can relax and get better performance at the same time because you have a system you trust and it does a better job than your brain could on its own.
Fifteen plus years of systematic use of systems changes a person, so my “systems” don’t look like they did when I first read GTD. In the beginning I had too much system because I needed extra system in order to be able to trust it. I didn’t know quite where the line was between me and the system and which could handle what best. But over time it’s evolved into something that works quite well. I don’t forget to do anything important, and my mind is not often busy worrying about the unfinished stuff.
So I’ve gone from specialized tools that were highly customizable to basically just a combo of Gmail, Google Keep, Google Calendar, and Pocket. I dump stuff in, it gets done, it goes out, and I regularly reprocess and review stuff (after doing it enough times, I morphed from scheduled reviews to continuous review). It works for me and the work I have, but something else might work for someone else. There’s lots of little details to getting it right, and I think each person has to discover a lot of what works for themselves because we’re all slightly differently shaped. Examples of other people’s systems are great for inspiration, though!
And interestingly, I find it relates a lot to my schedule. My schedule is just one more system designed to support my life and help make sure I do the things I intended without having to make choices or remember everything all the time. I long resented having a schedule, right up until I realized my schedule could support me rather than force me to do things, and that shift in mindset towards a support system to help rather than a coercive outside imposition made all the difference to increasing my happiness and success at shaping the world.
Nothing is quite so annoying as seeing another person do the thing you wish you yourself would not do.
I usually just call these “free variables” by analogy to that term’s usage in mathematics/statistics, but “aether” seems good too since, if nothing else, when I say “free variable” to people who don’t know the jargon they just sort of roll over it whereas they’d have a much harder time doing that if I said “aether”.
I might give this a read, but based on the abstract I am concerned that “has a perspective” is going to be one of those properties that’s so obvious that its presence can be left to human judgment, but that nonetheless contains all the complexity of the theory.
I read it more as pointing towards something like embedded agency.
Last summer when I was at the EA Hotel for TAISU I got the most value out of doing something similar. I’d host a session to “workshop” an idea I had, and it was roughly 20 minutes of setting it up and 40 minutes of back and forth with people pointing things out, stating objections, asking for clarifications, etc.. It was less structured than your approach, and I quite like this idea because it creates a level of safety my approach did not because it effectively bans the kinds of criticism (at least for the course of the conjecture workshop) that people sometimes jump to that also shut down fruitful idea development.
Thanks for sharing!
I have some experience with “okayness”, so maybe I can say a few useful things.
It’s hard to explain what it’s like to be okayness, which is I think a bit more accurate way of expressing the idea of “it’s okay” or “everything is okay”. I think it’s so hard because when we try we run up against all the way words create separation between this and not that, and separation, categories, and ontology fundamentally separate you from okayness because you have to make a judgement about what is this and what is that and making that judgement requires some fundamental mental motion of measuring how much something is something, but that’s antithetical to what okayness is like.
Okayness (I often use the word “perfection” as my pointer, although sometimes it appears to me as “enoughness”, and others have used “naturalness” or “Buddha nature” or some other term) exists prior to that measurement, prior to feedback, prior to our self-awareness.
You’re right that okayness is not a thing, and its thingness seems to be an after-the-fact attempt by our minds to label this non-dual experience. It’s not an emotional state, although there are often emotions felt while being it, nor is it really making any epistemology claims in itself because while being it the mechanisms of epistemology can’t function, yet strangely this doesn’t matter because you can only be what is when there is no map, so you have no need of epistemology because there is no map to be kept correlated with the territory, there is just territory.
I was going to come here to say to look at SENS. I feel like they do have an idea of a “blueprint” for solving aging and are working on executing on it, and they are from my outside perspective working in a way similar to MIRI: they have a problem to solve, some ideas about how to do it, and some research agendas they are executing on to move them towards the answer.
Radiation out from SENS is a constellation of folks working on this stuff. I’m not sure, though, that these folks also lack a blueprint, only that they work in field where they can’t easily publicly claim to have one.
I agree. I’m generally okay with the order (oracles do seem marginally safer than agents, for example, and more restrictions should generally be safer than less), but also think the marginal amount of additional safety doesn’t matter much when you consider the total absolute risk. Just to make up some numbers, I think of it like choosing between options that are 99.6%, 99.7%, 99.8%, and 99.9% likely to result in disaster. I mean of course I’ll pick the one with a 0.4% chance of success, but I’d much rather do something radically different that is orders of magnitude safer.
I also think it’s fair to have low expectations here. Although I generally like SEP, I also have experienced Gell-Mann moments with it enough times that now I think of it differently.
It’s not really like a regular encyclopedia written by anonymous authors with a “view from nowhere”. Each article has an author, those authors are allowed to have clear authorial bias, the whole thing has lots of selection bias (articles only exist because someone was willing to write them and they do count as publications though they are strangely influential and not at the same time because they are read by lots of people but most of those people avoid citing them and instead cite the work referenced by the SEP article based on my anecdotal data), and as a result you get things like, for example, articles written by people only because they are against some position not for it but no proponent of the position had written the article and those articles often fail to pass the ITT.
Combine this with the already low expectations around writing about AI safety topics in general and it makes it nicely surprising that this one turned out pretty good.
There’s nothing necessarily wrong with what you’re looking for here, but I think it’s worth pointing out that benefits or gains from meditation are antithetical to how many dedicated meditators think of their practice, and in fact progress is eventually hampered by the very idea of progress (or so many traditions claim).
A book I like to recommend to people interested in getting started in meditation is “A Path With Heart” by Jack Kornfield. It’s written by an author who is very decided not a rationalist and it’s filled with lots of references to supernatural things, but it’s also a very kind and gentle introduction to meditation and wider practice of the way. If you think of the supernatural stuff as metaphors rather than claims about physical reality, I think it can be quite helpful and teaches a lot of techniques and gives some good motivations for why and how they are useful.
Since you are aiming towards philosophy with this one, I’ll share something about my intuitions around emptiness (as opposed to form, in Buddhist Madhyamaka philosophy) as they relate to open sets in topology.
In my mind it has been fruitful to think of emptiness like openness and relate the two, specifically thinking of emptiness as describing the same aspect of reality that make “open” a good intuitive label for “open sets”. This has helped me understand what is pointed at by “emptiness” by understanding it as “openness” and reusing my topology intuitions as a grounding point.
I’m not sure I can be more precise, so I’ll have to leave it at that and hope it’s helpful.
Lots has been said within these servers on finding ways to say new exciting correct things. Or correcting things many think are right, but didn’t know were actually wrong. This is a relentlessly optimistic perspective. I have gotten grand mileage by merely trying to hold back from saying or acting on information or ideas I know are wrong, as opposed to worrying about thinking better, or not making mistakes, or being more rational when I am concerned with the truth. This other part is the meaty part, the intense part—the struggle to stop mouthing the words, to stop myself from willingly burning more money or time.
I think this perspective is appealing and useful advice, but only up to a point. At some point you get diminishing returns from ceasing to punch yourself in the face, as it were, and you find you can’t do the things that matter to you without sticking your neck out and risking being wrong by trying to say something new that might be correct.
I find nothing disagreeable in your post other than what I read as a sentiment that maybe just stop saying wrong things and you can improve. Sometimes you’ve already stopped doing that so much that you have to start saying some wrong things.
(For what it’s worth, my comment basically comes down to saying don’t forget about the law of equal and opposite advice.)
In my mind it’s something like you need:
strong interest in solving AI safety
being okay with breaking new ground and having to figure out what “right” means
strong mathematical reasoning skills
decent communication skills (you can less rely on strong existing publication norms and may have to get more creative to convey your ideas than in other fields)
the courage and care to work on something where the stakes are high and if you get it wrong things could go very badly
I think people tend to emphasize the technical skills the most, and I’m sure other answers will offer more specific suggestions there, but I also think there’s an import aspect of having the right mindset for this kind of work such that a person with the right technical skills might not make much progress on AI safety without these other “soft” skills.
Am I the right “kind” of researcher for working in AI Safety? Here, my main intuition is that the field needs more “theory-builders” than “problem-solvers”, to take the archetypes of Gower’s Two Cultures of Mathematics. By that I mean that AI Safety has not yet cristallize into a field where the main approaches and questions are well understood and known. Almost every researcher has a different perspective on what is fundamental in the field. Therefore, the most useful works will be the ones that clarify, deconfuse and characterize the fundamental questions and problems in the field.
To add on to this, it also means it’s going to be somewhat hard to know if you’re right kind of researcher or not because the feedback cycle is long and you may be doing good work but it’s work that will take months or years to come together in a way that can be easily evaluated by others.
This doesn’t mean it all looks maximally like this. This is less of an issue with, say, safety research focused on machine learning than safety research focused on theoretical AI systems we don’t know how to build yet or safety research focused on turning ideas about what safety looks like into something mathematical precise enough to build.
Thus a corollary of this answer might be something like “you might be the right kind of researcher only if you’re okay with long (multi-year) feedback cycles”.