Still haven’t heard a better suggestion than CEV.
TristanTrim
Karma: 447
Yeah. I think because of (3) there might be a perverse incentive to seek (1) at the detriment of (2). What do you think?
Reading this post inspires me to dream of two ideas in response:
Schelling points, rallying, hyperstition bait: A dream of an idea is not necessarily worthless, even if it is empty, if it can successfully serve as a Schelling point or rallying point drawing people with a shared desire to work on developing a useful idea to focus on a shared idea. The thing that drew them in to concerted focus need not itself be a useful idea if it drew them into a useful configuration for producing some other useful idea or ideas.
Sprawling, cascading, detailed ideas: Sometimes I go to write down a dream of the idea and it cascades into other dreams of ideas. The breadth and depth of the idea seems deep enough that the act of writing it down to exorcise it is a lengthy endeavour, and doesn’t immediately show the idea as useful or not. I guess the “write it down” test shows you when your idea is a simple empty dream of an idea, but for more complicated dreams of ideas, you need to invest more resources if you want to determine if the idea is useful or not.
I do not understand how “interesting or amusing factoids” helps me tactically, or do and finish tasks well.
It’s the “good ideas” aspect. Good ideas can be good because their useful, or just because they make you feel good. People feeling good is the ultimate terminal value imo. But honestly that sentence wasn’t the most salient thing to me while I was responding. If you feel it should have been I am sorry.
how does Anki reviewing or better recall help with integration?
Continuing with your example of online content creation facts for promoting a videography business, The facts have some relation to aspects of your strategy, but different aspects of your strategy will be salient to you on different days, so by reviewing that fact on different days you increase the chance of having the relevant strategy context in mind when also bringing the relevant fact to mind so that you can see how it applies to your strategy. That is assuming the fact has any relevance to your strategy. If it is indeed completely useless, then yeah, it was a waste of time to write it down, but I don’t think you can really get a sense of what things are going to be useful to you without writing down things and they checking, and remembering, whether they turned out to be useful or not.
( EDIT: Oh no, I’m responding to the same post 3 months apart without realizing it! That’s embarrassing. Sorry. )
Thanks for writing this!
It seems you are further than me in exploring math fields. I’d love pointers on what to study and how to study it!
My current math book reading list:
Skillicorn—Understanding High-Dimensional Spaces
Kosniowski—A first course in algebraic topology
Goldblatt—Topoi: The Categorical Analysis of Logic
Hutter—Universal Artificial Intelligence
But of course it’s so much effort to move through this kind of material and doesn’t give me any legible advantage in seeking funding, fellowship, or roles, so it’s hard to justify prioritizing it right now.
Neither field mentions this almost ever, and it’s unclear to me how much insight is being lost due to this.
I wonder about this kind of thing. It motivates me to want to find a formalism that is “one big, all-encompassing generalization that unites these frameworks.” (This is a red flag for being a crackpot, which is concerning, but it’s also a green flag for discovering useful new paradigms, so the whole situation seem a bit fraught.) I feel like if I’m going to continue down this path I should study more sociology to understand how fields develop, branch, and merge.
The thing I’m mainly focused on right now isn’t just systems evolving over time, but more specifically systems that funnel reality towards a subset of possible outcomes. I think “time” is a baked in requirement for such a system, but maybe the thing I care about is more like “influence” or “causality” and “time” seems to imply some specific topology over causality that may not be essential to what I’m interested in.
The “timeless” focus is cool to know about. It makes sense.
Oh, that’s a really interesting answer, like the difference is the difference between a named and anonymous function. I do think there is a kind of important semiotic power in naming things, so I can understand wanting to avoid that, but I also feel pretty comfortable writing a post and then slapping a random name on it based on how the vibe turned out. This is what I just did with Ball+Gravity has a “Downhill” Preference. It started as a quick take, but then became a long take, so I copied and pasted it into a post and gave it a name. That’s also what inspired this question.
I think I agree. I’m imagining short takes fill the role of quick, twitter like back and forth of idea snippits, and general questions like the one I just posed, but it seems like you can write entire article length content in them which makes me wonder what different people think the distinction should be.
I think this story might be a useful bit of propaganda for convincing people who are not already feeling future shock to start feeling it, which may be useful for getting political support.
Looking at the actual object level, and setting aside the massive complicated assumption “If all goes well”, I think this is an unfair perspective, because “If all goes well”, than AIs care about the wellbeing of humans and humanity, in which case there will be an incomprehensible collective of incomprehensible intelligence devoted to solving the problem of making humans feel comfortable adjusting to the future environment they now find themselves in.
It’s the marginal worlds between “things go well” and “things go poorly” where future shock is a concern.
If you haven’t already, maybe look at Bostrom’s Deep Utopia. I think his exploration of the “things go well” idea is quite good, although the format of the book seems optimized to amuse rather than to inform in an organized and efficient manner. I’m not sure I would have made the same decision.
I think “autopotency” is a relevant concept here. Moving from a “post-instrumental utopia” to a “plastic utopia” we would expect people to see people beginning to modify themselves in deep, repeatable ways that solve the issues of future shock.
How long do you think something should be before it is no longer a quick take and should instead be a top level post? Or is it not about the length? Maybe it’s about the amount of research and editing that goes into it?
This is heartwarming. It would be fun if my nephews asked me questions like this, but I think they have learned my answers are long winded and difficult to understand and sometimes have unsettling implications. So they would much rather learn about the world by watching people on youtube doing experiments but not calling it science and acting silly and rude or exploring video game world. I think it’s probably fine.
Thanks for writing this out and for helping inspire the younger generation. I’m hoping we can give them a future to explore and be creative in.
Ball+Gravity has a “Downhill” Preference
I like this post. I like the idea of bureaucracies as code that needs regular refactoring. I like acknowledgement of the amount of work refactoring is, but that the alternative is much worse. I like the focus on incentives, rather than rules. Thank you for this post.
A contradiction that isn’t a contradiction:
I hold both of these views:
More money should be directed to improving AI policy than is going into technical AI alignment.
I want to get paid to work on technical AI alignment.
Why might this seem like a contradiction? Either I should think that more money should be put into technical AI alignment so I and other people can get paid to do it, or I should conclude that that AI policy is more important and try to work on that instead.
Why do I believe this is not actually a contradiction? In my worldview, AI is a very important and potentially existentially dangerous technology, and the current shepherds of it’s development are not handling their responsibility with commensurate wisdom. AI policy is then, the more important and constrained focus. But I do not believe that technical AI alignment should be completely forgotten, and I do not believe I have the aptitude or desire to do policy work. I intrinsically like research and theory building.
I wonder how many other people feel the way I feel. It would be quite a problem if it was the majority of us.
I’ve never heard of 4E before, but I checked the wikipedia page and it sounds cool and like it relates to my ontology I’ve been trying to develop called “outcome influencing systems” (OISs), particularly in my observation that OISs are composed of one another and overlap with one another.
What would you suggest for someone who wants to engage more with 4E?
progress comes from building out a different ontology to the point where it can replace the old one. Good luck to anyone who’s trying to do that!
I’m trying to do that! And thanks! It’s much appreciated. I can’t tell if I’m succeeding and don’t expect other people can either, which is a kinda stressful place to be.
Question to people working on Technical AI Alignment: How are you currently making a living?
I want to, ideally, focus on Technical Alignment Research fulltime and feel a bit lost. Any advice or encouragement, or even discouragement, would be appreciated.
Edit: Here is johnswentworth’s answer to this question in 2021. I don’t know how much the space has changed since then.
I think there’s a perverse incentive around the learning and usage of math. This is majorly coloured by my experiences in calculus class where it seemed like most students were interested in memorizing how to use the formulas to get the correct answer to get good grades, without necessarily understanding anything about what the calculations they were doing represented or were used for.
Maybe the problem here is fully from Goodharting on grades, but I wonder if there may be a broader phenomenon.
There’s three objects here:
Understanding the symbolic language of some math
Understanding the point of that math: what does it relate to? What can it help you do? Where does it fit in the broader research community?
Social signalling of values and competence
The dynamic I’m hypothesizing is:
In general, (2) is more important than (1), and (1) is held in higher esteem than (2). Ideally people have a strong grasp of both (1) and (2), but people are intrinsically and extrinsically motivated to seek (3), and (2) is easier to fake than (1), so people are motivated to fake (2) and put their effort into signalling competence in (1) which would, ideally imply competence in (2), but doesn’t necessarily.
I feel this relates to what Grant Sanderson of 3b1b talked about in Math’s pedagogical curse. But of course I also worry this is something I’m imagining because I feel motivated to try to understand math that is too difficult for my level of skill and I want to rationalize away my incompetencies. Is it imposter syndrome or honest self knowledge?
What do you think? Is (2) more important than (1), or maybe it’s not important to be skilled at both, and we need people better at (1) and people better at (2)? Do you agree the dynamic I described exists, and does it feel common, or marginal? Or maybe my entire framing is flawed in some way?
Responding to “Misses the point” tag on
It kinda seems like the complaint of this post could be recontextualized as there being an impression that there is a retriever who wants a list of interesting and amusing factoids.
I think this maybe came across as a stronger sentiment than I meant it to be. Sorry.
I’m pretty interested in the model of a “future self” and “past self” or “retriever” and “submitter” who are trying to coordinate with each other. I’m not saying I’m correctly understanding the point of the post, but it seems like the problem of past self creating a bulk of poorly organized notes that future self doesn’t want to deal with can be looked at from different angles:
Past self was imprudently collecting ideas into notes like they were shiny stones picked up and put in a pocket and promptly forgotten about. (problem with submitter)
Past self is putting in work to build something but is let down by future selves inability to build on their work. (problem with retriever)
The system by which notes are getting from the past self to the future self is failing to connect the correct instances of submitting with the correct instances of receiving. (problem with indexer)
My above quote was looking at the first, problem with submitter, as the submitter having incorrect beliefs about what would be useful for the receiver. What might this incorrect belief look like? Possibly that the receiver will benefit just from lists of interesting and amusing factoids.
In response to “Seems Offtopic?” tag on
or maybe you could put factoids in an anki deck and review them periodically because doing so would bring you some sense of enjoyment or well being.
This was in response to this kind of idea:
I’ve instead burdened my future self with, what appears to be, rubbish.
It feels to me that you wouldn’t write something down if you thought it was rubbish. There was something about it that appealed to you, either because it seemed important or you had some other affection for the idea, but there is a mismatch between the sense of importance you felt writing it and later reading it.
At times in my life, I have put random ideas that I have affection for, like factoids or quotes, into an anki deck and reviewing them every morning. Thinking about it now, I might start the habit again. I wasn’t really trying to commit anything to memory like how anki is normally used, instead, I was using it to periodically remind myself of things. I really liked a few things about doing this:
It kept me connected to the context where the idea seemed useful, avoiding the phenomenon of looking at an old note and thinking “What does this even mean?” or “Why did I think this was important to write down?”
Reviewing the same ideas when I had different contexts let me notice different aspects of them and connect them to other ideas.
I could choose to delete notes if I no longer felt the sense of affection or usefulness that justified reviewing them, which gave me a better sense of what kinds of things I was trying to focus on and what things were ok to not focus on as much.
It made me very aware of the cost of trying to pick up ideas and make them part of my context when I was taking this kind of note, which helped the note taking version of myself better decide if things were worth writing down for general review, or if they should be archived in some other way, or just forgotten.
I don’t know if many good specific examples exist. I might point at gource as an example of a guise/view i would commonly use, except any tool I’ve actually used approaching from this direction has been clunky to the point of uselessness. It would need really good keybinding & UI like what is seen in video editing and 3d modelling software. Sorry if the description is super fuzzy. It’s because the idea is super fuzzy. Like I said, I abandoned this idea a long time ago because it is too ambitious & questionably possible.
Indeed, in many ways debuggers are much closer to what I wanted Naloe to be. Especially ones that work in the terminal with lots of nice keybindings for doing things quick, but they still very much keep the code stuck to this single klunky guise where it is text. I suppose you could call the callstack a different guise/view. That’s valid, but it’s a lot more constrained and klunky than the kind of flexible, mutable, visualization and interaction that I yearn for.