I guess I am partially trying to reject the frame of “contributing to all the different research directions that MIRI is pursuing.”
Ask not what you can do for agent foundations—ask what agent foundations can do for you, in your own personal quest to figure out how to save the world.
Most of the intelligence.org website has been updated very little over the last 4 years, and is thus at least a little out of date. The MIRI reading list in particular hasn’t really been updated since before I started at MIRI in 2015.I can summarize (what I think are the generators of) the MIRI reading list as:1) If you want to build a philosophically solid reductionist understanding of anything, you should probably start by learning math (theoretical CS is part of math).
2) If you want to work in a preparadigmatic field, focus on breadth and foundations.
3) If the thing you want to understand is about how minds work, things related to epistemics (logic, probability, information theory, topology), optimization (machine learning, game theory, calculus, decision theory), and algorithms maybe should get a little more attention.
(When you also bring in category theory because you want reductionism, which means you need to be consistently viewing the same thing at multiple different levels, this leads to basically the same conclusion as 1+2: learn all fields of math.)4) Here is our best guess at the best textbook on every subject.I personally basically haven’t read any of those books, so I don’t have much to say about point 4, but I believe that Nate learned a bunch of math from many of these exact books.
I stand behind points 1-3, as conditional statements, so if your goal (or your primary instrumental subgoal) is to build a philosophically solid reductionist understanding of how minds work, maybe you should read some of those books.
Note that this advice doesn’t have anything to do with MIRI, except in so far as I, and some others at MIRI, are reasonably well described as “trying to build a philosophically solid reductionist understanding of how minds work.”
If your plan is to spend 2 years learning all the math, and then reaching out to MIRI saying “I did my homework, can I have a job?” I recommend instead discussing this plan with an actual person at MIRI first.
If on the other hand, it feels obvious to you that you want a philosophically solid reductionist understanding of how minds work, enough that you would still want it if MIRI announced tomorrow that it was pivoting to focus on global poverty, then I recommend you start by learning some math, and I don’t have any better advice about what math to learn than what is on that webpage.
Note that the title is misleading. This is really countable dimension factored spaces, which is much better, since it allows for the possibility of something kind of like continuous time, where between any two points in time, you can specify a time strictly between them.
Yeah, also note that the history of X given Y is not actually a well defined concept. There is only the history of X given y for y∈Y. You could define it to be the union of all of those, but that would not actually be used in the definition of orthogonality. In this case hF(X|y), hF(V|y), and hF(Z|y) are all independent of choice of y∈Y, but in general, you should be careful about that.
I think that works, I didn’t look very hard. Yore histories of X given Y and V given Y are wrong, but it doesn’t change the conclusion.
I could do that. I think it wouldn’t be useful, and wouldn’t generalize to sub partitions.
I don’t know, the negation of the first thing? A system that can freely model humans, or at least perform computation indistinguishable from modeling humans.
I claim that conversations about good fuzzy abstract clusters are both difficult, and especially in the comparative advantage of LessWrong. I claim that LW was basically founded on analogy, specifically the analogy between human cognition and agent foundations/AI. The rest of the world is not very precise, and that is a point for LW having a precision comparative advantage, but the rest of the world seems to also be lacking abstraction skill.The claim that precision requires purity leads to the conclusion that you have to choose between precision and abstraction, and it also leads to the conclusion that precision is fragile, and thus that you should expect a priori for there to less of it naturally, but it is otherwise symmetric between precision and abstraction. It is only saying that we should pick a lane, not which lane it should be.
I claim that when I look at science academia, I see a bunch of precision directed in inefficient ways, together with a general failure to generalize across pre-defined academic fields, and even more of a failure to generalize out of the field of “science” and into the real world.
Warning: I am mostly conflating abstraction with analogy with lack of precision above, and maybe the point is that we need precise analogies.
Given a finite set Ω of cardinality n, find a computable upper bound on the largest finite factored set model that is combinatorially different from all smaller finite factored set models. (We say that two FFS models are combinatorially different if they say the same thing about the emptiness of all boolean combinations of histories and conditional histories of partitions of Ω.) (Such an upper bound must exist because there are only finitely many combinatorially distinct FFS models, but a computable upper bound, would tell us that temporal inference is computable.)
Prove the fundamental theorem for finite dimensional factored sets. (Seems likely combinatorial-ish, but I am not sure.)
Figure out how to write a program to do efficient temporal inference on small examples. (I suspect this requires a bunch of combinatorial insights. By default this is very intractable, but we might be able to use math to make it easier.)
Axiomatize complete consistent orthogonality databases (consistent means consistent with some model, complete means has an opinion on every possible conditional orthogonality) (To start, is it the case that compositional semigraphoid axioms already work?)
If by “pure” you mean “not related to history/orthogonality/time,” then no, the structure is simple, and I don’t have much to ask about it.
Yeah, this is the point that orthogonality is a stronger notion than just all values being mutually compatible. Any x1 and x2 values are mutually compatible, but we don’t call them orthogonal. This is similar to how we don’t want to say that x1 and (the level sets of) x1+x2 are compatible.
The coordinate system has a collection of surgeries, you can take a point and change the x1 value without changing the other values. When you condition on E, that surgery is no longer well defined. However the surgery of only changing the x4 value is still well defined, and the surgery of changing x1 x2 and x3 simultaneously is still well defined (provided you change them to something compatible with E).
We could define a surgery that says that when you increase x1, you decrease x2 by the same amount, but that is a new surgery that we invented, not one that comes from the original coordinate system.
Thanks for writing this.
On the finiteness point, I conjecture that “finite dimensional” (|B| is finite) is sufficient for all of my results so far, although some of my proofs actually use “finite” (|S| is finite). The example with real numbers is still finite dimensional, so I don’t expect any problems.