(...) the term technical is a red flag for me, as it is many times used not for the routine business of implementing ideas but for the parts, ideas and all, which are just hard to understand and many times contain the main novelties.
- Saharon Shelah
As a true-born Dutchman I endorse Crocker’s rules.
For my most of my writing see my short-forms (new shortform, old shortform)
Twitter: @FellowHominid
Personal website: https://sites.google.com/view/afdago/home
The cases of the Dutch republic and English development after the Civil war are highly confounded.
I wouldn’t put too much stock in this kind of correlation.
I think I speak for many when I ask you to please elaborate on this!
What did Yudkoswky get right?
The central problem of AI alignment. I am not aware of anything in subsequent work that is not already implicit in Yudkowsky’s writing.
Short timelines avant le lettre. Yudkowsky was predicting AGI in his lifetime from the very start when most academics, observers, AI scientists, etc considered AGI a fairytale.
Inherent and irreducible uncertainty of forecasting, foolishness of precise predictions.
The importance of (Pearlian) causality, Solomonoff Induction as theory of formal epistemology, Bayesian statistics, (Shannon) information theory, decision theory [especially UDT-shaped things].
(?nanotech, ?cryonics)
if you had a timemachine to go back to 2010 you should buy bitcoin and write Harry Potter fanfiction
Preach, brother.
One hundred twenty percent agreed. Hubris is the downfall of the rationalist project.
I’m usually a skeptic of the usefulness of this kind of speculation but I found this a good read. I am particularly intrigued hy the suggestion of decomposability of goals.
Excited to see this go live, Nick!
Played around with Pantheon for a couple minutes. I wrote a couple of lines but I didn’t get any daemons yet. How long should I have to wait for them to pop up?
On the word ‘theory’.
The word ‘theory’ is oft used and abused.
there is two ways ‘theory’ is used that are different and often lead to confusion.
Theory in thescientific sense
the way a physicist would use: it’s a model of the world that is either right or wrong. there might be competing theories and we neeed to have empirical evidence to figure out which one’s right. Ideally, they agree with empirical evidence or at least are highly falsifiable. Importantly, if two theories are to conflict they need to actually speak about the same variables, the same set of measurable quantities.
Theory in the mathematician’ sense; a formal framework
There is a related but different notion of theory that a mathematician would use: a theory of groups, of differential equations, of randomness, of complex systems, of etc etc. This is more like a formal framework for a certain phenomenon or domain.
It defines what the quantities, variables, features one is interested in even are.
One often hears the question whether this (mathematical) theory makes testable predictions. This sounds sensible but doesn’t really makes sense. It is akin to asking whether arithmetic or calculus makes testable predictions.*
Theories in the mathematician’s sense can’t really be wrong or right since (at least in theory) everything is proven. Of course, theories in this sense can fail to say much about the real world, they might bake in unrealistic assumptions of course etc.
Other uses of ‘Theory’
The world ‘theory’ is also used in other disciplines. For instance, in literature studies where it is a denotes free form vacuous verbiage; or in ML where ‘theory’ it is used for uninformed speculation.
*one could argue that the theory of Peano Arithmetic actually does make predictions about natural numbers in the scientific sense, and more generally theories in the mathematical sense in a deep sense really are theories in the scientific sense. I think there is something to this but 1. it hasn’t been developed yet 2. mostly irrelevant in the present context.
Whats FPT!=W[1] ?
I’m a computational complexity noob so apologies if this question is elementary.
I thought if one could solve one NP-complete problem then one can solve all of them. But you say that the treewidth doesn’t help at all with the Clique problem. Is the parametrized complexity filtration by treewidth not preserved by equivalence between different NP-complete problems somehow?
To me, if this is true it exposes a much more serious flaw in computational complexity classes and does not resolve the issue of whether P vs NP matters in practice at all. My takeaway is that the definitions are too coarse, aren’t really capturing what’s going on. But likely I’m confused.
Thanks for writing this up Dalcy!
My knowledge of computational complexity is unfortunately lacking. I too wonder to what degree the central definitions of the complexity classes truly reflect the deep algorithmic structures or perhaps are a rather coarse shadow of more complex invariants.
You mention treewidth—are there other quantities of similar importance?
I also wonder about treewidth. I looked up the definition on wikipedia but struggle to find a good intuitive story what it really measures. Do you have an intuition pump about treewidth you could share?
Current work on Markov blankets and Boundaries on LW is flawed and outdated. State of the art should factor through this paper on Causal Blankets; https://iwaiworkshop.github.io/papers/2020/IWAI_2020_paper_22.pdf
A key problem for accounts of blankets and boundaries I have seen on LW so far is the following elementary problem (from the paper):
”Therefore, the MB [Markov Blanket] formalism forbids interdependencies induced by past events that are kept in memory, but may not directly influence the present state of the blankets.
Thanks to Fernando Rosas telling me about this paper.
@Vanessa Kosoy knows more.
My question for you is: in the world we live in, the full causal history of any real event contains almost the whole history of Earth from the time of the event backwards, because the Earth is so small relative to the speed of light, and everything that could have interacted with the event is part of the history of that event. So in practice, won’t all counterfactable events need to be a more-or-less a full specification of the whole state of the world at a certain point in time?
I would argue that this is not in fact the case.
Our world is highly modular, much less entangled than other possible worlds [possibly because of anthropic reasons].
The way I think about it: in practice as you zoom in on a false counterfactual you will need to pay more and more bits for conflicting ‘coincidences’.
interesting. who do you think feature space is like a simplicial complex ?
Thanks. I’m looking forward to your paper!
The ‘surprise accounting’ framework sounds quite a lot like the Minimum Description Length principle (e.g. here). Do you have any takes on how surprise accounting compares and differs vis a vis MDL?
Do I understand correctly that the main issue is finding ~ a canonical prior on the set of circuits?
Why does the OR-gate cost only 1 bit?
One argument I can see is that for any binary gate you only consider OR and AND gates. If we make the (semi-reasonable) assumption that these are equally likely then an OR gate cost 1 -bit?
However, you also have to describe which gate is to be described which takes more bits. You;d need ~ worth of bits to describe an arbitrary gate.
You might be able to formalize this using algorithmic information theory /K-complexity.
Blocked and reported to the ichor-blooded mods. You will pay dearly for your hubris.
As suggested by @Mateusz Bagiński it is tempting to suggest that the proper reading of a coproof of should be a proof of the double negation of , i.e. is a proof .
The absence of evidence against the negation of is weaker than a proof however. The former allows for new evidence to still appear while the latter categorically rejects this possibility on pain of contradiction.
Coproofs as countermodels
How could absence of evidence be interpreted logically? One way is to interpret it as the provision of a countermodel. That is—a coproof of would be a model such that .
Currently, the countermodel prevents a proof of and the more countermodels we find the more we might hypothesize that in all models and therefore not . On the other hand, we may find new evidence in the future that expands our knowledge database and excludes the would-be countermodel . This would open the way of a proof of in our new context.
Coproofs as sets of countermodels
We can go further by defining some natural (probability) distribution on the space of models .
This is generically a tricky business but given a finite signature of propositional letters over a classical base logic the space of models is given by “ultrafilters/models/truth assignments” which assign or to the basic propsitional letters and are extended to compound propositions in the usual manner (i.e. etc).
A subset of models can now be interpreted as a coproof of if for all .
Probability distributions on propositions and distributions on models
We might want to generalize subsets of models to (generalized) distributions of models. Any distribution on the set of models now induces a distribution on the set of propositions.
In the simple case above we could also make an invocation of the principle of indifference to define a natural uniform distributions on . This would assigns a proposition the ratio .
Rk. Note that similar ideas appear in the van Horn-Cox theorem.
Crypticity, Reverse Epsilon Machines and the Arrow of Time?
[see https://arxiv.org/abs/0902.1209 ]
Our subjective experience of the arrow of time is occasionally suggested to be an essentially entropic phenomenon.
This sounds cool and deep but crashes headlong into the issue that the entropy rate and the excess entropy of any stochastic process is time-symmetric. I find it amusing that despite hearing this idea often from physicists and the like apparently this rather elementary fact has not prevented their storycrafting.
Luckily, computational mechanics provides us with a measure that is not time symmetric: the stochastic complexity of the epsilon machine C
For any stochastic process we may also consider the epsilon machine of the reverse process, in other words the machine that predicts the past based on the future. This can be a completely different machine whose reverse stochastic complexity Crev is not equal to C.
Some processes are easier to predict forward than backward. For example, there is considerable evidence that language is such a process. If the stochastic complexity and the reverse stochastic complexity differ we speak of a causally assymetric process.
Alec Boyd pointed out to me that the classic example of a glass falling of a table is naturally thought of in these terms. The forward process is easy to describe while the backward process is hard to describe where easy and hard are meant in the sense of stochastic complexity: bits needed to specify the states of perfect minimal predictor, respectively retrodictor.
rk. note that time assymmetry is a fundamentally stochastic phenomenon. THe underlyiing (let’s say classicially deterministic) laws are still time symmetric.
The hypothesis is then: many, most macroscopic processes of interest to humans, including other agents are fundamentally such causally assymetric (and cryptic) processes.