Vanessa Kosoy(Vanessa Kosoy)

Karma: 2,917

AI alignment researcher supported by MIRI and LTFF. Working on the learning-theoretic agenda.

Random factoids about Vanessa:

[insert typical rationalist / math nerd stuff here]
Loves fantasy and scifi, but romance novels especially
Rarely reads non-fiction books except for textbooks
Divorced with two children and a cat
Doesn’t eat vertebrates or vertebrate products
Polyamorous and bisexual
Has a weakness for pretty clothes. Also for tasty food
Speaks Hebrew, Russian and English. Has been trying to learn French

Vanessa Kosoy 27 Jan 2021 0:27 UTC
12 points
on: Simulacrum 3 As Stag-Hunt Strategy
I think that all of ethics works like this: we pretend to be more altruistic / intrinsically pro-social than we actually are, even to ourselves. And then there are situations like battle of the sexes, where we negotiate the Nash equilibrium while pretending it is a debate about something objective that we call “morality”.

Vanessa Kosoy 24 Jan 2021 18:10 UTC
LW: 2 AF: 1
AF
in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform
Actually the Schwartz–Zippel algorithm can easily be adapted to this case (just imagine that types are variables over $Q$ , and start from testing the identity of the types appearing inside parentheses), so we can validate expressions in randomized polynomial time (and, given standard conjectures, in deterministic polynomial time as well).

Vanessa Kosoy 23 Jan 2021 17:05 UTC
LW: 4 AF: 2
AF
in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform
Let’s also explicitly describe 0th order and 1st order infra-Bayesian logic (although they are ofc segments of higher-order).

0-th order

Syntax

Let $A$ be the set of propositional variables. We define the language $L$ :
- Any $a \in A$ is also in $L$
- $⊥ \in L$
- $⊤ \in L$
- Given $p, q \in L$ , $p \land q \in L$
- Given $p, q \in L$ , $p \lor q \in L$
Notice there’s no negation or implication. We define the set of judgements $J := L \times L$ . We write judgements as $p ⊢ q$ (” $q$ in the context of $p$ ”). A theory is a subset of $J$ .

Semantics

Given $T \subseteq J$ , a model of $T$ consists of a compact Polish space $X$ and a mapping $M : L \to □ X$ . The latter is required to satisfy:
- $M (⊥) = ⊥_{X}$
- $M (⊤) = ⊤_{X}$
- $M (p \land q) = M (p) \land M (q)$ . Here, we define $\land$ of infradistributions as intersection of the corresponding sets
- $M (p \lor q) = M (p) \lor M (q)$ . Here, we define $\lor$ of infradistributions as convex hull of the corresponding sets
- For any $p ⊢ q \in T$ , $M (p) ⪯ M (q)$
1-st order

Syntax

We define the language using the usual syntax of 1-st order logic, where the allowed operators are $\land$ , $\lor$ and the quantifier $\exists$ . Variables are labeled by types from some set $T$ . For simplicity, we assume no constants, but it is easy to introduce them. For any sequence of variables $(v_{1} \dots v_{n})$ , we denote $L_{v}$ the set of formulae whose free variables are a subset of $v_{1} \dots v_{n}$ . We define the set of judgements $J := ⋃_{v} L_{v} \times L_{v}$ .

Semantics

Given $T \subseteq J$ , a model of $T$ consists of
- For every $t \in T$ , a compact Polish space $M (t)$
- For every $p \in L_{v}$ where $v_{1} \dots v_{n}$ have types $t_{1} \dots t_{n}$ , an element $M_{v} (p)$ of $X_{v} := □ (\prod_{i = 1}^{n} M (t_{i}))$
It must satisfy the following:
- $M_{v} (⊥) = ⊥_{X_{v}}$
- $M_{v} (⊤) = ⊤_{X_{v}}$
- $M_{v} (p \land q) = M_{v} (p) \land M_{v} (q)$
- $M_{v} (p \lor q) = M_{v} (p) \lor M_{v} (q)$
- Consider variables $u_{1} \dots u_{n}$ of types $t_{1} \dots t_{n}$ and variables $v_{1} \dots v_{m}$ of types $s_{1} \dots s_{m}$ . Consider also some $σ : {1 \dots n} \to {1 \dots m}$ s.t. $s_{σ (i)} = t_{i}$ . Given $p \in L_{v}$ , we can form the substitution $q := p [x_{i} = y_{σ (i)}] \in L_{u}$ . We also have a mapping $f_{σ} : X_{v} \to X_{u}$ given by $f_{σ} (x_{1} \dots x_{m}) = (x_{σ (1)} \dots x_{σ (n)})$ . We require $M_{u} (q) = f^{*} (M_{v} (p))$
- Consider variables $v_{1} \dots v_{n}$ and $i \in {1 \dots n}$ . Denote $p r : X_{v} \to X_{v ∖ v_{i}}$ the projection mapping. We require $M_{v ∖ v_{i}} (\exists v_{i} : p) = {p r}_{*} (M_{v} (p))$
- For any $p ⊢ q \in T$ , $M_{v} (p) ⪯ M_{v} (q)$

Vanessa Kosoy 20 Jan 2021 20:33 UTC
2 points
on: Meetup Organizers, Our Virtual Garden is at Your Disposal
Are EA meetups welcome?

Vanessa Kosoy 16 Jan 2021 12:03 UTC
LW: 3 AF: 1
AF
in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform
When using infra-Bayesian logic to define a simplicity prior, it is natural to use “axiom circuits” rather than plain formulae. That is, when we write the axioms defining our hypothesis, we are allowed to introduce “shorthand” symbols for repeating terms. This doesn’t affect the expressiveness, but it does affect the description length. Indeed, eliminating all the shorthand symbols can increase the length exponentially.

Vanessa Kosoy 16 Jan 2021 11:54 UTC
LW: 3 AF: 1
AF
in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform
Instead of introducing all the “algebrator” logical symbols, we can define $T$ as the quotient by the equivalence relation defined by the algebraic laws. We then need only two extra logical atomic terms:
- For any $n \in N$ and $σ \in S_{n}$ (permutation), denote $n := \sum_{i = 1}^{n} 1$ and require $σ^{+} \in F_{n \to n}$
- For any $n \in N$ and $σ \in S_{n}$ , $σ_{α}^{\times} \in F_{α^{n} \to α^{n}}$
However, if we do this then it’s not clear whether deciding that an expression is a well-formed term can be done in polynomial time. Because, to check that the types match, we need to test the identity of algebraic expressions and opening all parentheses might result in something exponentially long.

Vanessa Kosoy 16 Jan 2021 0:01 UTC
LW: 2 AF: 1
AF
in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform
Infra-Bayesianism can be naturally understood as semantics for a certain non-classical logic. This promises an elegant synthesis between deductive/symbolic reasoning and inductive/intuitive reasoning, with several possible applications. Specifically, here we will explain how this can work for higher-order logic. There might be holes and/or redundancies in the precise definitions given here, but I’m quite confident the overall idea is sound.

For simplicity, we will only work with crisp infradistributions, although a lot of this stuff can work for more general types of infradistributions as well. Therefore, $□ X$ will denote the space of crisp infradistribution. Given $μ \in □ X$ , $S (μ) \subseteq Δ X$ will denote the corresponding convex set. As opposed to previously, we will include the empty-set, i.e. there is $⊥_{X} \in □ X$ s.t. $S (⊥_{X}) = \emptyset$ . Given $p \in Δ X$ and $μ \in □ X$ , $p : μ$ will mean $p \in S (μ)$ . Given $μ, ν \in □ X$ , $μ ⪯ ν$ will mean $S (μ) \subseteq S (ν)$ .

Syntax

Let $T^{ι}$ denote a set which we interpret as the types of individuals (we allow more than one). We then recursively define the full set of types $T$ by:
- $0 \in T$ (intended meaning: the uninhabited type)
- $1 \in T$ (intended meaning: the one element type)
- If $α \in T^{ι}$ then $α \in T$
- If $α, β \in T$ then $α + β \in T$ (intended meaning: disjoint union)
- If $α, β \in T$ then $α \times β \in T$ (intended meaning: Cartesian product)
- If $α \in T$ then $(α) \in T$ (intended meaning: predicates with argument of type $α$ )
For each $α, β \in T$ , there is a set $F_{α \to β}^{0}$ which we interpret as atomic terms of type $α \to β$ . We will denote $V_{α}^{0} := F_{1 \to α}^{0}$ . Among those we distinguish the logical atomic terms:
- ${i d}_{α} \in F_{α \to α}^{0}$
- $0_{α} \in F_{0 \to α}^{0}$
- $1_{α} \in F_{α \to 1}^{0}$
- ${p r}_{α β} \in F_{α \times β \to α}^{0}$
- $i_{α β} \in F_{α \to α + β}^{0}$
- Symbols we will not list explicitly, that correspond to the algebraic properties of $+$ and $\times$ (commutativity, associativity, distributivity and the neutrality of $0$ and $1$ ). For example, given $α, β \in T$ there is a “commutator” of type $α \times β \to β \times α$ .
- $=_{α} \in V_{(α \times α)}^{0}$
- ${d i a g}_{α} \in F_{α \to α \times α}^{0}$
- $()_{α} \in V_{((α) \times α)}^{0}$ (intended meaning: predicate evaluation)
- $⊥ \in V_{(1)}^{0}$
- $⊤ \in V_{(1)}^{0}$
- $\lor_{α} \in F_{(α) \times (α) \to (α)}^{0}$
- $\exists_{α β} \in F_{(α \times β) \to (β)}^{0}$
- $\forall_{α β} \in F_{(α \times β) \to (β)}^{0}$
- Assume that for each $n \in N$ there is some $D_{n} \subseteq □ [n]$ : the set of “describable” infradistributions (for example, it can be empty, or consist of all distributions with rational coefficients, or all distributions, or all infradistributions; EDIT: it is probably sufficient to only have the fair coin distribution in $D_{2}$ in order for it to be possible to approximate all infradistributions on finite sets). If $μ \in D_{n}$ then $┌ μ ┐ \in V_{(\sum_{i = 1}^{n} 1)}$
We recursively define the set of all terms $F_{α \to β}$ . We denote $V_{α} := F_{1 \to α}$ .
- If $f \in F_{α \to β}^{0}$ then $f \in F_{α \to β}$
- If $f_{1} \in F_{α_{1} \to β_{1}}$ and $f_{2} \in F_{α_{2} \to β_{2}}$ then $f_{1} \times f_{2} \in F_{α_{1} \times α_{2} \to β_{1} \times β_{2}}$
- If $f_{1} \in F_{α_{1} \to β_{1}}$ and $f_{2} \in F_{α_{2} \to β_{2}}$ then $f_{1} + f_{2} \in F_{α_{1} + α_{2} \to β_{1} + β_{2}}$
- If $f \in F_{α \to β}$ then $f^{- 1} : F_{(β) \to (α)}$
- If $f \in F_{α \to β}$ and $g \in F_{β \to γ}$ then $g \circ f \in F_{α \to γ}$
Elements of $V_{(α)}$ are called formulae. Elements of $V_{(1)}$ are called sentences. A subset of $V_{(1)}$ is called a theory.

Semantics

Given $T \subseteq V_{(1)}$ , a model $M$ of $T$ is the following data. To each $α \in T$ , there must correspond some compact Polish space $M (t)$ s.t.:
- $M (0) = \emptyset$
- $M (1) = p t$ (the one point space)
- $M (α + β) = M (α) ⊔ M (β)$
- $M (α \times β) = M (α) \times M (β)$
- $M ((α)) = □ M (α)$
To each $f \in F_{α \to β}$ , there must correspond a continuous mapping $M (f) : M (α) \to M (β)$ , under the following constraints:
- $0$ , $1$ , $i d$ , $p r$ , $i$ , $d i a g$ and the “algebrators” have to correspond to the obvious mappings.
- $M (=_{α}) = ⊤_{{d i a g}_{M (α)}}$ . Here, ${d i a g}_{X} \subseteq X \times X$ is the diagonal and $⊤_{C} \in □ X$ is the sharp infradistribution corresponding to the closed set $C \subseteq X$ .
- Consider $α \in T$ and denote $X := M (α)$ . Then, $M (()_{α}) = ⊤_{□ X} ⋉ {i d}_{□ X}$ . Here, we use the observation that the identity mapping ${i d}_{□ X}$ can be regarded as an infrakernel from $□ X$ to $X$ .
- $M (⊥) = ⊥_{p t}$
- $M (⊤) = ⊤_{p t}$
- $S (M (\lor) (μ, ν))$ is the convex hull of $S (μ) \cup S (ν)$
- Consider $α, β \in T$ and denote $X := M (α)$ , $Y := M (β)$ and $p r : X \times Y \to Y$ the projection mapping. Then, $M (\exists_{α β}) (μ) = {p r}_{*} μ$ .
- Consider $α, β \in T$ and denote $X := M (α)$ , $Y := M (β)$ and $p r : X \times Y \to Y$ the projection mapping. Then, $p : M (\forall_{α β}) (μ)$ iff ${p r}_{*}^{- 1} (p) \subseteq S (μ)$ .
- $M (f_{1} \times f_{2}) = M (f_{1}) \times M (f_{2})$
- $M (f_{1} + f_{2}) = M (f_{1}) ⊔ M (f_{2})$
- $M (f^{- 1}) (μ) = M (f)^{*} (μ)$ . Notice that pullback of infradistributions is always defined thanks to adding $⊥$ (the empty set infradistribution).
- $M (g \circ f) = M (g) \circ M (f)$
- $M (┌ μ ┐) = μ$
Finally, for each $ϕ \in T$ , we require $M (ϕ) = ⊤_{p t}$ .

Semantic Consequence

Given $ϕ \in V_{(1)}$ , we say $M ⊨ ϕ$ when $M (ϕ) = ⊤_{p t}$ . We say $T ⊨ ϕ$ when for any model $M$ of $T$ , $M ⊨ ϕ$ . It is now interesting to ask what is the computational complexity of deciding $T ⊨ ϕ$ . [EDIT: My current best guess is co-RE]

Applications

As usual, let $A$ be a finite set of actions and $O$ be a finite set of observation. Require that for each $o \in O$ there is $σ_{o} \in T^{ι}$ which we interpret as the type of states producing observation $o$ . Denote $σ_{*} := \sum_{o \in O} σ_{o}$ (the type of all states). Moreover, require that our language has the nonlogical symbols $s_{0} \in V_{(σ_{*})}^{0}$ (the initial state) and, for each $a \in A$ , $K_{a} \in F_{σ_{*} \to (σ_{*})}^{0}$ (the transition kernel). Then, every model defines a (pseudocausal) infra-POMDP. This way we can use symbolic expressions to define infra-Bayesian RL hypotheses. It is then tempting to study the control theoretic and learning theoretic properties of those hypotheses. Moreover, it is natural to introduce a prior which weights those hypotheses by length, analogical to the Solomonoff prior. This leads to some sort of bounded infra-Bayesian algorithmic information theory and bounded infra-Bayesian analogue of AIXI.

Vanessa Kosoy 11 Jan 2021 16:44 UTC
LW: 2 AF: 1
AF
in reply to: Davidmanheim’s comment on: Vanessa Kosoy’s Shortform

re: #5, that doesn’t seem to claim that we can infer U given their actions, which is what the impossibility of deducing preferences is actually claiming.

You misunderstand the intent. We’re talking about inverse reinforcement learning. The goal is not necessarily inferring the unknown $U$ , but producing some behavior that optimizes the unknown $U$ . Ofc if the policy you’re observing is optimal then it’s trivial to do so by following the same policy. But, using my approach we might be able to extend it into results like “the policy you’re observing is optimal w.r.t. certain computational complexity, and your goal is to produce an optimal policy w.r.t. higher computational complexity.”

(Btw I think the formal statement I gave for 5 is false, but there might be an alternative version that works.)

(And as pointed out elsewhere, it isn’t Stuart’s thesis, it’s a well known and basic result in the decision theory / economics / philosophy literature.)

I am referring to this and related work by Armstrong.

Vanessa Kosoy 11 Jan 2021 9:37 UTC
4 points
in reply to: Ben Pace’s comment on: Launching Forecast, a community for crowdsourced predictions from Facebook
Hmm, what is the reasoning? Assuming it’s a project that’s directly related to core interests of the community?

Vanessa Kosoy 11 Jan 2021 7:53 UTC
4 points
on: Launching Forecast, a community for crowdsourced predictions from Facebook
IMO this post should be on the front page

Vanessa Kosoy 28 Dec 2020 7:15 UTC
LW: 2 AF: 1
AF
in reply to: Charlie Steiner’s comment on: Vanessa Kosoy’s Shortform

Ah, okay, I see what you mean. Like how preferences are divisible into “selfish” and “worldly” components, where the selfish component is what’s impacted by a future simulation of you that is about to have good things happen to it.

...I brought up the histories->states thing because I didn’t understand what you were getting at, so I was concerned that something unrealistic was going on. For example, if you assume that the agent can remember its history, how can you possibly handle an environment with memory-wiping?

AMDP is only a toy model that distills the core difficulty into more or less the simplest non-trivial framework. The rewards are “selfish”: there is a reward function $r : (S \times A)^{*} \to R$ which allows assigning utilities to histories by time discounted summation, and we consider the expected utility of a random robot sampled from a late population. And, there is no memory wiping. To describe memory wiping we indeed need to do the “unrolling” you suggested. (Notice that from the cybernetic model POV, the history is only the remembered history.)

For a more complete framework, we can use an ontology chain, but (i) instead of $A \times O$ labels use $A \times M$ labels, where $M$ is the set of possible memory states (a policy is then described by $π : M \to A$ ), to allow for agents that don’t fully trust their memory (ii) consider another chain with a bigger state space $S^{'}$ plus a mapping $p : S^{'} \to N^{S}$ s.t. the transition kernels are compatible. Here, the semantics of $p (s)$ is: the multiset of ontological states resulting from interpreting the physical state $s$ by taking the viewpoints of different agents $s$ contains.

In fact, to me the example is still somewhat murky, because you’re talking about the subjective probability of a state given a policy and a timestep, but if the agents know their histories there is no actual agent in the information-state that corresponds to having those probabilities.

I didn’t understand “no actual agent in the information-state that corresponds to having those probabilities”. What does it mean to have an agent in the information-state?

Vanessa Kosoy 27 Dec 2020 18:17 UTC
LW: 2 AF: 1
AF
in reply to: Charlie Steiner’s comment on: Vanessa Kosoy’s Shortform
I’m not quite sure what are you trying to say here, probably my explanation of the framework was lacking. The robots already remember the history, like in classical RL. The question about the histories is perfectly well-defined. In other words, we are already implicitly doing what you described. It’s like in classical RL theory, when you’re proving a regret bound or whatever, your probability space consists of histories.

I’m still confused about what you mean by “Bayesian hypothesis” though. Do you mean a hypothesis that takes the form of a non-anthropic MDP?

Yes, or a classical RL environment. Ofc if we allow infinite state spaces, then any environment can be regarded as an MDP (whose states are histories). That is, I’m talking about hypotheses which conform to the classical “cybernetic agent model”. If you wish, we can call it “Bayesian cybernetic hypothesis”.

Also, I want to clarify something I was myself confused about in the previous comment. For an anthropic Markov chain (when there is only one action) with a finite number of states, we can give a Bayesian cybernetic description, but for a general anthropic MDP we cannot even if the number of states is finite.

Indeed, consider some $T : S \to Δ N^{S}$ . We can take its expected value to get $E T : S \to R_{+}^{S}$ . Assuming the chain is communicating, $E T$ is an irreducible non-negative matrix, so by the Perron-Frobenius theorem it has a unique-up-to-scalar maximal eigenvector $η \in R_{+}^{S}$ . We then get the subjective transition kernel:

$S T (t ∣ s) = \frac{E T (t ∣ s) η_{t}}{\sum_{t^{'} \in S} E T (t^{'} ∣ s) η_{t^{'}}}$

Now, consider the following example of an AMDP. There are three actions $A := {a, b, c}$ and two states $S := {s_{0}, s_{1}}$ . When we apply $a$ to an $s_{0}$ robot, it creates two $s_{0}$ robots, whereas when we apply $a$ to an $s_{1}$ robot, it leaves one $s_{1}$ robot. When we apply $b$ to an $s_{1}$ robot, it creates two $s_{1}$ robots, whereas when we apply $b$ to an $s_{0}$ robot, it leaves one $s_{0}$ robot. When we apply $c$ to any robot, it results in one robot whose state is $s_{0}$ with probability $\frac{1}{2}$ and $s_{1}$ with probability $\frac{1}{2}$ .

Consider the following two policies. $π_{a}$ takes the sequence of actions $c a c a c a \dots$ and $π_{b}$ takes the sequence of actions $c b c b c b \dots$ . A population that follows $π_{a}$ would experience the subjective probability $S T (s_{0} ∣ s_{0}, c) = \frac{2}{3}$ , whereas a population that follows $π_{b}$ would experience the subjective probability $S T (s_{0} ∣ s_{0}, c) = \frac{1}{3}$ . Hence, subjective probabilities depend on future actions. So, effectively anthropics produces an acausal (Newcomb-like) environment. And, we already know such environments are learnable by infra-Bayesian RL agents and, (most probably) not learnable by Bayesian RL agents.
What links here?
- Vanessa Kosoy's comment on Vanessa Kosoy’s Shortform by Vanessa Kosoy (26 Dec 2020 16:31 UTC; 2 points)

Vanessa Kosoy 26 Dec 2020 16:31 UTC
LW: 2 AF: 1
AF
in reply to: Charlie Steiner’s comment on: Vanessa Kosoy’s Shortform
I’m not sure what do you mean by that “unrolling”. Can you write a mathematical definition?

Let’s consider a simple example. There are two states: $s_{0}$ and $s_{1}$ . There is just one action so we can ignore it. $s_{0}$ is the initial state. An $s_{0}$ robot transition into an $s_{1}$ robot. An $s_{1}$ robot transitions into an $s_{0}$ robot and an $s_{1}$ robot. How will our population look like?

0th step: all robots remember $s_{0}$

1st step: all robots remember $s_{0} s_{1}$

2nd step: ¹⁄₂ of robots remember $s_{0} s_{1} s_{0}$ and ¹⁄₂ of robots remember $s_{0} s_{1} s_{1}$

3rd step: ¹⁄₃ of robots remembers $s_{0} s_{1} s_{0} s_{1}$ , ¹⁄₃ of robots remember $s_{0} s_{1} s_{1} s_{0}$ and ¹⁄₃ of robots remember $s_{0} s_{1} s_{1} s_{1}$

There is no Bayesian hypothesis a robot can have that gives correct predictions both for step 2 and step 3. Indeed, to be consistent with step 2 we must have $P r [s_{0} s_{1} s_{0}] = \frac{1}{2}$ and $P r [s_{0} s_{1} s_{1}] = \frac{1}{2}$ . But, to be consistent with step 3 we must have $P r [s_{0} s_{1} s_{0}] = \frac{1}{3}$ , $P r [s_{0} s_{1} s_{1}] = \frac{2}{3}$ .

In other words, there is no Bayesian hypothesis s.t. we can guarantee that a randomly sampled robot on a sufficiently late time step will have learned this hypothesis with high probability. The apparent transition probabilities keep shifting s.t. it might always continue to seem that the world is complicated enough to prevent our robot from having learned it already.

Or, at least it’s not obvious there is such a hypothesis. In this example, $\frac{P r [s_{0} s_{1} s_{1}]}{P r [s_{0} s_{1} s_{0}]}$ will converge to the golden ratio at late steps. But, do all probabilities converge fast enough for learning to happen, in general? I don’t know, maybe for finite state spaces it can work. Would definitely be interesting to check.

[EDIT: actually, in this example there is such a hypothesis but in general there isn’t, see below]

Vanessa Kosoy 25 Dec 2020 11:36 UTC
LW: 14 AF: 8
AF
on: Operationalizing compatibility with strategy-stealing
Notice that Eliezer’s definition + the universal prior is essentially the same as the computationally unbounded variant of my definition of goal-directed intelligence, if you fix the utility function and prior. I think that when you’re comparing different utility functions in your definition of strategy-stealing, it’s better to emulate my definition and correct by the description complexity of the utility function. Indeed, if you have an algorithm that works equally well for all utility functions but contains a specification of the utility function inside its source code, then you get “spurious” variance if you don’t correct for it (usually superior algorithms would also have to contain a specification of the utility function, so the more complex the utility function the less of them will be).

Vanessa Kosoy 6 Dec 2020 22:37 UTC
4 points
in reply to: abramdemski’s comment on: Cultural accumulation

Of particular interest would be the theory of computation, because the construction of a mechanical computer might be accomplished much earlier—although, the construction of suitable clockwork would be required.

If you mean a universal mechanical computer (like Babbage’s analytical engine) then, as far as I know, none was ever built, because it’s actually really hard to have clockwork that good.

Vanessa Kosoy 3 Dec 2020 12:02 UTC
2 points
in reply to: habryka’s comment on: The LessWrong 2018 Book is Available for Pre-order
Great, thank you :)

Vanessa Kosoy 3 Dec 2020 11:56 UTC
LW: 8 AF: 3
AF
on: The Parable of Predict-O-Matic
This is a rather clever parable which explains serious AI alignment problems in an entertaining form that doesn’t detract from the substance.

Vanessa Kosoy 3 Dec 2020 11:52 UTC
LW: 5 AF: 3
AF
on: Evolution of Modularity
I liked this post for talking about how evolution produces modularity (contrary to what is often said in this community!). This is something I suspected myself but it’s nice to see it explained clearly, with backing evidence.

Vanessa Kosoy 3 Dec 2020 11:41 UTC
6 points
on: Book Review: The Secret Of Our Success
This is a good review of an intriguing thesis about human culture, with a bunch of neat factoids. The thesis has some major flaws IMO, but it is a useful perspective to know.

Vanessa Kosoy 3 Dec 2020 11:33 UTC
2 points
on: Reframing the evolutionary benefit of sex
This post gives an elegant explanation of the evolutionary benefit of sexual reproduction that was new to me, at least. I like it, although I also wish some expert added their thoughts.