Antoine de Scorraille

Karma: 49

Antoine de Scorraille 29 Dec 2023 11:18 UTC
2 points
0
in reply to: rotatingpaguro’s comment on: Game Theory without Argmax [Part 1]
It’s fixed ;)

Antoine de Scorraille 28 Dec 2023 14:25 UTC
2 points
0
on: Game Theory without Argmax [Part 1]
Exercise 1:

The empty set is the only one. For any nonempty set X, you could pick $z e r o_{X} = λ^{x \in X} .0$ as a counterexample: $a r g m a x_{X} (z e r o_{X}) = X$

Exercise 2:

The agent will choose an option which scores better than the threshold $α$ .

It’s a generalization of satisficers, these latter are thresholders such as $u^{- 1} (α)$ is nonempty.

Exercise 3:

$m a j o r i t y_{X} = λ^{u : X \to R} . {x \in X | \forall x^{'} \in X, C a r d (u^{- 1} ({u (x)})) \geq C a r d (u^{- 1} ({u (x^{'})}))}$

Exercise 4:

I have discovered a truly marvelous-but-infinite solution of this, which this finite comment is too narrow to contain.

Exercise 5:

The generalisable optimisers are the following:

$a r g m i n_{X} = λ^{u : X \to R} . {x \in X | \forall x^{'} \in X, u (x) \leq u (x^{'}) ⟹ u (x^{'}) \leq u (x)}$

i.e. argmin will choose the minimal points.

$s a t i s f i c e_{s} = λ^{u : X \to R} . {x \in X | u (x) \geq u (s)}$

i.e. satisfice will choose an option $x \in X$ which dominates some fixed anchor point $s \in X$ . Note that since R is only equipped with a preorder, it means it might be a more selective optimiser (if not total, it’s harder to get an option better then the anchor). More interestingly, if there are indifferent options with the anchor (some x / $x \geq s$ and $s \geq x$ ), it could choose it rather than the anchor even if there is no gain to do so. This degree of freedom could be eventually not desirable.

$r o c k_{S} = λ^{u : X \to R} . S$

$s a t i s f i c e_{S} = λ^{u : X \to R} . {x \in X | \forall s \in S, u (x) \geq u (s)}$

$m a j o r i t y_{X} = λ^{u : X \to R} . {x \in X | \forall x^{'} \in X, C a r d (u^{- 1} ({u (x)})) \geq C a r d (u^{- 1} ({u (x^{'})}))}$

Exercise 6:

Interesting problem.

First of all, is there a way to generalize the trick?

The first idea would be to try to find some equivalent to the destested element $⊥$ . For context-dependant optimiser such as better-than-average, there isn’t.

A more general way to try to generalize the trick would be the following question:

For some fixed $ψ$ , $X_{l e g a l}$ and $u : X \to R$ , could we find some other $u^{'} : X \to R$ such as $ψ (u^{'}) = ψ (u)$ and $\forall x \in X_{l e g a l}, u^{'} (x) = u (x)$

i.e. is there a general way to replace values outside of $X_{l e g a l}$ without modify the result of the constrained optimisation?

Answer: no. Counter-example: The optimiser $m a j o r i t y_{X}$ for some infinite set X and finite nonempty sets $X_{l e g a l}$ and R.

So it seems there is no general trick here. But why bother? We should refocus on what we mean by constrained optimisation in general, and it has nothing to do with looking for some u’. What we mean is value outside $X_{l e g a l}$ are totally irrelevant.

How? For any of the current example we have here, what we actually want is not $ψ_{X} (u^{'})$ , but $ψ_{X_{l e g a l}} (u |_{X_{l e g a l}})$ : we only apply the optimiser on the legal set.

Problem: in the current formalism, an optimiser has type $(X \to R) \to X$ , so I don’t see obvious way to define the “same” optimiser on a different X. $a r g m a x_{X}$ and the others here are implicitly parametrized, so it’s not that a problem, but we have to specify this. This suggests to look for categories (e.g. $S e t$ for argmax...).

Antoine de Scorraille 28 Dec 2023 2:31 UTC
2 points
0
in reply to: rotatingpaguro’s comment on: Game Theory without Argmax [Part 1]
Even with finite sets, it doesn’t work because the idea to look for “closest to $X_{l e g a l}$ ” is not what we looking for.

Let $X = {A l i c e; B o b; C h a r l i e}$ a class of students, scoring within a range $R = {0; 1; 2; 3; 4; 5}$ , $u : X \to R$ . Let $ψ$ the (uniform) better-than-average optimizer, standing for the professor picking any student who scores better than the mean.

Let $X_{l e g a l} = {A l i c e; B o b}$ (the professor $ψ$ despises Charlie and ignores him totally).

If u(Alice) = 5 and u(Bob) = 4, their average is 4.5 so only Alice should be picked by the constrained optimisation.

Howewer, with your proposal, you run into trouble with u’ defined as u’(Alice) = u(Alice), u’(Bob) = u(Bob), and u’(Charlie) = 0.

The average value for this u’ is $(5 + 4 + 0) / 3 = 3$ , and both Alice and Bob scores better than 3: $ψ (u^{'}) = {A l i c e; B o b} = X_{l e g a l}$ . The size of the intersection with $X_{l e g a l}$ is then maximal, so your proposal suggests to pick this set as the result. But the actual result should be ${A l i c e}$ , because the score of Charlie is irrelevant to constrained optimisation.

Antoine de Scorraille 3 Jun 2023 13:21 UTC
5 points
0
in reply to: johnswentworth’s comment on: Interpretability/Tool-ness/Alignment/Corrigibility are not Composable
Natural language is lossy because the communication channel is narrow, hence the need for lower-dimensional representation (see ML embeddings) of what we’re trying to convey. Lossy representations is also what Abstractions are about.
But in practice, you expect Natural Abstractions (if discovered) cannot be expressed in natural language?

Antoine de Scorraille 3 Jun 2023 13:16 UTC
3 points
0
on: Shannon’s Surprising Discovery
There is one catch: in principle, there could be multiple codes/descriptions which decode to the same message. The obvious thing to do is then to add up the implied probabilities of each description which produces the same message. That indeed works great. However, it turns out that just taking the minimum description length—i.e. the length of the shortest code/description which produces the message—is a good-enough approximation of that sum in one particularly powerful class of codes: universal Turing machines.

Is this about K-complexity is silly; use cross-entropy instead?

Antoine de Scorraille 3 Jun 2023 12:01 UTC
4 points
1
in reply to: Ivan Vendrov’s comment on: Interpretability/Tool-ness/Alignment/Corrigibility are not Composable
Do we really have such good interpretations for such examples? It seems to me that we have big problems in the real world because we don’t.
We do have very high-level interpretations, but not enough to have solid guarantees. After all, we have a very high-level trivial interpretation of our ML models: they learn! The challenge is not just to have clues, but clues that are relevant enough to address safety concerns in relation to impact scale (which is the unprecedented feature of the AI field).

Antoine de Scorraille 3 Jun 2023 11:29 UTC
3 points
2
on: Four levels of understanding decision theory
Pretty cool!
Just to add, although I think you already know: we don’t need to have a reflexive understanding of your DT to put it into practice, because messy brains rather than provable algo etc....
And I always feel it’s kinda unfair to dismiss as orthogonal motivations “valuing friendliness or a sense of honor” because they might be evolutionarily selected heuristics to (sort of) implement such acausal DT concerns!

Antoine de Scorraille 30 May 2023 8:05 UTC
1 point
0
on: K-complexity is silly; use cross-entropy instead
Great! Isn’t it generalizable to any argmin/argmax issues? Especialy thinking about the argmax decision theories framework, which is a well-known difficulty for safety concerns.
Similarly, in EA/action-oriented discussions, there is a reccurent pattern like:

Eager-to-act padawan: If world model/moral theory X is best likely to be true (due to evidence y z...), we need to act accordingly with the controversial Z! Seems best EU action!
Experienced jedi: Wait for a minute. You have to be careful with this way of thinking, because there are unkwown unknown, unilateralist curse and so on. A better decision-making procedure is to listen several models, severals moral theories, and to look for strategies acceptable by most of them.

Antoine de Scorraille 30 May 2023 7:42 UTC
3 points
on: Study Guide
Breadth Over Depth → To reframe, is it about to optimize for known unknown?

Antoine de Scorraille 28 May 2023 22:51 UTC
1 point
0
on: Exploiting Newcomb’s Game Show
- This CDT protagonist is not winning the game in a predictable and avoidable way, so he’s a bad player
- Yes, in this example and many others, have legibility is a powerful strategy (big chunks of social skills are about that)

Antoine de Scorraille 8 Apr 2023 12:09 UTC
1 point
0
in reply to: Cleo Nardo’s comment on: The Waluigi Effect (mega-post)
I’m confused, could you clarify? I interpret your “Wawaluigi” as two successive layers of deception within a simulacra, which is unlikely if WE is reliable, right?
I didn’t say anything about Wawaluigis and I agree that they are not Luigis, because as I said, a layer of Waluigi is not a one-to-one operator. My guess is about a normal Waluigi layer, but with a desirable Waluigi rather than a harmful Waluigi.

Antoine de Scorraille 4 Apr 2023 15:39 UTC
1 point
0
in reply to: gwern’s comment on: The Waluigi Effect (mega-post)
“Good” simply means “our targeted property” here. So my point is, if WE is true to any property P, we could get a P-Waluigi through some anti-P (pseudo-)naive targeting.
I don’t get your second point, we’re talking about simulacra not agent, and obviously this idea would only be part of a larger solution at best. For any property P, I expect several anti-P so you don’t have to instanciate an actually bad Luigi, my idea is more about to trap deception as a one-layer only.

Antoine de Scorraille 13 Mar 2023 23:49 UTC
2 points
0
on: The Waluigi Effect (mega-post)
Honest Why-not-just question: if the WE is roughly “you’ll get exactly one layer of deception” (aka a Waluigi), why not just anticipate by steering through that effect? To choose an anti-good Luigi to get a good Waluigi?

Antoine de Scorraille 17 Jan 2023 14:14 UTC
3 points
0
on: Confusing the ideal for the necessary
This seems to generalize your recent Confusing the goal and the path take:
- the goal → the necessary: to achieve the goal, it’s tautologically necessary to achieve it
- the path → the ideal one wants: to achieve the goal, one draws some ideal path to do so, confusing it with the former.
To quote your post:
For it conjures obstacles that were never there.
Or, recalling the wisdom of good old Donald Knuth:
Premature optimization is the root of all evil

Antoine de Scorraille 12 Dec 2022 19:42 UTC
1 point
0
in reply to: TAG’s comment on: Formalization as suspension of intuition
Agreed. It’s the opposite assumption (aka no embeddedness) for which I wrote this; fixed.

Antoine de Scorraille 12 Dec 2022 19:11 UTC
4 points
3
on: Psychological Disorders and Problems
Cool! I like this as an example of difficult-to-notice problems are generally left unsolved. I’m not sure how serious this one is, though.

Antoine de Scorraille 12 Dec 2022 18:48 UTC
1 point
0
on: Confusing the goal and the path
And it feels like becoming a winner means consistently winning.

Reminds me strongly of the difficulty of accepting commitment strategies in decision theories as in Parfit’s Hitchhiker: one gets the impression that a rational agent win-oriented should win in all situations (being greedy); but in reality, this is not always what winning looks like (optimal policy rather than optimal actions).
For it conjures obstacles that were never there.
Let’s try apply this to a more confused topic. Risky. Recently I’ve slighty updated away from the mesa paradigm, reading the following in Reward is not the optimization target:
Stop worrying about finding “outer objectives” which are safe to maximize.^[9] I think that you’re not going to get an outer-objective-maximizer (i.e. an agent which maximizes the explicitly specified reward function).
1. Instead, focus on building good cognition within the agent.
2. In my ontology, there’s only one question: How do we grow good cognition inside of the trained agent?
How does this relate to the goal/path confusion? Alignment paths strategies:
Outer + Inner alignment aims to be an alignment strategy, but only the straight path one. Any homotopic alignment path could be safe as well, our only real concern.

Antoine de Scorraille 11 Dec 2022 17:03 UTC
13 points
3
on: Formalization as suspension of intuition
Surprisingly, this echoes to me a recent thought: “frames blind us”; it comes to mind when I think of how traditional agent models (RL, game theory...) blinded us for years to their implicit assumption of no embeddedness. As with non-Euclidean shift, the change has come from relaxing the assumptions.
This post seems to complete your good old Traps of Formalization. Maybe it’s a hegelian dialectic:
I) intuition domination (system 1)
II) formalization domination (system 2)
III) modulable formalization (improved system 1/cautious system 2)
Deconfusion is reached only when one is able to go from I to III.

Antoine de Scorraille 8 Dec 2022 15:48 UTC
7 points
3
on: If Wentworth is right about natural abstractions, it would be bad for alignment
In my view you misunderstood JW’s ideas, indeed. His expression “far away relevant”/”distance” is not limited to spatial or even time-spatial distance. It’s a general notion of distance which is not fully formalized (work’s not done yet).
We have indeed concerns about inner properties (like your examples), and it’s something JW is fully aware. So (relevant) inner structures could be framed as relevant “far away” with the right formulation.

Antoine de Scorraille 3 Dec 2022 22:21 UTC
1 point
on: Latent Variables and Model Mis-Specification
Find $θ$ to maximize the predictive accuracy on the observed data, $\sum_{i = 1}^{z} l o g p_{θ} (y_{i}, x_{i})$ , where $p_{θ} (y_{i} | x_{i}) = \sum_{z} p_{θ} (y_{i}, z | x_{i})$ . Call the result $θ_{0}$ .
Isn’t the z in the sum on the left a typo? I think it should be n