tailcalled

Karma: 826

tailcalled 17 Jan 2022 21:14 UTC
1 point
in reply to: gjm’s comment on: Entropy isn’t sufficient to measure password strength
It’s fair enough to call that probabilities, I guess that just means that the thing I was really confused about is how you derive the CDF from the probabilities. That is, this part:
then the corresponding pdf for our measure of attacker-capability q=1/p is f(1/q)/q^2; the relevant CDF here is the integral from 0 to q of this.
Under my framework, the distribution of amount of guesses that an attacker is willing to go through is a separate independent parameter that one can choose. I don’t see where the f(1/q)/q^2 expression comes from.

tailcalled 17 Jan 2022 18:29 UTC
1 point
in reply to: gjm’s comment on: Entropy isn’t sufficient to measure password strength
A little more detail seems worth providing. If you have a pdf (over probabilities) f(p), which may or may not actually integrate to 1 or to any finite value, then the corresponding pdf for our measure of attacker-capability q=1/p is f(1/q)/q^2; the relevant CDF here is the integral from 0 to q of this.
Conversely, if we start with a given cdf F(q), its pdf is F’(q), and the corresponding pdf for p is F’(1/p)/p^2.
I don’t think I understand. What do you mean by a pdf over probabilities? Like I get how one can have a pdf over probabilities in e.g. a Laplace’s law of succession-like situation, but I’m not sure what probabilities you have a pdf over here.

tailcalled 17 Jan 2022 18:24 UTC
1 point
in reply to: Stuart_Armstrong’s comment on: Different way classifiers can be diverse
I think it depends on your method. Most current methods would probably heavily benefit from a diverse dataset, as they tend to be based on somehow “compressing” or “clustering” the images. However, it seems like they should still work to an extent on the monotonous husky/lion datasets, just not as much.
However, if one is willing to go beyond current methods, then I feel like it should be possible to make unsupervised representation learning methods that are better able to deal with monotonous datasets. I’m sort of playing with some ideas for this in my spare time, because it seems like a promising approach for me, though I haven’t developed them much yet.

tailcalled 17 Jan 2022 16:56 UTC
1 point
on: Different way classifiers can be diverse
A fifth method: you could use unsupervised learning to learn some multidimensional representations for the images, and then understand the classifier’s diversity in terms of the diversity of these multidimensional representations.

tailcalled 17 Jan 2022 15:40 UTC
1 point
in reply to: dkirmani’s comment on: Why not align AI by putting it in a multipolar trap?
Well, the “why not” boils down to “because there doesn’t seem to be any clear applications of it”. The example in your original post fails because AGIs are not humans and therefore you have no reason to expect it to work. I don’t see any alternative approaches that are promising. By default stuff doesn’t have potential unless there’s something special about it.

tailcalled 17 Jan 2022 14:47 UTC
1 point
in reply to: dkirmani’s comment on: Why not align AI by putting it in a multipolar trap?
Presumably it would not be just like humans enforce norms on each other, because the agents wouldn’t be human.

tailcalled 17 Jan 2022 14:32 UTC
0 points
on: Why not align AI by putting it in a multipolar trap?

Imagine a multi-agent environment with two rules[1]: first, every agent must obey, in both spirit and letter, every human command given to it.

If you can code this in as part of the multipolar trap, and it gives you alignment (it wouldn’t, but let’s suppose it would), then what do you need the second part for?

The hard aprt of alignment isn’t that an AI would contradict its programming, because it wouldn’t. The hard part is programming it in such a way that it does what we want, rather than doing something we don’t want.

tailcalled 17 Jan 2022 12:29 UTC
4 points
on: Entropy isn’t sufficient to measure password strength
This seems like the exact sort of thing expected utility theory is made for.

Entropy is given by E[log 1/P]. Maximizing entropy can be seen as using a password-selection method with the assumption that log 1/P is the utility of a given password. Is that a reasonable utility function? What does it represent?

Well, 1/P seems to represent the rough number of guesses needed to guess a password. Which does seem like a relevant number for evaluating the utility of a password, as the difficulty in guessing it scales directly with the number of tries necessary. But should we take the logarithm of that? Should we just use 1/P directly? If we just use this directly, then we are risk-neutral with respect to guessing difficulty. Is that reasonable?

I don’t think so, and here’s a model: You might imagine that you have a distribution of adversaries. Some might just be script kiddies who are going to try 1-3 logical guesses and move on. Most might be using bots or botnets and guessing more, say 1000000 guesses or so. And small upper tail would consist of e.g. state actors who could spend an enormous amount of resources.

Assuming, unrealistically, that each of the adversaries would do equal amounts of harm when guessing your password, it seems like the value of the password difficulty would be in reducing the probability that they guess it, by locking out subset of the adversaries. For simplicity, we could take the cumulative distribution function of the adversary guess budget distribution as our measure of the probability that they will be defeated by the password.

So this model implies that a proper way of selecting is E[cdf(1/P)], where cdf is the cumulative distrobution function of attacker capabilities. The simple E[1/P] version can be seen as using an (improper) uniform prior where there is equal probability for any number of positive guesses, whether that is 1, 64827266195729462, a googolplex, Graham’s number, or whatever. Meanwhile, E[log 1/P] can be seen as using a (still improper) prior where the order of magnitude of the adversary’s capabilities are uniformly distributed. Both of these are unbounded, as tends to happen with improper priors, but if we used a proper prior, the utilities would be bounded, corresponding to strong risk-aversion.

tailcalled 17 Jan 2022 11:56 UTC
1 point
in reply to: Chris_Leong’s comment on: $1000 USD prize—Circular Dependency of Counterfactuals
I’m not familiar with Hume’s philosophy, but the idea that “things just are” without being restricted to follow some patterns/laws seems to lose badly in a Bayesian way to theories which accept the laws that exist.

tailcalled 16 Jan 2022 18:13 UTC
3 points
on: $1000 USD prize—Circular Dependency of Counterfactuals
My entry. Ultimately I’m not sure whether I agree or disagree with your point, but I hope I’ve bought up some valuable things.
I’m not sure how strong you are in physics; the “Causality is real, counterfactuals are not” section is a brief summary of some fairly abstract and general properties of physics, so we might need to discuss it further in the comments if they do not immediately ring true to you.

Some thoughts on “The Nature of Counterfactuals”

tailcalled16 Jan 2022 18:12 UTC

12 points

2 comments11 min readLW link

tailcalled 13 Jan 2022 20:31 UTC
1 point
in reply to: tailcalled’s comment on: The Unreasonable Feasibility Of Playing Chess Under The Influence
Wait derp, that’s the range for chess, but go appears to more have a range of around 3000 Elo points.

tailcalled 13 Jan 2022 20:27 UTC
1 point
in reply to: Jan’s comment on: The Unreasonable Feasibility Of Playing Chess Under The Influence
In psychology, these sorts of scales are often standardized relative to the standard deviation instead of the mean. Or you could pick some other range as a metric; for instance if I’m looking it up right (and I very well may not be), the human range of ability appears to span around 1000 Elo points. So the drop is about 2x that between beginner and master humans.

tailcalled 13 Jan 2022 10:28 UTC
11 points
on: The Unreasonable Feasibility Of Playing Chess Under The Influence

About a 40% reduction in rating.

Does Elo have a meaningful zero? I thought it was an interval scale.

tailcalled 12 Jan 2022 16:24 UTC
3 points
in reply to: lcmgcd’s comment on: Use Normal Predictions
They’re not equivalent if the distribution is skewed, because if you use a Laplace-distributed prediction, you maximize accuracy by guessing at the median value, rather than at the mean value.

tailcalled 12 Jan 2022 10:15 UTC
2 points
in reply to: lcmgcd’s comment on: Use Normal Predictions
Mean deviation corresponds to Laplace-distributed predictions, rather than normally-distributed predictions.

tailcalled 12 Jan 2022 9:24 UTC
3 points
on: Evolution of Modularity
There’s another form of modular variation with e.g. eating than just modularly varying foods in the environment, namely: we don’t eat all the time. Instead we have some periods where we eat and other periods where we do other stuff. I wonder if this also contributes to modularity, e.g. it makes it necessary for it to be possible to “activate” and “deactivate” an eating mode while the rest of the body just does whatever.

tailcalled 11 Jan 2022 14:30 UTC
2 points
in reply to: redbird’s comment on: Total compute available to evolution
No, I’m saying that AI might be much worse than us at long-term planning, because evolution has selected us on the basis of very long chains of causal effects, whereas we can’t train AIs on the basis of such long chains of causal effects.

tailcalled 11 Jan 2022 10:54 UTC
4 points
in reply to: dkirmani’s comment on: No Abstraction Without a Goal
“Who owns the land” has influences on many far away variables, as those who own the land can implent policies about what to do with the land. Similarly, “Are the buildings intact?” has influences on many far away variables, because it determines whether the people who live on the land continue to live on the land, and people who live in a place are the ones who influence the place the most.

If I wanted to understand the long-term future of an area that was currently at war, I’d want to know the information relevant for who wins the war and how destructive the war is, as that has a lot of effects. Meanwhile I don’t know of any major effects of bluejays.

tailcalled 11 Jan 2022 9:25 UTC
3 points
on: No Abstraction Without a Goal

The scout does not tell the general about the bluejays he saw. He reports the number of bombers in the enemy’s hangar.

Number of enemy bombers seems more relevant than number of bluejays for predicting most far-future variables to me? E.g. who will control the land, how much damage there will be to existing things in the area, etc.. Maybe it’s because I’m not an ornithologist so I don’t know anything about bluejays. But I’d think humans tend to be the dominant force in influencing an area with bluejays exerting only negligible influence.

tailcalled

Some thoughts on “The Na­ture of Coun­ter­fac­tu­als”

Some thoughts on “The Nature of Counterfactuals”