Interested in math, Game Theory, etc.
Pattern
Karma: 1,547
In principle, if something evolves, then I think it’s worth noticing. Also, recent events have shown just how impactful viruses can be. Which is interesting given how little they seem to do of:
‘collecting and organizing evidence to exert flexible influence over the future’
I think it’s fair to characterize them as ‘largely exploiting static features in the world’ - alas, we tend to create/are such things. And given our massive global success, things able to exploit what (weaknesses) we have in common can become quite formidable. For all our ‘immense’ differences, we aren’t so different after all.*
*Though I probably should look into the impacts of cultural variation.
From the perspective of thinking about broad strokes technological progress, Wright’s law seems like a strict upgrade to Moore’s law. However, Wright’s law still leaves unanswered the question “Why is cumulative production increasing exponentially?”
Jevon’s paradox isn’t quite the answer. (It only says that ‘this thing happens sometimes’.)
1. Technology is a broad category. We could also use it to characterize tech which is used ‘everywhere’. Like electricity, plumbing, or air conditioning. If there’s demand for something everywhere, that partially explains why production would keep growing. Another part would be that we (keep) build(ing) on older (universal) tech—from phones to the internet, from electric lightbulbs to electric everything, from just air conditioning so we can stand the heat to cooling computers and food as well.
2. Population growth? When people have more money to spend, they spend money on...more (types of) things?
Theoretical Turing machines are very simple, but have infinite resources, and are thus a bad way of determining the difficulty of things.
If you can’t do it with a Turing machine, maybe you can’t do it. This sort of hedges against ‘does someone come up with a better algorithm’, or a more efficient data structure, so things don’t get broken when something we thought was impossible is accomplished.
Other approaches include working out a minimum amount of computation, then looking at:
Practical costs (today)*
Minimum amount of energy used in theory, per physics laws re: deleting information
(Both might get thrown out of whack by quantum computers, if they pan out. Also, reversible computing might lower theoretical and practical costs.)
*This one can be affected by ‘if someone buys hardware specifically for doing this once, they can reuse it’ , if there’s not enough padding.
Not sure what you mean by ‘both uses can coexist’ - i.e. a chicken treated as a pet then eaten? Unlikely.
Multiple chickens owned, most eaten, one (particularly useful one) treated more like a (working) pet.
as some people of course already do
I think this used to be more common that it is today.
Aiming for a local maximum. (The best place to build a rocket launch probably isn’t Mount Everest, though I could be wrong about that.)
In one sense, you are making progress on a key metric: distance from the moon.
This seems like it’d be less than 1%. (Or at least, 10%.)
At that point, your program is basically a lookup table.
Can you actually produce a look up table for perfect chess play?
It might be useful to specify some of the effects that are specific. For instance:
Driving between a set of cities might be more Euclidean than the Traveling Salesman problem (which is more general).
In theory, it could be more difficult if driving costs are asymmetric. Perhaps in practice this would take the form of 1 or 2 roads which are closed for construction, which, by shaping the solution might make computation to find an optimal path faster.
Ditch the quest for the optimal path, say 3 times it in length is acceptable, and the problem gets easier.
If you’re talking about how in practice, the running time is a lot better than theory(’s worst case), then say that.
Even though the amount of knowledge, strategy and learning involved in these games is exactly the same!
Probabilistically, this isn’t the case. If that’s a ‘fair 3-sided coin’, then there’s only a one in three chance of playing chess. Also, playing 1 chess game, and 10 are different (if they’re done one after another).
(Ignore the unreality of the hypothetical, I’m trying to make a point.)
Go all out, versus pace yourself (so you’ll do better towards the end when other players will be running low). Might not be a hypothetical.
The obvious thing to do, would be to replace these probabilities with qualitative words that get the same results, then work out what the appropriate probability based on the words is.
The issue is that these students were only thinking about the happy path.
Or they were lying.
(And there is no plausible middle ground in which cows & chickens would be bred in large numbers and well treated but not eaten—i.e. get to live the lives of pets.)
The key word in that sentence is probably ‘in large numbers’. However, this seems to ignore the fact that:
‘Pets’ don’t usually produce food. (The prior sentence might be false.)
Both uses can coexist (leaning more towards eating than not, while instances of not still exist).
How many chickens would there be if everyone had a chicken? (More seriously, extrapolating from the past, what’s the upper bound on chicken population, under the ‘lots of people have (a few) chickens’ model?)
It doesn’t generally mean all possible lives need to be brought into existence.
1. A living animal has a right to life.
2. A living species has a right to life. This isn’t the same thing as ‘all possible lives need to be brought into existence’.
‘Actions based on internal information’ seems as descriptive of bacteria, as it does of viruses. Are they usually less complex, or something?
It looks like something happened to the formatting on this post.
Consider a model tasked with predicting characters in text with a set of 64 characters (52 uppercase and lowercase letters, along with some punctuation).
Wait, people are doing this, instead of just turning words into numbers and having ‘models’ learn those? Anything GPT sized and getting results?
Once you start training, the easiest win is to simply notice how frequent each character is; just noticing that uppercase letters are rare, spaces are common, vowels are common, etc. could get your error down to 4-5 bits.
A diverse dataset will also include mistakes. And the more common a mistake is, the more likely it is learned as correct. Like its versus it’s (also reasons to make that mistake structure wise maybe? There’s a reason we do it, after all) and whether or not name ending in s should be followed by ‘s or ’.
or example, it might learn that “George W” tends to be followed by “ashington”.
I am now imagining a sentence completion task where the answer is George Washington, but the model predicts George W Bush instead, or vice versa.
As a simple example of how the scaling hypothesis affects AI safety research, it suggests that the training objective (“predict the next word”) is relatively unimportant in determining properties of the trained agent; in contrast, the dataset is much more important. This suggests that analyses based on the “reward function used to train the agent” are probably not going to be very predictive of the systems we actually build.
A fascinating point (though how much compute it requires is relevant). Though, even if it was scaled up a lot, what could a program that plays GTA do?
Problem: This definition fails to account for cases of knowledge where the map is represented in a very different way that doesn’t resemble the territory, such as when a map is represented by a sequence of zeros and ones in a computer.
While a problem, it’s not obvious how to overcome it—probably because it’s a real problem. If you found a textbook in another language would you be able to recognize it? Figure out what it was about? If it was physical yes (looking at the pictures), if digital and it was stored in a way your computer couldn’t read it, probably not.
Problem: In the real world, nearly every region of space will have high mutual information with the rest of the world. For example, by this definition, a rock accumulates lots of knowledge as photons incident on its face affect the properties of specific electrons in the rock giving it lots of information.
It seems if I point a camera at a place, and have it send what it sees to a screen then this:
has a lot of mutual information with that place
Looks like what it ‘models’
More generally, if we could figure things out about the world by looking at a rock, then this definition might seem less of an issue.
Looking for something like an intersection of definitions seems to have worked well here.
Problem: A video camera that constantly records would accumulate much more knowledge by this definition than a human, even though the human is much more able to construct models and act on them.
A human will also move around, learn about the shape of the world, etc. While we don’t look at cameras and think ‘that’s a flat earther’, I expect there is a lot of contrast between ‘the information the system including the camera has’ and ‘international travelers’.
The next iteration of the idea would be a drone that flies around the world, perhaps with very powerful batteries or refueling stations. However, reaching this stage seems to have softened that problem somewhat. Flying requires knowing how to fly, there’s issues around avoiding weather, not getting destroyed, repair (or self repair)...
it seems wrong to say that during the map-making process knowledge was not accumulating.
Let’s say it was. Then the process that destroyed it worked against that. Similarly:
“sphex” characterizes activity taken by a (mindless automaton)
Nevermind, I just looked it up, and apparently that isn’t real.
I thought that was in The Sequences, but this yielded no results:
https://www.readthesequences.com/Search?q=sphex&action=search
It’s true that knowledge destroyed seems like it’s not useful for precipitating action (though we might like more of a distinction here between ‘knowledge’ and ‘life’ (that is, destroyed knowledge is ‘dead’)). Why this can’t be rescued with counterfactuals isn’t clear.
Also, if we send a ship out to map the coastline, and it doesn’t come back:
Maybe sailing around the coast is treacherous.
Or it ran into pirates, invaders, or something.
Repeated attempts, meeting with failure, may allow us to infer (‘through absence rather than presence’) some knowledge.
What are identities about if not beliefs? Values, goals, utility, etc.
When your beliefs are central to your identity, it is way more difficult to change your mind if they turn out to be wrong.
Arguably, what you value may never changed and so you may never change your mind about it. Only means of achieving your ends.
If you merely disagreed with the person making that argument instead of also identifying with your belief, you wouldn’t feel attacked.
How would you feel if this website was flooded with spam? A massive influx of ‘flat earth’-ers?
Julia Galef observes two ways in which beliefs become identities: when the belief causes the person to feel embattled or proud.
Another way you might believe something, if it seems to work, towards achieving your ends.
An obvious example is
groups that hold beliefs in line with scientific consensus, yet are unpopular.
In her book, Galef proposes a solution to the problem of identity interfering with truth-seeking. She suggests, in her words, to “hold your identity lightly.”
And we’re back to ‘what is identity?’ If anything, going after what you value...seems a driver for (related) truth seeking. If you don’t want to die, knowledge about covid19 and the vaccine are probably important to you, while you might be entirely indifferent to dinosaurs, their existence, or information about them as long as you know:
a) they don’t still exist and thus:
b) aren’t a threat.
This is just equivocating between ‘belief turned identity’ and ‘identity’.
or things that make you superior to other people.
“Look at us, the truth seekers. Better than everybody else because we’re right.”
Julia Galef makes a final suggestion: hold a scout identity.
There’s no way that could backfire.
And here it is! Acknowledgement that identity isn’t just belief.
Even if the practical rewards for studying math usually appear in the long term, your identity means you find the short-term actions more rewarding.
Or just find:
What you want that can be achieved that way, or how to use it to improve/advance toward your goals
The parts you enjoy studying? If you don’t enjoy studying some subset of math, then maybe...don’t? The issue also might be the way you’re studying. And if what you want doesn’t exist, you might have to make it yourself.
agentive behaviour.
Is a cat an agent?
Response broken up by paragraphs:
1)
If I write “The sun will explode in the year 5 billion AD” on a rock, the
possibly static property that depends upon interpretation
is that it says “The sun will explode in the year 5 billion AD”, and the ‘dependency on interpretation’ is ’the ability to read English.
a textbook doesn’t make predictions.
‘Technically true’ in that it may encode a record of past predictions by agents in addition to
encod[ing] some information in a way that allows an agent to make a prediction.
2)
Give the brain a voice, a body, or hook it up to sensors that detect what it thinks. The last option may not be what we think of as control, and yet (given further, feedback, visual or otherwise), one (such as a brain, in theory) may learn to control things.
3)
It’s a lot harder to see how the internal states of a lump of rock could be “hooked up” in any corresponding manner without essentially subsuming it into something that we already think of as an agent.
Break it up, extract those rare earth metals, make a computer. Is it an agent now?
And perhaps the times when you ask ‘how are you doing’ will be slightly more impactful.