PeterMcCluskey
Karma: 451
NewTop
Time Biases
I want to draw separate attention to chapter 40 of Drexler’s paper, which uses what looks like a novel approach to argue that current supercomputers likely have more raw processing power than a human brain. I find that scary.
I consider it important to further clarify the notion of a bounded utility function.
A deployed neural network has a utility function that can be described as outputting a description of the patterns it sees in its most recent input, according to whatever algorithm it’s been trained to apply. It’s pretty clear to any expert that the neural network doesn’t care about anything beyond a specific set of numbers that it outputs.
A neural network that is in the process of being trained is slightly harder to analyze, but essentially the same. It cares about generating an algorithm that will be used in a deployed neural network. At any one training step, it is focused solely on applying fixed algorithms to produce improvements to the deployable algorithm. It has no concept that would lead it to look beyond its immediate task of incremental improvements to that deployable algorithm.
And in some important sense, those steps are the main ways in which AI gets used to produce cars that have superhuman driving ability, and the designers can prove (at least to themselves) that the cars won’t go out and buy more processing power, or forage for more energy.
Many forms of AI will be more complex than neural networks (e.g. they might be a mix of RL and neural networks), and I don’t have the expertise to extend this analysis to those systems. I’m confident that it’s possible in principle to get general-purpose superhuman AIs using only this kind of bounded utility function, but I’m uncertain how practical that is compared to a more unified agent with a broader utility function.
MIRI has a lot invested in the idea that AI safety is a hard problem which must have a difficult solution. So there’s a sense in which the salaries of their employees depend on them not understanding how a simple solution to FAI might work.
That doesn’t sound correct. My understanding is that they’re looking for simple solutions, in the sense that quantum mechanics and general relativity are simple. What they’ve invested a lot in is the idea that it’s hard to even ask the right questions about how AI alignment might work. They’re biased against easy solutions, but they might also be biased in favor of simple solutions.
Any discussion of bailouts ought to note that some countries have much fewer banking crises than the US.
people generally identify as their characters, not their players.
I prefer to identify with my whole brain. I suspect that reduces my internal conflicts.
This seems a bit too strong. It seems to imply that I should ignore Bostrom’s writings about AI ethics, and only look to people such as Demis Hassabis.
Or if I thought that nobody was close to having the expertise to build a superintelligent AI, maybe I’d treat it as implying that it’s premature to have opinions about AI ethics.
Instead, I treat professional expertise as merely one piece of evidence about a person’s qualifications to inform us about ethics.
Book review: Artificial Intelligence Safety and Security
Based on focusing I have realized some feelings like tiredness are not really ‘real’. They’re just a felt sense of not wanting to keep programming.
Feelings such as tiredness involve more high-level processing than I used to think.
That doesn’t cause me to classify them as less real. Instead, I conclude that most, or maybe all, feelings of tiredness include some rather high-level predictions about the costs and benefits of whatever I’m doing now.
Perhaps the ability to recognize individuals isn’t as tied to being a social animal as I had thought
I expect multiple sources of evolutionary pressure for recognizing individuals. E.g. when a human chases an animal to exhaustion, the human needs to track that specific animal even if it disappears into a herd, so as to not make the mistake of chasing an animal that isn’t tired.
Being always hungry is a lousy way to loose weight. It means my body is always trying to conserve energy, as if this was a famine.
Part of the vicious cycle is addiction to food that doesn’t make us feel full (see the Satiety Index for ideas about which foods). Remember that obesity is virtually unknown in hunter-gatherers, even when they have plenty of food available. It takes modern foods to make obesity common (see Stephan Guyenet).
Intermittent hunger can work somewhat well for weight loss, but mainly I need to eat food that’s less addictive and that makes me feel full.
It is often difficult to get people to bet in markets. This looks like a case where employees will do approximately no betting unless there are unusual incentives. It’s hard to say whether these markets would produce enough benefit to pay for those incentives.
My intuition is that there’s likely some cheaper way of improving the situation.
I disagree with your analysis of “are we that ignorant?“.
For things like nuclear war or financial meltdown, we’ve got lots of relevant data, and not too much reason to expect new risks. For advanced nanotechnology, I think we are ignorant enough that a 10% chance sounds right (I’m guessing it will take something like $1 billion in focused funding).
With AGI, ML researchers can be influenced to change their forecast by 75 years by subtle changes in how the question is worded. That suggests unusual uncertainty.
We can see from Moore’s law and from ML progress that we’re on track for something at least as unusual as the industrial revolution.
The stock and bond markets do provide some evidence of predictability, but I’m unsure how good they are at evaluating events that happen much less than once per century.
By some strange coincidence, less than a week after posting this, I got another chance to observe the effects of being above the clouds. Technically, it was fog rather than clouds, but hiking up through and above fog has the same effect: in addition to creating the muted far-mode colors with fewer details that Josh mentions, it also enhances the impression of unusual height, by enabling my subconscious to imagine that the ground (or in this case ocean) beneath the clouds might be a half-mile below where I can see.
I wasn’t able to observe that I was more in far mode than on a typical hike. I’m pretty sure the unusual feelings were more along the lines of feeling high status, due to some correlation between being higher than my surroundings and being high status, or maybe due to the strategic advantage of having higher ground than any hostile forces. Or maybe just a feeling of accomplishment for having climbed so high.
At any rate, it feels quite good, and it seems similar to the effects from flying small aircraft.
The book is maybe 25% nostalgia, 30% blame. It seems pretty relevant to how we make long-term forecasts.
He’s talking SSTs. I’m unsure what propulsion methods he prefers for those. He claims sonic booms could have been cut in half compared to the Concorde. A good government would have set a fee for sonic booms that bore some resemblance to how much people disvalued loud noise. But the US government seems unwilling to say what conditions an SST would need to meet.
I’m unclear why I’d want undersea cities when there’s room for seasteads.
Lunar colonies would hedge against some catastrophic risks. They might be created for reasons similar to why Europeans settled the New World.
I got some good insights from the Charisma Tips website, which now seems to only be available via Wayback.
For writing a good profile, I liked: I Can’t Believe I’m Buying This Book: A Commonsense Guide to Successful Internet Dating, by Evan Marc Katz—mainly the examples in pages 36 through 39.
Authentic relating / circling workshops helped me, but they’re more oriented toward what to do when you’ve got a date.
Comsats seem to be working as predicted, based on an energy-intensive technology that stagnated in 1970 and on low energy electronics that have continued to progress well.
Asimov said in 1964:
Robots will neither be common nor very good in 2014, but they will be in existence.
Josh classifies current industrial robotics as 100% of what was predicted for today, home robots as 90%, and AI as 80%.
Arthur C. Clarke said in 1964:
It will be possible in that age, perhaps only 50 years from now, for a man to conduct his business from Tahiti or Bali just as well as he could from London … I am perfectly serious when I suggest that one day we may have brain surgeons in Edinburgh operating on patients in New Zealand.
Arthur C. Clarke in 1958 said we should have fusion power and a global library now, we should be getting weather control right about now, and gravity control in 2050.
Josh classifies our global library as 150% of what was predicted.
Other predictions Josh classifies as successful: home-based videophones, robocars, translating machines.
Other predictions that didn’t come close: lunar base, “Transportation 1000 mph and one cent per mile”, undersea cities.
There are lots of hypotheses about overall changes in innovation. But neither Josh nor I are talking about a general decline in innovation. I see patterns where there are big differences between industries, with innovation slowing in some and remaining good in others. I want to explain those differences.
I’m not talking about the range. Domain seems possibly right, but not as informative as I’d like. I’m talking about what parts of spacetime it cares about, and saying that it only cares about specific outputs of a specific process. Drexler refers to this as “bounded scope and duration”. Note that this will normally be an implicit utility function, that we infer from our understanding of the system.
“bounded utility function” is definitely not an ideal way of referring to this.