Interesting. The ability to fold it enough to fit in a backpack should reduce the hassle of storing it at my destination, which has been part of why I’ve been reluctant to use one.
PeterMcCluskey
Karma: 552 (LW), 0 (AF)
NewTop
I also ride my bike in Berkeley without a helmet.
Some other considerations which influence me:
-
helmet use seems to make crashes more likely (by making bikers and/or drivers less cautious), so it’s misleading to use data about harm that’s conditioned on there being a reported accident.
-
I’m fairly careful to avoid roads with heavy traffic, or with cars driving more than about 30 mph. I expect that fatality rates vary a lot by these road factors.
-
I use a cheap bike that doesn’t go as fast as the average bike.
Alas, I don’t have good evidence about how to quantify these considerations.
-
Economists usually treat minds as black boxes. That seems to help them develop their models, maybe via helping them to ignore issues such as “I’d feel embarrassed if my mind worked that way”.
There doesn’t seem to be much incentive for textbooks to improve their effectiveness at persuading the marginal student. The incentives might even be backwards, as becoming a good economist almost requires thinking in ways that seem odd to the average student.
Those concerns would have slowed adoption of agoric computing, but they seem to apply to markets in general, so they don’t seem useful in explaining why agoric computing is less popular than markets in other goods/services.
The central planner may know exactly what resources exist on the system they own, but they don’t know all the algorithms and data that are available somewhere on the internet. Agoric computing would enable more options for getting programmers and database creators to work for you.
One obstacle has been security. To develop any software that exchanges services for money, you need to put substantially more thought into the security risks of that software, and you probably can’t trust a large fraction of the existing base of standard software. Coauthor Mark S. Miller has devoted lots of effort to replacing existing operating systems and programming languages with secure alternatives, with very limited success.
One other explanation that I’ve wondered about involves conflicts of interest. Market interactions are valuable mainly when they generate cooperation among agents who have divergent goals. Most software development happens in environments where there’s enough cooperation that adding market forces wouldn’t provide much value via improved cooperation. I think that’s true even within large companies. I’ll guess that the benefits of the agoric approach only become interesting when large number of companies switch to using it, and there’s little reward to being the first such company.
Universities have tried something like this for tuition. See these mentions from Alex Tabarrok. They have some trouble with people defaulting.
Thanks, I’ve fixed those.
Drexler on AI Risk
After further rereading, I now think that what Drexler imagines is a bit more complex: (section 27.7) “senior human decision makers” would have access to a service with some strategic planning ability (which would have enough power to generate plans with dangerously broad goals), and they would likely restrict access to those high-level services.
I suspect Drexler is deliberately vague about the extent to which the strategic planning services will contain safeguards.
This, of course, depends on the controversial assumption that relatively responsible organizations will develop CAIS well before other entities are able to develop any form of equally powerful AI. I consider that plausible, but it seems to be one of the weakest parts of his analysis.
And presumably the publicly available AI services won’t be sufficiently general and powerful to enable random people to assemble them into an agent AGI? Combining a robocar + Google translate + an aircraft designer + a theorem prover doesn’t sound dangerous. But I’d prefer to have something more convincing than just “I spent a few minutes looking for risks, and didn’t find any”.
I was assuming that long term strategic planners (as described in section 27) are available as an AIS, and would be one of the components of the hypothetical AGI.
That’s not consistent with my understanding of section 27. My understanding is that Drexler would describe that as too dangerous.
suppose you asked the plan maker to create a plan to cure cancer.
I suspect that a problem here is that “plan maker” is ambiguous as to whether it falls within Drexler’s notion of something with a bounded goal.
CAIS isn’t just a way to structure software. It also requires some not-yet-common sense about what goals to give the software.
“Cure cancer” seems too broad to qualify as a goal that Drexler would consider safe to give to software. Sections 27 and 28 suggest that Drexler wants humans to break that down into narrower subtasks. E.g. he says:
By contrast, it is difficult to envision a development path in which AI developers would treat all aspects of biomedical research (or even cancer research) as a single task to be learned and implemented by a generic system.
My guess, based on crude extrapolations from reported nutrients of other dried foods, is that you’ll get half the nutrients of fresh mussels.
That ought to be a clear improvement on a vegan diet.
I suspect your main remaining reason for concern might be creatine.
My guesses about why vitamin pills tend to be ineffective (none of which apply to dried mussels):
* pills lack some important nutrients—ones which have not yet been recognized as important
* pills provide unnatural ratios of nutrients
* pills often provide some vitamins in a synthetic form, which not everyone converts to the biologically active form
Bundle your Experiments
Time Biases
I’m not talking about the range. Domain seems possibly right, but not as informative as I’d like. I’m talking about what parts of spacetime it cares about, and saying that it only cares about specific outputs of a specific process. Drexler refers to this as “bounded scope and duration”. Note that this will normally be an implicit utility function, that we infer from our understanding of the system.
“bounded utility function” is definitely not an ideal way of referring to this.
I want to draw separate attention to chapter 40 of Drexler’s paper, which uses what looks like a novel approach to argue that current supercomputers likely have more raw processing power than a human brain. I find that scary.
I consider it important to further clarify the notion of a bounded utility function.
A deployed neural network has a utility function that can be described as outputting a description of the patterns it sees in its most recent input, according to whatever algorithm it’s been trained to apply. It’s pretty clear to any expert that the neural network doesn’t care about anything beyond a specific set of numbers that it outputs.
A neural network that is in the process of being trained is slightly harder to analyze, but essentially the same. It cares about generating an algorithm that will be used in a deployed neural network. At any one training step, it is focused solely on applying fixed algorithms to produce improvements to the deployable algorithm. It has no concept that would lead it to look beyond its immediate task of incremental improvements to that deployable algorithm.
And in some important sense, those steps are the main ways in which AI gets used to produce cars that have superhuman driving ability, and the designers can prove (at least to themselves) that the cars won’t go out and buy more processing power, or forage for more energy.
Many forms of AI will be more complex than neural networks (e.g. they might be a mix of RL and neural networks), and I don’t have the expertise to extend this analysis to those systems. I’m confident that it’s possible in principle to get general-purpose superhuman AIs using only this kind of bounded utility function, but I’m uncertain how practical that is compared to a more unified agent with a broader utility function.
MIRI has a lot invested in the idea that AI safety is a hard problem which must have a difficult solution. So there’s a sense in which the salaries of their employees depend on them not understanding how a simple solution to FAI might work.
That doesn’t sound correct. My understanding is that they’re looking for simple solutions, in the sense that quantum mechanics and general relativity are simple. What they’ve invested a lot in is the idea that it’s hard to even ask the right questions about how AI alignment might work. They’re biased against easy solutions, but they might also be biased in favor of simple solutions.
Any discussion of bailouts ought to note that some countries have much fewer banking crises than the US.
Robin Hanson has, of course, written about how the simulation hypothesis should affect our behavior.