Arenamontanus

Karma: 420

Does davidad’s uploading moonshot work?

jacobjacob, lisathiergart, Anders_Sandberg, davidad and Arenamontanus

3 Nov 2023 2:21 UTC

145 points

32 comments25 min readLW link

Arenamontanus 21 Dec 2022 15:43 UTC
8 points
in reply to: weverka’s comment on: What 2026 looks like (Daniel’s Median Future)
We do not assume mirrors. As you say, there are big limits due to conservation of etendué. We are assuming (if I remember right) photovoltaic conversion into electricity and/or microwave beams received by rectennas. Now, all that conversion back and forth induces losses, but they are not orders of magnitude large.
In the years since we wrote that paper I have become much more fond of solar thermal conversion (use the whole spectrum rather than just part of it), and lightweight statite-style foil Dyson swarms rather than heavier collectors. The solar thermal conversion doesn’t change things much (but allows for a more clean-cut analysis of entropy and efficiency; see Badescu’s work). The statite style however reduces the material requirements many orders of magnitude: Mercury is safe, I only need the biggest asteroids.
Still, detailed modelling of the actual raw material conversion process would be nice. My main headache is not so much the energy input/waste heat removal (although they are by no means trivial and may slow things down for too concentrated mining operations—another reason to do it in the asteroid belt in many places), but how to solve the operations management problem of how many units of machine X to build at time t. Would love to do this in more detail!

Arenamontanus 28 Aug 2020 7:14 UTC
1 point
on: Multitudinous outside views
It seems to me that the real issue is rational weighing of reference classes when using multiple models. I want to assign them weights so that they form a good ensemble to build my forecasting distribution from, and these weights should ideally reflect my prior of them being relevant and good, model complexity, and perhaps that their biases are countered by other reference classes. In the computationally best of all possible world I go down the branching rabbit hole and also make probabilistic estimates of the weights. I could also wing it.
The problem is that the set of potential reference classes appears to be badly defined. The Tesla case potentially involves all possible subsets of stocks (2^N) over all possible time intervals (2^NT), but as the dictator case shows there is also potentially an unbounded set of other facts that might be included in selecting the reference classes. That means that we should be suspicious about having well-formed priors over the reference class set.
When I have some sensible reference classes pop up in my mind and I select from them I am doing naturalistic decision making where past experience gates availability. So while I should weigh their results together, I should be aware that they are biased in this way. I should broaden my model uncertainty for the weighing accordingly. But how much I broaden it depends on how large I allow the considerable set of potential reference classes to be, a separate meta-prior.

Arenamontanus 27 May 2020 19:17 UTC
14 points
on: Baking is Not a Ritual
I have been baking for a long time, but it took a surprisingly long while to get to this practical “not a ritual” stage. My problem was that I approached it as an academic subject: an expert tells you what you need to know when you ask, and then you try it. But the people around me knew how to bake in a practical, non-theoretical sense. So while my mother would immediately tell me how to fix a too runny batter and the importance of quickly working a pie dough, she could not explain why that worked in terms that I could understand. Much frustration ensued on both sides.
A while ago I came across Harold McGee’s “On Food and Cooking” and Jeff Potter’s “Cooking for Geeks”. These books explained what was going on in a format that made sense to me—starch gelation, protein denaturation, Maillard reactions, and so on - *and* linked it to the language of the kitchen. Suddenly I had freedom to experiment and observe but with the help of a framework of explicit chemistry and physics that helped me organise the observations. There has been a marked improvement in my results (although mother now finds me unbearably weird in the kitchen). It is also fun to share these insights: https://threadreaderapp.com/thread/1263895622433869827.html
The lesson of my experience is that sometimes it is important to seek out people who can explain and bootstrap your knowledge by speaking your “language”, even if they are not the conveniently close and friendly people around you. When you get non-working explanations they usually do not explain much, and hence just become ritual rules. Figuring out *why* explanations do not work for you is the first step, but then one needs to look around for sources of the right kind of explanations (which in my case took far longer). Of course, if you are not as theoretical explanation-dependent as I am but more of the practical, empirical bent, you can sidestep this issue to a large extent.

Arenamontanus 11 May 2020 23:55 UTC
1 point
in reply to: Aaro Salosensaari’s comment on: Maths writer/cowritter needed: how you can’t distinguish early exponential from early sigmoid
Awesome find! I really like the paper.
I had been looking at Fisher information myself during the weekend, noting that it might be a way of estimating uncertainty in the estimation using the Cramer-Rao bound (but quickly finding that the algebra got the better of me; it *might* be analytically solvable, but messy work).

Arenamontanus 6 May 2020 16:19 UTC
10 points
on: Assessing Kurzweil predictions about 2019: the results
I tried doing a PCA of the judgments, to see if there was any pattern in how the predictions were judged. However, the variance of the principal components did not decline fast. The first component explains just 14% of the variance, the next ones 11%, 9%, 8%… It is not like there are some very dominant low-dimensional or clustering explanation for the pattern of good or bad predictions.
No clear patterns when I plotted the predictions in PCA-space: https://www.dropbox.com/s/1jvhzcn6ngsw67a/kurzweilpredict2019.png?dl=0 (In this plot colour denotes mean assessor view of correctness, with red being incorrect, and size the standard deviation of assessor views, with large corresponding to more agreement). Some higher order components may correspond to particular correlated batches of questions like the VR ones.
(Or maybe I used the Matlab PCA routine wrong).

Arenamontanus 6 May 2020 9:54 UTC
28 points
on: Maths writer/cowritter needed: how you can’t distinguish early exponential from early sigmoid
Another nice example of how this is a known result but not presented in the academic literature:
https://constancecrozier.com/2020/04/16/forecasting-s-curves-is-hard/
The fundamental problem is not even distinguishing exponential from logistic: even if you *know* it is logistic, the parameters that you typically care about (inflexion point location and asymptote) are badly behaved until after the inflection point. As pointed out in the related twitter thread, you gain little information about the latter two in the early phase and only information about the first two in the mid phase: it is the sequential nature of the forecasting that is making this problem.
I find it odd that this does not have a classic paper. There are *lots* of Bass curves used in technology adoption studies, and serious business people are interested in using them to forecast—somebody ought to have told them they will get disappointed. It seems to be a result of the kind that everybody who knows the field would know but rarely mention since it is so obvious.

Arenamontanus 22 Apr 2020 13:55 UTC
4 points
on: Solar system colonisation might not be driven by economics
I think the argument can be reformulated like this: space has very large absolute amounts of some resources—matter, energy, distance (distance is a kind of resource useful for isolation/safety). The average density of these resources is very low (solar in space is within an order of magnitude of solar on Earth) and for matter it is often low-grade (Earth’s geophysics has created convenient ores). Hence matter and energy collection will only be profitable if (1) access gets cheap, (2) one can use automated collection with a very low marginal cost—plausibly robotic automation. (2) implies that a lot of material demands on Earth can be fulfilled that way too on Earth, making the only reason to go to space that one can get very large absolute amounts of stuff. That is fairly different from most material economics on Earth.
Typical ways of getting around this is either claiming special resources, like the Helium-3 lunar mining proposals (extremely doubtful; He3 fusion requires you to have solved lower energy fusion, which has plentiful fuels), or special services (zero gravity manufacturing, military, etc). I have not yet seen any convincing special resource, and while nice niche services may exist (obviously comms, monitoring and research; tourism? high quality fibre optics? military use?) they seem to be niche and near-Earth—not enough to motivate settling the place.
So I end up roughly with Stuart: the main reasons to actually settle space would be non-economic. That leads to another interesting question: do we have good data or theory for how often non-economic settlement occurs and works?
I think one interesting case study is Polynesian island settlement. The cost of exploratory vessels were a few percent of the local economy, but the actual settlement effort may have been somewhat costly (especially in people). Yet this may have reduced resource scarcity and social conflict: it was not so much an investment in getting another island as getting more of an island to oneself (and avoiding that annoying neighbour).

Arenamontanus 8 Jan 2020 23:13 UTC
3 points
on: Subscripting Typographic Convention For Citations/Dates/Sources/Evidentials: A Proposal
Overall, typographic innovations like all typography are better the less they stand out yet do their work. At least in somewhat academic text with references and notation subscripting appears to blend right in. I suspect the strength of the proposal is that one can flexibly apply it for readers and tone: sometimes it makes sense to say “I~2020~ thought”, sometimes “I thought in 2020″.
I am seriously planning to use it for inflation adjustment in my book, and may (publisher and test-readers willing) apply it more broadly in the text.

Arenamontanus 15 May 2019 7:09 UTC
18 points
on: Space colonization: what can we definitely do and how do we know that?
Looking back at our paper, I think the weakest points are (1) we handwave the accelerator a bit too much (I now think laser launching is the way to go), and (2) we also handwave the retro-rockets (it is hard to scale down nuclear rockets; I think a detachable laser retro-rocket is better now). I am less concerned about planetary disassembly and building destination infrastructure: this is standard extrapolation of automation, robotics and APM.
However, our paper mostly deals with sending a civilization’s seeds everywhere, it does not deal with near term space settlement. That requires a slightly different intellectual approach.
What I am doing in my book is trying to look at a “minimum viable product”—not a nice project worth doing (a la O’Neill/Bezos) but the crudest approach that can show a lower bound. Basically, we know humans can survive for years on something like the ISS. If we can show that an ISS-like system can (1) produce food and other necessities for life, (2) allow crew to mine space resources, (3) turn them into more habitat and life support material, (4) crew can thrive well enough to reproduce, and (5) this system can build more copies of itself with new crew at a faster rate than the system fails—then we have a pretty firm proof of space settlement feasibility. I suspect (1) is close to demonstration, (2) and (3) needs more work, (4) is likely a long term question that must be tested empirically, and (5) will be hard to strictly prove at present but can be made fairly plausible.
If the above minimal system is doable (and I am pretty firmly convinced it is—the hairy engineering problems are just messy engineering, rather than pushing against any limits of physics) then we can settle the solar system. Interstellar settlement requires either self-sufficient habitats that can last very long (and perhaps spread by hopping from Oort-object to Oort-object), AI-run mini-probes as in our paper, or extremely large amounts of energy for fast transport (I suspect having a Dyson sphere is a good start).
What links here?
- Which parts of the paper Eternity in Six Hours are iffy? by Ruby (6 May 2019 23:59 UTC; 18 points)

Arenamontanus 8 Jun 2017 15:50 UTC
0 points
in reply to: gathaung’s comment on: Hidden universal expansion: stopping runaways
I have not seen any papers about it, but did look around a bit while writing the paper.

However, a colleague and me analysed laser acceleration and it looks even better. Especially since one can do non-rigid lens systems to enable longer boosting. We developed the idea a fair bit but have not written it up yet.

I would suspect laser is the way to go.

Arenamontanus 1 Jun 2017 20:41 UTC
0 points
on: Regulatory lags for New Technology [2013 notes]
Another domain may be aviation. In the US, from the Wright brothers in 1903 to the Air Commerce Act 1926 it took 23 years.

Wikipedia: “In the early years of the 20th century aviation in America was not regulated. There were frequent accidents, during the pre-war exhibition era (1910–16) and especially during the barnstorming decade of the 1920s. Many aviation leaders of the time believed that federal regulation was necessary to give the public confidence in the safety of air transportation. Opponents of this view included those who distrusted government interference or wished to leave any such regulation to state authorities. Barnstorming accidents that led to such regulations during this period is accurately depicted in the 1975 film The Great Waldo Pepper.

At the urging of aviation industry leaders, who believed the airplane could not reach its full commercial potential without federal action to improve and maintain safety standards, President Calvin Coolidge appointed a board to investigate the issue. The board’s report favored federal safety regulation. To that end, the Air Commerce Act became law on May 20, 1926.”

The UK introduced regulation in 1920 and the Soviet Union in 1921. So a lag of 17-23 years seems to be a decent estimate here.

Arenamontanus 11 May 2017 10:48 UTC
2 points
on: Hidden universal expansion: stopping runaways
S. Jay Olson’s work on expanding civilizations is very relevant here, e.g. https://arxiv.org/abs/1608.07522 and https://arxiv.org/abs/1512.01521 That work suggests that even non-hidden civilizations will be fairly close to their light front.

Now, the METI application: if this scenario is true, then sending messages so that the expanding civilization notices us might be risky if they can quieten down and silently englobe or surprise us. (Surprise is likely more effective than englobement, since spamming the sky with quiet relativistic probes is hard to stop)

When does this matter? If we happen to be far away from the civilization, then they will notice the message late and we could have done all sorts of things in the meantime—escape, becoming an equivalently powerful civilization, gone extinct etc. We would have done the same even if they had not been there. So there is no change.

If they are already here there is only an effect if they react to the attempt at messaging (“Only talk to civs that want to talk to you”/”Only wipe out civs that might pollute the ether with deliberate messages”), so there is no change.

[ In fact, a civilization that deliberately tries to conceal itself may be particularly concerned with signalling civilizations since they might act as beacons: if they get cut off, that might tell other listeners where something is going on. Keeping them signalling even when well inside their englobement might be good camouflage. Until they are discreetly replaced with decoy young civilizations...]

So the only case that leads to nontrivial effects is the nearby but not here case. Friendly civilizations react by doing something friendly, but that would have happened anyway when they came here. Unfriendly civilizations use the information to be more efficiently unfriendly (like quiet englobement). The main thing that changes is that the opportunity for running away (or other reactive responses) decreases.

So the overall utility change is U(METI)-U(no METI) = -Pr[nearby but not here] * U(running away).

Now, Pr[nearby but not here] seems to be small for rapidly expanding civilizations. If they spread at speed v, then it is (v/c)^3. For v=0.9c, it is only 27.5%, and for 0.99c it is 2.97%. So it is a small net negative, assuming U(running away) is positive.

All of the above is conditioned on that interstellar/intergalactic expansion is doable. If it isn’t, we get informational game theory instead.

Arenamontanus 16 Sep 2015 15:10 UTC
3 points
on: A toy model of the control problem
It would be neat to actually make an implementation of this to show sceptics. It seems to be within the reach of a MSc project or so. The hard part is representing 2-5.

Arenamontanus 1 Jul 2015 9:17 UTC
4 points
in reply to: [deleted]’s comment on: Top 9+2 myths about AI risk
I think you will find this discussed in the Hanson-Yudkowsky foom debate. Robin thinks that distributed networks of intelligence (also known as economies) are indeed a more likely outcome than a single node bootstrapping itself to extreme intelligence. He has some evidence from the study of firms, which is a real-world example of how economies of scale can produce chunky but networked smart entities. As a bonus, they tend to benefit from playing somewhat nicely with the other entities.

The problem is that while this is a nice argument, would we want to bet the house on it? A lot of safety engineering is not about preventing the most likely malfunctions, but the worst malfunctions. Occasional paper jams in printers are acceptable, fires are not. So even if we thought this kind of softer distributed intelligence explosion was likely (I do) we could be wrong about the possibility of sharp intelligence explosions, and hence it is rational to investigate them and build safeguards.

Arenamontanus 1 Jul 2015 9:10 UTC
3 points
in reply to: Andy_McKenzie’s comment on: Top 9+2 myths about AI risk
I remember that we were joking at the NYC Singularity Summit workshop a few years back that maybe we should provide AI researchers with heroin and philosophers to slow them down.

As far as I have noticed, there are few if any voices in the academic/nearby AI safety community that promote slowing AI research as the best (or even a good) option. People talking about relinquishment or slowing seem to be far outside the main discourse, typically people who have only a passing acquaintance with the topic or a broader technology scepticism.

The best antidote is to start thinking about the details of how one would actually go about it: that generally shows why differential development is sensible.

Arenamontanus 30 Jun 2015 8:14 UTC
1 point
on: Top 9+2 myths about AI risk
I recently gave a talk at an academic science fiction conference about whether sf is useful for thinking about the ethics of cognitive enhancement. I think some of the conclusions are applicable to point 9 too:

(1) Bioethics can work in a “prophetic” and a “regulatory” mode. The first is big picture, proactive and open-ended, dealing with the overall aims we ought to have, possibilities, and values. It is open for speculation. The regulatory mode is about ethical governance of current or near-term practices. Ethicists formulate guidelines, point out problems, and suggest reforms, but their purpose is generally not to rethink these practices from the ground-up or to question the wisdom of the whole enterprise. As the debate about the role of speculative bioethics has shown, mixing the modes can be problematic. (Guyer and Moreno 2004) really takes bioethics to task for using science fiction instead of science to motivate arguments: they point out that this can actually be good if one does it inside the prophetic mode, but a lot of bioethics (like the President’s Council on Bioethics at the time) cannot decide what kind of consideration it is.

(2) Is it possible to find out things about the world by making stuff up? (Elgin 2014) argues fictions and thought experiments do exemplify patterns or properties that they share with phenomena in the real world, and hence we can learn something about the realized world from considering fictional worlds (i.e. there is a homeomorphism between them in some domain). It does require the fiction to be imaginative but not lawless: not every fiction or thought experiment has value in telling us something the real or moral world. This is of course why just picking a good or famous piece of fiction as a source of ideas is risky: it was selected not for how it reflects patterns in the real world, but for other reasons.

Considering Eliezer’s levels of intelligence in fictional characters is a nice illustration of this: level 1 intelligence characters show some patterns (being goal directed agents) that matter, and level 3 characters actually give examples of rational skilled cognition.

(3) Putting this together, if you want to use fiction in your argument, the argument better be in the more prophetic, open-ended mode (e.g. arguing that there is AI risks of various kind, what values are at stake etc.) and the fiction needs to have pretty not only high standards of not just internal consistency but actual mappability to the real world. If the discussion is on the more regulatory side (e.g. thinking of actual safeguards, risk assessment, institutional strategies) then fiction is unlikely to be helpful, and very likely (due to good story bias, easily inserted political agendas or different interpretations of worldview) introducing biasing or noise elements.

There are of course some exceptions. Hannu Rajaniemi provides a neat technical trick to the AI boxing problem in the second novel of his Quantum Thief trilogy (turn a classical computation into a quantum one that will decohere if it interacts with the outside world). But the fictions most people mention in AI safety discussions are unlikely to be helpful—mostly because very few stories succeed with point (2) (and if they are well written, they hide this fact convincingly!)

Arenamontanus 26 May 2015 15:23 UTC
3 points
on: No peace in our time?
Well, 70 years of ¹⁄₃₇ risk still has 13% chance of showing zero wars. Could happen. (Since we are talking about smaller ones rather than WWIII anthropics doesn’t distort the probabilities measurably.)

One could buy a Pinker improvement scenario and yet be concerned about a heavy tail due to nuclear or bio warfare of existential importance. The median cases might decline and the rate of events go down, yet the tail get nastier.

Arenamontanus 23 Oct 2014 0:50 UTC
3 points
in reply to: Stuart_Armstrong’s comment on: Anthropic signature: strange anti-correlations
This is incidentally another way of explaining the effect. Consider the standard diagram of the joint probability density and how it relates to correlation. Now take a bite out of the upper right corner of big X and big Y events: unless the joint density started in a really strange shape this will tend to make the correlation negative.

Arenamontanus 21 Oct 2014 23:53 UTC
11 points
in reply to: Stuart_Armstrong’s comment on: Anthropic signature: strange anti-correlations
It is pretty cute. I did a few Matlab runs with power-law distributed hazards, and the effect holds up well: http://aleph.se/andart2/uncategorized/anthropic-negatives/

Arenamontanus

Does davi­dad’s up­load­ing moon­shot work?

Does davidad’s uploading moonshot work?