This shortform serves as a repository for my initial considerations for my forecast on the following Metaculus question (* see below for question link):
Genome editing is a type of genetic engineering in which DNA is inserted, deleted, modified or replaced in the genome of a living organism (Wikipedia). The first gene-edited babies—Lulu and Nana—were reportedly born in October 2018.
How many gene-edited babies will have been born worldwide by the end of 2029?
Question resolves according to birth counts given in the first authoritative report (so judged by the admins) to cover the entire 2029 calendar year, as well as all years preceding it.
After reading this, the following questions come to mind:
How many gene-edited babies have been born thus far (as of February 19th 2022)? [base-rate]
What are people’s current attitudes towards human gene-editing?
How might people’s attitudes towards human gene-editing change?
How much do people’s attitudes towards human gene-editing affect the regulations and policies on human gene-editing?
Are there historical technologies that fill similar societal niches, and if so, how did they turn out?
How likely is human gene-editing to take off (also, given this, how much will it take off?)?
Given this information, I doubt this base-rate is of much use, and believe instead that the use of gene-editing in humans will follow a nonlinear growth trajectory, with initialization occurring when the first nation legalizes human gene-editing.
I believe that, should people come to accept or desire human gene-editing, be it for treatment or enhancement, the scientific community will be unable to prevent these technologies from being used, somewhere.
Next, I believe that India might be one of the first few countries to approve use of human gene-editing; should India, or a cohort of other nations, adopt human gene-editing, I believe that this might rapidly (within 1 year) shift the Overton Window towards acceptance of human gene-editing, especially if the results of the editing appear to be promising.
Okay, so will any nation widely adopted human gene-editing? A Google search of “human gene editing india” produces results that give credence to the idea that, while human gene-editing in banned in India, there are many ambiguities in the laws, and many laws do not seem readily enforced. Many other nations surveyed in the PEW report also seem to have regulations on human gene-editing existing in “legal limbo”.
(5 & forecast) I would put the probability of at least one country adopting human gene-editing in the next 8 years (2029 is about 8 years away) at 35% (adoption scenario). So, the probability that no country adopts human gene-editing in the next 8 years would 65% (non-adoption scenario).
The adoption scenario (some nation(s) adopt(s) human gene-editing before 2029): I believe that the number of gene-edited humans might grow at a similar rate to how Internet usage grew (https://en.wikipedia.org/wiki/History_of_the_Internet#1989%E2%80%932004:_Rise_of_the_global_Internet,_Web_1.0 & https://www.internetworldstats.com/emarketing.htm), i.e. adoption of human gene-editing will be limited for the first 2-5 years, perhaps to treatment oriented use cases, before truly taking off (I believe usage for enhancement purpose might follow treatment usage in ~3 years). I believe this because both human gene-editing and the Internet appear to both be transformative technologies, and sentiment on human gene-editing appears similar (maybe more negative than simply disinterested) to early Internet usage sentiment. Sentiment against human gene-editing globally seems strong enough to make me believe that any “initial usage” will not occur until at least 2025. I believe that initial usage (including scenarios where more than a single nation adopts human gene-editing) over these 3-4 years will very likely be less than 10000 use cases. I believe that the first year might see something on the order of 250-1000 cases (25% lower bound − 75% upper bound), and, following the pattern of Internet growth, will increase to ~560-2250, then to ~1090-4365, and finally to ~2290-9170.
The non-adoption scenario (no nation legalizes human gene-editing): In this scenario, I believe there may still be somewhere between 5 and 100 (25% lower bound and 75% upper bound, respectively) illegal gene-edited births in the 8 years leading up to 2029.
So, altogether, the expected lower bound is [0.35 x 2290] + [0.65 x 5] = 801.5 + 3.25 = 804.75 = ~805 births, and the expected upper bound is [0.35 x 9170] + [0.65 x 100] = 3209.5 + 6.5 = 3216.0.
Until I take another look at this question, I put my current forecast at 805-3216.
Constraining and/or lowering (via bans, information concealment, raised expenses, etc…) capabilities gains via regulation of certain AI research and production components (weights, chips, electricity, code, etc…) is a strategy pursued in part or fully by different AI Safety organizations.
One friend (who works in this space) and I were very recently reflecting on AI progress, along with strategies to contend with AI related catastrophes. While we disagree on the success probabilities of different AI Safety plans and their facets, including those pertaining to policy and governance, we broadly support similar measures. He does, however, believe “shut down” strategies ought to be prioritized much more than I do.
This friend has, in the last year, met with between 10-50 (providing this range for preservation of anonymity) congressional staffers; the stories of these meetings he has could make for both an entertaining and informative short book, and I was grateful for the experiences and the details of how he prepares for conversations and framing AI that he imparted on me.
The density of familiarity with AI risk across the staffers was concentrated on weaponization; most staffers (save for 3) did not have much if any sense of AI catastrophe. This point is interesting, but I found how my friend perceived his role in AI Safety with these meetings more intriguing.
To prelude, both him and I believe that, generally speaking, individual humans and governments (including the US government) require some (the more spontaneous, the more impactful) catastrophe to engender productive responses. Examples of this include near-death experiences (for people) and the 11 September 2001 bombings for the US government.
With this remark made: my friend perceives his role to be one primarily of priming the staffers i.e. “the US government” to respond more effectively to catastrophe (e.g. 100s of thousands to millions but not billions dead) than they otherwise would have been able.
Any immediate actions taken by the staffers towards AI Safety, especially with respect to a full cessation of certain lines of research, access to computational resources, or information availability, my friend finds excellent, but due to the improbability of these occurring, he believes the brunt of his impact comes to fruition if there is an AI catastrophe that humans can recover from.
This updated how I perceive the “slow down” focused crowd in AI Safety from being one focused on literally having many aspects of AI progress stalled partially or fully to one of governmental and institutional response enhancement in fire-alarm moments.
To provide a better response to your first question than the one I’ve provided below, I would need to ask him to explain more than he has already.
From what he has remarked, the first several meetings were very stressful (for most people, this would, of course, be the case!) but soon he adjusted and developed a routine for his meetings.
While the routine could go off course based on the responsiveness of the individual(s) present (one staffer kept nodding Yes, had no questions, and then 20 minutes later remarked that they “take into account” what had been said; another staffer remarked that the US simply needed to innovate AI as much as possible, and that safety that stifled this was not to be prioritized; these are statements paraphrased from my friend), I get the sense that in most instances he has been able to provide adequate context on his organization and the broader situation with AI first (not sure which of these two comes first and for how long they are discussed).
Concerning provision of a description of the AI context, I am not sure how dynamic they are; I think he mentioned querying the staffers on their familiarity, and the impression he had was that most staffers listened well and thought critically about his remarks.
After the aforementioned descriptions, my friend begins discussing measures that can be taken in support of AI Safety.
He mentioned that he tries to steer framings away from those invoking thoughts of arms races or weaponization and instead focuses on uncontrollability “race to the bottom” scenarios, since the former categorization(s) given by others to the staffers has, in his experience, in some instances downplayed concerns for catastrophe and increased focus on further expanding AI capabilities to “outcompete China”.
My friend’s strategy framings seem appropriate and he is a good orator, but I have not the nuanced suggestion that I wanted for my conversation with him, as I’ve not thought enough about which AI risk framings and safety proposals to have ready for staffers and as I believe talking to staffers qualifies as an instance of the proverb “an ounce of practice outweighs a pound of precept”.
Summary: Introduction (I introduce this shortform series), Year 0 for Human History (I discuss when years for humanity should begin to be counted)
Introduction
This shortform post marks the beginning of me trying to share on LessWrong some of the thoughts and notes I generate each day.
I suspect that every “thoughts and notes” shortform I write will contain a brief summary of its content at the start, and there will very likely be days where I post multiple shortforms of this nature, hence the (X) after the date.
As for the year in the date on these posts, I want to use something other than the Gregorian calendar’s current year. Moreover, I want to better capture the time of origin for a key moment in human history, such as the origin of agriculture, writing, or permanent settlement. The rest of this shortform consists of some notes on this topic.
A Starting Year for Our Calendars
In 2019, after I watched the Kurzgesagt—In a Nutshell video A New History for Humanity – The Human Era (2016), I opted to change the year in the date in my journal entries from 2019 to 12019. This Kurzgesagt video describes the idea that different choices for “year 0” for the “human era” result in different perceptions of human history.
Regarding this claim, I generally agree. If “year 0” for humanity began when the first anatomically modern humans appeared, then the year would be ~202022, and if “year 0″ began when the first nuclear weapon was deployed, the “human era” would be only 77 years old. These scenarios seem to strongly allocate my attention in different areas, with the former placing my attention on the thickness and mysteries of what we today call “prehistory” and the latter focusing my attention on the rapid progress and dangers that are characteristic of modernity.
The Kurzgesagt video explores the idea of setting “year 0” to 12000 years ago (the 10th millennium BC), which is apparently around the time the first large scale human construction project seems to have taken place. Having 12000 years ago be “year 0″ means that, when the current year is being considered, more attention would likely be allocated to the emergence of widespread agriculture, writing, and intensive construction of settlements and cities than is currently allocated.
Agriculture began independently in different parts of the globe, and included a diverse range of taxa. At least eleven separate regions of the Old and New World were involved as independent centers of origin. The development of agriculture about 12,000 years ago changed the way humans lived. They switched from nomadic hunter-gatherer lifestyles to permanent settlements and farming.[1]
Wild grains were collected and eaten from at least 105,000 years ago.[2] However, domestication did not occur until much later. The earliest evidence of small-scale cultivation of edible grasses is from around 21,000 BC with the Ohalo II people on the shores of the Sea of Galilee.
Following the emergence of agriculture, construction and architectural practices became more complex, leading to larger projects and settlements (see History of construction and Neolithic architecture)
The Neolithic, also known as the New Stone Age, was a time period roughly from 9000 BC to 5000 BC named because it was the last period of the age before woodworking began.
Architectural advances are an important part of the Neolithic period (10,000-2000 BC), during which some of the major innovations of human history occurred. The domestication of plants and animals, for example, led to both new economics and a new relationship between people and the world, an increase in community size and permanence, a massive development of material culture, and new social and ritual solutions to enable people to live together in these communities.
Located in southern Turkey. The tell includes two phases of use, believed to be of a social or ritual nature by site discoverer and excavator Klaus Schmidt, dating back to the 10th–8th millennium BC. The structure is 300 m in diameter and 15 m high.
Writing systems are believed to have emerged independently of each other, with the oldest instance of writing being in Mesopotamia potentially as early as 3.4k BCE.
However, the discovery of the scripts of ancient Mesoamerica, far away from Middle Eastern sources, proved that writing had been invented more than once. Scholars now recognize that writing may have independently developed in at least four ancient civilizations: Mesopotamia (between 3400 and 3100 BCE), Egypt (around 3250 BCE),[4][5][2]China (1200 BCE),[6] and lowland areas of Southern Mexico and Guatemala (by 500 BCE).[7]
Given that these historical developments I have outlined above seem very valuable to consider in context of modern civilizational progress, I’ve decided to take “year 0” to be 12000 years ago. The official name for this calendar system is actually the Holocene calendar, which was developed by Cesare Emiliani in 1993. The current year in the Holocene calendar is 12022 HE. Below are two comments on the benefits and accuracy, respectively, of the Holocene calendar’s Wikipedia page:
Human Era proponents claim that it makes for easier geological, archaeological, dendrochronological, anthropological and historical dating, as well as that it bases its epoch on an event more universally relevant than the birth of Jesus. All key dates in human history can then be listed using a simple increasing date scale with smaller dates always occurring before larger dates. Another gain is that the Holocene Era starts before the other calendar eras, so it could be useful for the comparison and conversion of dates from different calendars.
When Emiliani discussed the calendar in a follow-up article in 1994, he mentioned that there was no agreement on the date of the start of the Holocene epoch, with estimates at the time ranging between 12,700 and 10,970 years BP.[5] Since then, scientists have improved their understanding of the Holocene on the evidence of ice cores and can now more accurately date its beginning. A consensus view was formally adopted by the IUGS in 2013, placing its start at 11,700 years before 2000 (9701 BC), about 300 years more recent than the epoch of the Holocene calendar.[6]
So, why is the year on this shortform 0012022 and not just 12022? There are two reasons for this. The first is that I would like for myself to think more deeply and frequently about my own future and about humanity’s long-term future.
An organization developed around the idea of thinking about and safeguarding humanity’s future is the Long Now Foundation (LNF), which most LWers have likely heard of. This is its description:
The Long Now Foundation
is a nonprofit established in 01996 to foster long-term thinking.
Our work encourages imagination at the timescale of civilization — the next and last 10,000 years — a timespan we call the long now.
The LNF’s foundation year consists of 1996 with a 0 appended to the front, indicating that the timeframe under consideration − 10k years—is slowly being reached, one year at a time.
I aim to do a similar thing but believe that the timescale of 10k years is too short, so I instead opt for 1 million years, given that 1 million years is roughly the base rate for hominin species survival duration. It is also very interesting to imagine what humanity will be doing (should they persist) 1 million years following the start of the agricultural revolution. So, 12022 → 0012022.
Snyder-Beattie, Andrew E., Toby Ord, and Michael B. Bonsall. “An upper bound for the background rate of human extinction.” Scientific reports 9, no. 1 (2019): 1-9.
Hominin survival times. Next, we evaluate whether the upper bound is consistent with the broader hominin fossil record. There is strong evidence that Homo erectus lasted over 1.7 Myr and Homo habilis lasted 700 kyr [21], indicating that our own species’ track record of survival exceeding 200 kyr is not unique within our genus. Fossil record data indicate that the median hominin temporal range is about 620 kyr, and after accounting for sample bias in the fossil record this estimate rises to 970 kyr [22] . Although it is notable that the hominin lineage seems to have a higher extinction rate than those typical of mammals, these values are still consistent with our upper bound. It is perhaps also notable that some hominin species were likely driven to extinction by our own lineage [34], suggesting an early form of anthropogenic extinction risk.
I will close this shortform post here, but definitely want to parse out my thoughts concerning humanity’s future more in subsequent posts, and enjoyed writing this first post.
I am a someone who has participated informally in self-experimentation and in quantified-self activities.
Reading work composed by Gwern (e.g. this piece) and niplav’s (e.g. this piece), alongside writing in the fields of medicine (e.g. this paper) and psychology (e.g. this paper), has been both illuminating and motivating.
I am trying to formalize my data-recording for SE/QS and have found that the following approaches have not “stuck”:
(1) automated emails or Google Forms
(2) cron-tabs on my MacBook for digital EOD entry of data
(3) alarms at EOD for manual entry of data into .csv, .txt, or .json files
One approach that has given me optimism and feels relatively more entertaining (i.e. that seems to “stick” better) has been the following: once per week or month I generate a LaTeX PDF document (using Python to generate the document w/ configuration and command-line arguments) with checkboxes next to categorical variables I want to track (e.g. whether I have consumed at least 2 liters of water; these are loaded from a config.json file) and EOD I fill in the boxes with a pencil for what I did. The goal is to eventually be able to scan the PDF sheets and automatically extract the data for analysis.
Here is a section of an example sheet: here. Here are some of the variables: □ Reading for at least an hour and a half. □ Studying Anki cards at least once. □ Bathing at least once. □ Making at least one comment on an on-line forum. □ Consuming at least a coffee’s worth of
caffeine. □ Going to bed before 1:00 AM EST. □ Drinking at least 2 liters of water. □ Meditating for at least 15 minute
(Forecast Object) What is the object that we want to forecast? Is it a time series, such as sales of a firm recorded over time, or an event, such as devaluation of a currency, or something else? Appropriate forecasting strategies depend on the nature of the object being forecast.
(Information Set) On what information will the forecast be based? In a time series environment, for example, are we forecasting one series, several, or thousands? And what is the quantity and quality of the data? Appropriate forecasting strategies depend on the information set, broadly interpreted to not only quantitative data but also expert opinion, judgment, and accumulated wisdom.
(Model Uncertainty and Improvement) Does our forecasting model match the true GDP? Of course not. One must never, ever, be so foolish as to be lulled into such a naive belief. All models are false: they are intentional abstractions of a much more complex reality. A model might be useful for certain purposes and poor for others. Models that once worked well may stop working well. One must continually diagnose and assess both empirical performance and consistency with theory. The key is to work continuously toward model improvement.
(Forecast Horizon) What is the forecast horizon of interest, and what determines it? Are we interested, for example, in forecasting one month ahead, one year ahead, or ten years ahead (called h-step-ahead fore- casts, in this case for h = 1, h = 12 and h = 120 months)? Appropriate forecasting strategies likely vary with the horizon.
(Structural Change) Are the approximations to reality that we use for forecasting (i.e., our models) stable over time? Generally not. Things can change for a variety of reasons, gradually or abruptly, with obviously important implications for forecasting. Hence we need methods of detecting and adapting to structural change.
(Forecast Statement) How will our forecasts be stated? If, for exam- ple, the object to be forecast is a time series, are we interested in a single “best guess” forecast, a “reasonable range” of possible future values that reflects the underlying uncertainty associated with the forecasting prob- lem, or a full probability distribution of possible future values? What are the associated costs and benefits?
(Forecast Presentation) How best to present forecasts? Except in the simplest cases, like a single h-step-ahead point forecast, graphical methods are valuable, not only for forecast presentation but also for forecast construction and evaluation.
(Decision Environment and Loss Function) What is the decision environment in which the forecast will be used? In particular, what decision will the forecast guide? How do we quantify what we mean by a “good” forecast, and in particular, the cost or loss associated with forecast errors of various signs and sizes?
(Model Complexity and the Parsimony Principle) What sorts of models, in terms of complexity, tend to do best for forecasting in business, finance, economics, and government? The phenomena that we model and forecast are often tremendously complex, but it does not necessarily follow that our forecasting models should be complex. Bigger forecasting models are not necessarily better, and indeed, all else equal, smaller models are generally preferable (the “parsimony principle”).
(Unobserved Components) In the leading time case of time series, have we successfully modeled trend? Seasonality? Cycles? Some series have all such components, and some not. They are driven by very different factors, and each should be given serious attention.
3.
Question: How should I measure the long-term civilizational importance of the subject of a forecasting question?
I’ve used the Metaculus API to collect my predictions on open, closed, and resolved questions.
I would like to organize these predictions; one way I want to do this is by the “civilizational importance” of the forecasting question’s content.
Right now, I’ve thought to given subjective ratings of importance on logarithmic scale, but want a more formal system of measurement.
Another idea for each question is to give every category a score of 0 (no relevance), 1 (relevance), or 2 (relevant and important). For example, if all of my categories “Biology, Astronomy, Space_Industry, and Sports”, then the question—Will SpaceX send people to Mars by 2030? - would have this dictionary {”Biology”:0, “Space_Industry”:2, “Astronomy”:1, “Sports”:0}. I’m unsure whether this system is helpful.
Summary: An assessment of the conditions under which civilizational collapse may occur due to climate change would greatly improve the ability of the public and policymakers to address the threats from climate change, according to academic researchers Steela et al. in a PNAS opinion piece. While literature on climate change (e.g., reports from the Intergovernmental Panel on Climate Change) typically covers the deleterious effects that climate change is having or will have on human activities, there has been much less focus on exactly how climate change might factor into different scenarios for civilization collapse. Given the deficits in this research topic, Steela et al. outline three civilizational collapse scenarios that could stem from climate change—local collapse, broken world, and global collapse—and then discuss three groups of mechanisms—direct impacts, socio-climate feedbacks, and exogenous shock vulnerability—for how these scenarios might be realised. (6 October 2022)
Policy comment:Just as governments and policymakers have directed funding and taken action to mitigate the harmful, direct effects of climate change, it seems natural that they should take the next step and address making the aspects of civilization most vulnerable to climate change more robust. The recommendation in this paper for policymakers and researchers alike to promote more rigorous scientific investigation of the mechanisms and factors of civilizational collapse involving climate change seems keen. While this paper does not perform a detailed examination of the scenarios and mechanisms of civilizational collapse that it proposes, it is a call-to-action for more work to understand how climate change affects civilization stability and the role of climate change in civilization collapse.
2.
A condensed version of the summary and policy comment in (1)
Summary: Humanity must understand how climate change (CC) could engender civilizational collapse. Coverage of this topic is sparse relative coverage of CC’s direct effects. Steela et al.’s PNAS opinion piece is a call to action for more research on this topic; they contribute an outline of 3 collapse scenarios—local collapse, broken world, and global collapse—and 3 collapse mechanisms—direct impacts, socio-climate feedbacks, and exogenous shock vulnerability (6 October 2022).
Policy comment: Policymakers and researchers need to promote research on the effects of climate change on civilizational stability so that critical societal institutions and infrastructure are protected from collapse. Such research efforts would include further investigations of the many scenarios and mechanisms through which civilization may collapse due to climate change; Steela et al. lay some groundwork in this regard, but fail to provide a detailed examination.
3.
One issue I have is being concise with my writing. This was recently pointed out to me by my friend Evan, when I asked him to read (1), and I want to write some thoughts of mine that were evoked by the conversation.
My first thought: What do I want myself and others to get from my writing?
I want to learn, and writing helps with this. I want to generate novel and useful ideas and to share them with people. I want to show people what I’ve done or am doing. I want a record of my thinking on certain topics
I want my writing to help others learn efficiently and I want to tell people entertaining stories, ones that engender curiosity.
My next thought: How is my writing inadequate?
I aim for transparency, informativeness, clarity, and efficiency in my writing, but feel that my writing is much less transparent, informative, clear, and efficient than it could be.
W.r.t. transparency, my model is Reasoning Transparency. My writing sometimes includes answers to these questions[1] (this comment).
W.r.t. informativeness, I assume someone has already thought about or attempted what I am working on, so I try not to repeat (Don’t repeat yourself) and to synthesize works when synthesizing has not yet occurred or has occurred but inadequately.
W.r.t. clarity, I try to edit my work multiple times and make it clear what I want to be understood. I read my writing aloud to determine if hearing it is pleasurable.
W.r.t. efficiency, my sense of where to allocate attention across my writing is fuzzy. I use editing and footnotes to consolidate, but still have trouble.
I don’t have good ways to measure or assess these things in my writing, and I haven’t decided which hypothetical audiences to gear my writing towards; I believe this decision affects how much effort I expend optimizing at least transparency and efficiency.
I will address my writing again at some point, but think it best I read the advice of others first.
4.
My friend Evan on concision:
Yelling at people on the internet is a general waste of time, but it does teach concision. No matter how sound your argument, if you say something in eight paragraphs and then your opponent comes in and summarizes it perfectly in twenty words, you look like an idiot. Don’t look like an idiot in front of people! Be concise.
Coverage of this topic is sparse relative to coverage of CC’s direct effects.
The idea is that the corpus of work on how climate change is harmful to civilization includes few detailed analyses of the mechanisms through which climate change leads to civilizational collapse but does includes many works on the direct effects of climate change.
For the former comment, I am not sure what you mean w.r.t “engender”.
Glad I’ve helped with the part where I was not ignorant and confused myself, that is with not knowing the word engender and the use of it. Thanks for pointing it out clearly. By the way it seems “cause” would convey the same meaning and might be easier to congest in general.
Purpose
This shortform serves as a repository for my initial considerations for my forecast on the following Metaculus question (* see below for question link):
Forecast
How many gene-edited babies will have been born worldwide by the end of 2029?
This question was authored by Pablo on Metaculus.
After reading this, the following questions come to mind:
How many gene-edited babies have been born thus far (as of February 19th 2022)? [base-rate]
What are people’s current attitudes towards human gene-editing?
How might people’s attitudes towards human gene-editing change?
How much do people’s attitudes towards human gene-editing affect the regulations and policies on human gene-editing?
Are there historical technologies that fill similar societal niches, and if so, how did they turn out?
How likely is human gene-editing to take off (also, given this, how much will it take off?)?
(1) This source (https://getanimated.uk.com/meet-lulu-and-nana-the-worlds-first-crispr-genome-edited-babies/), along with Eli’s comment on this question (https://www.metaculus.com/questions/3289/how-many-gene-edited-babies-will-have-been-born-worldwide-by-the-end-of-2029/#comment-79822), make me believe that the base rate is 2 (I count the twins, Lula and Nana, as a single instance of gene-edited babies) in 2022 − 2019 = 3 years (the question was written in 2019).
(2-4 & 6) Human gene-editing seems to be highly divisive in the scientific community (see https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.3000224&type=printable). Also, generally, people seem to be averse to gene-editing in humans for enhancement purposes, but seem to agree that gene-editing may be useful for treatment of disease (see https://www.pewresearch.org/science/2020/12/10/biotechnology-research-viewed-with-caution-globally-but-most-support-gene-editing-for-babies-to-treat-disease/). This previous source from the PEW Research Center also indicates that religion is the dominant factor in people’s acceptance of human gene-editing. Indian survey respondents supported gene-editing in humans the most, and deemed it appropriate by a large margin.
Given this information, I doubt this base-rate is of much use, and believe instead that the use of gene-editing in humans will follow a nonlinear growth trajectory, with initialization occurring when the first nation legalizes human gene-editing.
I believe that, should people come to accept or desire human gene-editing, be it for treatment or enhancement, the scientific community will be unable to prevent these technologies from being used, somewhere.
Next, I believe that India might be one of the first few countries to approve use of human gene-editing; should India, or a cohort of other nations, adopt human gene-editing, I believe that this might rapidly (within 1 year) shift the Overton Window towards acceptance of human gene-editing, especially if the results of the editing appear to be promising.
Okay, so will any nation widely adopted human gene-editing? A Google search of “human gene editing india” produces results that give credence to the idea that, while human gene-editing in banned in India, there are many ambiguities in the laws, and many laws do not seem readily enforced. Many other nations surveyed in the PEW report also seem to have regulations on human gene-editing existing in “legal limbo”.
(5 & forecast) I would put the probability of at least one country adopting human gene-editing in the next 8 years (2029 is about 8 years away) at 35% (adoption scenario). So, the probability that no country adopts human gene-editing in the next 8 years would 65% (non-adoption scenario).
The adoption scenario (some nation(s) adopt(s) human gene-editing before 2029): I believe that the number of gene-edited humans might grow at a similar rate to how Internet usage grew (https://en.wikipedia.org/wiki/History_of_the_Internet#1989%E2%80%932004:_Rise_of_the_global_Internet,_Web_1.0 & https://www.internetworldstats.com/emarketing.htm), i.e. adoption of human gene-editing will be limited for the first 2-5 years, perhaps to treatment oriented use cases, before truly taking off (I believe usage for enhancement purpose might follow treatment usage in ~3 years). I believe this because both human gene-editing and the Internet appear to both be transformative technologies, and sentiment on human gene-editing appears similar (maybe more negative than simply disinterested) to early Internet usage sentiment. Sentiment against human gene-editing globally seems strong enough to make me believe that any “initial usage” will not occur until at least 2025. I believe that initial usage (including scenarios where more than a single nation adopts human gene-editing) over these 3-4 years will very likely be less than 10000 use cases. I believe that the first year might see something on the order of 250-1000 cases (25% lower bound − 75% upper bound), and, following the pattern of Internet growth, will increase to ~560-2250, then to ~1090-4365, and finally to ~2290-9170.
The non-adoption scenario (no nation legalizes human gene-editing): In this scenario, I believe there may still be somewhere between 5 and 100 (25% lower bound and 75% upper bound, respectively) illegal gene-edited births in the 8 years leading up to 2029.
So, altogether, the expected lower bound is [0.35 x 2290] + [0.65 x 5] = 801.5 + 3.25 = 804.75 = ~805 births, and the expected upper bound is [0.35 x 9170] + [0.65 x 100] = 3209.5 + 6.5 = 3216.0.
Until I take another look at this question, I put my current forecast at 805-3216.
(*)
Constraining and/or lowering (via bans, information concealment, raised expenses, etc…) capabilities gains via regulation of certain AI research and production components (weights, chips, electricity, code, etc…) is a strategy pursued in part or fully by different AI Safety organizations.
One friend (who works in this space) and I were very recently reflecting on AI progress, along with strategies to contend with AI related catastrophes. While we disagree on the success probabilities of different AI Safety plans and their facets, including those pertaining to policy and governance, we broadly support similar measures. He does, however, believe “shut down” strategies ought to be prioritized much more than I do.
This friend has, in the last year, met with between 10-50 (providing this range for preservation of anonymity) congressional staffers; the stories of these meetings he has could make for both an entertaining and informative short book, and I was grateful for the experiences and the details of how he prepares for conversations and framing AI that he imparted on me.
The density of familiarity with AI risk across the staffers was concentrated on weaponization; most staffers (save for 3) did not have much if any sense of AI catastrophe. This point is interesting, but I found how my friend perceived his role in AI Safety with these meetings more intriguing.
To prelude, both him and I believe that, generally speaking, individual humans and governments (including the US government) require some (the more spontaneous, the more impactful) catastrophe to engender productive responses. Examples of this include near-death experiences (for people) and the 11 September 2001 bombings for the US government.
With this remark made: my friend perceives his role to be one primarily of priming the staffers i.e. “the US government” to respond more effectively to catastrophe (e.g. 100s of thousands to millions but not billions dead) than they otherwise would have been able.
Any immediate actions taken by the staffers towards AI Safety, especially with respect to a full cessation of certain lines of research, access to computational resources, or information availability, my friend finds excellent, but due to the improbability of these occurring, he believes the brunt of his impact comes to fruition if there is an AI catastrophe that humans can recover from.
This updated how I perceive the “slow down” focused crowd in AI Safety from being one focused on literally having many aspects of AI progress stalled partially or fully to one of governmental and institutional response enhancement in fire-alarm moments.
> I was grateful for the experiences and the details of how he prepares for conversations and framing AI that he imparted on me.
I’m curious, what was his strategy for preparing for these discussions? What did he discuss?
> This updated how I perceive the “show down” focused crowd
possible typo?
Thank you for the typo-linting.
To provide a better response to your first question than the one I’ve provided below, I would need to ask him to explain more than he has already.
From what he has remarked, the first several meetings were very stressful (for most people, this would, of course, be the case!) but soon he adjusted and developed a routine for his meetings.
While the routine could go off course based on the responsiveness of the individual(s) present (one staffer kept nodding Yes, had no questions, and then 20 minutes later remarked that they “take into account” what had been said; another staffer remarked that the US simply needed to innovate AI as much as possible, and that safety that stifled this was not to be prioritized; these are statements paraphrased from my friend), I get the sense that in most instances he has been able to provide adequate context on his organization and the broader situation with AI first (not sure which of these two comes first and for how long they are discussed).
Concerning provision of a description of the AI context, I am not sure how dynamic they are; I think he mentioned querying the staffers on their familiarity, and the impression he had was that most staffers listened well and thought critically about his remarks.
After the aforementioned descriptions, my friend begins discussing measures that can be taken in support of AI Safety.
He mentioned that he tries to steer framings away from those invoking thoughts of arms races or weaponization and instead focuses on uncontrollability “race to the bottom” scenarios, since the former categorization(s) given by others to the staffers has, in his experience, in some instances downplayed concerns for catastrophe and increased focus on further expanding AI capabilities to “outcompete China”.
My friend’s strategy framings seem appropriate and he is a good orator, but I have not the nuanced suggestion that I wanted for my conversation with him, as I’ve not thought enough about which AI risk framings and safety proposals to have ready for staffers and as I believe talking to staffers qualifies as an instance of the proverb “an ounce of practice outweighs a pound of precept”.
Thoughts and Notes: October 10th 0012022 (1)
Summary: Introduction (I introduce this shortform series), Year 0 for Human History (I discuss when years for humanity should begin to be counted)
Introduction
This shortform post marks the beginning of me trying to share on LessWrong some of the thoughts and notes I generate each day.
I suspect that every “thoughts and notes” shortform I write will contain a brief summary of its content at the start, and there will very likely be days where I post multiple shortforms of this nature, hence the (X) after the date.
As for the year in the date on these posts, I want to use something other than the Gregorian calendar’s current year. Moreover, I want to better capture the time of origin for a key moment in human history, such as the origin of agriculture, writing, or permanent settlement. The rest of this shortform consists of some notes on this topic.
A Starting Year for Our Calendars
In 2019, after I watched the Kurzgesagt—In a Nutshell video A New History for Humanity – The Human Era (2016), I opted to change the year in the date in my journal entries from 2019 to 12019. This Kurzgesagt video describes the idea that different choices for “year 0” for the “human era” result in different perceptions of human history.
Regarding this claim, I generally agree. If “year 0” for humanity began when the first anatomically modern humans appeared, then the year would be ~202022, and if “year 0″ began when the first nuclear weapon was deployed, the “human era” would be only 77 years old. These scenarios seem to strongly allocate my attention in different areas, with the former placing my attention on the thickness and mysteries of what we today call “prehistory” and the latter focusing my attention on the rapid progress and dangers that are characteristic of modernity.
The Kurzgesagt video explores the idea of setting “year 0” to 12000 years ago (the 10th millennium BC), which is apparently around the time the first large scale human construction project seems to have taken place. Having 12000 years ago be “year 0″ means that, when the current year is being considered, more attention would likely be allocated to the emergence of widespread agriculture, writing, and intensive construction of settlements and cities than is currently allocated.
Some notes for the preceding paragraph:
Agriculture seems to have started roughly 12k years ago (see History of agriculture).
Following the emergence of agriculture, construction and architectural practices became more complex, leading to larger projects and settlements (see History of construction and Neolithic architecture)
The oldest known surviving manmade building is Göbekli Tepe, which was make between 12k to 10k years ago (this is the structure alluded to in the Kurzgesagt video I mentioned earlier).
Writing systems are believed to have emerged independently of each other, with the oldest instance of writing being in Mesopotamia potentially as early as 3.4k BCE.
Given that these historical developments I have outlined above seem very valuable to consider in context of modern civilizational progress, I’ve decided to take “year 0” to be 12000 years ago. The official name for this calendar system is actually the Holocene calendar, which was developed by Cesare Emiliani in 1993. The current year in the Holocene calendar is 12022 HE. Below are two comments on the benefits and accuracy, respectively, of the Holocene calendar’s Wikipedia page:
So, why is the year on this shortform 0012022 and not just 12022? There are two reasons for this. The first is that I would like for myself to think more deeply and frequently about my own future and about humanity’s long-term future.
An organization developed around the idea of thinking about and safeguarding humanity’s future is the Long Now Foundation (LNF), which most LWers have likely heard of. This is its description:
The LNF’s foundation year consists of 1996 with a 0 appended to the front, indicating that the timeframe under consideration − 10k years—is slowly being reached, one year at a time.
I aim to do a similar thing but believe that the timescale of 10k years is too short, so I instead opt for 1 million years, given that 1 million years is roughly the base rate for hominin species survival duration. It is also very interesting to imagine what humanity will be doing (should they persist) 1 million years following the start of the agricultural revolution. So, 12022 → 0012022.
From An upper bound for the background rate of human extinction (Snyder-Beattie et al., 2019)
Snyder-Beattie, Andrew E., Toby Ord, and Michael B. Bonsall. “An upper bound for the background rate of human extinction.” Scientific reports 9, no. 1 (2019): 1-9.
I will close this shortform post here, but definitely want to parse out my thoughts concerning humanity’s future more in subsequent posts, and enjoyed writing this first post.
I am a someone who has participated informally in self-experimentation and in quantified-self activities.
Reading work composed by Gwern (e.g. this piece) and niplav’s (e.g. this piece), alongside writing in the fields of medicine (e.g. this paper) and psychology (e.g. this paper), has been both illuminating and motivating.
I am trying to formalize my data-recording for SE/QS and have found that the following approaches have not “stuck”:
(1) automated emails or Google Forms
(2) cron-tabs on my MacBook for digital EOD entry of data
(3) alarms at EOD for manual entry of data into
.csv,.txt, or.jsonfilesOne approach that has given me optimism and feels relatively more entertaining (i.e. that seems to “stick” better) has been the following: once per week or month I generate a LaTeX PDF document (using Python to generate the document w/ configuration and command-line arguments) with checkboxes next to categorical variables I want to track (e.g. whether I have consumed at least 2 liters of water; these are loaded from a
config.jsonfile) and EOD I fill in the boxes with a pencil for what I did. The goal is to eventually be able to scan the PDF sheets and automatically extract the data for analysis.Here is a section of an example sheet: here. Here are some of the variables: □ Reading for at least an hour and a half. □ Studying Anki cards at least once. □ Bathing at least once. □ Making at least one comment on an on-line forum. □ Consuming at least a coffee’s worth of caffeine. □ Going to bed before 1:00 AM EST. □ Drinking at least 2 liters of water. □ Meditating for at least 15 minute
Thoughts, Notes: 10/14/0012022 (2)
Contents:
Track Record, Binary, Metaculus, 10/14/0012022
Quote: Universal Considerations [Forecasting]
Question: on measuring importance of forecasting questions
Please tell me how my writing and epistemics are inadequate.
1.
My Metaculus Track Record, Binary, [06/21/0012021 − 10/14/0012022]
2.
The Universal Considerations for forecasting in Chapter 2 of Francis X. Diebold’s book Forecasting in Economics, Business, Finance and Beyond:
3.
Question: How should I measure the long-term civilizational importance of the subject of a forecasting question?
I’ve used the Metaculus API to collect my predictions on open, closed, and resolved questions.
I would like to organize these predictions; one way I want to do this is by the “civilizational importance” of the forecasting question’s content.
Right now, I’ve thought to given subjective ratings of importance on logarithmic scale, but want a more formal system of measurement.
Another idea for each question is to give every category a score of 0 (no relevance), 1 (relevance), or 2 (relevant and important). For example, if all of my categories “Biology, Astronomy, Space_Industry, and Sports”, then the question—Will SpaceX send people to Mars by 2030? - would have this dictionary {”Biology”:0, “Space_Industry”:2, “Astronomy”:1, “Sports”:0}. I’m unsure whether this system is helpful.
Does anyone have any thoughts for this?
Thoughts, Notes: 10/14/0012022 (1)
Contents:
Summary, comment: Climate change and the threat to civilization (10/06/2022)
Compression of (1)
Thoughts: writing and condensing information
Quote: my friend Evan on concision
To the reader: Please point out inadequacies in my writing.
1.
Article: Climate change and the threat to civilization (10/06/2022)
Context: My work for Rumtin Sempasspour (gcrpolicy.com) includes summarizing articles relevant to GCRs and GCR policy.
Summary: An assessment of the conditions under which civilizational collapse may occur due to climate change would greatly improve the ability of the public and policymakers to address the threats from climate change, according to academic researchers Steela et al. in a PNAS opinion piece. While literature on climate change (e.g., reports from the Intergovernmental Panel on Climate Change) typically covers the deleterious effects that climate change is having or will have on human activities, there has been much less focus on exactly how climate change might factor into different scenarios for civilization collapse. Given the deficits in this research topic, Steela et al. outline three civilizational collapse scenarios that could stem from climate change—local collapse, broken world, and global collapse—and then discuss three groups of mechanisms—direct impacts, socio-climate feedbacks, and exogenous shock vulnerability—for how these scenarios might be realised. (6 October 2022)
Policy comment: Just as governments and policymakers have directed funding and taken action to mitigate the harmful, direct effects of climate change, it seems natural that they should take the next step and address making the aspects of civilization most vulnerable to climate change more robust. The recommendation in this paper for policymakers and researchers alike to promote more rigorous scientific investigation of the mechanisms and factors of civilizational collapse involving climate change seems keen. While this paper does not perform a detailed examination of the scenarios and mechanisms of civilizational collapse that it proposes, it is a call-to-action for more work to understand how climate change affects civilization stability and the role of climate change in civilization collapse.
2.
A condensed version of the summary and policy comment in (1)
Summary: Humanity must understand how climate change (CC) could engender civilizational collapse. Coverage of this topic is sparse relative coverage of CC’s direct effects. Steela et al.’s PNAS opinion piece is a call to action for more research on this topic; they contribute an outline of 3 collapse scenarios—local collapse, broken world, and global collapse—and 3 collapse mechanisms—direct impacts, socio-climate feedbacks, and exogenous shock vulnerability (6 October 2022).
Policy comment: Policymakers and researchers need to promote research on the effects of climate change on civilizational stability so that critical societal institutions and infrastructure are protected from collapse. Such research efforts would include further investigations of the many scenarios and mechanisms through which civilization may collapse due to climate change; Steela et al. lay some groundwork in this regard, but fail to provide a detailed examination.
3.
One issue I have is being concise with my writing. This was recently pointed out to me by my friend Evan, when I asked him to read (1), and I want to write some thoughts of mine that were evoked by the conversation.
My first thought: What do I want myself and others to get from my writing?
I want to learn, and writing helps with this. I want to generate novel and useful ideas and to share them with people. I want to show people what I’ve done or am doing. I want a record of my thinking on certain topics
I want my writing to help others learn efficiently and I want to tell people entertaining stories, ones that engender curiosity.
My next thought: How is my writing inadequate?
I aim for transparency, informativeness, clarity, and efficiency in my writing, but feel that my writing is much less transparent, informative, clear, and efficient than it could be.
W.r.t. transparency, my model is Reasoning Transparency. My writing sometimes includes answers to these questions[1] (this comment).
W.r.t. informativeness, I assume someone has already thought about or attempted what I am working on, so I try not to repeat (Don’t repeat yourself) and to synthesize works when synthesizing has not yet occurred or has occurred but inadequately.
W.r.t. clarity, I try to edit my work multiple times and make it clear what I want to be understood. I read my writing aloud to determine if hearing it is pleasurable.
W.r.t. efficiency, my sense of where to allocate attention across my writing is fuzzy. I use editing and footnotes to consolidate, but still have trouble.
I don’t have good ways to measure or assess these things in my writing, and I haven’t decided which hypothetical audiences to gear my writing towards; I believe this decision affects how much effort I expend optimizing at least transparency and efficiency.
I will address my writing again at some point, but think it best I read the advice of others first.
4.
My friend Evan on concision:
Why does this writing exist?
Who is this writing for?
What does this content claim?
How good is this content?
Can you trust the author?
What are the author’s priors?
What beliefs ought to updated?
What has the author contributed here?
“engender”—funny typo!+)
This sentence seems hard to read, lacks coherency, IMO.
> Coverage of this topic is sparse relative coverage of CC’s direct effects.
Thank you for taking a look Martin Vlach.
For the latter comment, there is a typo. I meant:
The idea is that the corpus of work on how climate change is harmful to civilization includes few detailed analyses of the mechanisms through which climate change leads to civilizational collapse but does includes many works on the direct effects of climate change.
For the former comment, I am not sure what you mean w.r.t “engender”.
Glad I’ve helped with the part where I was not ignorant and confused myself, that is with not knowing the word engender and the use of it. Thanks for pointing it out clearly. By the way it seems “cause” would convey the same meaning and might be easier to congest in general.