J. Zachary Mazlish - LessWrong 2.0 viewer
https://www.greaterwrong.com/
J. Zachary Mazlish - LessWrong 2.0 viewerxml-emitteren-usAGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years by basil.halperin
https://www.greaterwrong.com/posts/ngpC5PFAgxHJMhicM/agi-and-the-emh-markets-are-not-expecting-aligned-or-1
<style>.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0}
.MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0}
.mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table}
.mjx-full-width {text-align: center; display: table-cell!important; width: 10000em}
.mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0}
.mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left}
.mjx-numerator {display: block; text-align: center}
.mjx-denominator {display: block; text-align: center}
.MJXc-stacked {height: 0; position: relative}
.MJXc-stacked > * {position: absolute}
.MJXc-bevelled > * {display: inline-block}
.mjx-stack {display: inline-block}
.mjx-op {display: block}
.mjx-under {display: table-cell}
.mjx-over {display: block}
.mjx-over > * {padding-left: 0px!important; padding-right: 0px!important}
.mjx-under > * {padding-left: 0px!important; padding-right: 0px!important}
.mjx-stack > .mjx-sup {display: block}
.mjx-stack > .mjx-sub {display: block}
.mjx-prestack > .mjx-presup {display: block}
.mjx-prestack > .mjx-presub {display: block}
.mjx-delim-h > .mjx-char {display: inline-block}
.mjx-surd {vertical-align: top}
.mjx-surd + .mjx-box {display: inline-flex}
.mjx-mphantom * {visibility: hidden}
.mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%}
.mjx-annotation-xml {line-height: normal}
.mjx-menclose > svg {fill: none; stroke: currentColor; overflow: visible}
.mjx-mtr {display: table-row}
.mjx-mlabeledtr {display: table-row}
.mjx-mtd {display: table-cell; text-align: center}
.mjx-label {display: table-row}
.mjx-box {display: inline-block}
.mjx-block {display: block}
.mjx-span {display: inline}
.mjx-char {display: block; white-space: pre}
.mjx-itable {display: inline-table; width: auto}
.mjx-row {display: table-row}
.mjx-cell {display: table-cell}
.mjx-table {display: table; width: 100%}
.mjx-line {display: block; height: 0}
.mjx-strut {width: 0; padding-top: 1em}
.mjx-vsize {width: 0}
.MJXc-space1 {margin-left: .167em}
.MJXc-space2 {margin-left: .222em}
.MJXc-space3 {margin-left: .278em}
.mjx-test.mjx-test-display {display: table!important}
.mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px}
.mjx-test.mjx-test-default {display: block!important; clear: both}
.mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex}
.mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left}
.mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right}
.mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0}
.MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal}
.MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal}
.MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold}
.MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold}
.MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw}
.MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw}
.MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw}
.MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw}
.MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw}
.MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw}
.MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw}
.MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw}
.MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw}
.MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw}
.MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw}
.MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw}
.MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw}
.MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw}
.MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw}
.MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw}
.MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw}
.MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw}
.MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw}
.MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw}
.MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw}
@font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')}
@font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')}
@font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold}
@font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')}
@font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')}
@font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold}
@font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')}
@font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic}
@font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')}
@font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')}
@font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold}
@font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')}
@font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic}
@font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')}
@font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')}
@font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')}
@font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')}
@font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold}
@font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')}
@font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic}
@font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')}
@font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')}
@font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic}
@font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')}
@font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')}
@font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')}
@font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')}
@font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')}
@font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')}
@font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')}
@font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold}
@font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}
</style><p><i>by </i><a href="https://tmychow.com/"><i>Trevor Chow</i></a><i>, </i><a href="https://basilhalperin.com/"><i>Basil Halperin</i></a><i>, and </i><a href="https://jzmazlish.substack.com/"><i>J. Zachary Mazlish</i></a></p><p>In this post, we point out that short AI timelines would cause <i>real interest rates</i> to be high, and would do so under expectations of either unaligned or aligned AI. However, <u>30- to 50-year real interest rates are low</u>. We argue that this suggests one of two possibilities:</p><ol><li><p><strong>Long(er) timelines.</strong> Financial markets are often highly effective information aggregators (the “efficient market hypothesis”), and therefore real interest rates accurately reflect that transformative AI is unlikely to be developed in the next 30-50 years.</p></li><li><p><strong>Market inefficiency.</strong> Markets are radically underestimating how soon advanced AI technology will be developed, and real interest rates are therefore too low. There is thus an opportunity for philanthropists to borrow while real rates are low to cheaply do good today; and/or an opportunity for anyone to earn excess returns by betting that real rates will rise.</p></li></ol><p>In the rest of this post we flesh out this argument.</p><ol><li><p>Both intuitively and under every mainstream economic model, the “<a href="https://www.openphilanthropy.org/research/could-advanced-ai-drive-explosive-economic-growth/">explosive growth</a>” caused by <i>aligned</i> AI would cause high real interest rates.</p></li><li><p>Both intuitively and under every mainstream economic model, the existential risk caused by <i>unaligned</i> AI would cause high real interest rates.</p></li><li><p>We show that in the historical data, indeed, real interest rates have been correlated with future growth.</p></li><li><p>Plugging the <a href="/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines">Cotra probabilities</a> for AI timelines into the baseline workhorse model of economic growth implies substantially higher real interest rates today.</p></li><li><p>In particular, we argue that markets are decisively rejecting the shortest possible timelines of 0-10 years.</p></li><li><p>We argue that <a href="https://twitter.com/ESYudkowsky/status/1455949250320240641">the efficient market hypothesis (EMH) is a reasonable prior</a>, and therefore one reasonable interpretation of low real rates is that since markets are simply <i>not</i> forecasting short timelines, neither should we be forecasting short timelines.</p></li><li><p>Alternatively, if you believe that financial markets are wrong, then you have the opportunity to (1) borrow cheaply today and use that money to e.g. fund AI safety work; and/or (2) earn alpha by betting that real rates will rise.</p></li></ol><p>An order-of-magnitude estimate is that, <u>if markets are getting this wrong, then there is easily $1 trillion lying on the table in the US treasury bond market alone</u> – setting aside the enormous implications for every other asset class.</p><p><strong>Interpretation.</strong> We view our argument as the best existing <i>outside view</i> evidence on AI timelines – but also as only <i>one</i> model among a mixture of models that you should consider when thinking about AI timelines. The logic here is a simple implication of a few basic concepts in orthodox economic theory and some supporting empirical evidence, which is important because the unprecedented nature of transformative AI makes “reference class”-based outside views difficult to construct. This outside view approach contrasts with, and complements, an inside view approach, which attempts to build a detailed structural model of the world to forecast timelines (e.g. <a href="/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines">Cotra 2020</a>; see also <a href="https://nostalgebraist.tumblr.com/post/693718279721730048/on-bio-anchors">Nostalgebraist 2022</a>).</p><p><strong>Outline. </strong>If you want a short version of the argument, sections I and II (700 words) are the heart of the post. Additionally, the section titles are themselves summaries, and we use text formatting to highlight key ideas.<br> </p><h1><strong>I. Long-term real rates would be high if the market was pricing advanced AI</strong></h1><p>Real interest rates reflect, among other things: </p><ol><li><p>Time discounting, which includes the probability of death</p></li><li><p>Expectations of future economic growth </p></li></ol><p>This claim is compactly summarized in the “<a href="https://en.wikipedia.org/wiki/Keynes%E2%80%93Ramsey_rule">Ramsey rule</a>” (and the only math that we will introduce in this post), a version of the “Euler equation” that in one form or another lies at the heart of <i>every</i> theory and model of dynamic macroeconomics:</p><p><span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="r=\rho+\sigma g"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">r</span></span><span class="mjx-mo MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.077em; padding-bottom: 0.298em;">=</span></span><span class="mjx-mi MJXc-space3"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em;">ρ</span></span><span class="mjx-mo MJXc-space2"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.298em; padding-bottom: 0.446em;">+</span></span><span class="mjx-mi MJXc-space2"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em; padding-right: 0.001em;">σ</span></span><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em; padding-right: 0.003em;">g</span></span></span></span></span></span></span><span></span></p><p>where:</p><ul><li><p><span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="r"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">r</span></span></span></span></span></span></span> is the real interest rate over a given time horizon</p></li><li><p><span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="\rho"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em;">ρ</span></span></span></span></span></span></span> is time discounting over that horizon</p></li><li><p><span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="\sigma"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em; padding-right: 0.001em;">σ</span></span></span></span></span></span></span> is a (positive) preference parameter reflecting how much someone cares about smoothing consumption over time</p></li><li><p><span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="g"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em; padding-right: 0.003em;">g</span></span></span></span></span></span></span> is the growth rate</p></li></ul><p>(Internalizing the meaning of these Greek letters is wholly not necessary.)</p><p>While more elaborate macroeconomic theories vary this equation in interesting and important ways, it is common to all of these theories that the real interest rate is higher when either (1) the time discount rate is high or (2) future growth is expected to be high.</p><p>We now provide some intuition for these claims.</p><p><strong>Time discounting and mortality risk.</strong> Time discounting refers to how much people discount the future relative to the present, which captures both (i) <a href="https://d101vc9winf8ln.cloudfront.net/documents/27957/original/Cowen___Parfit_-_Against_the_social_discount_rate.pdf?1523454279"><i>intrinsic</i> preference for the present</a> relative to the future and (ii) the probability of death.</p><p>The intuition for why the probability of death raises the real rate is the following. Suppose we expect with high probability that humanity will go extinct next year. Then there is no reason to save today: no one will be around to use the savings. This pushes up the real interest rate, since there is less money available for lending.</p><p><strong>Economic growth.</strong> To understand why higher economic growth raises the real interest rate, the intuition is similar. If we expect to be wildly rich next year, then there is also no reason to save today: we are going to be tremendously rich, so we might as well use our money today while we’re still comparatively poor.</p><p>(For the formal math of the Euler equation, <a href="https://www.brookings.edu/wp-content/uploads/2005/01/2005a_bpea_baker.pdf">Baker, Delong, and Krugman 2005</a> is a useful reference. The core intuition is that either mortality risk or the prospect of utopian abundance reduces the supply of savings, due to <i>consumption smoothing</i> logic, which pushes up real interest rates.)</p><p><strong>Transformative AI and real rates.</strong> Transformative AI would either raise the risk of extinction (if unaligned), or raise economic growth rates (if aligned).</p><p><u>Therefore, based on the economic logic above, the prospect of transformative AI – unaligned or aligned – will result in high real interest rates</u>. This is the key claim of this post.</p><p>As an example in the aligned case, Davidson (2021) usefully defines AI-induced “explosive growth” as an increase in growth rates to at least 30% annually. Under a baseline calibration where <span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="\sigma=1"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em; padding-right: 0.001em;">σ</span></span><span class="mjx-mo MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.077em; padding-bottom: 0.298em;">=</span></span><span class="mjx-mn MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">1</span></span></span></span></span></span></span> and <span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="\rho=0.01"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em;">ρ</span></span><span class="mjx-mo MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.077em; padding-bottom: 0.298em;">=</span></span><span class="mjx-mn MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">0.01</span></span></span></span></span></span></span>, and importantly assuming growth rates are known with certainty, the Euler equation implies that moving from 2% growth to 30% growth would raise real rates from 3% to 31%!</p><p>For comparison, real rates in the data we discuss below have never gone above 5%.</p><p>(In using terms like “transformative AI” or “advanced AI”, we refer to the cluster of concepts discussed in <a href="https://intelligence.org/files/AIPosNegFactor.pdf">Yudkowsky 2008</a>, <a href="https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies">Bostrom 2014</a>, <a href="https://docs.google.com/document/d/1IJ6Sr-gPeXdSJugFulwIpvavc0atjHGM82QjIfUSBGQ/edit">Cotra 2020</a>, <a href="https://arxiv.org/abs/2206.13353">Carlsmith 2021</a>, <a href="https://www.openphilanthropy.org/research/could-advanced-ai-drive-explosive-economic-growth/">Davidson 2021</a>, <a href="https://www.cold-takes.com/most-important-century/">Karnofsky 2022</a>, and <a href="https://www.openphilanthropy.org/research/some-background-on-our-views-regarding-advanced-artificial-intelligence/#Sec1">related literature</a>: AI technology that precipitates a transition comparable to the agricultural or industrial revolutions.)</p><h1><strong>II. But: long-term real rates are low</strong></h1><p>The <a href="https://fred.stlouisfed.org/series/DFII30">US 30-year real interest rate</a> ended 2022 at 1.6%. Over the full year it averaged 0.7%, and as recently as March was below zero. Looking at a shorter time horizon, the US <a href="https://fred.stlouisfed.org/series/DFII10">10-year</a> real interest rate is 1.6%, and similarly was below <i>negative </i>one percent as recently as March.</p><p class="imgonly"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1673285569/mirroredImages/ngpC5PFAgxHJMhicM/iogfekhkllvzx3z9byyp.png" loading="lazy"></p><p><i>(Data sources used here are explained in section V.)</i></p><p>The UK in autumn 2021 sold a <i>50</i>-year real bond <a href="https://www.reuters.com/markets/europe/uk-sells-new-2073-index-linked-gilt-with-record-low-volume-2021-11-23/">with</a> a −2.4% rate at the time. Real rates on analogous bonds in other developed countries in recent years have been similarly low/negative for the longest horizons available. <a href="https://www.tradingview.com/symbols/TVC-AT100Y/">Austria</a> has a <i>100</i>-year nominal bond – being <a href="https://www.youtube.com/watch?v=VYcgcWS_NBQ">nominal</a> should make its rate higher due to expected inflation – with yields less than 3%.</p><p>Thus the conclusion previewed above: <u>financial markets, as evidenced by real interest rates, are not expecting a high probability of either AI-induced growth acceleration or elevated existential risk, on</u> <i>at least</i> <u>a 30-50 year time horizon</u>. </p><h1><strong>III. Uncertainty, takeoff speeds, inequality, and stocks</strong></h1><p>In this section we briefly consider some potentially important complications.</p><p><strong>Uncertainty. </strong>The Euler equation and the intuition described above assumed <i>certainty</i> about AI timelines, but taking into account uncertainty does not change the core logic. With uncertainty about the future economic growth rate, then the real interest rate reflects the <i>expected</i> future economic growth rate, where importantly the expectation is taken over the <a href="https://en.wikipedia.org/wiki/Risk-neutral_measure">risk-neutral measure</a>: in brief, probabilities of different states are reweighted by their marginal utility. We return to this in our quantitative model below.</p><p><strong>Takeoff speeds.</strong> Nothing in the logic above relating growth to real rates depends on <a href="/posts/vwLxd6hhFvPbvKmBH/yudkowsky-and-christiano-discuss-takeoff-speeds">slow vs. fast takeoff speed</a>; the argument can be reread under either assumption and nothing changes. Likewise, when considering the case of aligned AI, rates should be elevated whether economic growth starts to rise more rapidly <i>before</i> advanced AI is developed <a href="/posts/aFaKhG86tTrKvtAnT/against-gdp-as-a-metric-for-timelines-and-takeoff-speeds">or only does so afterwards</a>. What matters is that GDP – or really, consumption – <i>ends up</i> high within the time horizon under consideration. As long as future consumption will be high within the time horizon, then there is less motive to save today (“consumption smoothing”), pushing up the real rate.</p><p><strong>Inequality. </strong>The logic above assumed that the development of transformative AI affects everyone equally. This is a reasonable assumption in the case of unaligned AI, where it is thought that all of humanity will be evaporated. However, when considering <i>aligned</i> AI, it may be thought that only some will benefit, and therefore real interest rates will not move much: if only an elite Silicon Valley minority is expected to have utopian wealth next year, then everyone else may very well still choose to save today.</p><p>It is indeed the case that inequality in expected gains from transformative AI would <i>dampen</i> the impact on real rates, but this argument should not be overrated. First, asset prices can be <a href="https://www.journals.uchicago.edu/doi/abs/10.1086/680996">crudely</a> thought of as reflecting a <i>wealth-weighted average</i> across investors. Even if only an elite minority becomes fabulously wealthy, it is <i>their</i> desire for consumption smoothing which will end up dominating the determination of the real rate. Second, <i>truly</i> transformative AI leading to 30%+ economy-wide growth (“<a href="https://moores.samaltman.com/">Moore’s law for everything</a>”) would not be possible without having economy-wide benefits. </p><p><strong>Stocks</strong>. One naive objection to the argument here would be the claim that real interest rates sound like an odd, arbitrary asset price to consider; certainly stock prices are the asset price that receive the most media attention.</p><p>In <a href="/posts/ngpC5PFAgxHJMhicM/agi-and-the-emh-markets-are-not-expecting-aligned-or-1#Appendix_1__Against_using_stock_prices_to_forecast_AI_timelines">appendix 1</a>, we explain that the level of the real interest rate affects <i>every</i> asset price: stocks for instance reflect the <i>present discounted value</i> of future dividends; and real interest rates <i>determine</i> <i>the discount rate used to discount</i> those future dividends. Thus, <u>if real interest rates are ‘wrong’,</u> <i>every</i> <u>asset price is wrong. If real interest rates are wrong,</u> <i>a lot</i> <u>of money is on the table, a point to which we return in section X</u>. </p><p>We also argue that stock prices in particular are not a useful indicator of market expectations of AI timelines. Above all, high stock prices of <a href="https://twitter.com/robertwiblin/status/1577782210568003585">chipmakers</a> or companies like Alphabet (parent of DeepMind) could only reflect expectations for <i>aligned</i> AI and could not be informative of the risk of <i>unaligned</i> AI. Additionally, as we explain further in the appendix, aligned AI could even <i>lower</i> equity prices, by pushing up discount rates.</p><h1><strong>IV. Historical data on interest rates supports the theory: preliminaries</strong></h1><p>In section I, we gave theoretical <i>intuition</i> for why higher expected growth or higher existential risk would result in higher interest rates: expectations for such high growth or mortality risk would lead people to want to save less and borrow more today. In this section and the next two, we showcase some simple empirical evidence that the predicted relationships hold in the available data.</p><p><strong>Measuring real rates.</strong> To compare historical real interest rates to historical growth, we need to measure real interest rates.</p><p>Most bonds historically have been <i>nominal</i>, where the yield is not adjusted for changes in inflation. Therefore, the vast majority of research studying real interest rates starts with <i>nominal</i> interest rates, attempts to construct an estimate of expected inflation using some statistical model, and then subtracts this estimate of expected inflation from the nominal rate to get an <i>estimated</i> real interest rate. However, <u>constructing measures of inflation expectations is extremely difficult, and as a result most papers in this literature are not very informative</u>. </p><p>Additionally, most bonds historically have had some risk of default. Adjusting for this default premium is also extremely difficult, which in particular complicates analysis of long-run interest rate trends. </p><p>The difficulty in measuring real rates is one of the main causes, in our view, of <a href="https://marginalrevolution.com/marginalrevolution/2015/04/tyler-cowens-three-laws.html">Tyler Cowen’s Third Law</a>: “all propositions about real interest rates are wrong”. Throughout this piece, we are badly violating this (Gödelian) Third Law. In <a href="/posts/ngpC5PFAgxHJMhicM/agi-and-the-emh-markets-are-not-expecting-aligned-or-1#Appendix_2__Tyler_Cowen_s_Third_Law">appendix 2</a>, we expand on our argument that the source of Tyler’s Third Law is measurement issues in the extant literature, together with some separate, frequent conceptual errors. </p><p><strong>Our approach.</strong> We take a more direct approach.</p><p><strong>Real rates. </strong>For our primary analysis, we instead use market real interest rates from <i>inflation-linked bonds</i>. Because we use interest rates <i>directly</i> from inflation-linked bonds – instead of constructing shoddy estimates of inflation expectations to use with nominal interest rates – this approach avoids the measurement issue just discussed (and, we argue, allows us to escape Cowen’s Third Law).</p><p><u>To our knowledge, prior literature has not used real rates from inflation-linked bonds only because these bonds are comparatively new</u>. Using inflation-linked bonds confines our sample to the last ~20 years in the <a href="https://www.federalreserve.gov/pubs/feds/2008/200805/200805abs.html">US</a>, the last ~30 in the <a href="https://www.bankofengland.co.uk/statistics/yield-curves">UK</a>/Australia/Canada. Before that, inflation-linked bonds didn’t exist. Other countries have data for even fewer years and less liquid bond markets.</p><p>(The yields on inflation-linked bonds are not perfect measures of real rates, because of <a href="https://www.frbsf.org/economic-research/files/wp08-34bk.pdf">risk premia</a>, <a href="https://www.basilhalperin.com/essays/efficient-restaurant-hypothesis-mental-model-finance-food.html">liquidity issues</a>, and some <a href="https://www.ijcb.org/journal/ijcb12q4a2.pdf">subtle</a> <a href="https://www.cambridge.org/core/journals/journal-of-financial-and-quantitative-analysis/article/abs/tips-from-tips-the-informational-content-of-treasury-inflationprotected-security-prices/76EBC9C8B0FEF951704DCCA4A4781CEC">issues</a> with the way these securities are structured. You can build a model and attempt to strip out these issues; here, we will just use the raw rates. If you prefer to think of these empirics as “are <i>inflation-linked bond yields</i> predictive of future real growth” rather than “are <i>real rates</i> predictive of future real growth”, that interpretation is still sufficient for the logic of this post.)</p><p><strong>Nominal rates.</strong> Because there are only 20 or 30 years of data on <i>real</i> interest rates from inflation-linked bonds, we supplement our data by also considering unadjusted <i>nominal</i> interest rates. Nominal interest rates reflect real interest rates plus inflation expectations, so it is not appropriate to compare nominal interest rates to <i>real</i> GDP growth. </p><p>Instead, analogously to comparing <i>real</i> interest rates to <i>real</i> GDP growth, we compare <i>nominal</i> interest rates to <i>nominal</i> GDP growth. The latter is not an ideal comparison under economic theory – and inflation variability could swamp real growth variability – but we argue that this approach is simple and transparent.</p><p>Looking at nominal rates allows us to have a very large sample of countries for many decades: we use <a href="https://data.oecd.org/interest/long-term-interest-rates.htm">OECD data</a> on nominal rates available for up to 70 years across 39 countries.<br> </p><h1><strong>V. Historical data on interest rates supports the theory: graphs</strong></h1><p>The goal of this section is to show that real interest rates have correlated with future real economic growth, and secondarily, that nominal interest rates have correlated with future nominal economic growth. We also briefly discuss the state of empirical evidence on the correlation between real rates and existential risk.</p><p><strong>Real rates vs. real growth. </strong>A first cut at the data suggests that, indeed, higher real rates today predict higher real growth in the future:<br> </p><p class="imgonly"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1673285569/mirroredImages/ngpC5PFAgxHJMhicM/ibwp0rirhrvjx4cqvc6g.png" loading="lazy"></p><p>To see how to read these graphs, take the left-most graph (“10-year horizon”) for example. The x-axis shows the level of the real interest rate, as reflected on 10-year inflation linked bonds. The y-axis shows average real GDP growth over the <i>following</i> 10 years.</p><p>The middle and right hand graphs show the same, at the 15-year and 20-year horizons. The scatter plot shows all available data for the US (<a href="https://www.federalreserve.gov/pubs/feds/2008/200805/200805abs.html">since 1999</a>), the UK (<a href="https://www.bankofengland.co.uk/statistics/yield-curves">since 1985</a>), Australia (since 1995), and Canada (since 1991). (Data for Australia and Canada is only available at the 10-year horizon, and comes from <a href="https://augurlabs.com/curve/">Augur Labs</a>.)</p><p>Eyeballing the figure, <u>there appears to be a strong relationship between real interest rates today and future economic growth over the next 10-20 years</u>.</p><p>To our knowledge, this simple stylized fact is novel.</p><p><strong>Caveats. </strong>“Eyeballing it” is not a formal econometric method; but, this is a blog post not a journal article (TIABPNAJA). We do not perform any formal statistical tests here, but we do want to acknowledge some important statistical points and other caveats.</p><p>First, the data points in the scatter plot are not statistically independent: real rates and growth are both persistent variables; the data points contain overlapping periods; and growth rates in these four countries are correlated. These issues are evident even from eyeballing <a href="https://raw.githubusercontent.com/basilhalperin/agi_emh/main/r_figs/ts_r_vs_gdp_10.png">the time series</a>. Second, of course this relationship is not causally identified: we do not have exogenous variation in real growth rates. (If you have ideas for identifying the causal effect of higher real growth expectations on real rates, we would love to discuss with you.)</p><p>Relatedly, many other things are changing in the world which are likely to affect real rates. <a href="https://www.forourposterity.com/best-chad-jones-papers/">Population growth</a> is slowing, <a href="https://worthwhile.typepad.com/worthwhile_canadian_initi/2013/10/asset-prices-and-the-retirement-revolution.html">retirement</a> is lengthening, the population is <a href="http://web.stanford.edu/~aauclert/demowealth21.pdf">aging</a>. But under AI-driven “explosive” growth – again say 30%+ annual growth, following the excellent analysis of <a href="https://www.openphilanthropy.org/could-advanced-ai-drive-explosive-economic-growth">Davidson (2021)</a> – then, we might reasonably expect that this massive of an increase in the growth rate would drown out the impact of any other factors.</p><p><strong>Nominal rates vs. nominal growth.</strong> Turning now to evidence from nominal interest rates, recall that the usefulness of this exercise is that while there only exists 20 or 30 years of data on <i>real</i> interest rates for two countries, there is much more data on <i>nominal</i> interest rates.</p><p>We simply take all available data on 10-year nominal rates from the set of <a href="https://data.oecd.org/interest/long-term-interest-rates.htm">39 OECD countries since 1954</a>. The following scatterplot compares the 10-year nominal interest versus nominal GDP growth over the succeeding ten years by country:</p><p class="imgonly"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1673285569/mirroredImages/ngpC5PFAgxHJMhicM/jnq2mf9viyogujx6ww3w.png" alt="Chart, scatter chart
Description automatically generated" loading="lazy"></p><p>Again, there is a strong positive – if certainly not perfect – relationship. (For example, the outlier brown dots at the bottom of the graph are Greece, whose high interest rates despite negative NGDP growth reflect high default risk during an economic depression.) </p><p>The same set of nontrivial caveats apply to this analysis as above.</p><p>We consider this data from nominal rates to be significantly weaker evidence than the evidence from real rates, but corroboration nonetheless.</p><p><strong>Backing out market-implied timelines.</strong> Taking the univariate pooled OLS results from the real rate data <i>far</i> too seriously, the fact that the 10-year real rate in the US ended 2022 at 1.6% would predict average annual real GDP growth of 2.6% over the next 10 years in the US;<strong> </strong>the analogous interest rate of −0.2% in the UK would predict 0.7% annual growth over the next 10 years in the UK. Such growth rates, clearly, are not compatible with the arrival of transformative aligned AI within this horizon.</p><h1><strong>VI. Empirical evidence on real rates and mortality risk</strong></h1><p>We have argued that in the theory, real rates should be higher in the face of high economic growth or high mortality risk; empirically, so far, we have only shown a relationship between real rates and growth, but not between real rates and mortality.</p><p>Showing that real rates accurately reflect changes in existential risk is very difficult, because there is no word-of-god measurement of how existential risk has evolved over time.</p><p>We would be very interested in pursuing new empirical research examining “asset pricing under existential risk”. In <a href="/posts/ngpC5PFAgxHJMhicM/agi-and-the-emh-markets-are-not-expecting-aligned-or-1#Appendix_3__Asset_pricing_under_existential_risk__a_literature_review">appendix 3</a>, we perform a scorched-earth literature review and find essentially zero existing empirical evidence on real rates and <i>existential</i> risks.</p><p><strong>Disaster risk. </strong>In particular, the extant literature does not study existential risks but instead “merely” <i>disaster risks</i>, under which real assets are devastated but humanity is not exterminated. <a href="https://www.aeaweb.org/articles?id=10.1257/pol.5.4.306">Disaster risks</a> do <i>not</i> necessarily raise real rates – indeed, such risks are thought to <i>lower</i> real rates due to precautionary savings. That notwithstanding, some highlights of the appendix review include <a href="https://www.jstor.org/stable/pdf/2117594.pdf?">a small set of papers</a> <a href="https://www.jstor.org/stable/pdf/2600866.pdf">finding</a> <a href="https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1465-7295.1990.tb00824.x">that</a> <a href="https://www.nber.org/system/files/working_papers/w0887/w0887.pdf">individuals</a> with a higher perceived risk of nuclear conflict during the Cold War saved less, as well as <a href="https://davidandrewfiner.com/jmp/">a paper</a> noting that equities which were headquartered in cities more likely to be targeted by Soviet missiles did worse during the Cuban missile crisis (<a href="https://www.sciencedirect.com/science/article/abs/pii/S1062976922000217">see also</a>). Our assessment is that these and the other available papers on <i>disaster</i> risks discussed in the appendix have severe limitations for the purposes here.</p><p><strong>Individual mortality risk. </strong>We judge that the best evidence on this topic comes instead from examining the relationship between <i>individual mortality risk</i> and <i>savings/investment</i> behavior. The logic we provided was that if humanity will be extinct next year, then there is no reason to save, pushing up the real rate. Similar logic says that at the <i>individual</i> level, a higher risk of death for any reason should lead to lower savings and less investment in human capital. Examples of lower savings at the <i>individual </i>level need not raise interest rates at the <i>economy-wide</i> level, but do provide evidence for the <i>mechanism</i> whereby extinction risk should lead to lower saving and thus higher interest rates. </p><p>One example comes <a href="https://www.aeaweb.org/articles?id=10.1257/app.20150369">from Malawi</a>, where the provision of a new AIDS therapy caused a significant increase in life expectancy. Using spatial and temporal variation in where and when these therapeutics were rolled out, it was found that increased life expectancy results in more savings and more human capital investment in the form of education spending. Another experiment in Malawi <a href="https://repository.upenn.edu/psc_publications/39/">provided information</a> to correct pessimistic priors about life expectancy, and found that higher life expectancy directly caused more investment in agriculture and livestock.</p><p>A third example comes from <a href="https://www.jstor.org/stable/23469683">testing for Huntington’s disease</a>, a disease which causes a meaningful drop in life expectancy to around 60 years. Using variation in when people are diagnosed with Huntington’s, it has been found that those who learn they carry the gene for Huntington’s earlier are 30 percentage points less likely to finish college, which is a significant fall in their human capital investment.</p><p>Studying the effect on savings and real rates from increased life expectancy <i>at the population level</i> is potentially intractable, but would be interesting to consider further. Again, in our assessment, the best empirical evidence available right now comes from the research on individual “existential” risks and suggests that real rates should increase with existential risk.</p><h1><strong>VII. Plugging the Cotra probabilities into a simple quantitative model of real interest rates predicts very high rates</strong></h1><p>Section VI used <u>historical data</u> to go from <i>the current real rate</i> to a very crude <i>market-implied forecast</i> of growth rates; in this section, we instead use a <u>model</u> to go from <i>existing forecasts</i> of AI timelines to <i>timeline-implied real rates.</i> We aim to show that under short AI timelines, real interest rates would be unrealistically elevated.</p><p>This is a useful exercise for three reasons. First, the historical data is only able to speak to growth forecasts, and therefore only able to provide a forecast under the possibly incorrect assumption of <i>aligned</i> AI. Second, the empirical forecast assumes a <i>linear </i>relationship between the real rate and growth, which may not be reasonable for a massive change caused by transformative AI. Third and quite important, the historical data cannot transparently tell us anything about uncertainty and the market’s beliefs about the full probability <i>distribution</i> of AI timelines.</p><p>We <a href="https://docs.google.com/spreadsheets/d/10ULqcFRKXD-UT5JYDMrQeLtUfzkyzNUxCPQvYqbEdig/edit#gid=850885346">use</a> the canonical (and nonlinear) version of the Euler equation – the model discussed in section I – but now allow for <i>uncertainty</i> on both how soon transformative AI will be developed and whether or not it will be aligned. The model takes as its key inputs (1) a probability of transformative AI each year, and (2) a probability that such technology is aligned.</p><p>The model is a simple application of the stochastic Euler equation under an <a href="https://en.wikipedia.org/wiki/Isoelastic_utility">isoelastic utility function</a>. We use the following as a baseline, before considering alternative probabilities:</p><ul><li><p>We use smoothed <a href="/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines"><strong>Cotra (2022)</strong></a><strong> probabilities for transformative AI</strong> over the next 30 years: a 2% yearly chance until 2030, a 3% yearly chance through 2036, and a 4% yearly chance through 2052.</p></li><li><p>We use the <strong>FTX Future Fund’s median </strong><a href="https://archive.ph/9UQTA"><strong>estimate</strong></a><strong> of 15% for the probability that AI is </strong><i><strong>un</strong></i><strong>aligned</strong> conditional on the development of transformative AI.</p></li><li><p>With the arrival of <i>aligned</i> AI, we use the <strong>Davidson (2020) assumption of 30% annual economic growth</strong>; with the arrival of unaligned AI, we assume human extinction. In the absence of the development of transformative AI, we assume a steady 1.8% growth rate.</p></li><li><p>We calibrate the pure rate of subjective time preference to 0.01 and the consumption smoothing parameter (i.e. inverse of the elasticity of intertemporal substitution) as 1, following the economic literature.</p></li></ul><p>Thus, to summarize: by default, GDP grows at 1.8% per year. Every year, there is some probability (based on Cotra) that transformative AI is developed. If it is developed, there is a 15% probability the world ends, and an 85% chance GDP growth jumps to 30% per year.</p><p>We have built a spreadsheet <a href="https://docs.google.com/spreadsheets/d/10ULqcFRKXD-UT5JYDMrQeLtUfzkyzNUxCPQvYqbEdig/edit#gid=850885346">here</a> that allows you to tinker with the numbers yourself, such as adjusting the growth rate under aligned AI, to see what your timelines and probability of alignment would imply for the real interest rate. (It also contains the full Euler equation formula generating the results, for those who want the mathematical details.) We first estimate real rates under the baseline calibration above, before considering variations in the critical inputs.</p><p><strong>Baseline results.</strong> The model predicts that under <i>zero </i>probability of transformative AI, the real rate at any horizon would be 2.8%. In comparison, <u>under the baseline calibration just described based on Cotra timelines, the real rate at a 30-year horizon would be pushed up to 5.9% – roughly three percentage points higher</u>.</p><p>For comparison, the 30-year real rate in the US is currently 1.6%. </p><p>While the simple Euler equation somewhat overpredicts the level of the real interest rate even under zero probability of transformative AI – the 2.8% in the model versus the 1.6% in the data – this overprediction is explainable by the radical simplicity of the model that we use and is a known issue in the literature. Adding other factors (e.g. <a href="https://en.wikipedia.org/wiki/Precautionary_savings">precautionary savings</a>) to the model would lower the level. Changing the level does not change its <i>directional</i> predictions, which help <a href="https://ideas.repec.org/a/ijc/ijcjou/y2017q3a1.html">quantitatively</a> <a href="https://www.nber.org/papers/w30024">explain</a> the fall in real rates over the past ~30 years.</p><p>Therefore, what is most informative is the three percentage point <i>difference</i> between the real rate under Cotra timelines (5.9%) versus under no prospect of transformative AI (2.8%): Cotra timelines imply real interest rates substantially higher than their current levels.</p><p>Now, from this baseline estimate, we can also consider varying the key inputs.</p><p><strong>Varying assumptions on P(misaligned|AGI).</strong> First consider changing the assumption that advanced AI is 15% likely to be unaligned (conditional on the development of AGI). <a href="https://docs.google.com/spreadsheets/d/1YNzk_p-IHtzhV6Eupahhpo-hRSwDmTA3iAGd0ASAdCA/edit#gid=1179700904">Varying this parameter</a> does not have a large impact: moving from 0% to 100% probability of misalignment raises the model’s predicted real rate from 5.8% only to 6.3%.</p><p class="imgonly"><s><div class="imgonly"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1673285569/mirroredImages/ngpC5PFAgxHJMhicM/glqbphp61njiewdrhcdv.png" loading="lazy"></div></s></p><p><strong>Varying assumptions on timelines.</strong> Second, consider making timelines shorter or longer. In particular, consider varying the <i>probability of development by 2043</i>, which we use as a benchmark per the <a href="https://archive.ph/9UQTA">FTX Future Fund</a>.</p><p>We scale the Cotra timelines up and down to vary the probability of development by 2043. (Specifically: we target a specific <i>cumulative </i>probability of development by 2043; and, following Cotra, if the <i>annual </i>probability up until 2030 is <span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="x"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">x</span></span></span></span></span></span></span>, then it is <span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="1.5x"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mn"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">1.5</span></span><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">x</span></span></span></span></span></span></span> in the subsequent seven years up through 2036, and it is <span class="mathjax-inline-container"><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="2x"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mn"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">2</span></span><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">x</span></span></span></span></span></span></span> in the remaining years of the 30-year window.)</p><p>As <a href="https://docs.google.com/spreadsheets/d/1eC1xLpb_UWJ5rO-wScHjM5ERwYldw6Qxa3x0tzao_tk/edit#gid=1601822419">the next figure</a> shows and as one might expect, shorter AI timelines have a very large impact on the model’s estimate for the real rate. </p><p class="imgonly"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1673285569/mirroredImages/ngpC5PFAgxHJMhicM/gbzn21meugu8paqgz4wn.png" loading="lazy"></p><ul><li><p>The original baseline parameterization from Cotra corresponds to the FTX Future Fund “upper threshold” of a 45% chance of development by 2043, which generated the 3 percentage point increase in the 30-year real rate discussed above.</p></li><li><p>The Future Fund’s median of a 20% probability by 2043 generates a 1.1 percentage point increase in the 30-year real rate.</p></li><li><p>The Future Fund’s “lower threshold” of a 10% probability by 2043 generates a 0.5 percentage point increase in the real rate.</p></li></ul><p>These results strongly suggest that any timeline shorter than or equal to the Cotra timeline is <i>not </i>being expected by financial markets.</p><h1><strong>VIII. Markets are decisively rejecting the shortest possible timelines</strong></h1><p>While it is not possible to back out <i>exact </i>numbers for the market’s implicit forecast for AI timelines, it is reasonable to say that <u>the market is</u> <i>decisively</i> <u>rejecting – i.e., putting very low probability on – the development of transformative AI in the</u> <i>very</i> <u>near term, say within the next ten years</u>. </p><p>Consider the following examples of extremely short timelines: </p><ol><li><p>Five year timelines: With a 50% probability of transformative AI by 2027, and the same yearly probability thereafter, the model predicts 13.0pp higher 30-year real rates today! </p></li><li><p>Ten year timelines: With a 50% probability of transformative AI by 2032, and the same yearly probability thereafter, the model predicts 6.5pp higher 30-year real rates today. </p></li></ol><p>Real rate movements of these magnitudes are wildly counterfactual. As previously noted, real rates in the data used above have never gone above even 5%.</p><p><strong>Stagnation.</strong> As a robustness check, <a href="https://docs.google.com/spreadsheets/d/10ULqcFRKXD-UT5JYDMrQeLtUfzkyzNUxCPQvYqbEdig/edit#gid=850885346">in the configurable spreadsheet</a> we allow you to place some yearly probability on the economy stagnating and growing at 0% per year from thereon. Even with a 20% chance of stagnation by 2053 (higher than realistic), under Cotra timelines, the model generates a 2.1% increase in 30-year rates. </p><p><strong>Recent market movements. </strong>Real rates have increased around two percentage points since the start of 2022, with the <a href="https://fred.stlouisfed.org/series/DFII30">30-year real rate</a> moving from −0.4% to 1.6%, approximately the pre-covid level. This is a large enough move to merit discussion. While this rise in long-term real rates could reflect changing market expectations for timelines, it seems much more plausible that high inflation, the Russia-Ukraine war, and monetary policy tightening have together worked to drive up short-term real rates and <a href="https://www.federalreserve.gov/econres/notes/feds-notes/tips-from-tips-update-and-discussions-20190521.html">the risk premium on long-term real rates</a>.</p><h1><strong>IX. Financial markets are the most powerful information aggregators produced by the universe (so far)</strong></h1><p>Should we update on the fact that markets are not expecting very short timelines? </p><p>Probably!</p><p>As a prior, we think that <a href="https://twitter.com/ESYudkowsky/status/1455949250320240641">market efficiency is reasonable</a>. We do not try to provide a full defense of the efficient markets hypothesis (EMH) in this piece given that it has been debated ad nauseum elsewhere, but here is a scaffolding of what such an argument would look like.</p><p>Loosely, the EMH says that the current price of any security incorporates all public information about it, and as such, <a href="https://twitter.com/ESYudkowsky/status/1278486324018704384">you should not expect to systematically make money by trading securities</a>.</p><p>This is simply a no-arbitrage condition, and certainly no more radical than supply and demand: if something is over- or under-priced, you’ll take action based on that belief until you no longer believe it. In other words, <a href="https://twitter.com/ESYudkowsky/status/1426612130975870981">you’ll buy and sell it until you think the price is right</a>. Otherwise, there would be an <a href="https://yudkowsky.tumblr.com/writing/inexploitability">unexploited opportunity</a> for profit that was being left on the table, and <a href="/posts/h24JGbmweNpWZfBkM/markets-are-anti-inductive">there are no free lunches when the market is in equilibrium</a>.</p><p>As a corollary, the current price of a security should be the best available risk-adjusted predictor of its future price. Notice we didn’t say that the price is equal to the “correct” fundamental value. In fact, the current price is almost certainly wrong. What we did say is that it is the best guess, i.e. no one knows if it should be higher or lower.</p><p>Testing this hypothesis is difficult, in the same way that testing any equilibrium condition is difficult. Not only is the equilibrium always changing, there is also a joint hypothesis problem which <a href="https://www.jstor.org/stable/2325486">Fama (1970)</a> outlined: comparing actual asset prices to “correct” theoretical asset prices means you are simultaneously testing whatever asset pricing model you choose, alongside the EMH.</p><p>In this sense, it makes no sense to talk about “testing” the EMH. Rather, the question is how quickly prices converge to the limit of market efficiency. In other words, how fast is information diffusion? Our position is that for most things, this is pretty fast!</p><p>Here are a few heuristics that support our position:</p><ol><li><p>For our purposes, the earlier evidence on the link between real rates and growth is a highly relevant example of market efficiency.</p></li><li><p>There are notable examples of markets seeming to be eerily good at forecasting hard-to-anticipate events:<ol><li><p>In the wake of the Challenger explosion, despite no definitive public information being released, <a href="http://wisdomofcrowds.blogspot.com/2009/12/stock-market-reaction-to-challenger.html">the market</a> <a href="https://slate.com/business/2003/08/the-disaster-market.html">seems to have identified</a> which firm was responsible.</p></li><li><p><a href="https://www.sciencedirect.com/science/article/pii/S0929119914000546#f0005">Economist Armen Alchian observed</a> that the stock price of lithium producers spiked 461% following the public announcement of the first hydrogen bomb tests in 1954, while the prices of producers of other radioactive metals were flat. He circulated a paper within RAND, where he was working, identifying lithium as the material used in the tests, before the paper was suppressed by leadership who were apparently aware that indeed lithium was used. The market was prescient even though <i>zero </i>public information was released about lithium’s usage. </p></li></ol></p></li></ol><p>Remember: if real interest rates are wrong, <i>all</i> financial assets are mispriced. If real interest rates “should” rise three percentage points or more, that is easily hundreds of billions of dollars worth of revaluations. It is unlikely that sharp market participants are leaving billions of dollars on the table.</p><h1><strong>X. If markets are not efficient, you could be earning alpha and philanthropists could be borrowing</strong></h1><p>While our prior in favor of efficiency is fairly strong, the market could be currently<i> </i>failing to anticipate transformative AI, due to various <a href="https://scholar.harvard.edu/files/shleifer/files/limitsofarbitrage.pdf">limits to arbitrage</a>.</p><p>However, if you do believe the market is currently wrong about the probability of short timelines, then we now argue there are two courses of action you should consider taking:</p><ol><li><p>Bet on real rates rising (“get rich or die trying”)</p></li><li><p>Borrow today, including in order to fund philanthropy (“impatient philanthropy”)</p></li></ol><h2>1. Bet on real rates rising (“get rich or die trying”)</h2><p>Under the logic argued above, if you genuinely believe that AI timelines are short, then you should consider putting your money where your mouth is: bet that real rates will rise when the market updates, and potentially earn a lot of money if markets correct. Shorting (or going underweight) government debt is the simplest way of expressing this view.</p><p>Indeed, AI safety researcher Paul Christiano has <a href="https://ea.greaterwrong.com/posts/KdxGwxwY3t7iw9xjB/three-impacts-of-machine-intelligence#comment-aYCNP5PgDYsZpxsbX">written publicly</a> that he is (or was) short 30-year government bonds.</p><p>If short timelines are your true belief in your heart of hearts, and not merely a <a href="/posts/CqyJzDZWvGhhFJ7dY/belief-in-belief">belief in a belief</a>, then you should seriously consider how much money you could earn here and what you could do with those resources.</p><p><strong>Implementing the trade.</strong> For retail investors, betting against treasuries via ETFs is perhaps simplest. Such trades can be done easily with retail brokers, like Schwab.</p><p>(i) For example, one could simply short the <a href="https://www.pimco.com/en-us/investments/etf/15-year-us-tips-index-exchange-traded-fund/">LTPZ ETF</a>, which holds long-term real US government debt (effective duration: 20 years).</p><p>(ii) Alternatively, if you would prefer to avoid engaging in shorting yourself, there are ETFs which will do the shorting for you, with nominal bonds: <a href="https://www.proshares.com/our-etfs/leveraged-and-inverse/tbf">TBF</a> is an ETF which is short 20+ year treasuries (duration: 18 years); <a href="https://www.proshares.com/our-etfs/leveraged-and-inverse/tbt">TBT</a> is the same, but levered 2x; and <a href="https://www.proshares.com/our-etfs/leveraged-and-inverse/ttt">TTT</a> is the same, but levered 3x. There are a number of <a href="https://etfdb.com/etfdb-category/inverse-bonds/">other similar options</a>. Because these ETFs do the shorting for you, all you need to do is purchase shares of the ETFs.</p><p><strong>Back of the envelope estimate.</strong> A rough estimate of how much money is on the table, just from shorting the US treasury bond market <i>alone</i>, suggests <u>there is easily $1 trillion in value at stake from betting that rates will rise</u>.</p><ul><li><p>In response to a 1 percentage point rise in interest rates, the price of a bond falls in percentage terms by its “<a href="https://www.blackrock.com/fp/documents/understanding_duration.pdf"><i>duration</i></a>”, to a first-order approximation.</p></li><li><p>The average value-weighted duration of (privately-held) US treasuries <a href="https://www.brookings.edu/blog/up-front/2022/07/27/projecting-the-structure-of-us-treasury-debt/">is approximately 4 years</a>.</p></li><li><p>So, to a first-order approximation, if rates rise by 3 percentage points, then the value of treasuries will fall by 12% (that is, 3*4).</p></li><li><p>The market cap of (privately-held) treasuries <a href="https://www.dallasfed.org/research/econdata/govdebt#data">is approximately $17 trillion</a>.</p></li><li><p>Thus, if rates rise by 3 percentage points, then the total value of treasuries can be expected to fall by $2.04 trillion (that is, 12%*17 trillion).</p></li><li><p>Slightly more than half (55%) of the interest rate sensitivity of the treasury market <a href="https://www.brookings.edu/blog/up-front/2022/07/27/projecting-the-structure-of-us-treasury-debt/">comes from</a> bonds with maturity beyond 10 years. Assuming that the 3 percentage point rise occurs only at this horizon, and rounding down, we arrive at the $1 trillion estimate.</p></li></ul><p>Alternatively, returning to the LTPZ ETF with its duration of 20 years, a 3 percentage point rise in rates would cause its value to fall by 60%. Using the 3x levered TTT with duration of 18 years, a 3 percentage point rise in rates would imply a mouth-watering cumulative return of 162%.</p><p>While fully fleshing out the trade analysis is beyond the scope of this post, this illustration gives an idea of how large the possibilities are.</p><p>The alternative to this order-of-magnitude estimate would be to build a complete bond pricing model to estimate more precisely the expected returns of shorting treasuries. This would need to take into account e.g. the convexity of price changes with interest rate movements, the varied maturities of outstanding bonds, and the different varieties of instruments issued by the Treasury. Further refinements would include trading derivatives (e.g. interest rate futures) instead of shorting bonds directly, for capital efficiency, and using leverage to increase expected returns. </p><p>Additionally, the analysis could be extended beyond the US government debt market, again since changes to real interest rates would plausibly impact the price of every<i> </i>asset: stocks, commodities, real estate, <i>everything</i>.</p><p>(If you would be interested in fully scoping out possible trades, we would be interested in talking.)</p><p><strong>Trade risk and foom risk.</strong> We want to be clear that – unless you are risk neutral, or can borrow without penalty at the risk-free rate, or believe in short timelines with 100% probability – then such a bet would not be a free lunch: this is not an “arbitrage” in the technical sense of a <a href="/posts/Afwaj6sGfxcQZrmYF/chapter-4-the-efficient-market-hypothesis">risk-free profit</a>. One risk is that the market moves in the other direction in the short term, before correcting, and that you are unable to roll over your position for liquidity reasons. </p><p>The other risk that could motivate not making this bet is the risk that the market – for some unspecified reason – never has a chance to correct, because (1) transformative AI ends up <i>unaligned </i>and (2) humanity’s conversion into paperclips occurs <i>overnight</i>. This would prevent the market from ever “waking up”.</p><p>However, to be clear, expecting this specific scenario requires both: </p><ol><li><p>Buying into specific stories about how takeoff will occur: specifically, Yudkowskian <a href="/tag/the-hanson-yudkowsky-ai-foom-debate">foom</a>-type scenarios with fast takeoff.</p></li><li><p>Having a lot of skepticism about the optimization forces pushing financial markets towards informational efficiency.</p></li></ol><p>You should be sure that your beliefs are actually congruent with these requirements, if you want to refuse to bet that real rates will rise. Additionally, we will see that the second suggestion in this section (“impatient philanthropy”) is not affected by the possibility of foom scenarios.</p><h2>2. Borrow today, including in order to fund philanthropy (“impatient philanthropy”)</h2><p>If prevailing interest rates are lower than your subjective discount rate – which is the case if you think markets are underestimating prospects for transformative AI – then simple cost-benefit analysis says you should save less or even borrow today.</p><p><strong>An illustrative example.</strong> As an extreme example to illustrate this argument, imagine that you think that there is a 50% chance that humanity will be extinct next year, and otherwise with certainty you will have the same income next year as you do this year. Suppose the market real interest rate is 0%. That means that if you borrow $10 today, then in expectation you only need to pay $5 off, since 50% of the time you expect to be dead.</p><p>It is only if the market real rate is 100% – so that your $10 loan requires paying back $20 next year, or exactly $10 in expectation – that you are indifferent about borrowing. If the market real rate is less than 100%, then you want to borrow. <u>If interest rates are “too low” from your perspective, then on the margin this should encourage you to borrow</u>, or at least save less.</p><p>Note that this logic is not affected by whether or not the market will “correct” and real rates will rise before everyone dies, unlike the logic above for trading.</p><p><strong>Borrowing to fund philanthropy today.</strong> While you may want to borrow today simply to fund wild parties, a natural alternative is: borrow today, locking in “too low” interest rates, in order to fund philanthropy today. For example: to fund AI safety work.</p><p>We can call this strategy “impatient philanthropy”, in analogy to the concept of “<a href="https://80000hours.org/podcast/episodes/phil-trammell-patient-philanthropy/">patient philanthropy</a>”.</p><p>This is not a call for philanthropists to radically rethink their cost-benefit analyses. Instead, we merely point out: <u>ensure that your financial planning properly accounts for any difference between</u> <i>your</i> <u>discount rate and</u> <i>the market real rate at which you can borrow</i>. You should not be using the market real rate to do your financial planning. If you have a higher effective discount rate due to your AI timelines, that could imply that you should be borrowing today to fund philanthropic work. </p><p><strong>Relationship to impatient philanthropy.</strong> The logic here has a similar flavor to <a href="https://80000hours.org/podcast/episodes/phil-trammell-patient-philanthropy/">Phil Trammell’s “patient philanthropy” argument</a> (<a href="https://globalprioritiesinstitute.org/wp-content/uploads/Trammell-Dynamic-Public-Good-Provision-under-Time-Preference-Heterogeneity.pdf">Trammell 2021</a>) – but with a sign flipped. Longtermist philanthropists with a zero discount rate, who live in a world with a positive real interest rate, should be willing to save all of their resources for a long time to earn that interest, rather than spending those resources today on philanthropic projects. Short-timeliners have a higher discount rate than the market, and <a href="https://80000hours.org/podcast/episodes/phil-trammell-patient-philanthropy/#what-about-groups-who-have-a-particular-sense-of-urgency-014046">therefore</a> should be <i>im</i>patient philanthropists.</p><p>(The point here is not an exact analog to Trammell 2021, because the paper there considers strategic game theoretic considerations and also takes the real rate as exogenous; here, the considerations are not strategic and the endogeneity of the real rate is the critical point.)</p><h1><strong>XI. Conclusion: outside views vs. inside views & future work</strong></h1><p>We do not claim to have special technical insight into forecasting the likely timeline for the development of transformative artificial intelligence: we do not present an inside view on AI timelines. </p><p>However, we do think that market efficiency provides a powerful <i>outside view</i> for forecasting AI timelines and for making financial decisions. Based on prevailing real interest rates, the market seems to be strongly rejecting timelines of less than ten years, and does not seem to be placing particularly high odds on the development of transformative AI even 30-50 years from now.</p><p>We argue that market efficiency is a reasonable benchmark, and consequently, this forecast serves as a useful prior for AI timelines. If markets are wrong, on the other hand, then there is an enormous amount of money on the table from betting that real interest rates will rise. In either case, this market-based approach offers a useful framework: either for forecasting timelines, or for asset allocation.</p><p><strong>Opportunities for future work.</strong> We could have put 1000 more hours into the empirical side or the model, but, TIABPNAJA. Future work we would be interested in collaborating on or seeing includes:</p><ol><li><p>More careful empirical analyses of the relationship between real rates and growth. In particular, (1) analysis of data samples with larger variation in growth rates (e.g. with the Industrial Revolution, China or the East Asian Tigers), where a credible measure of <i>real </i>interest rates can be used; and (2) <i>causally identified </i>estimates of the relationship between real rates and growth, rather than correlations. Measuring historical real rates is the key challenge, and the main reason why we have not tried to address these here.</p></li><li><p>Any empirical analysis of how real rates vary with changing existential risk. Measuring changes in existential risk is the key challenge.</p></li><li><p>Alternative quantitative models on the relationship between real interest rates and growth/x-risk with <a href="https://arxiv.org/abs/2201.10673">alternative preference specifications</a>, incomplete markets, or disaster risk.</p></li><li><p>Tests of market forecasting ability at longer time horizons for any outcome of significance; and comparisons of market efficiency at shorter versus longer time horizons.</p></li><li><p>Creation of sufficiently-liquid genuine <i>market </i>instruments for directly measuring outcomes we care about like long-horizon GDP growth: e.g. GDP swaps, GDP-linked bonds, or binary GDP prediction markets. (We emphasize <i>market</i> instruments to distinguish from forecasting platforms like Metaculus or play-money sites like Manifold Markets where the forceful logic of financial market efficiency simply <a href="https://basilhalperin.com/essays/metaculus-monetary-policy.html">does not hold</a>.)</p></li><li><p>An analysis of the most capital-efficient way to bet on short AI timelines and the possible expected returns (“the greatest trade of all time”).</p></li><li><p>Analysis of the informational content of infinitely-lived assets: e.g. the discount rates embedded in land prices and rental contracts. There is an existing literature related to this topic: <a href="https://academic.oup.com/qje/article/130/1/1/2337985">[1],</a> <a href="https://faculty.haas.berkeley.edu/amir/discussion/Discussion_Slides_Very_long_run_risk.pdf">[2],</a> <a href="https://www.sciencedirect.com/science/article/abs/pii/S0165176522002919?dgcid=raven_sd_via_email">[3],</a> <a href="https://www.sciencedirect.com/science/article/pii/S0197397507000574?via%3Dihub">[4]</a>, <a href="https://academic.oup.com/ej/article/128/613/1820/5088336">[5],</a> <a href="https://onlinelibrary.wiley.com/doi/10.1002/jae.2867">[6],</a> <a href="https://academic.oup.com/rfs/article/34/8/3527/6187965">[7]</a>.<ul><li><p>This literature estimates <i>risky, nominal</i> discount rates embedded in rental contracts out as far as 1000 years, and finds surprisingly low estimates – certainly less than 10%. This is potentially extremely useful information, though this literature is not without caveats. Among many other things, we cannot have the presumption of informational efficiency in land/rental markets, unlike financial markets, due to severe frictions in these markets (e.g. inability to short sell).</p></li></ul> </p></li></ol><p>Thanks especially to <a href="https://www.forourposterity.com/">Leopold Aschenbrenner</a>, <a href="https://thegoodblog.substack.com/">Nathan Barnard</a>, <a href="https://www.linkedin.com/in/jackson-w-barkstrom/">Jackson Barkstrom</a>, <a href="https://joel-becker.com/">Joel Becker</a>, <a href="https://danicaratelli.github.io">Daniele Caratelli</a>, James Chartouni, <a href="https://www.tamaybesiroglu.com/">Tamay Besiroglu</a>, <a href="https://economics.mit.edu/people/phd-students/joel-flynn">Joel Flynn</a>, <a href="https://jamesrhowevi.me/">James Howe</a>, <a href="https://www.linkedin.com/in/charles-hyland/?originalSubdomain=au">Chris Hyland</a>, <a href="https://stephenmalina.com/">Stephen Malina</a>, <a href="https://twitter.com/ptr_mcl">Peter McLaughlin</a>, <a href="https://www.jacksonmejia.com/">Jackson Mejia</a>, <a href="https://www.lauranicolae.com/">Laura Nicolae</a>, <a href="https://www.linkedin.com/in/sam-henry-lazarus-1a6617150/">Sam Lazarus</a>, <a href="https://lehrer.substack.com/">Elliot Lehrer</a>, <a href="https://economics.mit.edu/people/phd-students/jett-pettus">Jett Pettus</a>, <a href="https://brettongoods.substack.com/">Pradyumna Prasad</a>, <a href="http://www.tejassubramaniam.com/">Tejas Subramaniam</a>, <a href="https://karthiktadepalli.com/">Karthik Tadepalli</a>, <a href="https://philiptrammell.com/">Phil Trammell</a>, and participants at <a href="https://ea.greaterwrong.com/posts/L3WPuztkSMohBTWqZ/etgp-2022-materials-feedback-and-lessons-for-2023">ETGP 2022</a> for very useful conversations on this topic and/or feedback on drafts.</p><p>Update 1: we have now posted a <a href="https://ea.greaterwrong.com/posts/8c7LycgtkypkgYjZx/agi-and-the-emh-markets-are-not-expecting-aligned-or#comment-6sgwspRx5irn5APNB">comment</a> summarising our responses to the feedback we have received so far.</p><p><br><br><strong>Postscript</strong></p><p>OpenAI’s ChatGPT model on what will happen to real rates if transformative AI is developed:</p><p class="imgonly"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1673285569/mirroredImages/ngpC5PFAgxHJMhicM/rcqopiocw5ft5qnnbciw.png" loading="lazy"></p><p>🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔</p><p>Some framings you can use to interpret this post:</p><ol><li><p>“This blog post takes Fama seriously” [a la <a href="https://eml.berkeley.edu/~dromer/papers/MRW_QJE1992.pdf">Mankiw-Romer-Weil</a>]</p></li><li><p>“The market-clearing price does not hate you nor does it love you” [a la <a href="https://www.goodreads.com/quotes/499238-the-ai-does-not-hate-you-nor-does-it-love">Yudkowsky</a>]</p></li><li><p>“Existential risk and asset pricing” [a la <a href="https://globalprioritiesinstitute.org/leopold-aschenbrenner-existential-risk-and-growth/">Aschenbrenner 2020</a>, <a href="https://philiptrammell.com/static/ExistentialRiskAndExogenousGrowth.pdf">Trammell 2021</a>]</p></li><li><p>“Get rich or hopefully don’t die trying” [a la <a href="https://en.wikipedia.org/wiki/Get_Rich_or_Die_Tryin%27">50 Cent</a>]</p></li><li><p>“You <i>can</i> short the apocalypse.” [contra <a href="https://ea.greaterwrong.com/posts/Fucn4TDYLCAxLybvL/x-risk-and-peter-thiel-s-optimistic-investment-thesis#Anthropic_logic_means_markets_ignore_the_risk_">Peter Thiel</a>, cf <a href="https://marginalrevolution.com/marginalrevolution/2017/08/can-short-apocalypse.html">Alex Tabarrok</a>]</p></li><li><p>“Tired: <a href="https://en.wikipedia.org/wiki/Market_monetarism">market monetarism</a>. Inspired: <i>market longtermism</i>.” [a la Scott Sumner]</p></li><li><p>“This is not <i>not</i> financial advice.” [a la <a href="https://money.stackexchange.com/questions/144674/why-do-some-opinion-pieces-say-this-is-not-financial-advice">the standard disclaimer</a>]</p></li></ol><h2><strong>Appendix 1. Against using stock prices to forecast AI timelines</strong></h2><p><a href="/posts/yFkNYyspBBqfSeBx9/against-using-stock-prices-to-forecast-ai-timelines">Link to separate LW post</a></p><h2><strong>Appendix 2. Explaining Tyler Cowen’s Third Law</strong></h2><p><a href="https://basilhalperin.com/essays/cowens-third-law.html">Link to separate post</a></p><h2><strong>Appendix 3. Asset pricing under existential risk: a literature review</strong></h2><p><a href="https://docs.google.com/document/d/1sVkVxhQ179XNieu1zAiwzDsFec5r6UQn0dTKJjgCbOk/edit">Link to Google Doc</a></p><h2><strong>Appendix 4. Supplementary Figures</strong></h2><p><a href="https://docs.google.com/document/d/1CWw4t_SzQKDaEhiXGxZBtZQ7vkLddL2aGGqbqVibI-g/edit?usp=sharing">Link to Google Doc</a></p>basil.halperinngpC5PFAgxHJMhicMTue, 10 Jan 2023 16:06:52 +0000Against using stock prices to forecast AI timelines by basil.halperin
https://www.greaterwrong.com/posts/yFkNYyspBBqfSeBx9/against-using-stock-prices-to-forecast-ai-timelines
<p><br><i>by </i><a href="https://tmychow.com/"><i>Trevor Chow</i></a><i>, </i><a href="https://basilhalperin.com/"><i>Basil Halperin</i></a><i>, and </i><a href="https://jzmazlish.substack.com/"><i>J. Zachary Mazlish</i></a></p><p>[Note: This is an appendix to <a href="/posts/ngpC5PFAgxHJMhicM/agi-and-the-emh-markets-are-not-expecting-aligned-or-1">“AGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years”</a>]</p><p>One naive objection would be the claim that real interest rates sound like an odd, arbitrary asset price to consider. Certainly, real interest rates are not frequently featured in newspaper headlines – if any interest rates are quoted, it is typically <i>nominal</i> interest rates – and stock prices receive by far the most popular attention.</p><p><strong>The importance of real rates.</strong> However, even if real interest rates are not often discussed, real interest rates affect <i>every </i>asset price. This is because asset prices always reflect some <i>discounted</i> value of future cash flows: for example, the price of Alphabet stock reflects the present <i>discounted</i> value of future Alphabet dividend payments. These future dividend payments are discounted using a discount rate <i>which is determined by the prevailing real interest rate</i>. Thus the claim that real interest rates affect every asset price. </p><p>As a result, if real interest rates are ‘wrong’, <i>every</i> asset price is wrong. If real interest rates are wrong, <i>a lot</i> of money is on the table.</p><p><strong>Stocks are hard to interpret. </strong>It may nonetheless be tempting to look at <i>stock</i> prices to attempt to interpret how the market is thinking about AI timelines (e.g. <a href="https://www.airr.io/quote/636d77b9249836a41ad9e0e1">Ajeya Cotra</a>; <a href="https://twitter.com/MatthewJBar/status/1599394102881456130">Matthew Barnett</a>; <a href="https://old.reddit.com/r/slatestarcodex/comments/v7omil/if_we_think_agi_is_imminent_in_which_companies/">/r/ssc</a>). It may be tempting to consider the high market capitalization of Alphabet as reflecting market expectations for large profits generated by DeepMind’s advancing capabilities, or TSMC’s market cap as reflecting market expectations for the <a href="https://twitter.com/robertwiblin/status/1577782210568003585">chipmaker</a> to profit from AI progress.</p><p>However, extracting AI-related expectations from stock prices is a very challenging exercise – to the point that we believe it is simply futile – for four reasons.</p><ol><li><p>First, and most importantly, these companies will only have the possibility of high profits if transformative AI is <i>aligned</i>; under unaligned AI, the value of stocks along with everything else is converted to zero. </p></li><li><p>Second, it is not obvious that even in the aligned case that these companies will earn high profits. For instance, OpenAI has committed to a capped profit model, and others <a href="https://www.metaculus.com/questions/4061/will-any-major-ai-company-commit-to-an-ai-windfall-clause-by-2025/">may</a> sign on to a similar ‘<a href="https://www.fhi.ox.ac.uk/windfallclause/">Windfall Clause</a>’. Beyond corporate altruism, it seems extremely plausible that if a private company develops truly transformative AI technology then the state will (attempt to) nationalize and expropriate it to distribute the benefits more broadly, preventing profits. </p></li><li><p>Third, stock valuations are extremely idiosyncratic: which stock should we be looking at? And critically, even if we take a basket of tech companies and average over them, then this only includes <i>public</i> companies. If the market expects transformative AI in 12 months, but only because it will be developed by OpenAI – a company which is not traded publicly – then this will not show up in any equity index.</p></li><li><p>Fourth, and quite importantly, <i>it is not obvious whether expectations of transformative AI would raise or lower stock prices</i>. This is because, as described in the previous subsection, stock prices reflect the present-discounted value of future profits; and advanced AI may raise those future profits, but – as the central thesis of this piece argues – advanced AI would <i>also</i> raise the interest rate used to discount those profits. The net effect on stock prices is not immediately obvious. <ol><li><p>(In math, briefly: if the price P is the value of future profits D discounted at rate r, i.e. P=D/r, then transformative AI may raise future profits D but it could raise the discount rate r by even more.)</p></li><li><p>(Higher growth causes lower average stock prices if the <a href="https://eml.berkeley.edu/~enakamura/teaching/EstimatingIES.pdf">intertemporal elasticity of substitution</a> is greater than one, rather than less than one. This parameter is subject to significant debate; see the linked slides for useful discussion. John Cochrane offers additional intuition <a href="https://johnhcochrane.blogspot.com/2018/02/stock-gyrations.html">here</a> and argues that the empirically-relevant case is the one where higher growth causes lower equity prices: expectations for transformative AI would <i>lower</i> equity prices.)</p></li></ol></p></li></ol><p>If you want to use market prices to predict AI timelines, using equities is not a great way to do it.</p><p>In contrast, real interest rates do not suffer from these problems. </p>basil.halperinyFkNYyspBBqfSeBx9Tue, 10 Jan 2023 16:03:31 +0000