given that AI-associated x-risk largely depends on our level of mathematical competence, this is also arguably the most important time in history for the social role of mathematicians.
Strongly doubt that.
FlorianH
Karma: 709
Your views on Data Science studies future?
Do you have advice for a university thinking about what curriculum to teach at the interface of Data Science/AI/Business?
I guess the most basic obvious points are covered, but maybe you have interesting points/opportunities one might be overlook and that might realistically be taken up by a university today, aiming to turn out graduates attractive to a labor market in the next 4-10 years (obviously in a not-full-AI-take-off-in-the-next-5-years or so scenario)?
“Emptier than your average flight would have you naively believe” should be true I reckon.
Again don’t fully disagree: yes exposing people, if you manage to get them to listen at all, a bit longer to longtermism, could easily make some of them agree with you.
But
Most people who disagree with longtermism probably do so because partisans on their side of the political spectrum disagree with longtermism for x reason [..]
seems nearly a bit conspiratorial, while reality seems to me so much more banal instead: Most of us are simply not wired to care a lot about these questions. Before anyone would have to exercise some propaganda on you, natural instinct is to care about much more tangible things than an (felt) abstract future expansion of humans and so on. It’s just not what commoners think about. Not because sb tells them it’s not important. Simply because natural drive to even think and care about such things is lacking. When you write most “haven’t been exposed to a fair analysis of how longtermists resolve most of the complaints brought up” you’re for sure right, but that ‘haven’t been exposed’ in itself is signal, it’s a symptom of their own preferences, maybe more than of external system factors: People don’t care about longtermism questions and neither about arguments. Had people sought wisdom about longtermism, the market would trivially flood them with articles and media highlighting arguments like the ones we may make here for it.
So the problem is one level deeper in our psyche and in our moral behavior: as humans, we won’t typically engage with this thing for a start, and thus, also not be educated about it.
FWIW, I think this complacency need also not be surprising at all from an evolutionary perspective. We have mostly ad-hoc moral interests and tastes, those that have enabled us to highlight why we are the ones in the right and of importance and a broad set of things of that sort. Longtermism is really far out from that set of evolutionary pressures we were exposed to.
It seems absurd that most people don’t find longtermism as an ideology compelling
Not fully disagreeing. But it doesn’t mean it’s wrong to suggest longtermism is counterintuitive for us all in all.
Instead: Yes, human moral taste simply is completely absurd—when judged in the way that you implicitly call out.Meet human moral tastes:
Rmbr how we want to give that one single poor child on the picture—yet ignore how we could save many more efficiently? And how our motivation pales just once we see there are these many to be fed instead of that one?
Rmbr how we all love to care for our cat and feed it the most evilly abused chicken day by day? And it’s not that we’d not know how the chicken is abused. Many really do know all while continuing.
Rmbr how we’re all happy going to protest against Shell, not for corrupting policy and cheating us, potentially, but merely for having sold big quantities of oil—that exact oil we’re until today continuing to order from them every day? Ok, this one a super random example in my mind for a seaparate reason today, but I just wanted to take one absurdity among the million you could pick.
Let’s not bore everyone with continuing this obvious list further.
--> So while I find your individual points do exactly make sense, and it seems like longtermism should be the most natural thing: In the aggregate, it is not in any way an absurd claim that longtermism is counterintuitive for many—it really is exactly that. But really, it’s also not a surprise, given what I’d guess even you—or at least most readers here—might most plausibly be able to observe (i) even within oneself—despite the slightly higher rationality than average humans’ - or, else, at the very least, (ii) among pretty much everyone else near and dear you.
OP has a theoretical point but let’s fleshman and not steelman it and then it seems to me, it is for most bits and purposeses simply too abstract. In most cases where we like to steelman an idea it is because we want to give the subject matter, the core idea itself the benefit of the doubt, maybe exactly—and for good reason—to separate it from its flawed proponents. This could theoretically be stupid in some cases but in most I think it’s exactly the careful conversational rule that can help prevent talking past each other or ignoring useful insights.
Say: Marx still to be taken seriously? Don’t hang the idea with it’s brutal historic fellows, or with it’s well-meaning but heart-not-brain-led current advocates’ rosy claims. Let’s see whether in absolute, his core tenets and claims may hold true under today’s or tomorrow’s unmovable parts of the technological and biological framework conditions. Let’s steelman it to see whether we can really fully or only mostly throw him out without regret
Yes, whether we’d actively support _a specific group_ of course would more depend on fleshwomaning but I don’t think that’s most often the point of discussion (even elsewhere than on LW), and where it is, I don’t think the “steelmanning” as concept tends to sneak in.
Maybe one could say much of OP’s intuitive traction therefore hinged on strawmanig a bit steelman’s case.
Weakly pushing back on bare optimism about this—not saying it makes no sense in any form:
Within OP a rare concrete advantage mentioned is ‘tax code’. And rightly, tax is a super key domain. Issue: First solve asymmetric tax competition, else you just introduce a new type of race-to-the-bottom elements that are more parasitic than genuinely productive.
It’s trival to attract big money as minuscule charter city within the large EU pond. But it subverts the idea of a functioning overall redistributive ecosystem.
Stability vs. Instability as ideal ground: Yes stability is helpful. BUT, the other way round: Charter cities’ theoretical potential is also greatest where we have instability and a failure to enable labor to be productive and where sunk investments are subject to hold up/expropriation one way or other. The latter is usually not an issue in EU. And there are many places in Europe that allow labor to work reasonably productively and without prohibitive taxes. Backfiring labor protection in many places yes, but some states even design template engagement forms to purposely circumvent official employment in what clearly is employment. So you’re free to hire persons close to how you want. High taxes in many places yes, but some places have simply reasonably low or maybe arguably too low taxes e.g. in some places due to three-fold tax competition (country/state/commune each in competition with their peers at the same level). Basic economic liberty mostly given—well, arguably a ton of bad laws wherever you look, but does lifting them really make the big bang for most of society? Not automatically convinced about that.
Finally: We anyway already have a ton of implicit, small special regional zones in Europe. Many are countries or other subnational jurisdictions whose main advantage is not innovation but indeed: simply low taxes/great conditions for the rich. One of them is indeed so successful atm they literally ran out of good ideas how to spend the excess money they get[1] - though for its original citizens the situation isn’t automatically great as many complain they cannot afford housing anymore as the region is too attractive (if you search a room you will stumble upon people offering one on the condition for you to not be there—as many seek the location just to benefit from low taxes).
Surely you can theoretically with smart policies find great ways to foster innovation etc., and to reign in govt overreach by having competing small jurisdictions. All for it. Before you get my support though, show how you avoid simply becoming yet another Monaco or Zug or what have you with doubt-able net benefit for the world.
- ^
Sounds implausible right? It’s true still. They were searching for ways to redistribute the money back to the economy/people. They now even open a new fund of dozens of millions for “blockchain research”—doesn’t quite appear to be the lowest hanging fruit for what to do with random spare money.
Thanks for the education, I’ve read quite some Frankish et al. but somehow I had missed that part. Yes, Kammerer interviews it was! Looking fwd to reading your two posts! Background: I always found Illusionists, but if you thus want Kammerer, ‘stupid’ in claiming, say, ‘actual reason to genuinely care’ may not fall apart if (thus, say, Kammerer’s strong) illusionism held, and Kammerer’s talk about the reasons I found really low-grade philo confabulation. Since a few years I, let’s say ‘intellectually’, put quite some credence to that type of illusionism, in a way (emotionally rather not at all so far), and find it super dangeorous. (fwiw I actually think rather than ‘machine is conscious’, the AI development may in my expectation also nudge, say, careful enquirers more towards such strong illusionism and it may be terrible—and be terrible even if potentially true, to the degree that ‘terrible’ then still deserves a place in language). May my other comment be downvoted appropriately :)
Not so clearcut. Clearly there are e.g. known illusionist philosophers providing wishiwsshy reasons as to why illusionists ought to still be altruistic. Their arguments there tens to not be that intrinsic value there is but actually seriously esoteric stuff. I’ll provide link if the name of they French guy interviewed on it pops up again—and he’s just an example. Illusionists portray very clearly the situation often as one of absence of intrinsically morally loaden feelings.
Update: Glad to see the post Voluntary Paternalism (original here) - reinventing this wheel one could say, but in particular providing an imho ideal name for what I propose!
I’ve proposed THIS here: Alienable (not Inalienable) Right to Buy!
Thanks to your post I now have a better name for it! I love “Voluntary Paternalism”!
Every sentence you write gets from me 💯💯💯.
We should feel sad that our posts don’t get more upvotes—as it’s exactly an important idea!
It addresses not only inividual well-being but also democratic buy-in to our market system that evilly over-tempts us with stuff we cannot well enough resist to and that commoners—at least superficially—often hate, and that makes people cynical about economy and society and, in the democratic cases, democracy, I’d wager from anecdotal evidences.
You might btw find interesting another post I’ve written that touches upon: Why paternalism anyway, even the non-voluntary one. I start with a weird-seeming trip re what ‘self’ actually is when we really are skeptical observers rather than naive believers in what we feel, but you can even disregard that a bit, but it ends in an argument as to why the fact that there’s simply no reason to treat “you” now and “you” in 30 years as a fully integrated “single entity” in a most fundamental way; if you’re myopic in your view and not careful for the future, there’s no reason to keep the you-in-30-years unabatedly hostage by your current you; we must intervene. Your own fully pragmatic ‘it’s simply better to intervene’ can suffice, fwiw here that extended (implied) argument pro paternalism: Relativity Theory for What the Future ‘You’ Is and Isn’t
(would be keen to read also your potential agreement or disagremeents).I think btw while even our LW co-smarties seem to underappreciate the relevance of the thing—call me biased—tech will of course help us solve it at least partly. The bank account that let’s you easily pre-program what you can and cannot spend on is surely just around the corner, tbc.
been hearing about PV and wind “exploding” for over a decade now, that makes me skeptical. If it happens I’ll be pleasantly surprised.
I work in modelling energy markets since 10-20y and yes we ‘saw it explode’ since 1-2 decades indeed; it has been rapid exponential growth, and now its a few % of global power (more and more countries going rapidly towards high double-digit %age), and if nothing fundamentally changes, soon a few and then many % of global energy, I see nothing to stop this. My honest strong gut feeling (FWIW!!) is: “OP is a bit late to the trash renewables (RES) party” (again, despite the still slightly humbling graph). 20y ago it started with claims as to how it’d be a physical tautology that RES can’t make a serious dent ever. ‘We’d have to cover this high double-digit share of all agri surface and have wind turbines EVERYwhere’ or things like that. Then they were unaffordable. Then we cannot store them. BUT TODAY, REALLY: really cheap, not so storable anymore, IOT naturally enables a ton of flexibilities; we’re super dumb societally and don’t dare strong dynamic pricing but with that we’d adjust on a whim I’m sure—though this cannot be super well studied without having (i) social scale (ii) long-ish term daring attempts to expose firms, industries, households to dynamic prices. And I don’t have the impression the expansion is just from strong subsidies or something; often it simply seems like a very natural option to go for. Then again, your graph surprised me more than it should so maybe I’m too much in the weeds, in the individual numbers to see part of the big pic.
re tax wedge: The optimal tax vis-a-vis pollution is not 0 but the level that coresponds to the marginal pollution cost, no matter whether some emissions will eventually be CCS-ed or not. There’s anyway no ‘distortion’ as the basic distortion you fear from taxing is ‘qty reduction’ but that’s exactly what’s optimal in the face of a negative externality. Bread shouldn’t cost 0 in the optimum only because somebody (in this case the baker himself but that’s more accidental in terms of discussing optimal prize) bears a negativity (his effort); carbon emissions exactly equally simply shouldn’t cost 0 as broad society bears a negativity. You want taxes as “fair” but I see nothing unfair here.
Btw, I’m fully agreeing we should exactly skim resource extraction rents in a way you describe, not contesting at all. (and interestingly it would also be a very natural & undistortive way to deal with AI creating a lot - a lot from our earth as inputs and the inputs it should pay for so we have lots of public money and thus random people can naturally own much of the eventual produce, even if the AI owners as intermediaries may keep a competitive, incentivizing margin remunerating their contribution)
Love the challenge and I think good points in order to not forget the reality, but I think it’s still relveant that:
60$/tCO2 sequestered feels a bit cherrypicked; significantly too low. Cheap biological sequestration with strong durability questions about. Sequestering fossil fuel CO2 to underground often significantly more expensive, and for good or for bad is mostly the main removal route considered afaik; I’ve worked on financial estimates for projects and I think we were usually significantly above 100€/tCO2. The EU has carbon tax above 60$/tCO2 but I don’t think we’re on track to getting much of it w/o special subsidies for it (all projects I’ve so far seen were for special support funds)
Plus, there’s topic of ‘depletion’ of good reservoirs for e.g. liquefied CO2 which is still a mainstay of CCS
Yes energy mix pic so far doesn’t look great, and I admit upon looking it up to my surprise your graph is one with the substitution method that shows the proportions even in a relatively renewables-favorable way. Oopsie. Still, PV deployment rate is massively exploding now and it seems like it’s not everywhere only because of subsidies. I hear often about it being cheapest scalable source in many locations. Look at record breaking competitive auction prices. Yes, intermittency, but nowadays even with batteries getting seriously cheap. Add in EVs as natural batteries. Plus, how easily we can flexibilize demand if need be—if we get used to it on a societal level.
More generally, I think your ‘rent’ topic is mostly overcomplicating the narrative. We want energy, that much we know. We can see whether it’s cheaper to build renewable energy or fossil+carbon removal and the one that is cheaper, indeed, “wins” the economic-sensemaking-competition, as simple as that—on first order approximation (massive simplification but not more than in your narrative), and if we’re assuming all elements considered would be available at any required scale.
I’m not alone to want to live in a city with EVs just because of local pollution & noise & accumulating blackness and because I then myself can have a car without having too much annoying maintenance. Just to say: Gosh, how dirty dirty fuel is while electrification has nice side benefits—I think in partiuclar in transport sector, which you emphasize: electrification as +- inevitable part of the renewables transition is a super large obvious extra benefit along quite a few dimensions. I personally believe once we get rid of cars in our cities we’ll look back like we do in my country to the time where we had cigarettes in all trains and bars: it was not nice.
You cite Norway, and you may live in the US, but mind: for large parts of the world continuing with fossil fuels as main energy sources means maintaining an uncanny dependence from despots and what have you.
Looking fwd to seeing where it goes in 20 years—well, that’s today of course a bit a weird thing to say, and reminds me yes expontential data centres may also make it more challenging to go only renewables. Damn, yet another point for you :).
Update: Just found https://community.samharris.org/c/general/self-is-an-illusion-and-this-will-make-you-feel-it proposing roughly the same argumentation and same conclusions—and leading you nicely up to it with a few small thought experiments. Not really surprised somebody else also came up with it independently: It really is such an obvious point even if it is all set up for it to feel totally impossible.
My comment is slightly harsh-toned, because the concept of moral “obligation” tends to rub me slightly the wrong way. Either I appeal to your kindness in the hope you’ll WANT to be good, or I have the WHIP to materially impose something on you—philosophical “obligation” in the abstract sense in between feels often unhelpful, which strikes me as part of the issue in OP.
An original sin is your shallow definition of obligation. “A thing one must (or must not) do” is too thin, the crucial question is who obliges whom if there is a ‘must’, and leaving this open makes the whole argument nearly a question of definition. Different sources of obligation—a community’s claim, one’s own commitments, institutions—would give different verdicts on exactly the cases your argument leans on.
Key point re core logic though: Your key point is about parity, and it’s intuitively super appealing but imho as bottom line actually misguided; too theoretical and ignoring randomness: by your own numbers, practices we accept to kill animals in the same ballpark as meat-eating (driving: 75 vertebrates per lifetime; windowed houses: hundred of birds; power lines another handful), so by parity meat should be fine too, and this may hold esp. when using relative personal benefit/overall harm type ratios. But parity needs those other practices to actually be ‘permissible’ (using similar type of language as the philosophical ‘obligation’), not just currently accepted. In a super brutal and slavery-tolerant society, “don’t hit your wife slightly” wouldn’t stop being an obligation just because much worse usually goes unchecked—doesn’t pass my smell test. Moral progress is uneven and a bit random across domains, our psychology individual and collective is patchy about what it tolerates. Maybe driving and electricity are domains we’re just still backwards on, not proof the practice is fine. In sum, a ton of evil things being seen as ‘permissible’, would not mean ‘that other less evil thing’ should ‘thus also’ be permissible; let’s not hold us back on moral progress if in some domain we really manage to socially somehow—yes weirdly—we happened to be able to make it.
In sum: lacking operational definition of “obligation,” and a relevant insight about uneven moral demands imho overinterpreted due to ignoring, more interestingly, a crucial point about flexibility and randomness of domain-specific (or even detailed question-specific) moral progress.
Note: My comment should not be misinterpreted as to saying anything as to whether or not society ought to impose veganism or higher farm animal welfare standards; I agree with you simply that animal welfare is a super important topic, and my personal opinion is that regulating morally relevant things is very much the domain of democratic decision making, but it’s of course a super complex topic tooo.
Thanks, great warning that puts into perspective some of the techno-optimism often dominating the discussion. Less necessarily changing the assessment of those anyway looking at the risk at a more granular level, but maybe ideal rebuttal of those claims that take the shortcut of ‘tech always been good, so I need not listen further’.
One counter to your, say, it’s often been bad at least initially so a superficial prior could rather be ‘bad at least temporarily than simple-and-always good’, could be a parallel to what we’d say about how today’s society & institutions can prevent resource curse: ‘yes, in less developed societies oil diamonds what have you ends terrible for its population (in fact in partly quite similar ways to bunch of things you point out: hierarchical dominance, exploitation/inequality, …), BUT, now with our modern developed & strong institutions, we can handle it better’. ‘We’re now Norway, not a guerilla war state, when we discover the oil’ → here being ‘we now have strong democratic decision making to prevent the bad from AI/ASI and we’ll prevent many otherwise natural mistakes we’d have made centuries ago’.
The devil lies in the details as to which degree these modern institutions can or cannot really help us here. My confidence is weak that our institutions truly will improve the picture, and I remain pessimistic, but I find it a worthy complication for refinement of your argument.
I love economics (I’m an engineer and accidentally stumbled into it and now am PhD econ since 15 y or so and lecturer) and much of me believes DI quote exactly hits the nail. Sometimes, say 20% of me*, thinks that even reasonably advanced economics doesn’t help not so smart people really be smart. Still, I think the statement survives.
PM me if you’re interested in me sharing some materials personally with you! I’m also happy to have brief chats on things if useful.
Smartness + econ really is king.
One thing I can say upfront: I’m convinced on in LW or EA circles I have the impression to often find smarter quasi-economics analysis than among economists opinions/statements, so it’s all a bit a funny world with econ (a rant e.g. here).
*My exact literal and ungrammatical statement makes little sense but I reckon you see how I mean it.
Yes. But OBAA might also somewhat counterintuitively increase public trust instead. A bit like May labor day protest might appease people and make them accept capitalism more in daily live (and—maybe bit more remotely related—a bit like shallow and effectless ‘green product’ options and publicities might appease the average consumer giving him false warm comfort that environmental concerns are, after all, taken care of one way or other). I believe that’s not just a theoretical possibility but quite plausible.
Women who have reproduced recently are less likely to have passed the menopause.
I guess that was exactly part of what I meant.
Other mammals don’t live for decades after the menopause like we do.
Interesting point!! Apparently women really live exceptionally long, while their menopause (and its timing) itself is not unusual at all, and the most prevalent hypothesis is “grandmothering”, which posits what its name suggests, and essentially is, for our purposes, yet another symptom of them (say, our genes, or figuratively thus ‘our children’) desperately wanting us to stick around to care for these slow-up-growing and nerve-and-everything-wrecking offspring. In that sense, indirect extra, tiny support of the idea ‘they want to lure into having sex anytime as a loyalty encouraging devise and preventing us from straying too far’ or something like that. I’m vary if it sounds too simple but it does all seem to make sense (?).
Except that it actually mostly/likely enough isn’t? If anything it’s mostly a proxy for bad things (and ☘️ might accidentally help with some of them but not others)