Thrasymachus

Karma: 1,427

Thrasymachus 23 Dec 2025 12:08 UTC
13 points
3
on: How to game the METR plot
I also applaud the effort to interrogate the underlying data. I have also been dismayed at people hanging dramatic updates off (what usually should be?) 1-few bits of surprisal. (I don’t think METR can be fairly blamed for others ~hunting noise in the ‘last’ datapoint—the CIs are clearly printed on the graph.)
Per other comments, I think the more theoretical worries in the OP miss the mark: you should end up with something like logistic curve if task length is unbounded but success probability is (0, 1); logging does a fairly good job at linearizing the data (although at least for sonnet 3.7 the fit collapses in the 2hr+ region, and eyeballing the other histograms suggests this might generalize).
Yet I think they may be in right neighbourhood of a ‘construct validity’ worry around time horizons. In precis (hopefully a full post someday):
- Unlike (e.g.) ‘how fast can you run?’ or ‘how much can you lift?’ there’s seldom a handy cardinal scale for intellectual performance: IQ = 0 does not mean ‘zero intelligence’, nor you having double my chess ELO means you are twice as good at chess as I am. (Even if you’re happy not having a meaningful zero, meaningful interval scales don’t exist either.)
- Besides issues of general overprediction, it seems hard to tell how meaningful a D increment on X benchmark is. The function from ‘benchmark score’ to ‘irl importance’ (or ‘AI capabilities’) could be almost anything monotonic: from “any nonzero score is a cataclysmic breakthrough (but any further increment matters little on the margin”, to “long march through the ‘nines’ (so all scores <99.9% are ~equally worthless), and everything in between.
- Hence the utility of METR’s time horizons as a (/the only?) cardinal measure: ‘doubling’ is meaningful, and (if treated, as it often is—and I suspect more than METR would like it to be—as a proxy for ‘AI capabilities in general’) it shows a broad trend of exponentially increasing capabilities over last few years. (+/- discourse over whether recent data points indicate even more dramatic acceleration, ‘hitting the wall’, etc.)
- What is load-bearing for this account is the essentially exponential transformation between ‘raw’ scores on HCAST etc. to time horizons. Per OP (and comments), you can get a similar plot with just the raw scores, and it is largely the transformation from that to time horizons which gives (e.g.) Opus 4.5, scoring 75%, ~double the time horizon of GPT5 (70%), or ~treble the time horizon of o3 (66%). If the y-axis of the figure was instead “composite accuracy (SWAA+HCAST+REBench)”, the figure might be grist for the mill of folks like Gary Marcus: “A whole year of dramatically increasing investment and computation, and all it got you was another 10%.”
- It goes without saying METR didn’t simply stipulate ’linear score improvement = exponentially increasing time horizons”: it arose from a lot of admirable empirical work demonstrating the human completion time is roughly log-distributed.
- But at least when taken as the colloquial byword for AI capabilities, this crucial contour feels a bit too mechanistic to me. I take that you can generalise the technique widely to other benchmarks deepens rather than alleviates this concern: if human benchmarking exercises would give log-distributed horizons across the items in many (/most?) benchmarks, such that progressive linear increments in model performance would give a finding of exponentially improving capabilities, maybe too much is being proven.
- Taking the horizons (and their changes) literally has dubious face validity by my lights:
  - It doesn’t seem to me the frontier has gotten ~3x more capable over this year, and although I’m no software engineer, it doesn’t look from the outside like e.g. Opus 4.5 is 2x better at SWE than Opus 4.1, etc.
  - Presumably we could benchmark humans against the time horizons (IIRC not everyone used in the benchmarking could successfully complete the task), or at least the benchmarks from which time horizons could be imputed from. I’d at least be doubtful our best guess should be Alice (who cracks 75%) is 3x the SWE of Bob who hits 65%, etc.
  - That said, given our grasp of the ‘true cardinal scale of intellect’ is murky—or fictitious—even if my vibes are common, it looks reasonable to deny them rather than the facially contradicting data.
- Perhaps the underlying moral of the jagged frontier is there isn’t some crisp (at least crisp + practically accessible) measure out there re. ‘general intelligence’ (or even general measures of intelligence when particularly applied: cf. ‘twice as good at chess’), and we should focus on metrics specific to whatever real-world impact we are interested in (maybe for ‘AI generally’, just trend extrapolate from ‘economy generally’?). But if the story of benchmarks over the last while is they are missing whatever intellectual dark matter intervenes between ‘benchmark assessing X’ and ‘actually Xing’, maybe you can’t derive sturdy synthetic y-axis yardsticks from their distorted timber: the transfer function from ‘time horizon’ to ‘irl importance’ is a similar value of ”??” as the original benchmarks were.

Thrasymachus 24 Aug 2025 16:19 UTC
19 points
−7
on: Banning Said Achmiz (and broader thoughts on moderation)
As a tenured (albeit perhaps now ‘emeritus’) member of the “generally critical commentator crew”, I think this is the wrong decision (cf.). As the OP largely anticipates the reasons I would offer against it, I think the disagreement is a matter of degrees among the various reasons pro and con. For a low resolution sketch of why I prefer my prices of ‘pro tanto’ to the moderators:
1. I don’t think Said’s commenting, in aggregate, strays that close to the sneer attractor. “Pointed questions with an undercurrent of disdain” may not be ideal, but I have seen similar-to-worse antics^[1] (e.g. writing posts which are thinly veiled/naked attacks on other users, routine abuse of subtext then ‘going meta’ to mire any objection to this game with interminable rules-lawyering) from others on this site who have prosecuted ‘campaigns’ against ideologies/people they dislike.^[2]
2. The principal virtue of Said doing this for LW is calling bullshit on things which are, in fact, bullshit. I think there remains too much (e.g.) ‘post’-‘rationalist’ woo on LW, and it warrants robustly challenging/treating with the disdain it deserves. I don’t see many others volunteering for duty.
3. The principal cost is when this misfires, so the author ends up led into a subthread wasteland by Said thanks to him taking an odd, (unintentionally?) tendentious? line of questioning. In principle, this should not be that costly: if a comment asks for a clarification where I am confident other readers would agree with me the questioner is being very dumb, willfully obtuse, or making a ‘cheap shot’, I can ignore them without fear of third parties making an adverse inference. This applies whether this is the initial exchange or 1+ plys deep.^[3]
Even if ‘in principle’ this is fine, maybe (per OP) the scales tilt the other way in practice. But I don’t think doing more to be ‘writer friendly’ by squashing putative gadflies like Said gets you enough marginal high-quality community content to be worth it across the scales of the (admittedly-nebulous) ‘taxing criticism’.
The track record of hewing moderation to cater for authors has not borne much fruit so far: the mod tools were introduced in large part to entice Eliezer back. He’s not. I think I recall a lot of mod effort has been spent on mediating spats between high profile users/contributors, but I think the usual end result is these dramatis personae have faded away.
#
Regardless of all that, it’s your website, and I’m barely a stakeholder worth considering (my last substantial contribution was over a decade ago). I wouldn’t hold it against Pace or Habryka if from arguments we had on the EA forum^[4] they thought my judgements better to inverse, and my absence satisfying.

I expect I will continue participating very little in LW, although Said getting banned has little to do with it. Basically I don’t find enough yield of ‘good generalist (i.e. not principally focused on AI) content’ here anymore. I think Said incrementally helped by reducing the volume/prominence of not-so-good generalist content, so this seems a step in the wrong direction.^[5] Happy days, and more fool me, if the future proves me wrong.
1. ^
  Although I appreciate mitigating circumstances (and an isolated case), moderator behaviour on this post has been ‘similar-to-worse antics’ too. It seems bad form to (as it appears Habryka has done) strong downvote a large number (most?) comments by Said in the threads he is arguing with him in (can I do this if I get into a fight with someone with much lower vote power than me?). Ditto (as Pace did) use site-admin info to score points against a dissenting user he wanted to be snide to, especially when that user seems to be dissenting in the manner OP requested they do.
2. ^
  I’m not giving examples to avoid prompting a subthread wasteland on whatever I bring up. If widely disbelieved and crucial to the discussion, I am open to being cajoled into naming some names
3. ^
  Aside: it is perhaps unfortunate ‘tapping out’ is the lingo for dropping a discussion. In martial arts, (notwithstanding the gloss on the wiki that it can mean ‘one is tired, or at risk of injury, or has simply had one’s fill’) tapping is typically an admission of defeat.
  Regardless of the lingo, there is still the advantage of having the ‘last word’ (cf. OP). I could be odd, but I feel this gets outweighed by the much lower visibility of (e.g.) the 5th+ nested comment being seldom more than ‘you and your interlocutor’. In terms of ‘discussion as social fight’, whoever got ratioed in the first 1-2 back and forths on the thread is the loser, even if they make the last ‘rebuttal’.
4. ^
  FWIW I don’t have the impression that the EA forum is more ‘linkedin-y’ than LW nowadays. Besides roughly similar levels of spats/drama, many of my comments there are much meaner towards the OP than Said’s, and I haven’t had the moderators generally ‘on my case’ about them (e.g.).
5. ^
  But there are secular explanations which likely overdetermine, e.g.:
  Maybe we’ve run out of useful general things to say, so useful conversation inevitably gets more and more sub-specialised?
  Maybe the noughties internet just developed a lot of surplus for places like LW, but nowadays gifted writers want to cultivate their own substack or whatever.
  Maybe things have professionalized so the typical commenter who could share interesting takes on AI alignment (or whatever) as an amateur has been recruited to a think tank as a professional.

Thrasymachus 31 May 2024 8:57 UTC
16 points
13
in reply to: Jacob_Hilton’s comment on: Non-Disparagement Canaries for OpenAI
I see the concerns as these:
1. The four corners of the agreement seem to define ‘disparagement’ broadly, so one might reasonably fear (e.g.) “First author on an eval especially critical of OpenAI versus its competitors”, or “Policy document highly critical of OpenAI leadership decisions” might ‘count’.
2. Given Altman’s/OpenAI’s vindictiveness and duplicity, and the previous ‘safeguards’ (from their perspective) which give them all the cards in terms of folks being able to realise the value of their equity, “They will screw me out of a lot of money if I do something they really don’t like (regardless of whether it ‘counts’ per the non-disparagement agreement)” seems a credible fear.
  1. It appears Altman tried to get Toner kicked off the board for being critical of OpenAI in a policy piece, after all.
3. This is indeed moot for roles which require equity to be surrendered anyway. I’d guess most roles outside government (and maybe some within it) do not have such requirements. A conflict of interest roughly along the lines of the first two points makes impartial performance difficult, and credible impartial performance impossible (i.e. even if indeed Alice can truthfully swear “My being subject to such an agreement has never influenced my work in AI policy”, reasonable third parties would be unwise to believe her).
4. The ‘non-disclosure of non-disparagement’ makes this worse, as it interferes with this conflict of interest being fully disclosed. “Alice has a bunch of OpenAI equity” is one thing, “Alice has a bunch of OpenAI equity, and has agreed to be beholden to them in various ways to keep it” is another. We would want to know the latter to critically appraise Alice’s work whenever it is relevant to OpenAI’s interests (and I would guess a lot of policy/eval/reg/etc. would be sufficiently relevant that we’d like to contemplate whether Alice’s commitments colour her position). Yet Alice has also promised to keep these extra relevant details secret.

Thrasymachus 14 Apr 2022 9:41 UTC
0 points
0
in reply to: swarriner’s comment on: A Quick Guide to Confronting Doom
I can’t help with the object level determination, but I think you may be overrating both the balance and import of the second-order evidence.

As far as I can tell, Yudkowsky is a (?dramatically) pessimistic outlier among the class of “rationalist/rationalist-adjacent” SMEs in AI safety, and probably even more so relative to aggregate opinion without an LW-y filter applied (cf.). My impression of the epistemic track-record is Yudkowsky has a tendency of staking out positions (both within and without AI) with striking levels of confidence but not commensurately-striking levels of accuracy.

In essence, I doubt there’s much epistemic reason to defer to Yudkowsky more (or much more) than folks like Carl Shulman, or Paul Christiano, nor maybe much more than “a random AI alignment researcher” or “a superforecaster making a guess after watching a few Rob Miles videos” (although these have a few implied premises around difficulty curves/ subject matter expertise being relatively uncorrelated to judgemental accuracy).
I suggest ~all reasonable attempts at idealised aggregate wouldn’t take a hand-brake turn to extreme pessimism on finding Yudkowsky is. My impression is the plurality LW view has shifted more from “pretty worried” to “pessimistic” (e.g. p(screwed) > 0.4) rather than agreement with Yudkowsky, but in any case I’d attribute large shifts in this aggregate mostly to Yudkowsky’s cultural influence on the LW-community plus some degree of internet cabin fever (and selection) distorting collective judgement.

None of this is cause for complacency: even if p(screwed) isn’t ~1, > 0.1 (or 0.001) is ample cause for concern, and resolution on values between (say) [0.1 0.9] is informative for many things (like personal career choice). I’m not sure whether you get more yield for marginal effort on object or second-order uncertainty (e.g. my impression is the ‘LW cluster’ trends towards pessimism, so adjudicating whether this cluster should be over/under weighted could be more informative than trying to get up to speed on ELK). I would guess, though, that whatever distils out of LW discourse in 1-2 months will be much more useful than what you’d get right now.

Thrasymachus 20 Oct 2021 11:27 UTC
7 points
0
in reply to: Adam Scholl’s comment on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
Looking back, my sense remains that we basically succeeded—i.e., that we described the situation about as accurately and neutrally as we could have. If I’m wrong about this… well, all I can say is that it wasn’t for lack of trying.
I think CFAR ultimately succeeded in providing a candid and good faith account of what went wrong, but the time it took to get there (i.e. 6 months between this and the initial update/apology) invites adverse inferences like those in the grandparent.
A lot of the information ultimately disclosed in March would definitely have been known to CFAR in September, such as Brent’s prior involvement as a volunteer/contractor for CFAR, his relationships/friendships with current staff, and the events as ESPR. The initial responses remained coy on these points, and seemed apt to give the misleading impression CFAR’s mistakes were (relatively) much milder than they in fact were. I (among many) contacted CFAR leadership to urge them to provide more candid and complete account when I discovered some of this further information independently.

I also think, similar to how it would be reasonable to doubt ‘utmost corporate candour’ back then given initial partial disclosure, it’s reasonable to doubt CFAR has addressed the shortcomings revealed given the lack of concrete follow-up. I also approached CFAR leadership when CFAR’s 2019 Progress Report and Future Plans initially made no mention of what happened with Brent, nor what CFAR intended to improve in response to it. What was added in is not greatly reassuring:
And after spending significant time investigating our mistakes with regard to Brent, we reformed our hiring, admissions and conduct policies, to reduce the likelihood such mistakes reoccur.
A cynic would note this is ‘marking your own homework’, but cynicism is unnecessary to recommend more self-scepticism. I don’t doubt the Brent situation indeed inspired a lot of soul searching and substantial, sincere efforts to improve. What is more doubtful (especially given the rest of the morass of comments) is whether these efforts actually worked. Although there is little prospect of satisfying me, more transparency over what exactly has changed—and perhaps third party oversight and review—may better reassure others.

Thrasymachus 1 Feb 2020 8:32 UTC
9 points
0
in reply to: Ben Pace’s comment on: REVISED: A drowning child is hard to find
The malaria story has fair face validity if one observes the wider time series (e.g.). Further, the typical EA ‘picks’ for net distribution are generally seen as filling around the edges of the mega-distributors.
FWIW: I think this discussion would be clearer if framed in last-dollar terms.
If Gates et al. are doing something like last dollar optimisation, trying to save as many lives as they can allocating across opportunities both now and in the future, leaving the right now best marginal interventions on the table would imply they expect to exhaust their last dollar on more cost-effective interventions in the future.
This implies the right now marginal price should be higher than the (expected) last dollar cost effectiveness (if not, it should be reallocating some of the ‘last dollars’ to interventions right now). Yet this in turn does not imply we should see 50Bn of marginal price lifesaving lying around right now. So it seems we can explain Gates et al. not availing themselves of the (non-existent) opportunity to (say) halve communicable diseases for 2Bn a year worldwide (extrapolating from the right now marginal prices) without the right now marginal price being lied about or manipulated. (Obviously, even if we forecast the Gates et al. last dollar EV to be higher than the current marginal price, we might venture alternative explanations of this discrepancy besides them screwing us.)

Thrasymachus 12 Jan 2020 18:45 UTC
8 points
0
in reply to: Bendini’s comment on: Please Critique Things for the Review!
I also buy the econ story here (and, per Ruby, I’m somewhat pleasantly surprised by the amount of reviewing activity given this).
General observation suggests that people won’t find writing reviews that intrinsically motivating (compare to just writing posts, which all the authors are doing ‘for free’ with scant chance of reward, also compare to academia—I don’t think many academics find peer review/refereeing one of the highlights of their job). With apologies for the classic classical econ joke, if reviewing was so valuable, how come people weren’t doing it already? [It also looks like ~25%? of reviews, especially the most extensive, are done by the author on their own work].
If we assume there’s little intrinsic motivation (I’m comfortably in the ‘you’d have to pay me’ camp), the money doesn’t offer that much incentive. Given Rudy’s numbers suppose each of the 82 reviews takes an average of 45 minutes or so (factoring in (re)reading time and similar). If the nomination money is ~roughly allocated by person time spent, the marginal expected return of me taking an hour to review is something like $40. Facially, this isn’t too bad an hourly rate, but the real value is significantly lower:
- The ‘person-time lottery’ model should not be denominated by observed person-time so far, but one’s expectation how much will be spent in total once reviewing finishes, which will be higher (especially conditioned on posts like this).
- It’s very unlikely the reward is going to allocated proportionately to time spent (/some crude proxy thereof like word count). Thus the EV would be discounted by whatever degree of risk aversion one has (I expect the modal ‘payout’ for a review to be $0).
- Opaque allocation also incurs further EV-reducing uncertainty, but best guesses suggest there will be Pareto-principle/tournament dynamic game dynamics, so those with (e.g.) reasons to believe they’re less likely to impress the mod team’s evaluation of their ‘pruning’ have strong reasons to select themselves out.
What links here?
- Reviewing the Review by Raemon (26 Feb 2020 2:51 UTC; 45 points)

Thrasymachus 21 Dec 2019 19:56 UTC
4 points
0
in reply to: Pattern’s comment on: Polio and the controversy over randomized clinical trials
Sure—there’s a fair bit of literature on ‘optimal stopping’ rules for interim results in clinical trials to try and strike the right balance.
It probably wouldn’t have helped much for Salk’s dilemma: Polio is seasonal and the outcome of interest is substantially lagged from the intervention—which has to precede the exposure, and so the ‘window of opportunity’ is quickly lost; I doubt the statistical methods for conducting this were well-developed in the 50s; and the polio studies were already some of the largest trials ever conducted, so even if available these methods may have imposed even more formidable logistical challenges. So there probably wasn’t a neat pareto-improvement of “Let’s run an RCT with optimal statistical control governing whether we switch to universal administration” Salk and his interlocutors could have agreed to pursue.

Thrasymachus 20 Dec 2019 22:23 UTC
11 points
0
on: Polio and the controversy over randomized clinical trials
Mostly I just find it fascinating that as late as the 1950s, the need for proper randomized blind placebo controls in clinical trials was not universally accepted, even among scientific researchers. Cultural norms matter, especially epistemic norms.
This seems to misunderstand the dispute. Salk may have had an overly optimistic view of the efficacy of his vaccine (among other foibles your source demonstrates), but I don’t recall him being a general disbeliever in the value of RCTs.
Rather, his objection is consonant with consensus guidelines for medical research, e.g. the declaration of Helsinki (article 8): [See also the Nuremberg code (art 10), relevant bits of the Hippocratic Oath, etc.]
While the primary purpose of medical research is to generate new knowledge, this goal can never take precedence over the rights and interests of individual research subjects.
This cashes out in a variety of ways. The main one is a principle of clinical equipoise—one should only conduct a trial if there is genuine uncertainty about which option is clinically superior. A consequence of this is that clinical trials conducted are often stopped early if a panel supervising the trial finds clear evidence of (e.g.) the treatment outperforming the control (or vice versa) as continuing the trial continues to place those in the ‘wrong’ arm in harm’s way—even though this comes at an epistemic cost as the resulting data is poorer than that which could have been gathered if the trial continued to completion.
I imagine the typical reader of this page is going to tend unsympathetic to the virtue ethicsy/deontic motivations here, but there is also a straightforward utilitarian trade-off: better information may benefit future patients, at the cost of harming (in expectation) those enrolled in the trial. Although RCTs are the ideal, one can make progress with less (although I agree it is even more treacherous), and the question of the right threshold for these is fraught. (There also also natural ‘slippery slope’ style worries about taking a robust ‘longtermist’ position in holding the value of the evidence for all future patients is worth much more than the welfare of the much smaller number of individuals enrolled in a given trial—the genesis of the Nuremberg Code need not be elaborated upon.)
A lot of this ethical infrastructure post-dates Salk, but this suggests his concerns were forward-looking rather than retrograde (even if he was overconfident in the empirical premise that ‘the vaccine works’ which drove these commitments). I couldn’t in good conscience support a placebo-controlled trial for a treatment I knew worked for a paralytic disease either. Similarly, it seems very murky to me what the right call was given knowledge-at-the-time—but if Bell and Francis were right, it likely owed more to them having a more reasonable (if ultimately mistaken) scepticism of the vaccine efficacy than Salk, rather him just ‘not getting it’ about why RCTs are valuable.

Thrasymachus 30 Nov 2019 18:41 UTC
7 points
0
on: Neural Annealing: Toward a Neural Theory of Everything (crosspost)
I’m afraid I couldn’t follow most of this, but do you actually mean ‘high energy’ brain states in terms of aggregate neural activity (i.e. the parentheticals which equate energy to ‘firing rates’ or ‘neural activity’)? If so, this seems relatively easy to assess for proposed ‘annealing prompts’ - whether psychedelics/meditation/music/etc. tend to provoke greater aggregate activity than not seems open to direct calorimetry, leave alone proxy indicators.
Yet the steers on this tend very equivocal (e.g. the evidence on psychedelics looks facially ‘right’, things look a lot more uncertain for meditation and music, and identifying sleep as a possible ‘natural annealing process’ looks discordant with a ‘high energy state’ account, as brains seem to consume less energy when asleep than awake). Moreover, natural ‘positive controls’ don’t seem supportive: cognitively demanding tasks (e.g. learning an instrument, playing chess) seem to increase brain energy consumption, yet presumably aren’t promising candidates for this hypothesised neural annealing.
My guess from the rest of the document is the proviso about semantically-neutral energy would rule out a lot of these supposed positive controls: the elevation needs to be general rather than well-localized. Yet this is a lot harder to use as an instrument with predictive power: meditation/music/etc. have foci too in the neural activity it provokes.

Thrasymachus 19 Sep 2019 4:51 UTC
7 points
0
on: The unexpected difficulty of comparing AlphaStar to humans
Thanks for this excellent write-up!
I’m don’t have relevant expertise in either AI or SC2, but I was wondering whether precision might still be a bigger mechanical advantage than the write-up notes. Even if humans can (say) max out at 150 ‘combat’ actions per minute, they might misclick, not be able to pick out the right unit in a busy and fast battle to focus fire/trigger abilities/etc, and so on. The AI presumably won’t have this problem. So even with similar EAPM (and subdividing out ‘non-combat’ EAPM which need not be so accurate), Alphastar may still have a considerable mechanical advantage.
I’d also be interested in how important, beyond some (high) baseline, ‘decision making’ is at the highest levels of SC2 play. One worry I have is although decision-making is important (build orders, scouting, etc. etc.) what decides many (?most) pro games is who can more effectively micro in the key battles, or who can best juggle all the macro/econ tasks (I’d guess some considerations in favour would be that APM is very important, and that a lot of the units in SC2 are implicitly balanced by ‘human’ unit control limitations). If so, unlike Chess and Go, there may not be some deep strategic insights Alphastar can uncover to give it the edge, and ‘beating humans fairly’ is essentially an exercise in getting the AI to fall within the band of ‘reasonably human’, but can still subtly exploit enough of the ‘microable’ advantages to prevail.

Thrasymachus 30 Aug 2019 5:53 UTC
31 points
0
on: Is there a type of utilitarianism that combines the sum and median utility?
Combining the two doesn’t solve the ‘biggest problems of utilitarianism’:
1) We know from Arrhenius’s impossibility theorems you cannot get an axiology which can avoid the repugnant conclusion without incurring other large costs (e.g. violations of transitivity, dependence of irrelevant alternatives). Although you don’t spell out ‘balance utilitarianism’ enough to tell what it violates, we know it—like any other population axiology—will have very large drawbacks.
2) ‘Balance utilitarianism’ seems a long way from the frontier of ethical theories in terms of its persuasiveness as a population ethic.
a) The write-up claims that actions that only actions that increase sum and median wellbeing are good, those that increase one or the other are sub-optimal, and those that decrease both are bad. Yet what if we face choices where we don’t have an option that increases both sum and median welfare (such as Parfit’s ‘mere addition’), and we have to choose between them? How do we balance one against the other? The devil is in these details, and a theory being silent on these cases shouldn’t be counted in its favour.
b) Yet even as it stands we can construct nasty counter-examples to the rule, based on very benign versions of mere addition. Suppose Alice is in her own universe at 10 welfare (benchmark this as a very happy life). She can press button A or button B. Button A boosts her up to 11 welfare. Button B boosts her to 10^100 welfare, and brings into existence 10^100 people at (10-10^-100) welfare (say a life as happy as Alice but with a pinprick). Balance utilitarianism recommends button A (as it increases total and median) as good, but pressing button B as suboptimal. Yet pressing button B is much better for Alice, and also instantiates vast numbers of happy people.
c) The ‘median criterion’ is going to be generally costly, as it is insensitive to changing cardinal levels outside the median person/pair so long as ordering is unchanged (and vice-versa).
d) Median views (like average ones) also incur costs due to their violation of separability. It seems intuitive that the choiceworthiness of our actions shouldn’t depend on whether there is an alien population on Alpha Centauri who are happier/sadder than we are (e.g. if there’s lots of them and they’re happier, any act that brings more humans into existence is ‘suboptimal’ by the lights of balance util).

Thrasymachus 9 Jun 2019 20:49 UTC
9 points
0
in reply to: Davis_Kingsley’s comment on: Asymmetric Weapons Aren’t Always on Your Side
(Very minor inexpert points on military history, I agree with the overall point there can be various asymmetries, not all of which are good—although, in fairness, I don’t think Scott had intended to make this generalisation.)
1) I think you’re right the German army was considered one of the most effective fighting forces on a ‘man for man’ basis (I recall pretty contemporaneous criticism from allied commanders on facing them in combat, and I think the consensus of military historians is they tended to outfight American, British, and Russian forces until the latest stages of WW2).
2) But it’s not clear how much the Germany owed this performance to fascism:
- Other fascist states (i.e. Italy) had much less effective fighting forces.
- I understand a lot of the accounts to explain how German army performed so well sound very unstereotypically facist—delegating initiative to junior officers/NCOs rather than unquestioning obedience to authority (IIRC some historical comment was the American army was more stiflingly authoritarian than the German one for most of the war), better ‘human resource’ management of soldiers, combined arms, etc. etc. This might be owed more to Prussian heritage than Hitler’s rise to power.
3) Per others, it is unclear ‘punching above one’s weight’ for saying something is ‘better at violence’. Even if the US had worse infantry, they leveraged their industrial base to give their forces massive material advantages. If the metric for being better at violence is winning in violent contests, the fact the German’s were better at one aspect of this seems to matter little if they lost overall.

Thrasymachus 8 Jun 2019 16:12 UTC
12 points
0
on: The Schelling Choice is “Rabbit”, not “Stag”
It’s perhaps worth noting that if you add in some chance of failure (e.g. even if everyone goes stag, there’s a 5% chance of ending up −5, so Elliott might be risk-averse enough to decline even if they knew everyone else was going for sure), or some unevenness in allocation (e.g. maybe you can keep rabbits to yourself, or the stag-hunt-proposer gets more of the spoils), this further strengthens the suggested takeaways. People often aren’t defecting/being insufficiently public spirited/heroic/cooperative if they aren’t ‘going to hunt stags with you’, but are sceptical of the upside and/or more sensitive to the downsides.
One option (as you say) is to try and persuade them the value prop is better than they think. Another worth highlighting is whether there are mutually beneficial deals one can offer them to join in. If we adapt Duncan’s stag hunt to have a 5% chance of failure even if everyone goes, there’s some efficient risk-balancing option A-E can take (e.g. A-C pool together to offer some insurance to D-E if they go on a failed hunt with them).
[Minor: one of the downsides of ‘choosing rabbit/stag’ talk is it implies the people not ‘joining in’ agree with the proposer that they are turning down a (better-EV) ‘stag’ option.]

Thrasymachus 4 Jun 2019 17:38 UTC
8 points
0
in reply to: Benquo’s comment on: Drowning children are rare
A marginalist analysis that assumes that the person making the decision doesn’t know their own intentions & is just another random draw of a ball from an urn totally misses this factor.
Happily, this factor has not been missed by either my profile or 80k’s work here more generally. Among other things, we looked at:
- Variance in impact between specialties and (intranational) location (1) (as well as variance in earnings for E2G reasons) (2, also, cf.)
- Areas within medicine which look particularly promising (3)
- Why ‘direct’ clinical impact (either between or within clinical specialties) probably has limited variance versus (e.g.) research (4), also
I also cover this in talks I have given on medical careers, as well as when offering advice to people contemplating a medical career or how to have a greater impact staying within medicine.
I still think trying to get a handle on the average case is a useful benchmark.

Thrasymachus 4 Jun 2019 14:58 UTC
21 points
0
in reply to: Benquo’s comment on: Drowning children are rare
[I wrote the 80k medical careers page]
I don’t see there as being a ‘fundamental confusion’ here, and not even that much of a fundamental disagreement.
When I crunched the numbers on ‘how much good do doctors do’ it was meant to provide a rough handle on a plausible upper bound: even if we beg the question against critics of medicine (of which there are many), and even if we presume any observational marginal response is purely causal (and purely mediated by doctors), the numbers aren’t (in EA terms) that exciting in terms of direct impact.
In talks, I generally use the upper 95% confidence bound or central estimate of the doctor coefficient as a rough steer (it isn’t a significant predictor, and there’s reasonable probability mass on the impact being negative): although I suspect there will be generally unaccounted confounders attenuating ‘true’ effect rather than colliders masking it, these sort of ecological studies are sufficiently insensitive to either to be no more than indications—alongside the qualitative factors—that the ‘best (naive) case’ for direct impact as a doctor isn’t promising.
There’s little that turns on which side of zero our best guess falls, so long as we be confident it is a long way down from the best candidates: on the scale of intervention effectiveness, there’s not that much absolute distance between estimates (I suspect) Hanson or I would offer. There might not be much disagreement even in coarse qualitative terms: Hanson’s work here—I think—focuses on the US, and US health outcomes are a sufficiently pathological outlier in the world I’m also unsure whether marginal US medical effort is beneficial; I’m not sure Hanson has staked out a view on whether he’s similarly uncertain about positive marginal impact in non-US countries, so he might agree with my view it is (modestly) net-positive, despite its dysfunction (neither I nor what I wrote assumes the system ‘basically knows what it’s doing’ in the common-sense meaning).
If Hanson has staked out this broader view, then I do disagree with it, but I don’t think this disagreement would indicate at least one of us has to be ‘deeply confused’ (this looks like a pretty crisp disagreement to me) nor ‘badly misinformed’ (I don’t think there are key considerations one-or-other of us is ignorant of which explains why one of us errs to sceptical or cautiously optimistic). My impressions are also less sympathetic to ‘signalling accounts’ of healthcare than his (cf.) - but again, my view isn’t ‘This is total garbage’, and I doubt he’s monomaniacally hedgehog-y about the signalling account. (Both of us have also argued for attenuating our individual impressions in deference to a wider consensus/outside view for all things considered judgements).
Although I think the balance of expertise leans against archly sceptical takes on medicine, I don’t foresee convincing adjudication on this point coming any time soon, nor that EA can reasonably expect to be the ones to provide this breakthrough—still less for all the potential sign-inverting crucial considerations out there. Stumbling on as best we can with our best guess seems a better approach than being paralyzed until we’re sure we’ve figured it all out.

Thrasymachus 7 Apr 2019 15:19 UTC
11 points
0
on: What are the advantages and disadvantages of knowing your own IQ?
It looks generally redundant in most cases to me: Given how pervasive IQ-correlations are, I think most people can get a reasonable estimate of their IQ by observing their life history so far. E.g.
- Educational achievement
- Performance on other standardised tests
- Job type and professional success
- Peer esteem/reputation
Obviously, none of these are perfect signals, but I think taking them together usually gives a reasonable steer to a credible range not dramatically larger than test-restest correlations of an IQ test. An IQ test would still provide additional information, but I’m not sure there are many instances where (say) knowing the answer in a 5 point band versus a 10 point band is that important.
The case where I think it could be worthwhile is for those whose life history hasn’t generated the usual signals to review: maybe one was initially homeschooled and became seriously ill before starting employment/university, etc.

Thrasymachus 26 Feb 2019 1:55 UTC
36 points
0
on: How good is a human’s gut judgement at guessing someone’s IQ?
Googling around phrases like ‘perception of intelligence’ seems to be a keyword for a relevant literature. On a very cursory skim (i.e. no more than what you see here) it seems to suggest “people can estimate intelligence of strangers better than chance (but with plenty of room for error and bias), even with limited exposure”. E.g.:
Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women (Note in this study the assessment was done purely on looking at a photograph of someone’s face)
Accurate Intelligence Assessments in Social Interactions: Mediators and Gender Effects (Abstract starts with: “Research indicates that people can assess a stranger’s measured intelligence more accurately than expected by chance, based on minimal information involving appearance and behavior.”)
Thin Slices of Behavior as Cues of Personality and Intelligence. (Short 1-2min slices of behaviour in a variety of contexts leads to assessments by strangers that positively correlate with administered test scores for IQ and big 5)

Thrasymachus 19 Feb 2019 10:39 UTC
6 points
0
on: Epistemic Tenure
As you say, Bob’s good epistemic reputation should count when he says something that appears wild, especially if he has a track record that endorses him in these cases (“We’ve thought he was crazy before, but he proved us wrong”). Maybe one should think of Bob as an epistemic ‘venture capitalist’, making (seemingly) wild epistemic bets which are right more often than chance (and often illuminating even if wrong), even if they aren’t right more often than not, and this might be enough to warrant further attention (“well, he’s probably wrong about this, but maybe he’s onto something”).
I’m not sure your suggestion pushes in the right direction in the case where—pricing all of that in—we still think Bob’s belief is unreasonable and he is unreasonable for holding it. The right responses in this case by my lights are two-fold.
First, you should dismiss (rather than engage with) Bob’s wild belief—as (ex hypothesi) all things considered it should be dismissed.
Second, it should (usually) count against Bob’s overall epistemic reputation. After all, whatever it was that meant despite Bob’s merits you think he’s saying something stupid is likely an indicator of epistemic vice.
This doesn’t mean it should be a global black mark to taking Bob seriously ever again. Even the best can err badly, so one should weigh up the whole record. Furthermore, epistemic virtue has a few dimensions, and Bob’s weaknesses in something need not mean his strengths in others be sufficient for attention esteem going forward: An archetype I have in mind with ‘epistemic venture capitalist’ is someone clever, creative, yet cocky and epistemically immodest—has lots of novel ideas, some true, more interesting, but many ‘duds’ arising from not doing their homework, being hedgehogs with their preferred ‘big idea’, etc.
I accept, notwithstanding those caveats, this still disincentivizes epistemic venture capitalists like Bob to some degree. Although I only have anecdata, this leans in favour of some sort of trade-off: brilliant thinkers often appear poorly calibrated and indulge in all sorts of foolish beliefs; interviews with superforecasters (e.g.) tend to emphasise things like “don’t trust your intuition, be very self sceptical, canvass lots of views, do lots of careful research on a topic before staking out a view”. Yet good epistemic progress relies on both—and if they lie on a convex frontier, one wants to have a division of labour.
Although the right balance to strike re. second order norms depends on tricky questions on which sort of work is currently under-supplied, which has higher value on the margin, and the current norms of communal practice (all of which may differ by community), my hunch is ‘epistemic tenure’ (going beyond what I sketch above) tends disadvantageous.
One is noting the are plausible costs in both directions. ‘Tenure’-esque practice could spur on crack pots, have too lax a filter for noise-esque ideas, discourage broadly praiseworthy epistemic norms (cf. virtue of scholarship), and maybe not give Bob-like figures enough guidance so they range too far and unproductively (e.g. I recall one Nobel Laureate mentioning the idea of, “Once you win your Nobel Prize, you should go and try and figure out the hard problem of consciousness”—which seems a terrible idea).
The other is even if there is a trade-off, one still wants to reach the one’s frontier on ‘calibration/accuracy/whatever’. Scott Sumner seems to be able to combine researching on the inside view alongside judging on the outside view (see). This seems better for Sumner, and the wider intellectual community, than Sumner* who could not do the latter.

Thrasymachus 14 Jan 2019 17:53 UTC
15 points
0
in reply to: ChristianKl’s comment on: What are the open problems in Human Rationality?
FWIW: I’m not sure I’ve spent >100 hours on a ‘serious study of rationality’. Although I have been around a while, I am at best sporadically active. If I understand the karma mechanics, the great majority of my ~1400 karma comes from a single highly upvoted top level post I wrote a few years ago. I have pretty sceptical reflexes re. rationality, the rationality community, etc., and this is reflected in that (I think) the modal post/comment I make is critical.
On the topic ‘under the hood’ here:
I sympathise with the desire to ask conditional questions which don’t inevitably widen into broader foundational issues. “Is moral nihilism true?” doesn’t seem the right sort of ‘open question’ for “What are the open questions in Utilitarianism?”. It seems better for these topics to be segregated, no matter the plausibility or not for the foundational ‘presumption’ (“Is homeopathy/climate change even real?” also seems inapposite for ‘open questions in homeopathy/anthropogenic climate change’). (cf. ‘This isn’t a 101-space’).
That being said, I think superforecasting/GJP and RQ/CART etc. are at least highly relevant to the ‘Project’ (even if this seems to be taken very broadly to normative issues in general—if Wei_Dai’s list of topics are considered elements of the wider Project, then I definitely have spent more than 100 hours in the area). For a question cluster around “How can one best make decisions on unknown domains with scant data”, the superforecasting literature seems some of the lowest hanging fruit to pluck.
Yet community competence in these areas has apparently declined. If you google ‘lesswrong GJP’ (or similar terms) you find posts on them but these posts are many years old. There has been interesting work done in the interim: here’s something on the whether the skills generalise, and something else of a training technique that not only demonstrably improves forecasting performance, but also has a handy mnemonic one could ‘try at home’. (The same applies to RQ: Sotala wrote a cool sequence on Stanovich’s ‘What intelligence tests miss’, but this is 9 years old. Stanovich has written three books since expressly on rationality, none of which have been discussed here as best as I can tell.)
I don’t understand, if there are multiple people who have spent >100 hours on the Project (broadly construed), why I don’t see there being a ‘lessons from the superforecasting literature’ write-up here (I am slowly working on one myself).
Maybe I just missed the memo and many people have kept abreast of this work (ditto other ‘relevant-looking work in academia’), and it is essentially tacit knowledge for people working on the Project, but they are focusing their efforts to develop other areas. If so, a shame this is not being put into common knowledge, and I remain mystified as to why the apparent neglect of these topics versus others: it is a lot easier to be sceptical of ‘is there anything there?’ for (say) circling, introspection/meditation/enlightenment, Kegan levels, or Focusing than for the GJP, and doubt in the foundation should substantially discount the value of further elaborations on a potentially unedifying edifice.
[Minor] I think the first para is meant to be block-quoted?