Ap­proval and score vot­ing are both 100% clone im­mune and 100% cen­ter-squeeze im­mune. Score vot­ing is liter­ally as good as you can get (at least in VSE terms!) if there’s no strate­gic vot­ing, and de­volves into ap­proval vot­ing un­der strate­gic vot­ing. So, if there is some kind of trade-off be­tween cen­ter squeeze and clone prob­lems, it must be in the ter­ri­tory of meth­ods that are bet­ter than ap­proval vot­ing (even given strate­gic vot­ers).

I don’t un­der­stand the con­clu­sion here. Score and ap­proval don’t ex­hibit the trade-off. And some other meth­ods do. But what do you mean about it speci­fi­cally be­ing in the ter­ri­tory of meth­ods that are bet­ter than ap­proval that the trade-off ex­ists?

Stress tests

Many sys­tems get “spot-checked” by ar­tifi­cially forc­ing them into a rare but im­por­tant-to-cor­rectly-han­dle stressed state un­der con­trol­led con­di­tions where more mon­i­tor­ing and re­cov­ery re­sources are available (or where the stakes are lower) than would be the case dur­ing a real in­stance of the stressed state.

Th­ese serve to prac­tice pro­ce­dures, yes, but they also serve to eval­u­ate whether the pro­ce­dures would be fol­lowed cor­rectly in a crisis, and whether the pro­ce­dures even work.

• Drills

  • Fire/​tor­nado/​earth­quake/​nu­clear-at­tack drills

  • Mili­tary drills (the kind where you tell ev­ery­one to get to bat­tle sta­tions, not the use­less march­ing around in for­ma­tion kind)

• Large cloud com­put­ing com­pa­nies I’ve worked at need to stay on­line in the face of loss of a sin­gle com­puter, or a sin­gle dat­a­cen­ter. They pe­ri­od­i­cally check to see that these failures are sur­viv­able by di­rectly pow­er­ing off com­put­ers, dis­con­nect­ing en­tire dat­a­cen­ters from the net­work, or sim­ply run­ning through a dat­a­cen­ter failover pro­ce­dure be­gin­ning to end to check that it works.

• https://​​en.wikipe­dia.org/​​wiki/​​Stress_test_(fi­nan­cial)

Thank you for ex­tend­ing more char­ity than you did pre­vi­ously. It was hard for me to re­spond fairly when you made ar­gu­ments like “Well ac­tu­ally, you shouldn’t use the word ‘never’ be­cause no prob­a­bil­ity is liter­ally zero.”

I hon­estly don’t think point [2. You’re not try­ing your best if you’re not happy.] is speci­fied well enough to be con­fused with an in­ter­est­ing truth claim, as op­posed to a helpful heuris­tic. For ex­am­ple, one can eas­ily make the case that that no hu­man is “try­ing their best,” and there­fore the state­ment is vac­u­ously true be­cause a true state­ment is im­plied by any­thing. I think the most rea­son­able way to in­ter­pret the sen­tence is “no­body is try­ing their best, and hap­piness is a par­tic­u­larly high ROI di­men­sion along which to no­tice this.”

I ten­ta­tively ac­cept the ap­pli­ca­bil­ity of this claim to the re­stricted do­main of peo­ple who are phys­i­cally/​men­tally healthy, eco­nom­i­cally/​so­cially priv­ileged and fo­cused on their long-term im­pact. Since I’m in that cat­e­gory, it may well be use­ful for me ac­tu­ally, so I’ll try think about it more; thanks for rais­ing the ar­gu­ment.

Given that you be­lieve this part, I sus­pect that we pretty much agree on ev­ery claim about the ter­ri­tory. If you think so too, then I think there’s not much left to dis­pute other than word choice.

For “what task should I do next,” it’s O(n), be­cause you just go down the list once do­ing a pair­wise com­par­i­son. “What task should I do next” seems more im­por­tant for ac­tu­ally do­ing things than sort­ing the en­tire list at first to avoid de­liber­a­tion time /​ in­de­ci­sion /​ harder 3-way value com­par­i­sons.

I have a friend who has been tak­ing HCQ for a chronic ill­ness since long be­fore the pan­demic, as it’s been his­tor­i­cally used for. They have to go in for reg­u­lar reti­nal checks to en­sure that they aren’t go­ing blind, and they are other sig­nifi­cant con­cerns. They’re on it be­cause those risks aren’t as bad as the effects of their chronic ill­ness that HCQ re­lieves. But those risks are still bad. And if HCQ doesn’t help, then they definitely don’t want to give it out to ev­ery­one.

Cur­rent open mar­ket price of an asset

Public, highly liquid mar­kets for as­sets cre­ate lots of in­for­ma­tion about the value of those as­sets, which is ex­tremely use­ful for both in­di­vi­d­u­als and firms that are try­ing to un­der­stand:

• the state of their finances

• how suc­cess­ful a ven­ture deal­ing in those as­sets has been

• whether to ac­cept a deal (a fi­nan­cial trans­ac­tion, or some co­op­er­a­tive ven­ture) in­volv­ing those assets

• (if the as­sets are stock in some com­pany) how suc­cess­ful the com­pany has been so far

• Ad­vanced Place­ment tests

• SAT sub­ject tests

• Pr­ereg­is­tra­tion of studies

When I al­low my­self to have in­con­sis­ten­cies in my be­liefs and at­ti­tudes, I’m just us­ing my brain the way it evolved to work. Ac­cept­ing that I can’t un­tan­gle ev­ery­thing is nec­es­sary to make any progress.

When other peo­ple let them­selves have in­con­sis­ten­cies, it is out of self-serv­ing bias; it is anti-so­cial and they ought to do bet­ter. Their per­sis­tence in try­ing to have it both ways causes ex­cess harm.

The Hitch­hiker’s Guide to the Galaxy has “DON’T PANIC” printed in “large, friendly let­ters.” The de­scrip­tion of the PANIC but­ton here is similar. Might not be re­lated.

Share a triv­ial in­con­ve­nience that might have dra­matic consequences

• I didn’t have my band’s email logged in on my phone so I never checked it. I missed a few im­por­tant emails/​op­por­tu­ni­ties.

I have an im­por­tant “ugh” task I have to do by early next week, so I picked that as one of my things to try this on. I felt like I needed more time, so I changed my rules slightly: I listed aver­sions for 5 min­utes each, then spent 10 min­utes work­ing on the sub-aver­sions for each. I haven’t finished yet.

This sets off a small alarm bell in my head—I might be spend­ing more time on the ex­er­cise as a way to avoid ac­tu­ally do­ing the things. Re­gard­less, I’m go­ing to try to finish this ex­er­cise to­day even if it takes a lit­tle longer than ex­pected.

I feel like this is a pow­er­ful ex­er­cise. I need to prac­tice it a few more times to find ways to tai­lor it more to me.

Also, I’m a bit fuzzy on when to say I don’t en­dorse some­thing. Is the heuris­tic some­thing like “when the best way to solve the sub-aver­sion is to just find a way to (emo­tion­ally) get over it?”

“Post­mortem cul­ture” from the Google SRE book: https://​​sre.google/​​sre-book/​​post­mortem-cul­ture/​​

This book has some other sec­tions that are also about eval­u­a­tion, but this chap­ter is pos­si­bly my fa­vorite chap­ter from any cor­po­rate hand­book.

Seems like it’s a lazy sort to me (with ob­vi­ous wrin­kles from the fact that the list can grow). It also seems to be a var­i­ant of drop sort (which is $O\left(n\right)$ via cheat­ing) de­signed for re­peated passes on the re­main­ing list

I have a go-to eval­u­a­tion sys­tem for best ROI items in a brain­stormed list amongst team mem­bers. First we gen­er­ate the list, which ends up with, say, three dozen items from 6 of us. Then name a rea­son­ably small but large-enough num­ber like 10. Every­one may put 10 stars by items, max 2 per item, for any rea­son they like, in­clud­ing “this would be best for my morale”. Sort, pick the top three to use. Any sur­prises? Dis­cuss them. (Mod­ify num­bers like 8, 2, and 3 as ap­pro­pri­ate.)

This eval­u­a­tion sys­tem is sim­ple to im­ple­ment in many con­texts, eas­ily un­der­stood with­out much ex­pla­na­tion at all, fast, and pro­duces perfectly ac­cept­able if not nec­es­sar­ily op­ti­mal re­sults. It is pretty de­cent at grab­bing info from folks in­tu­itions with­out re­quiring them to in­tro­spect enough to make those in­tu­itions ex­plicit.

Two that are fo­cused on cri­tique rather than eval­u­a­tion per se:

• “the crit­i­cal re­sponse pro­cess” is about use­ful cri­tique in the arts: https://​​li­zler­man.com/​​crit­i­cal-re­sponse-pro­cess/​​

• “best prac­tices for code re­view”: https://​​smart­bear.com/​​learn/​​code-re­view/​​best-prac­tices-for-peer-code-re­view/​​

Thanks for the good work!

I es­pe­cially like the choice set pa­per.

I am of the same sen­ti­ment, and I’ve been afraid to post a less-than-perfect ar­gu­men­ta­tion. So thanks for be­ing the nail that stands out. I’ll add these reser­va­tions:

• HCQ seems dis­qual­ified on LW, but not for me. The fact that a fraud­u­lent study was posted in The Lancet is weak ev­i­dence that it works. Same for the fact that French gov­ern­ment un­duly for­bid its sel­l­ing right af­ter the pub­li­ca­tion of that ar­ti­cle but didn’t can­cel that in­ter­dic­tion af­ter the ar­ti­cle was pul­led. To this day, French gov­ern­ment and mass me­dia keep pre­tend­ing that a drug that was freely sold for decades sud­denly be­came dan­ger­ous. From what I’ve ab­sorbed pas­sively, Afri­can coun­tries do a lot bet­ter than us, and they also use HCQ.

• On av­er­age so far, my gov­ern­ment has taken mea­sures that sig­nal care but deeply harm us cit­i­zens. The last thing they’re try­ing is to crim­i­nal­ize filming po­lice in a way that al­lows iden­ti­fi­ca­tion. There­fore, if they sup­port a vac­cine, this is medium ev­i­dence that a vac­cine is against my best in­ter­ests.

I have a ques­tion: viruses mu­tate all the time. There are already mul­ti­ple strands around the world. Will the same vac­cine work on all of them? How long is it guaran­teed to work?

The short ver­sion of my cur­rent stance on Vas­sar is that:

(1) I would not trust him to con­form to lo­cal rules or norms. He also still seems to me to pre­cip­i­tate psy­chotic epi­sodes in his in­ter­locu­tors sur­pris­ingly of­ten, to come closer to ad­vo­cat­ing phys­i­cal vi­o­lence than I would like (e.g. this tweet), and to have con­ver­sa­tional pat­terns that of­ten di­s­ori­ent his in­ter­locu­tors and leave them be­liev­ing differ­ent things while talk­ing to Michael than they do a bit later.

(2) I don’t have over­all ad­vice that peo­ple ought to avoid Vas­sar, in spite of (1), be­cause it now seems to me that he is try­ing to help him­self and oth­ers to­ward truth, and I think we’re bot­tle­necked on that enough that I could eas­ily imag­ine (2) over­shad­ow­ing (1) for in­di­vi­d­u­als who are in a ro­bust place (e.g., who don’t feel like they are trapped or “have to” talk to a per­son or do a thing) and who are choos­ing who they want to talk to. (There were parts of Michael’s con­ver­sa­tional pat­terns that I was in­ter­pret­ing as less truth-con­ducive a cou­ple years ago than I am now. I now think that this was partly be­cause I was over­an­chored on the (then-re­cent) ex­am­ple of Brent, as well as be­cause I didn’t un­der­stand part of how he was do­ing it, but it is pos­si­ble that it is cur­rent-me who is wrong.) (As one ex­am­ple of a con­sid­er­a­tion that moved me here: a friend of mine whose epistemics I trust, and who has known Vas­sar for a long time, said that she usu­ally in the long-run ended up agree­ing with her while-in-the-con­ver­sa­tion self, and not with her af­ter-she-left-the-con­ver­sa­tion self.)

Also I was a bit dis­com­fited when my pre­vi­ous LW com­ment was later cited by folks who weren’t all that LW-y in their con­ver­sa­tional pat­terns as a gen­eral “de­nounce­ment” of Vas­sar, al­though I should prob­a­bly have pre­dicted this, so, that’s an­other rea­son I’d like to try to pub­li­cly state my re­vised views. To be clear, I do not cur­rently wish to “de­nounce” Vas­sar, and I don’t even think that’s what I was try­ing to do last time, al­though I think the fault was mostly mine that some peo­ple read my pre­vi­ous com­ment as a gen­eral de­nounce­ment.

Also, to be clear, what I am say­ing here is just that on the strength of my own ev­i­dence (which is not all ev­i­dence), (1) and (2) seem true to me. I am not at all try­ing to be a court here, or to eval­u­ate any ob­jec­tions any­one else may have to Vas­sar, or to claim that there are no valid ob­jec­tions some­one else might have, or any­thing like that. Just to share my own re­vised im­pres­sion from my own limited first-hand ob­ser­va­tions.

Some ad­di­tional ideas: There’s a large va­ri­ety of “loss func­tions” that are used in ma­chine learn­ing to score the qual­ity of solu­tions. There are a lot of these, but some of the most pop­u­lar are be­low. A good overview is at https://​​medium.com/​​udac­ity-py­torch-challengers/​​a-brief-overview-of-loss-func­tions-in-py­torch-c0ddb78068f7
* Mean Ab­solute Er­ror (a.k.a. L1 loss)
* Mean squared er­ror
* Nega­tive log-like­li­hood
* Hinge loss
* KL di­ver­gence
* BLEU loss for ma­chine trans­la­tion (https://​​www.wik­iwand.com/​​en/​​BLEU)

There’s also a large set of “good­ness of fit” mea­sures that eval­u­ate the qual­ity of a model, in­clud­ing sim­ple things like r^2 but also more ex­otic tests to do things like com­pare dis­tri­bu­tions. Wikipe­dia again has a good overview (https://​​www.wik­iwand.com/​​en/​​Good­ness_of_fit)

Thanks for pre­sent­ing this al­gorithm! I had not heard about it, and might try it at some point.

One thing I’m cu­ri­ous about is its in­ter­pre­ta­tion as a sort­ing al­gorithm. Be­cause ba­si­cally what you are do­ing is sort­ing your list us­ing pair­wise com­par­i­sons. It can­not be bet­ter than the clas­si­cal sort­ing al­gorithms, be­cause com­par­i­son based sort­ing takes at least $O\left(nlog\right(n\left)\right)$. But maybe the benefits is that it’s more in­tu­itive to do for a hu­man be­ing than quick­sort or merge­sort? Or that it re­quire a weaker sense of tran­si­tivity than those? Also, I won­der if this isn’t a known sort­ing al­gorithm.

Have moved to com­ments. Thank you both for the feed­back.