A high integrity/epistemics political coalition?
I have goals that are much easier to reach with a powerful political bl. Probably a lot of other people around here share them. (Goals include “ensure no powerful dangerous AI get built”, “ensure governance of the US and world are broadly good / not decaying”, “have good civic discourse that plugs into said governance.”)
I think it’d be good if there was a powerful, high integrity political bloc with good epistemics, trying to make those things happen.
Unfortunately the naive ways of doing that would destroy the good things about the rationalist intellectual scene. This post lays out some thoughts on how to have a political bloc with good epistemics and integrity.
Recently, I gave to the Alex Bores campaign. It turned out to raise a quite serious, surprising amount of money.
I donated to Alex Bores fairly confidently. A few years ago, I donated to Carrick Flynn, feeling kinda skeezy about it. Not because there’s necessarily anything wrong with Carrick Flynn, but, because the process that promoted “donate to Carrick Flynn” to my attention was a self-referential “well, he’s an EA, so it’s good if he’s in office.” (There might have been people with more info than that, but I didn’t hear much about it).
Ultimately, I kinda agreed, but, I wouldn’t have publicly defended the choice. This was during FTX era where money was abundant and we were starting to attract grifters (i.e. hearing explicit comments like “oh man all you have to do is say you care about causes X and Y and you can get free money.”) It was not sustainable to keep donating to people “because they were EA” or “because they mouthed the words ‘AI Safety’.”
Alas, there are important political goals I want to accomplish. Political goals require getting a lot of people moving in lockstep. Rationalists hate moving in lockstep. For good reason. At the time, my solution was “donate to Carrick Flynn, but feel skeezy about it.”
One option is leave this to “The EA community” rather than trying to invoke “the rationalists.” Alas, I just… don’t really trust the EA community to do a good job here. Or, rather, them succeeding at this requires them to lean into the rationalist-y traits, which would reintroduce all the same allergies and handwringing. My political goals are nuanced. I don’t want to go the route of environmentalism that bans nuclear power and ends up making things worse.
The AI Safety Case
AI Safety isn’t the only thing you might want a powerful political bloc with good epistemics to support. Maybe people want to be ambitious and do something much more openended than that. But, this is the motivating case for why it’s in my top-5 things to maybe do, and it’s useful to dissect motivating cases.
I think many people around here agree we need to stop the development of unsafe, overwhelmingly powerful superintelligence. (We might disagree about a lot about the correct steps to achieve that).
Here are some ways to fail to do that:
you create a molochian Moral Maze that’s in charge of “regulating AI”, which isn’t even trying to do the right thing, staffed by self-serving bureaucrats that hand out favors that have nothing to do with regulating unsafe, overwhelmingly powerful superintelligence.
you create a highly trusted set of technocrats who, unfortunately, are just wrong about what types of training runs,compute controls, or other interventions will actually work, because that’s a complex question.
you create some system that does approximately the right thing on Day 1 but still needs to be making “live” choices 2 decades later and has ossified.
you never got buy-in for the thing, because you didn’t know how to compromise and build alliances.
you built alliances that accomplish some superficially similar goal that isn’t solving the right problem.
That’s rough. Wat do?
What I think Wat Do is, figure out how to build a political network that is powerful enough to have leverage, but, is still based on a solid foundation of epistemic trust.
How do that?
Well, alas I dunno. But it feels very achievable to me to do better than both “don’t play the game” or “naively play the game, short sightedly.” Here are some thoughts on that
Some reason things are hard
This is difficult for lots of reasons. Here are some easier to articulate ones:
Mutual Reputation Alliances
A lot of the world runs on implicit alliances, where people agree to recommend each other as good people, and not to say bad things about each other.
One big reason ornery rationalists are like “politics is Real Hard to do without intellectual compromise” (while other people might be like “I see why you’d be worried, but, you seem to be exaggerating the worry”), is that this is a very pernicious. It fucks with epistemics in a way that is invisible if you’re not actively tracking it, and the mutual reputation alliances don’t want you to be tracking it so it requires active effort to make it possible to track.
People feel an incentive to gain power generally
There are good (naive) reasons to gain power. You do need political power to get shit done. But, also, people feel an attraction to power for normal, boring, selfish reasons. It is easy to deceive yourself about your motivations here, and about what your motivations will be in the future when you’ve enmeshed yourself in a political alliance.
Lots of ways of gaining power involve Mutual Reputation Alliances, or other compromises.
(Oliver Habryka has argued to me that there are ways of gaining conditional power (as opposed to unconditional power) which involve less compromise. This post is mostly about gaining unconditional power but seemed worth flagging the difference)
Private information is very relevant
There is some public info available, but for “will this broad political project work longterm”, it’s going to depend on things like “does so-and-so keep their word?”, “will so-and-so keep keeping their word if the political situation changes, or they see an opportunity for power?”
This requires subtle details about their character, which you can only really get from people who have worked with them a bunch, who are often part of a mutual reputation alliance, won’t want their name attached to the info if you share it, and will only give you the info if you can share it in a way that won’t make it obvious that they were the one sharing it.
Powerful people can be vindictive
In addition to “embedded in a mutual reputation alliance”, powerful people can be vindictive if you try to share negative information about their character. And, since they are powerful, if they want to hurt you, they probably can.
People don’t share bad information about powerful people out of fear, not just loyalty.
(One specific case of this is “they can sue you for libel, or at least threaten to.”)
Politics is broadly adversarial
There will be rival actors who don’t want your preferred candidate to be elected or your preferred policy to be implemented. They will actively make it hard for you to do this. They may do so with underhanded tactics that are difficult to detect, just under the threshold for feeling “unreasonable” so it’s hard to call out.
It also means that sometimes you want to raise funds or maneuver in secret.
Lying and Misleadingness are contagious
Mutual reputation alliances are costly because they radiate out of the alliance. In practice, there is not a sharp divide between the politicians and the rationalists. The people rallying support and finding private information will (by default, probably) radiate some pressure to not question the narrative, and to avoid making someone regret having shared information.
See also: Entangled Truths, Contagious Lies
Politics is the Mind Killer / Hard Mode
This is hard-mode enough when we’re just trying to be a corner of the internet talking about some stuff. It’ll matter a lot more if you are trying to achieve a political goal.
See: Politics is the Mind-Killer and Politics is hard mode
A high integrity political bloc needs to work longterm, not just once
A lot of these problems aren’t that bad if you’re doing a one-time political maneuver. You might make some enemies and risk a bit of tribal groupthink, but, eh, then you go back to doing other things and the consequences are bounded.
But, the whole point of building a Good Epistemics/Integrity political bloc is to keep persistently doing stuff. This will attract enemies, if it succeeds. It will also attract…
Grift
People will try to manipulate into giving them money. Some instances of this might be well intentioned. You need to be able to defend against it anyway.
Passwords should be costly to fake
If it’s known that there’s a High Integrity/Epistemics Political Bloc that’s on the lookout for sociopaths and subtle corruption, people will try to mouth the words that make it sound like they are avoiding sociopathy/subtle-corruption. This includes both candidates, and people running the rallying-campaigns to get candidates funded.
“I believe in AI safety” or “I care about epistemics” is an easy password to fake.
An example of a harder password to fake is “I have made many public statements about my commitments that would look bad for me if I betrayed them.”
For people running PACs or other orgs, “here are the incentives I have constructed to make it hard for myself / The Org to betray it’s principles” is even better. (i.e. OpenAI’s nonprofit governance structure did make it at least difficult and take multiple years, for the org to betray it’s principles).
Example solution: Private and/or Retrospective Watchdogs for Political Donations
A sometimes-difficulty with political fundraising is early on it’s often important to happen in a low-key way, since if rival politicians know your plan they can work against it. But,
I think part of the process should be, there are people involved in low-key-private-political-fundraising who are playing a watchdog role, helping establish mutual knowledge of things like whether a given politician...
Top Tier:
...has ever made a political costly decision to stand by a principle
...does NOT have any track record of various flavors of sociopathy
...has ever gotten a bill passed that looks like it’d actually help with x-risk or civilizational sanity or other relevant things.
Mid Tier
...has ever stated out loud “I want to pass a bill that helps with x-risk or related stuff”, that establishes a reputation you can at least call them on later.
...has a reputation for consistently saying things that make sense, and not saying things that don’t make sense.
Minimum Tier:
...in private conversations, they seem to say things that make sense, promise to work on AI risk or important related things, etc… and, ideally, this is vouched for by someone who has a track record of successfully noticing sociopaths who claimed such things, but later betrayed their principles.
…they seem generally qualified for the office they’re running for.
I didn’t trust the people advocating for Alex Bores to have noticed sociopathy. But, he did in fact pass the Raise Act. Scott Wiener tried to pass SB 1047 twice and succeeded the second time, sorta. They might still betray their principles later, but, their track record indicates they are at least willing to ever put their actions where their mouth was, and the bills looked pretty reasonable.
That seemed good enough to me to be worth $7000 (given the other analysis arguing that the money would help them win).
If I imagine a high Integrity Political Bloc, I think it probably involves some sort of evaluator watchdog who a) privately researches and circulates information about candidates during the Low Key period, and b) writes public writeups afterwards that allow for retrospective sanity checking, and noticing if the political bloc is going astray.
I’d want the watchdogs to split up observations and inferences, and split up particular observations about Cause A vs Cause B (i.e. make it easy for people who want to support AI safety but don’t care about veganism, or, vice versa, to track which candidates are good by their lights, rather than aggregating them into a general vector of Goodness).
People in charge of PACs/similar needs good judgment
The actual motivating example here was thinking about supporting PACs, as opposed to candidates.
I don’t actually understand PACs very well. But, as I understand it, they need to be deciding which candidates to support, which means you need all the same apparatus for evaluating candidates and thinking through longterm consequences.
Any broad political org needs a person in charge of it who is responsible for making sure it is high integrity. I have a particularly high bar for this.
If you want to run a PAC or org that gets money from a hypothetical High Epistemics/Integrity Political Bloc, it is not merely your job to “not lie” or “not mess up in the obvious ways.” Politics is hard mode. You need to be tracking the incentives, tracking whether your org is evolving into a moral maze, and proactively work to make sure it doesn’t get eaten by an egregore.
This requires taste, as well as effort.
Taste is hard to acquire. Often, “just try harder” won’t realistic work. If you don’t have good enough judgment, you either need to find another person to be in charge, or you might need to go try doing some projects that will enable you to learn from experience and become wiser / more cynical / etc.
Don’t share reputation / Watchdogs shouldn’t be “an org”
An earlier draft described this as “GiveWell for retroactive political action assessment”. But, the word “Givewell” implies there is an org. Orgs bundle up people’s reputation together, such that every person involved feels pressure to not risk the reputation of everyone else at the org. This has been a failure mode at OpenPhil (from what I understand).
Watchdogs will need to make some tradeoff on gaining access to private information, vs making various promises and compromises. But, they can do that individually, so the results aren’t as contagious.
Different “Watchdogs” and “Rally-ers”
I would ideally like everyone involved to have maximally good epistemics. But, in order for this to succeed, you need some people who are really good at rallying large numbers of people to do a thing (i.e donate to candidates, vote). Rallying is a different skill from maintaining-good-epistemics-while-evaluating. It’s hard to be good at both. It’s hard because a) it’s just generally harder to have two skills than one, and b) “rallying” just does often require a mindset that is more Mindkiller-y.
So, I would like at least some people who are spec’d into “watchdog-ing”/”evaluation”, who are not also trying to rally people.
I want rally people to be more careful on the margin. I think it is possible to skill up at inspiring conviction/action without having distorted beliefs. But, I think the project can work if the rally-ers aren’t maximally good at that.
Watchdog-ing the ecosystem, not just candidates
One way for this to fail is for individual candidates to turn out to be grifters who extract money, or sociopaths who end up net-negative.
Another way for this to fail is for the system to become subtly corrupted over time, making individual little compromises that don’t seem that bad but add up to “now, this is just a regular ol’ political bloc, with the word ‘epistemics/integrity’ taped to the front door.”
There needs to be watchdogs who are modeling the whole ecosystem, and speaking out if it is sliding towards failure.
Donors/voters have a responsibility not to get exploited
It’s not enough for watchdogs to periodically say “hey, this candidate seems sus” or “we seem to be sliding towards worse epistemics collectively.” The people voting with their ballots or wallets need to actually care. This means a critical mass of them need to actually care about the system not sliding towards corruption.
Prediction markets for integrity violation
This could be an entirely separate idea for “watchdog evaluators”, but it dovetails nicely. For candidates that a powerful high-integrity political bloc are trying to help, it probably makes sense to have public prediction markets about whether they will keep their word about various promises.
If individual watchdogs gain a track record for successfully noticing “so and so is going to betray their principles” and “so and so probably won’t betray their principles”, those people can also then maybe be trusted more to represent private information (“I talked to Candidate Alice, and I really do get a sense of them knowing what they’re talking about and committing to Cause A”).
The main problem with doing that publicly is that powerful people might be vindictive about it. I’m most worried about people being vindictive when they kind grew up with the rationalsphere, so having rationalists criticize them or estimate them as low integrity, feels personal, rather than just cost-of-doing-business as a politician.
I do think the norm and vibe should be “this is a cost of doing business. If you want money/support from the high integrity political engine, you should expect people to be evaluating you, this is nothing personal, the standards are very exacting and you may not meet them.”
Handling getting sued for libel
A problem I’m not 100% sure how to handle, is getting sued for evaluating people/orgs as sociopathic.
I’m not sure what the legal standing is, if a prediction market reads:
“Within 5 years, I will judge that OpenAI’s nonprofit board no longer has teeth”
or
“Within 5 years, I will think [Candidate X] betrayed a campaign promise.”
or:
“Within 5 years, CEO Charlie will have violated one of these principles they established.”
A serious political engine could have money to defend against lawsuits, but, also, the more money you have, the more it’s worth suing you. (I think at the very least having someone who specializes in handling all the hassle of getting sued would be worth it).
My hope is that, unlike previous instances of people trying to claim an individual did bad things, this project is in some sense “big enough to be clearly worth protecting” (whereas a random person in a vague community scene being maybe a bad actor doesn’t have anyone incentivized to make it their job to defend)
LessWrong is for evaluation, and (at best) a very specific kind of rallying
Sometimes people get annoyed that LessWrong isn’t letting them do a particular kind of rallying, or saying something with one voice. They read Why Our Kind Can’t Cooperate and are like “okay, so, can we have a culture where people publicly support things and there isn’t this intense allergic criticism?”.
I think maybe there should be another forum or tool for doing that sort of thing. But, it’s definitely not LessWrong’s job. LessWrong definitely should not be synonymous with a political agenda.
I think posts like these are fine and good:
I feel wary of posts like this:
Statement of Support for “If Anyone Builds It, Everyone Dies”
We must be very clear: fraud in the service of effective altruism is unacceptable
I think the difference is:
Posts that argue the object level of ‘this candidate or project will have good/bad consequences’ are fine.
Some things are less fine include: trying to shame people without making arguments, or laying plausibly deniable traps that make it hard to argue back, or that try to establish a false consensus.
Posts that argue about what is socially acceptable to think/say on LessWrong ARE fine. The difference between this and the previous one can be subtle. I still find John Wentworth’s comments from Power buys you distance from the crime pretty good:
> Who’s at fault for the subcontractor(^3)’s slave labor?
[...] My instinct says DO NOT EVER ASK THAT QUESTION, it is a WRONG QUESTION, you will be instantly mindkilled every time you ask “who should be blamed for X?”.
… on reflection, I do not want to endorse this as an all-the-time heuristic, but I do want to endorse it whenever good epistemic discussion is an objective. Asking “who should we blame?” is always engaging in a status fight. Status fights are generally mindkillers, and should be kept strictly separate from modelling and epistemics.
Now, this does not mean that we shouldn’t model status fights. Rather, it means that we should strive to avoid engaging in status fights when modelling them. Concretely: rather than ask “who should we blame?”, ask “what incentives do we create by blaming <actor>?”. This puts the question in an analytical frame, rather than a “we’re having a status fight right now” frame.
To be clear, LessWrong doesn’t prevent you from posting rallying / status-fighty / social-reality-manipulating posts. But, it is setup to discourage it on the margin, and prevent a lot of the upside from trying to do it. You won’t be on the frontpage, you won’t get curated, etc. If it seems like you’re doing it in a way that mods think is bad for the culture, we might yell at you.
(But also note, I did not run this by the rest of the Lightcone team and we have a policy of speaking for ourselves, since orgs don’t actually have “beliefs”)
Recap
Just to restate all the premises in one place:
A political bloc is a system that coordinates lots of people to produce a political outcome. (If you don’t need to coordinate lots of people, you just have a political actor, not a bloc)
It’s hard to build a high integrity/epistemics political bloc, because:
There is a pull towards mutual reputation alliances
There are incentives to gain power that distort our thinking
There are incentives towards simple grift.
You need access to private information (which often lives within the mutual reputation alliances)
Powerful people might try to punish you for exposing subtle character flaws
Enemies will be trying to sabotage you, while maintaining plausible deniability
And this all needs to keep working longterm, if you want a longterm powerful impact, so, it needs to be robust to all the adversarial failure modes.
Some potential solutions:
Have private evaluator people who check in on whether candidates seem good, and whether the whole political bloc seems sane.
Avoid sharing reputation as much as possible, so people feel more free to speak/think independently.
Maybe try prediction markets for commitment-violation.
Donors/voters will need to decide which candidates to support, and need to actually be trying to form their own judgments to avoid getting consumed by an egregore.
Maybe I should spell out some background/inside-baseball context:
In the past year, I’ve been aware of some waves of effort to coordinate ~hundreds of people to donate to political candidates, that successfully raised a lot of money, generally through a mix of circulating google docs making arguments, and manually DMing thousands of people through existing social networks.
It was all pretty lowkey, for the reasons stated in this post.
This accomplished some pretty impressive stuff. When I write this post, I’m not like “it’d be cool if some magical rational/EA political arm came out of nowhere.” I’m like “It’d be cool if the existing political networks that are actually pretty competent, also developed some processes for group epistemics and integrity (while handling the reality that it’s operating in an adversarial environment). Here are some ideas on how to do that.”
A lot of the comments here seem to be responding to this like a pie-in-the-sky ideation. I’m approaching it from a pretty brass tacks practical mindset, but it makes sense if that feels weird to most readers.
Good post, but do you really want to use the specific term political machine for this? “Bloc” or “institution” seem more like what you’re talking about here, unless I’m misunderstanding.
Huh, I never knew that term had any such narrowly construed meaning. Good to know! Given that, I agree that this is a bad name for the thing Ray is talking about.
Ah, alas. Well I just replaced most of the instances with “bloc” but I’m not sure if the connotations of that are quite right either.
I thought “political machine” was a category that included the corrupt bad thing in the wiki article but also included other things, and, man now I’m not sure there’s even a good word for exactly what I want.
How bad is this, actually? My impression is that everyone complains about politicians lying but in practice in doesn’t cost you much. E.g. no one says “well I otherwise like him but he betrayed this position I hate, so I won’t support him”
well I said “harder to fake”, not ironclad or “sufficiently hard to fake.” It’s better than “in private, he said he cared about My Pet Cause”
I do think people sometimes get mad at and vote out politicians that betrayed a principle they care about, esp. if they are a single-issue voter.
I think this definitely can happen. The Liberal Democrats in the UK (third party, colloquially the Lib Dems) gained support in the 2010 election by pledging not to vote for increases in tuition fees for UK universities. They were able to form a coalition government with the Conservative party after the 2015 election. The Lib Dems hadn’t been in power before (though their predecessor, the Liberal party, hadn’t been in power since the 1920s).
The coalition government increased tuition fees by a factor of 3. There were widespread protests at universities, mostly peaceful though one bloke did lob a fire extinguisher off a roof. The Liberal Democrats never recovered, going from 57 seats to 8 at the next election in 2015. I was a member of the party around 2017, and many people I spoke to said “I used to vote for them but then they increased tuition fees”.
This doesn’t prove a causal link (people might have just been annoyed at the coalition government’s austerity policies, and used tuition fees as a totem of that) but people do literally say “I liked Nick Clegg but he betrayed his position on tuition fees and I’ll never vote Lib Dem again.”
I think quantifying how strong a particular signal is will be an important part of having a self-correcting political coalition. My guess is that public statements of this kind are epsilon, but I’m open to data I’m wrong.
The reason I thought they were non-epsilon was “it sure seems like people are not willing to go on record as saying AI x-risk is important” (going on record in favor of “AI safety” is probably easy).
Generally, going on record saying something out-of-the-overton window I think counts for nontrivial integrity. (But, this isn’t what I really said in the post, to be fair)
(edit: agreed that quantifying how strong a signal is is important, and not at all sure the strengths I implied here are correct, although I think they are in at least a relative fashion)
I think it helps if
1. The standards are legible, and clearly written down.
2. The standards are regarded as a separate category from “are you a good guy”.
Like, it should feel like “yep, trying to meet these standards is an actual cost, and it might not be worth it for you, and that doesn’t mean that we can’t be friendly, or be allies, but it does mean that you don’t get to have the resources that are allocated specifically for the people who decide to buy into these costs (in exchange for the support).”
It helps if they’re legible and written down because otherwise they will tend to slide into “are you a good guy”.
Ah, well I agree with that as the dominant failure mode but I think we are on the harder level of “there are opaque gestalt Be A Good Guy For Actualz that are hard to distill into rules, and, you need to successfully sus out ‘are they actually a good guy?’ without actually instead asking ‘do they vibe like a good guy?’”
(I don’t know that I believe that exactly for this context, but, I think that’s at least often a situation I find myself in)
I don’t have much to say except that I think it would be good to create a bloc with the proposed goals and standards, but that it would be hard to adhere to those standards and get anywhere in today’s politics.
Also, if I was an American and interested in the politics of AI, I would be interested in the political stories surrounding the two movements that actually made a difference to executive-branch AI policy, namely effective altruism during the Biden years and e/acc during Trump 2.0. I think the EAs got in because the arrival of AI blindsided normie society and the EAs were the only ones who had a plan to deal with it, and then the e/accs managed to reverse that because the tech right wanted to get rich from a new technological revolution, and were willing to bet on Trump.
Also, for the record, the USA actually had a state politician who was a rationalist, ten years ago.
Huh. Presumably there’s some space for proposing and arguing for norms that you think should become part of the culture. (Noting that this is a very importantly different thing from shaping the culture by making normative statement in a way that ambiguously implies that they are already the socially normative, which shifts people’s expectations about what behavior is rewarded, praised, punished, or shamed.)
Yeah my statement as-worded is too strong. (Actually, upon reflection I am surprised I didn’t trigger Oliver coming in and saying “WHAT!? Ray that is wrong!”)
Yeah, posts saying “hey, here are some reasons I think this should be a norm” are extremely fine. Posts doing that while also using emotionally laden language are, like, kinda fine depending on context. Posts that are directly wielding shame and implying anyone who disagrees with the post is a bad person in ways that feel socially hard to push back against are generally not fine.
It was honestly a statement that was so clearly wrong in the straightforward interpretation that my guess was you obviously meant to convey something different than the obvious interpretation, but then I didn’t really put in the effort to reflect on that.
(I nearly wrote a comment saying it was blatantly wrong. For instance, I think I often write things with the intent to expand the space of things that people feel comfortable saying on LessWrong. But I didn’t comment because I felt bad being so agro about one line in a post I hadn’t read in case I misunderstood the context.)
This seems like the sort of thing that would be feasible to do publicly, at least in large part, since politicians make many public decisions. You already have fact checkers like PolitiFact which rates the veracity of individual statements and also tracks promises made and kept by presidents, although keeping promises isn’t the same thing as honesty or standing by principles (there are lots of reasons why it’s hard for presidents to keep promises).
I don’t think PolitiFact actually does the thing you’re pointing at here, but it’s a proof of concept that it’s possible to do similar things. A PolitiFact-esque org could track politicians’ honesty and sticking-to-principles-ness.
Remembering that PolitiFact actually reminds me to be more worried about this. My impression is PolitiFact started off reasonably neutral and then veered into being a partisan mouthpiece.
(I am maybe less worried about a version that has more specific goal of “decide between candidates you plausibly like”, but, it’s the sort of the thing that would have a natural tendency to turn into an org, and then get audience-captured)
Random aside: I did recently find out about a thing called “Center for Effective Lawmaking” that seems to rate legislators based on how well they accomplish the policies they (or their party?) set out to do. I haven’t looked into it at all but it seemed like another angle from PolitiFact of “past example of someone trying to do the thing”.
https://thelawmakers.org/find-representatives
That sounds very useful, but it also sounds like an intractably difficult thing to figure out, even putting aside the issue of motivation that you brought up.
Suppose I’m a politician with a single issue that I care about. Is sponsoring ten bills on that issue a plus even if none of them get passed? But what if I’m in a party facing a 49-51 minority, and no bill I sponsor is going to get passed regardless of what I do—is there no way to evaluate me, if non-passed legislation doesn’t count? What about my ability to shift the Overton Window, or talk my party members into supporting my position in the future? I would imagine that the targeted function looks like...
∫∞t=nowpolicy_alignmentt−∫∞t=nowpolicy_alignment′t
...where
policy_alignmentis some approximation of the U.S. government’s alignment with my stated positions, andpolicy_alignment'is the same, but in a counterfactual world where my opponent won the general election, such that my political impact across the entire collective future of America is computed. But that’s a very difficult function to build a proxy for, because, even putting the credit assignment problem aside, who can say whether a line item giving my issue of choice $50 million in federal funding is going to have more long-term influence than a high-profile showdown that wins me no friends on Capitol Hill but forces the next president to adopt it as part of his platform, let alone backing down quietly on the line-item in exchange for a favor from a senior congressman that I can use later on?This suggest that the organization that has the money to defend against lawsuits is not the same organization as the organization making the potentially libelous claims.
There are broad organizations like Foundation for Individual Rights and Expression (FIRE) that can do that. You could also fund an organization that’s more specific about defending people within your coalition.
Yeah, to be clear I have not thought that hard about how to handle the lawsuits. Even with a functioning lawsuit defense org-thingy, I think Evaluator People will probably need to have courage / conflict-readiness, and part of the post is a call for that.
I think the best model here is a constellation of individuals and micro-orgs, and some donors who are are serious about supporting the entire endeavor (which does unfortunately involve some modeling of the “what counts as the endeavor”).
“Medieval institutions” seem like they have a lot of room for improvement. Can we build high integrity/epistemics institutions at all? Or are we still lacking some key social tech we’ve yet to find or develop? (Do we e.g. need Rationalist alternatives to “mutual reputation alliances”?) There seem to have been several attempts by the Rationalists, with mixed results. LessWrong itself being one.
Can anyone suggest some of which have been the most successful and why? Which have failed spectacularly and why? Which are failing more subtly? (Or if this has been answered or partially answered already, can you link?)
Do we have a catalog of social tech that might help with institution building? E.g., I heard about prediction markets and dominant assurance contracts first on LW, etc. Seems like a lot of game theory is applicable.
I think, beyond the issues you bring up, is the probability that people who care about integrity, epistemics, and competence may have fundamentally different core interests that make coalitions between them impossible. In other words, many people sincerely care about epistemics as a means, but their end goals are so wildly different that one faction’s idea of “successful AI deployment, followed by total eudaimonia” is another’s idea of “disastrous AI misuse that is on the same order of badness as human extinction”. Even a universal pause is difficult to negotiate, as, if the terms of the pause increase the eventual probability of faction A’s desired outcome, faction B wouldn’t consider it an improvement. Two sets of truly sincere, selfless, and intelligent people may still have entirely incompatible value functions.
Historically, formally bipartisan movements that have found relevance have often been torn apart as they decide on the often-binary question of which major political bloc is a more acceptable coalition partner. Others have survived this by more-or-less explicitly taking a side, but been subsumed completely shortly thereafter. The fate of Sierra Club springs to mind, with a fracture following David Gelbaum’s $200M donation and subsequent ultimatum on immigration. Following the fracture, the victorious faction used its newly-established internal power to transform it into an intersectional left-leaning activist organization rather than a bipartisan group of environmentalists. It could be argued that the backlash against this shift was catastrophic for the state of environmentalism on the political right, and thus the political fate of environmentalism in general. It could also be argued that acceptance of Gelbaum’s donation and Sierra Club’s subsequent vassalization to a major party allowed the Sierra Club to take actions that it might otherwise have been unable to take.
The debate over whether to support Scott Weiner is a more immediate example. If his name became associated with a political upstart, then empowering that group would amount to empowering him, and many people are categorically unwilling to do that. For those aligned with him politically, consider what his Republican counterpart would look like, and consider whether you would be willing to donate money to a group that proudly endorsed his campaign because he reliably backed AI safety bills. Moreover, consider what this candidate’s idea of “Aligned” AI would look like, and whether this is an improvement relative to your current expectations of the future.
If there is a single-issue group that has managed to avoid the fates described and still meaningfully exercise influence, it would be worth investigating how they did so.
Have you ever read Tocqueville’s “Democracy in America”? You can have as many checks and balances and clever system frameworks as you want, but at some point you just need people to believe in democracy. That is to say, your model of good government is never going to be complete without some kind of model of culture and shared values.
Maybe talking about culture—which is fuzzy, historical, symbolic, and social—is outside your wheelhouse. So what? Maybe clever system design and creating good incentive structures is outside my competency—that doesn’t mean I can create meaningful political change by writing fiction alone or going to rallies. Don’t fall into the failure mode of: ‘to the hammer everything looks like a nail.’
To concretise this, consider your proposal “Have private evaluator people who check in on whether candidates seem good, and whether the whole political bloc seems sane.”
Who watches the watchmen? Who ensures they are sane and good? No-one, it can’t be checks-and-balances all the way down: at some point you just need to trust experts will have high integrity and aligned values. Not everyone can be elected, not everyone can be hired by a market process, not everyone can be monitored all the time, an objective bureaucratic examination can never filter for everything you want in a leader.
Does that mean your proposals are wrong or unhelpful? No, they seem well thought-out and plausible. It is merely to say your world-model seems to be missing a few gears.
The watchmen publish their work publicly, people read the writeups and check that they make sense, pretty much anyone can become a watchman if they want. (The whole idea is that the writeups can be private during low-key-fundraising periods but public afterwards so it’s easier to sanity-check. Also, during the lowkey periods, there can be private mailing lists for discussion)
The answer to “who watches the watchmen” is “distributed spotchecks by readers.” Will that be perfect? No. It just has to be good enough to be make it worth making a lot more political donations at scale.
It feels like you’re a) assuming I’m more absolutist about this than I am, b) that I haven’t thought about the stuff you mention here, and I don’t really know why.
I think this might explain the difference in framing? From the quote below, I assumed you were trying to come up with a fully general solution to the problem you specify:
But I see now that you were taking the existence of a community of sane, reasonable, and mostly value-aligned participants as a given, and instead focusing on a framework which could make their interaction with the wider political process saner.
The uncharitable reading is that you are assuming a can opener, but, from reading your other writing, evidently the better reading is that you do have a model for producing/widening this community elsewhere (inter alia).
Yeah. I’d phrase it as “reasonably sane, reasonably reasonable, and reasonably value-aligned.” I don’t think the LW commentariat is perfect, but, I think they are within a basin where “aiming for a sane political coalition” is a reasonble aspirational goal. (and, while I’d like to succeed at the most ambitious version of the thing, all it needs to succeed at is “be a better use of people’s time/attention than other things” (given that there are pretty compelling alternatives).
I know a lot of people around here with similar-ish political goals, and similar-ish ideals of what you might hope a rationalist political bloc to look like, such that “okay, translate that into implementation details” feels like the right step.
Here in NZ, we have a minor party that’s been trying to win influence and elections, called The Opportunities Party (TOP). TOP has not been very successful, and I believe it’s due to not having a core identity. Trying to resonate with people on the basis of Epistemics isn’t very effective in practice.
In this case, I’m not saying “let’s make an Epistemics Party.” I’m saying, rationalsphere people who agree on political goals should coordinate to achieve those goals as effectively as possible (which includes preserving rationality).
I expect this to look more like supporting ordinary Democrat or Republican candidates (in the US), or otherwise mostly engaging with the existing political apparatus.
perhaps a PAC run by dath ilan-style liquid democracy principles.
A quick heuristic to filter low-integrity politicians: check if they take bribes. The vast majority of U.S. politicians openly take legal bribes to their campaign funds, so this is a strong filter.
The US system is build so that taking campaign donations is important to getting elected. It’s not like for example the German system where politician can really on support of his party to get into parliament.
If you want to affect the US politician system it’s likely necessary to cooperate with politicians who do take donations to their campaign funds.
It’s also worth noting that that donations to PACs are probably worse than direct campaigns when it comes to lobbyist power.
I have a very simple opinion about why pushes for AI regulation fail. Ready?
Because nobody knows what they are asking for, or what they are asking for is absurd.
Here are the absurdities:
“Stop all AI research forever”
“Stop all AI research until it’s safe to do, and I say when it’s safe (i will never say it’s safe)”
“Stop all AI research except mine, where I sit around and think about it”
“Pause all AI research until we have a functional genetic engineering project so our smarter descendants can do AI research”
Once we get past absurd asks we get into easily coopted asks, like “restrict compute” which turns into “declare an arms race with the largest industrial power on the planet” and “monitor training runs” which turns into “tell every potential competitor you are going to have the government crush them”.
What will convince me there is a sane block pushing for AI-related regulations is when they propose a regulation that is sane. I cannot emphasize enough how much these things have failed at the drafting stage. The part everyone is allegedly good at, that part, where you write down a good idea that it might be a good idea to do? That’s the part where this has failed.