Linch

• LinchJan 2, 2025, 9:08 AM

  LW: 3 AF: 1

  1

  AF

  in reply to: David Scott Krueger (formerly: capybaralet)’s comment on: Eval­u­at­ing the his­tor­i­cal value mis­speci­fi­ca­tion argument

  I think I’m relatively optimistic that the difference between a system that “can (and will) do a very good job with human values when restricted to the text domain: vs “system that can do a very good job, unrestricted” isn’t that high. This is because I’m personally fairly skeptical about arguments along the lines of “words aren’t human thinking, words are mere shadows of human thinking” that people put out, at least when it comes to human values.
  
  (It’s definitely possible to come up with examples that illustrates the differences between all of human thinking and human-thinking-put-into-words; I agree about their existence, I disagree about their importance).

An­nounc­ing the Q1 2025 Long-Term Fu­ture Fund grant round

• LinchDec 10, 2024, 9:20 PM

  6 points

  0

  on: Scale Was All We Needed, At First

  So there was a lot of competitive pressure to keep pushing to make it work. A good chunk of the Superalignment team stayed on in the hope that they could win the race and use OpenAI’s lead to align the first AGI, but many of the safety people at OpenAI quit in June. We were left with a new alignment lab, Embedded Intent, and an OpenAI newly pruned of the people most wanting to slow it down.”

  “And that’s when we first started learning about this all?”

  “Publicly, yes. The OpenAI defectors were initially mysterious about their reasons for leaving, citing deep disagreements over company direction. But then some memos were leaked, SF scientists began talking, and all the attention of AI Twitter was focused on speculating about what happened. They pieced pretty much the full story together before long, but that didn’t matter soon. What did matter was that the AI world became convinced there was a powerful new technology inside OpenAI.”

  Yarden hesitated. “You’re saying that speculation, that summer hype, it led to the cyberattack in July?”

  “Well, we can’t say for certain,” I began. “But my hunch is yes. Governments had already been thinking seriously about AI for the better part of a year, and their national plans were becoming crystallized for better or worse. But AI lab security was nowhere near ready for that kind of heat.

  Wow. So many of the sociological predictions became true even though afaict the technical predictions are (thankfully) lagging behind.

• LinchDec 4, 2024, 6:55 PM

  2 points

  1

  in reply to: Martin Randall’s comment on: Eval­u­at­ing the his­tor­i­cal value mis­speci­fi­ca­tion argument

  Thanks, I’d be interested in @Matthew Barnett’s response.

A Qual­i­ta­tive Case for LTFF: Filling Crit­i­cal Ecosys­tem Gaps

• LinchNov 27, 2024, 9:54 PM

  4 points

  0

  in reply to: Going Durden’s comment on: Are the ma­jor­ity of your an­ces­tors farm­ers or non-farm­ers?

  Interesting! I didn’t consider that angle

• LinchNov 18, 2024, 5:29 AM

  4 points

  0

  in reply to: Zach Stein-Perlman’s comment on: A Qual­i­ta­tive Case for LTFF: Filling Crit­i­cal Ecosys­tem Gaps

  Agreed, I was trying to succinctly convey something that I think is underrated, unfortunately going to miss some nuances.

• LinchNov 15, 2024, 11:33 PM

  4 points

  0

  in reply to: johnswentworth’s comment on: The Me­dian Re­searcher Problem

  If the means/​medians are higher, the tails are also higher as well (usually).

  Norm(μ=115, σ=15) distribution will have a much lower proportion of data points above 150 than Norm(μ=130, σ=15). Same argument for other realistic distributions. So if all I know about fields A and B is that B has a much lower mean than A, by default I’d also assume B has a much lower 99th percentile than A, and much lower percentage of people above some “genius” cutoff.

• LinchNov 15, 2024, 8:07 PM

  5 points

  1

  on: The Me­dian Re­searcher Problem

  Again using the replication crisis as an example, you may have noticed the very wide (like, 1 sd or more) average IQ gap between students in most fields which turned out to have terrible replication rates and most fields which turned out to have fine replication rates.

  This is rather weak evidence for your claim (“memeticity in a scientific field is mostly determined, not by the most competent researchers in the field, but instead by roughly-median researchers”), unless you additionally posit another mechanism like “fields with terrible replication rates have a higher standard deviation than fields without them” (why?).

• LinchNov 15, 2024, 7:36 PM

  3 points

  0

  on: Seven les­sons I didn’t learn from elec­tion day

  Some people I know are much more pessimistic about the polls this cycle, due to herding. For example, nonresponse bias might just be massive for Trump voters (across demographic groups), so pollsters end up having to make a series of unprincipled choices with their thumbs on the scales.

• LinchNov 14, 2024, 2:52 AM

  2 points

  0

  in reply to: Viktor Rehnberg’s comment on: Sur­vival with­out dignity

  There’s also a comic series with explicitly this premise, unfortunately this is a major plot point so revealing it will be a spoiler:

  Transdimensional Brain-Chip

• LinchNov 13, 2024, 10:07 PM

  3 points

  0

  in reply to: Cole Wyeth’s comment on: Sur­vival with­out dignity

  Yeah this was my first thought halfway through. Way too many specific coincidences to be anything else.

• LinchNov 3, 2024, 5:28 PM

  7 points

  −2

  in reply to: TheCookieLab’s comment on: SwapYourVote: trade your vote with swing-state vot­ers to stop Trump

  Constitutionally protected free speech, efforts opposing it were ruled explicitly unconstitutional.
  
  God LW standards sure are slipping. 8 years ago people would be geeking out about the game theory implications, commitments, decision theory, alternative voting schemas, etc. These days after the first two downvotes it’s just all groupthink, partisan drivel, and people making shit up, apparently.

• LinchNov 3, 2024, 5:27 PM

  3 points

  0

  on: SwapYourVote: trade your vote with swing-state vot­ers to stop Trump

  See Scott Aaronson and Julia Galef on “vote trading” in 2016: https://​​www.happyscribe.com/​​public/​​rationally-speaking-podcast/​​rationally-speaking-171-scott-aaronson-on-the-ethics-and-strategy-of-vote-trading

• LinchOct 27, 2024, 3:57 PM

  7 points

  10

  in reply to: davekasten’s comment on: dav­ekas­ten’s Shortform

  My guess is that we wouldn’t actually know with high confidence before (and likely even some time after) things-will-definitely-be-fine.
  
  E.g. 3 months after safe ASI people might still be publishing their alignment takes.

• LinchOct 27, 2024, 3:54 PM

  9 points

  1

  on: What are some good ways to form opinions on con­tro­ver­sial sub­jects in the cur­rent and up­com­ing era?

  There are also times where “foreign actors” (I assume by that term you mean actors interested in muddying the waters in general, not just literal foreign election interference) know that it’s impossible to push a conversation towards their preferred 1)A or 5)B, at least among informed/​educated voices, so they try to muddy the waters and push things towards 3). Climate change^[1] and covid vaccines are two examples that comes to mind.

  1. ^

     Though the correct answer for climate change is closer to 2) than 1)

• LinchOct 20, 2024, 12:32 AM

  2 points

  1

  in reply to: Roko’s comment on: What ac­tual bad out­come has “ethics-based” RLHF AI Align­ment already pre­vented?

  They were likely using inferior techniques to RLHF to implement ~Google corporate standards; not sure what you mean by “ethics-based,” presumably they have different ethics than you (or LW) does but intent alignment has always been about doing what the user/​operator wants, not about solving ethics.

• LinchOct 3, 2024, 1:54 PM

  2 points

  0

  in reply to: M. Y. Zuo’s comment on: Would catch­ing your AIs try­ing to es­cape con­vince AI de­vel­op­ers to slow down or un­de­ploy?

  I’m not suggesting to the short argument should resolve those background assumptions, I’m suggesting that a good argument for people who don’t share those assumptions roughly entails being able to understand someone else’s assumptions well enough to speak their language and craft a persuasive and true argument on their terms.

• LinchOct 3, 2024, 2:06 AM

  4 points

  0

  in reply to: MichaelDickens’s comment on: MichaelDick­ens’s Shortform

  Reverend Thomas Bayes didn’t strike me as a genius either, but of course the bar was a lot lower back then.

• LinchOct 3, 2024, 2:04 AM

  4 points

  0

  in reply to: MichaelDickens’s comment on: MichaelDick­ens’s Shortform

  Norman Borlaug (father of the Green Revolution) didn’t come across as very smart to me. Reading his Wikipedia page, there didn’t seem to be notable early childhood signs of genius, or anecdotes about how bright he is.