harfe

• harfe 21 Apr 2026 12:03 UTC

  1 point

  0

  on: To­day’s Ring Sig­na­tures and Re­lated Tools

  For the purpose of whistle-blowing, I wonder whether Signal could be used as a source of keys tied to identities of real people. The advantage would be that lots of OpenBrain employees might already use signal. But there are certainly some difficulties:

  • Ring signatures can not be publicly verified, but they could be verified by a journalist who gets their hands on lots of Signal contacts of OpenBrain employees.

  • Verifying that all the signal contacts in the ring actually belong to OpenBrain employees is difficult.

  • The public keys are not visible in the normal signal client, so specialized tools would need to be created.

• harfe 10 Apr 2026 8:29 UTC

  5 points

  0

  on: harfe’s Shortform

  There was report that the CIA used a new tool called Ghost Murmur to detect the electromagnetic signals of a human heart from (40?) miles away, using long-range quantum magnetometry.

  See also Wikipedia.

  My first guess (and still a hypothesis) is that this is deliberate disinformation by the US, but i do not have the expertise required to judge the plausibility. In any case, it could have been an interesting question on the “Could a superintelligence do that?” quiz show.

• harfe 10 Apr 2026 5:55 UTC

  1 point

  1

  in reply to: Mo Putera’s comment on: Mo Putera’s Shortform

  Is any of the lean code public? That could give a better sense of what to expect. Saying that they are working on a “skeletal Lean code” could be very little compared to what would be required to convince other mathematicians.

• harfe 19 Mar 2026 12:45 UTC

  2 points

  0

  in reply to: Mikhail Samin’s comment on: Mikhail Samin’s Shortform

  Yesterday they also did an exclusive interview with Sam Altman

• harfe 21 Jan 2026 15:34 UTC

  4 points

  0

  on: harfe’s Shortform

  I learned that undersea data centers are possible. Microsoft had Project Natick, but it looks like they abandoned it. There is also a chinese project and a western startup. The main benefit seems to be reduced cooling costs.

• harfe 13 Jan 2026 11:18 UTC

  5 points

  3

  in reply to: lilkim2025’s comment on: Eliz­a­beth’s Shortform

  Canada also uses FPTP, so this is not the example you should be using for examining alternatives.

  Proportional representation, which is common in continental Europe, does result in a diversity of parties in practice.

• harfe 28 Oct 2025 14:27 UTC

  9 points

  0

  on: Heuris­tics for as­sess­ing how much of a bub­ble AI is in/​will be

  There’s no bigger narrative than the one AI industry leaders have been pushing since before the boom: AGI will soon be able to do just about anything a human can do, and will usher in an age of superpowerful technology the likes of which we can only begin to imagine. Jobs will be automated, industries transformed, cancer cured, climate change solved; AI will do quite literally everything.

  The article unfortunately does not seriously consider the possibility that AGI has the potential to automate most jobs in a few years. The large investments into AI would be justified in this case, even if current revenue is small! I think this is an important difference to past bubbles.

  OpenAI, Anthropic, and the AI-embracing tech giants are burning through billions, inference costs haven’t fallen (those companies still lose money on nearly every user query), and the long-term viability of their enterprise programs are a big question mark at best.

  The part about inference costs seems false, unless they mean total inference costs of all their instances.

• harfe 27 Oct 2025 15:11 UTC

  1 point

  0

  on: Un­com­mon Utili­tar­i­anism #3: Bounded Utility Functions

  Most^[1] problems with unbounded utility functions go away if you restrict yourself to summable utility functions^[2]. Summable utility functions can still be unbounded.

  For example, if each planet in the universe gives you 1 utility, and $P\left(\text{universe has exactly}n\text{planets}\right)=2^{-n}$ for $n\ge 1$, then your utility function is unbounded but summable. In such a universe it would be very unlikely for a casino to hand out a large number of planets.

  Your proof relies on the assumption

  assuming that the casino has unbounded utility to hand out.

  and this assumption would be wrong in my example.

  ---

  1. ↩︎

     In fact, I do not know of an exception.

  2. ↩︎

     A summable function is a measurable function for which the integral of its absolute value is finite (using the probability measure for the integral in this context).

• harfe 29 Apr 2025 10:59 UTC

  4 points

  0

  in reply to: Zach Stein-Perlman’s comment on: What are the best stan­dard­ised, re­peat­able bets?

  Note that GWWC is shutting down their donor lottery, among other things: https://​​forum.effectivealtruism.org/​​posts/​​f7yQFP3ZhtfDkD7pr/​​gwwc-is-retiring-10-initiatives

• harfe 14 Feb 2025 21:21 UTC

  1 point

  2

  in reply to: plex’s comment on: harfe’s Shortform

  Mid 2027 seems too late to me for such a candidate to start the official campaign.

  For the 2020 presidential election, many democratic candidates announced their campaign in early 2019, and Yang already in 2017. Debates happened already in June 2019. As a likely unknown candidate, you probably need a longer run time to accumulate a bit of fame.

• harfe 12 Feb 2025 17:20 UTC

  12 points

  0

  in reply to: MichaelDickens’s comment on: harfe’s Shortform

  Also Musk’s regulatory plan is polling well

  What plan are you referring to? Is this something AI safety specific?

• harfe 12 Feb 2025 13:05 UTC

  1 point

  0

  in reply to: Mateusz Bagiński’s comment on: harfe’s Shortform

  I wouldn’t say so, I don’t think his campaign has made UBI advocacy more difficult.

  But an AI notkilleveryoneism campaign seems more risky. It could end up making the worries look silly, for example.

• harfe 12 Feb 2025 3:40 UTC

  5 points

  0

  in reply to: Mitchell_Porter’s comment on: harfe’s Shortform

  Their platform would be whatever version and framing of AI notkilleveryoneism the candidates personally endorse, plus maybe some other smaller things. They should be open that they consider the potential human disempowerment or extinction to be the main problem of our time.

  As for the concrete policy proposals, I am not sure. The focus could be on international treaties, or banning or heavy regulation of AI models who were trained with more than a trillion quadrillion (10^27) operations. (not sure I understand the intent behind your question).

• harfe 11 Feb 2025 18:19 UTC

  81 points

  13

  on: harfe’s Shortform

  A potentially impactful thing: someone competent runs as a candidate for the 2028 election on an AI notkilleveryoneism^[1] platform. Maybe even two people should run, one for the democratic primary, and one in the republican primary. While getting the nomination is rather unlikely, there could be lots of benefits even if you fail to gain the nomination (like other presidential candidates becoming sympathetic to AI notkilleveryoneism, or more popularity of AI notkilleveryoneism in the population, etc.)

  On the other hand, attempting a presidential run can easily backfire.

  A relevant previous example to this kind of approach is the 2020 campaign by Andrew Yang, which focussed on universal basic income (and downsides of automation). While the campaign attracted some attention, it seems like it didn’t succeed in making UBI a popular policy among democrats.

  ---

  1. ↩︎

     Not necessarily using that name.

  What links here?

  • The Sorry State of AI X-Risk Ad­vo­cacy, and Thoughts on Do­ing Better by Thane Ruthenis (21 Feb 2025 20:15 UTC; 158 points)
  • 2028 Should Not Be AI Safety’s First Fo­ray Into Politics by Jesse Richardson (4 Mar 2025 16:46 UTC; 5 points)

• harfe 7 Feb 2025 14:22 UTC

  LW: 2 AF: 2

  1

  AF

  on: De­bate, Or­a­cles, and Obfus­cated Arguments

  This can easily be done in the cryptographic example above: B can sample a new number $y=p^{\prime }\cdot q^{\prime }$, and then present $y$ to a fresh copy of A that has not seen the transcript for $x$ so far.

  I don’t understand how this is supposed to help. I guess the point is to somehow catch a fresh copy of A in a lie about a problem that is different from the original problem, and conclude that A is the dishonest debater?

  But couldn’t A just answer “I don’t know”?

  Even if it is a fresh copy, it would notice that it does not know the secret factors, so it could display different behavior than in the $x$ case where A knows the secret factors $p,q$.

• harfe 3 Feb 2025 16:26 UTC

  9 points

  2

  in reply to: johnswentworth’s comment on: What do co­her­ence ar­gu­ments ac­tu­ally prove about agen­tic be­hav­ior?

  Some of these are very easy to prove; here’s my favorite example. An agent has a fixed utility function and performs Pareto-optimally on that utility function across multiple worlds (so “utility in each world” is the set of objectives). Then there’s a normal vector (or family of normal vectors) to the Pareto surface at whatever point the agent achieves. (You should draw a picture at this point in order for this to make sense.) That normal vector’s components will all be nonnegative (because Pareto surface), and the vector is defined only up to normalization, so we can interpret that normal vector as a probability distribution. That also makes sense intuitively: larger components of that vector (i.e. higher probabilities) indicate that the agent is “optimizing relatively harder” for utility in those worlds. This says nothing at all about how the agent will update, and we’d need a another couple sentences to argue that the agent maximizes expected utility under the distribution, but it does give the prototypical mental picture behind the “Pareto-optimal → probabilities” idea.

  Here is an example (to point out a missing assumption): Lets say you are offered to bet on the result of a coin flip for $1$ dollar. You get $3$ dollars if you win, and your utility function is linear in dollars. You have three actions: “Heads”, “Tails”, and “Pass”. Then “Pass” performs Pareto-optimally across multiple worlds. But “Pass” does not maximize expected utility under any distribution.

  I think what is needed for the result is an additional convexity-like assumption about the utilities. This could be the set of achievable utility vectors is convex'', or even something weaker like every convex combination of achievable utility vectors is dominated by an achievable utility vector” (here, by utility vector I mean $(u_w{)}_{w\in W}$ if $u_w$ is the utility of world $w$). If you already accept the concept of expected utility maximization, then you could also use mixed strategies to get the convexity-like assumption (but that is not useful if the point is to motivate using probabilities and expected utility maximization).

  Or: even if you do expect powerful agents to be approximately Pareto-optimal, presumably they will be approximately Pareto optimal, not exactly Pareto-optimal. What can we say about coherence then?

  The underlying math statement of some of these kind of results about Pareto-optimality seems to be something like this:

  If $\overline{x}$ is Pareto-optimal wrt utilities $u_i$, $i=1,\dots n$ and a convexity assumption (e.g. the set $\left\{\right(u_i\left(x\right){)}_{i=1}^n:x\}$ is convex, or something with mixed strategies) holds, then there is a probability distribution $\mu$ so that $\overline{x}$ is optimal for $U\left(x\right)=E_{i\sim \mu }{u}_i\left(x\right)$.

  I think there is a (relatively simple) approximate version of this, where we start out with approximate Pareto-optimality.

  We say that $\overline{x}$ is Pareto$\epsilon$—optimal if there is no (strong) Pareto-improvement by more than $\epsilon$ (that is, there is no $x$ with $u_i\left(x\right)>u_i\left(\overline{x}\right)+\epsilon$ for all $i$).

  Claim: If $\overline{x}$ is Pareto$\epsilon$—optimal and the convexity assumption holds, then there is a probability distribution $\mu$ so that $\overline{x}$ is $\epsilon$-optimal for $U\left(x\right)=E_{i\sim \mu }{u}_i\left(x\right)$.

  Rough proof: Define $Y:=\left\{\right(u_i\left(x\right){)}_{i=1}^n:x\}$ and $\overline{Y}$ as the closure of $Y$. Let $\tilde{y}\in \overline{Y}$ be of the form $\tilde{y}=\left(u_i\right(\overline{x})+\delta {)}_{i=1}^n$ for the largest $\delta$ such that $\tilde{y}\in \overline{Y}$. We know that $\delta \le \epsilon$. Now $\tilde{y}$ is Pareto-optimal for $Y$, and by the non-approximate version there exists a probability distribution $\mu$ so that $\tilde{y}$ is optimal for $y\mapsto E_{i\sim \mu }{y}_i$. Then, for any $x$, we have $\mathbb{E}{i\sim\mu} u_i(x) \leq \mathbb{E}{i\sim\mu} \tilde y_i = \mathbb{E}{i\sim\mu} (u_i(\bar x) + \delta)\le \varepsilon + \mathbb{E}{i\sim\mu} u_i(\bar x), $ that is, $\overline{x}$ is $\epsilon$-optimal for $U$.

• harfe 23 Jan 2025 17:03 UTC

  LW: 13 AF: 9

  2

  AF

  in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform

  I think there are some subtleties with the (non-infra) bayesian VNM version, which come down to the difference between “extreme point” and “exposed point” of $D$. If a point is an extreme point that is not an exposed point, then it cannot be the unique expected utility maximizer under a utility function (but it can be a non-unique maximizer).

  For extreme points it might still work with uniqueness, if, instead of a VNM-decision-maker, we require a slightly weaker decision maker whose preferences satisfy the VNM axioms except continuity.

• harfe 22 Jan 2025 16:27 UTC

  LW: 12 AF: 8

  2

  AF

  in reply to: Vanessa Kosoy’s comment on: Vanessa Kosoy’s Shortform

  For any $\Phi ,\Psi \in \hat{D}$, if ${\Theta }^{\ast }=\Phi \vee \Psi$ then either $\Phi \subseteq \Psi$ or $\Psi \subseteq \Phi$.

  I think this condition might be too weak and the conjecture is not true under this definition.

  If ${\Phi }_1\subseteq {\Phi }_2$, then we have $E_{y\sim \xi }{min}_{\mu \in {\Phi }_2}{E}_{x\sim \mu }u(x,y)\le E_{y\sim \xi }{min}_{\mu \in {\Phi }_1}{E}_{x\sim \mu }u(x,y)$ (because a minimum over a larger set is smaller). Thus, ${\Phi }_2$ can only be the unique argmax if ${\Phi }_1={\Phi }_2$.

  Consider the example $\hat{D}=\left\{\right[0,x]:x\in [0,1\left]\right\}$. Then $\hat{D}$ is closed. And ${\Theta }^{\ast }=[0,1]$ satisfies ${\Theta }^{\ast }=\Phi \vee \Psi ⟹\Phi \subseteq \Psi \vee \Psi \subseteq \Phi$. But per the above it cannot be a unique maximizer.

  Maybe the issue can be fixed if we strengthen the condition so that ${\Phi }^{\ast }$ has to be also minimal with respect to $\subseteq$.

• harfe 6 Dec 2024 13:52 UTC

  1 point

  0

  in reply to: quila’s comment on: quila’s Shortform

  For a provably aligned (or probably aligned) system you need a formal specification of alignment. Do you have something in mind for that? This could be a major difficulty. But maybe you only want to “prove” inner alignment and assume that you already have an outer-alignment-goal-function, in which case defining alignment is probably easier.

• harfe 15 Nov 2024 11:09 UTC

  7 points

  2

  in reply to: Garrett Baker’s comment on: D0TheMath’s Shortform

  insofar as the simplest & best internal logical-induction market traders have strong beliefs on the subject, they may very well be picking up on something metaphysically fundamental. Its simply the simplest explanation consistent with the facts.

  Theorem 4.6.2 in logical induction says that the “probability” of independent statements does not converge to $1$ or $0$, but to something in-between. So even if a mathematician says that some independent statement feels true (eg some objects are “really out there”), logical induction will tell him to feel uncertain about that.