Rafael Harth

Karma: 2,136

Rafael Harth 21 Jul 2021 16:06 UTC
2 points
in reply to: TAG’s comment on: Is the argument that AI is an xrisk valid?
I think that’s a decent argument about what models we should build, but not an argument that AI isn’t dangerous.

Rafael Harth 21 Jul 2021 14:52 UTC
2 points
in reply to: TAG’s comment on: Is the argument that AI is an xrisk valid?

No necessarily, since AIs can be WBEs or otherwise anthropomorphic. An AI with an explicitly coded goal is possible , but not the only kind.

While I think this is 100% true, it’s somewhat misleading as a counter-argument. The single-goal architecture of one model of AI that we understand, and a lot of arguments focus on how that goes wrong. You can certainly build a different AI, but that comes at the price of opening yourself up to a whole different set of failure modes. And (as far as I can see), it’s also not what the literature is up to right now.

Rafael Harth 19 Jul 2021 20:14 UTC
7 points
in reply to: Steven Byrnes’s comment on: Is the argument that AI is an xrisk valid?
Reading this, I feel somewhat obligated to provide a different take. I am very much a moral realist, and my story for why the quoted passage isn’t a good argument is very different from yours. I guess I mostly want to object to the idea that [believing AI is dangerous] is predicated on moral relativism.

Here is my take. I dispute the premise:

In the proposed picture of singularity claim & orthogonality thesis, some thoughts are supposed to be accessible to the system, but others are not. For example:

I’ll grant that most of the items on the inaccessible list are, in fact, probably accessible to an ASI, but this doesn’t violate the orthogonality thesis. The Orthogonality thesis states that a system can have any combination of intelligence and goals, not that it can have any combination of intelligence and beliefs about ethics.

Thus, let’s grant that an AI with a paperclip-like utility function can figure out #6-#10. So what? How is [knowing that creating paperclips is morally wrong] going to make it behave differently?

You (meaning the author of the paper) may now object that we could program an AI to do what is morally right. I agree that this is possible. However:

(1) I am virtually certain that any configuration of maximal utility doesn’t include humans, so this does nothing to alleviate x-risks. Also, even if you subscribe to this goal, the political problem (i.e., convincing AI people to implement it) sounds impossible.

(2) We don’t know how to formalize ‘do what is morally right’.

(3) If you do black box search for a model that optimizes for what is morally right, this still leaves you with the entire inner alignment problem, which is arguably the hardest part of the alignment problem anyway.

Unlike you (now meaning Steve), I wouldn’t even claim that letting an AI figure out moral truths is a bad approach, but it certainly doesn’t solve the problem outright.

Rafael Harth 19 Jul 2021 12:58 UTC
4 points
on: Rafael Harth’s Shortform
Instead of explaining something to a rubber duck, why not explain it via an extensive comment? Maybe this isn’t practical for projects with multiple people, but if it’s personal code, writing it down seems better as a way to force rigor from yourself, and it’s an investment into a possible future in which you have to understand the code once again.

Rafael Harth 17 Jul 2021 7:44 UTC
2 points
in reply to: Przemysław Czechowski’s comment on: How to Sleep Better
Yeah, I use wax earplugs every night. The downside is that they’re gross and they sometimes cause itches, but they always go away if I scratch a bit. The upside is that you cut out a large portion of unexpected sounds, which to me way outweighs the problems.

Rafael Harth 7 Jul 2021 22:24 UTC
2 points
in reply to: AllAmericanBreakfast’s comment on: Chess and cheap ways to check day to day variance in cognition
I don’t think this is in conflict with what I said. Blundering is itself a matter of luck. You can have both players play sufficiently reckless that they risk 1-move blunders, but then one person gets lucky that they’ve never made one. I assume you’re familiar with the situation where you make a move that’s only not a blunder due to $X$ where $X$ is something you didn’t think about while making the move.

Also, I don’t know how literal you meant your post, but I don’t think it’s true that you can get there by ‘just’ avoiding one-move blunders. If your positional play is sufficiently worse than your opponent’s, you should lose even if your opponent blunders a minor piece at some point and you don’t. I think it’s more like, most people around 1500 are somewhat even in positional skill, and you can separate yourself from them by avoiding blunders.

(Out of curiosity (or perhaps because I’ll challenge you), what time format are you 1550 in?)

Rafael Harth 7 Jul 2021 19:28 UTC
4 points
on: Chess and cheap ways to check day to day variance in cognition
In addition to variance of opponents’ skill, there is also a substantial luck factor. Any search algorithm that can’t search the entire tree has uncertainty about how good moves are, which effectively translates into luck.

I think there’s a good chance your observations are still valid (also because you have intuition about how well you play on top of results), but it is a possible factor.

Rafael Harth 5 Jul 2021 11:08 UTC
4 points
in reply to: Viliam’s comment on: Rafael Harth’s Shortform
The IQ objection is a really good one that hasn’t occurred to me at all. Although I’d have estimated less than half as large of a difference.

On maintaining order, it’s worth pointing out that insofar as this is the relative strength of the highschool teacher, it probably doesn’t have much to do with what the teacher learned from the literature.

Rafael Harth 3 Jul 2021 19:50 UTC
2 points
in reply to: Viliam’s comment on: Rafael Harth’s Shortform
I did mean both. Comparing just tutoring to just regular school would be pretty unfair.

Rafael Harth 3 Jul 2021 19:20 UTC
2 points
in reply to: Dagon’s comment on: Rafael Harth’s Shortform

Do you mean that smart untrained people would teach an average high school class better than a trained teacher?

Yeah.

“the same” in math or physics is about learning the topic, or learning to teach the topic.

It’s mostly like applying the knowledge somewhere. Suppose you have to solve a real problem that requires knowing physics.

Of course you can also read the literature, but my post was about when it’s possible to do better without having done so.

Rafael Harth 3 Jul 2021 11:21 UTC
10 points
on: Rafael Harth’s Shortform
It seems to me that many smart people could ignore the existing literature on pedagogy entirely and outperform most people who have obtained a formal degree in the area (like highschool teachers), just by relying on their personal models. Conversely, I’d wager that no-one could do the same in physics, and (depending on how ‘outperforming’ is measured) no-one or almost no-one could do it in math.

I would assume most people on this site have thought about this kind of stuff, but I don’t recall seeing many posts about it, and I don’t anyone sharing their estimates for where different fields place on this spectrum.

There is some discussion for specific cases like prediction markets, covid models, and economics. And now that I’m writing this, I guess Inadequate Equilibria is a lot about answering this question, but it’s only about the abstract level, i.e., how do you judge the competence of a field, not about concrete results. Which I’ll totally grant is the more important part, but I still feel like comparing rankings of fields on this spectrum could be valuable (and certainly interesting).

Rafael Harth 23 Jun 2021 14:01 UTC
3 points
in reply to: chaosmage’s comment on: Irrational Modesty

And why would you refuse to seek justified status?

One reason would be that seeking status will lead to you having less of it, which I strongly think is true insofar as ‘seeking’ means ‘having it as the driving motivation’. Think about how many of the high-status people in the rationalist sphere are relatively status-blind. If we draw from fiction, note that this is true for Harry in hpmor, too.

It’s also not always true that high status people are worth imitiating or listening to, but I would agree if you just meant on average.

Rafael Harth 22 Jun 2021 12:27 UTC
17 points
on: Less Wrong needs footnotes
You can do this if you write with the markdown editor.^[1] You can activate it in the user settings.
1. Like this! ↩︎

Rafael Harth 19 Jun 2021 11:28 UTC
3 points
on: Covid vaccine safety: how correct are these allegations?
This is the video, right? You could link to that instead of the removed youtube link.

Rafael Harth 19 Jun 2021 11:22 UTC
2 points
in reply to: DPiepgrass’s comment on: Covid vaccine safety: how correct are these allegations?

This is relevant because I read that “FDA requires healthcare providers to report any death after COVID-19 vaccination to VAERS”

How does this square with OpenVaers’s claim that only about 1% of injuries are reported?

Without knowing the reporting rate, it’s difficult to interpret the data. If we take the 1% and 5869 numbers at face value, it implies that the vaccines killed about 560.000 people, whereas if we assume 100% reporting rate, it looks like they’re an amazing preventer of unrelated causes of death. Is there any reasonable way to estimate what % to use?

Rafael Harth 18 Jun 2021 8:41 UTC
LW: 2 AF: 1
AF
on: Open problem: how can we quantify player alignment in 2x2 normal-form games?
I’ll take a shot at this. Let $A$ and $B$ be the sets of actions of Alice and Bob. Let $o_{n} : B \to {1, . . . n}$ (where ‘n’ means ‘nice’) be function that orders $B$ by how good the choices are for Alice, assuming that Alice gets to choose second. Similarly, let $o_{s} : B \to {1, . . ., n}$ (where ‘s’ means ‘selfish’) be the function that orders $B$ by how good the choices are for Bob, assuming that Alice gets to choose second. Choose some function $ψ$ measuring similarity between two orderings of a finite set (should range over $[- 1, 1]$ ); the alignment of $B$ with $A$ is then $ψ (o_{n}, o_{s})$ .

Example: in the prisoner’s dilemma, $B = {c, d}$ , and $o_{n}$ orders $c > d$ whereas $o_{s}$ orders $d > c$ . Hence $ψ (o_{n}, o_{m})$ should be $- 1$ , i.e., Bob is maximally unaligned with Alice. Note that this makes it different from Mykhailo’s answer which gives alignment $0.5$ , i.e., medium aligned rather than maximally unaligned.

This seems like an improvement over correlation since it’s not symmetrical. In the game where Alice and Bob both get to choose numbers $x, y \in {1, 2}$ and Alice’s utility function outputs $y + x$ whereas Bob’s outputs $y - x$ , Bob would be perfectly aligned with Alice (his $o_{n}$ and $o_{s}$ both order $2 > 1$ ) but Alice perfectly unaligned with Bob (her $o_{n}$ orders $1 > 2$ but her $o_{s}$ orders $2 > 1$ ).

I believe this metric meets criteria 1,3,4 you listed. It could be changed to be sensitive to players’ decision theories by changing $o_{s}$ (for alignment from Bob to Alice) to be the order output by Bob’s decision theory, but I think that would be a mistake. Suppose I build an AI that is more powerful than myself, and the game is such that we can both decide to steal some of the other’s stuff. If the AI does this, it leads to −10 utils for me and +2 for it (otherwise ⁰⁄₀); if I do it, it leads to −100 utils for me because the AI kills me in response (otherwise ⁰⁄₀). This game is trivial: the AI will take my stuff and I’ll do nothing. Also, the AI is maximally unaligned with me. Now suppose I become as powerful as the AI and my ‘take AI’s stuff’ becomes −10 for AI, +2 for me. This makes the game a prisoner’s dilemma. If we both run UDT or FDT, we would now cooperate. If $o_{s}$ is the ordering of the AI’s decision theory, this would mean the AI is now aligned with me, which is odd since the only thing that changed is me getting more powerful. With the original proposal, the AI is still maximally unaligned with me. More abstractly, game theory assumes your actions have influence on the other player’s rewards (else the game is trivial), so if you cooperate for game-theoretical reasons, this doesn’t seem to capture what we mean by alignment.

Rafael Harth 7 Jun 2021 19:50 UTC
2 points
in reply to: chinese4’s comment on: The dumbest kid in the world (joke)
You only have two votes right now, but they counted for −10, so probably 2 strong downvotes. You can see the number of votes by hovering your mouse over the number.

Rafael Harth 7 Jun 2021 14:22 UTC
8 points
in reply to: Davidmanheim’s comment on: We need a standard set of community advice for how to financially prepare for AGI
This seems like a very surprising claim to me. You can make money on stocks by knowing things above pure chance. Do you really think that $P (stock rises | AGI) \approx P (stock rises)$ for all stocks?

Rafael Harth 7 Jun 2021 13:28 UTC
18 points
on: Often, enemies really are innately evil.
I don’t believe a significant percentage of people is innately evil, and at the end of part I of this post, I don’t think you’ve given me significant evidence to chance my mind. The study is not convincing, and not because of effect size—people could have misunderstood the game or just pressed the red button for fun since we’re talking about cents. I would have predicted few people to press the red button if the payouts were significant (thinking at least 100$ difference); I genuinely don’t know what I would have predicted for the game as-is.

there are many, many, MANY more pieces of evidence from (almost) every internet troll, bully, and rapist, and many other criminals too.

I mean, rape has a pretty obvious advantage for the rapist. “Troll” is so overloaded that I think you’d have to define it before I can consider it seriously for anything. Bullying is the most convincing case, but my model of bullies, especially if they’re young, isn’t that they’re innately evil. If I remember correctly, I have participated in bullying a couple of times before thinking about it and deciding that it’s morally indefensible. I imagine most bullies are similar except that they skipped the part where they think about it, or that they have thought about it, maybe decided to stop, but then proceeded anyway because the instinct was too strong.

Anyway, this is quite speculative, but my point is that I don’t think you’re making a strong case for your leading claim. I realize that this may come across as nitpicking details in a mountain of obvious evidence, but that’s often just how it feels like if someone doubts what you consider an obvious truth.

There’s also an issue that we may have different ideas of what ‘innately evil’ means.

Rafael Harth 6 Jun 2021 8:03 UTC
5 points
in reply to: gilch’s comment on: The dumbest kid in the world (joke)
If you just cut everything from “Later” in the third-to-last paragraph onward, smart readers would probably still get it but it would be less obvious.