Anonymous

Karma: 145

Anonymous 27 Jan 2023 15:47 UTC
3 points
2
in reply to: Lone Pine’s comment on: All AGI Safety questions welcome (especially basic ones) [~monthly thread]
Normal, standard causal decision theory is probably it. You can make a case that people sometimes intuitively use evidential decision theory (“Do it. You’ll be glad you did.”) but if asked to spell out their decision making process, most would probably describe causal decision theory.

Anonymous 10 Dec 2022 8:00 UTC
3 points
0
in reply to: localdeity’s comment on: Where’s the economic incentive for wokism coming from?
Fandom people on Tumblr, AO3, etc. really responded to The Last Jedi (because it was targeted to them). Huge phenomenon. There are now bestselling romance novels that started life as TLJ fanfiction. Everything worked just like it does for the Marvel movies, very profitably.
However there was an additional group of Star Wars superfans outside of fandom, who wanted something very different, hence the backlash. This group is somewhat more male and conservative, and then everything polarized on social media so this somehow became a real culture war issue. Of course, Disney did not like the backlash, and tried to make the 3rd movie more palatable to this group.
That kind of fan doesn’t organically exist for most things outside of Star Wars though. For most things, you only get superfans in this network of fan communities which skew towards social justice. And for any new genre story without a pre-existing fanbase, there’s an opportunity to get fandom people excited about it, which is very valuable.

Anonymous 9 Dec 2022 13:12 UTC
1 point
0
on: Where’s the economic incentive for wokism coming from?
As far as running a media company goes, fandom is extremely profitable, increasingly so in an age where enormous sci-fi/fantasy franchises drive everything. And there’s been huge overlap between fandom communities and social justice politics for a long time.
It’s definitely in Disney’s interest to appeal to Marvel superfans who write fanfiction and cosplay and buy tons of merchandise, and those people tend to also be supporters of social justice politics.
Like, nothing is being forced on this audience—there are large numbers of people who get sincerely excited when a new character is introduced that gives representation for the first time to a new minority group, or something like that.
As with so many businesses, the superfans are worth quite a few normies who might be put off by this. I think this is the main explanation.

Anonymous 8 Dec 2022 18:54 UTC
8 points
7
in reply to: LawrenceC’s comment on: Where to be an AI Safety Professor
The “canonical” rankings that CS academics care about would be csrankings.org (also not without problems but the least bad).

Anonymous 3 Nov 2022 17:11 UTC
1 point
0
in reply to: MathiasKB’s comment on: Adversarial Policies Beat Professional-Level Go AIs
The KataGo paper says of its training, “Self-play games used Tromp-Taylor rules modified to not require capturing stones within pass-alive territory”.
It sounds to me like this is the same scoring system as used in the adversarial attack paper, but I don’t know enough about Go to be sure.

Anonymous 23 Oct 2022 9:41 UTC
2 points
0
on: The Last Year - is there an existing novel about the last year before AI doom?
The Sprawl trilogy by William Gibson (starting with Neuromancer) is basically about this, and is a classic for a reason. It’s not exactly hard sci-fi though.

Anonymous 10 Oct 2022 17:04 UTC
7 points
0
on: Signaling Guilt
If you don’t signal the expected way then you are, if not being dishonest, at least misleading people — in many cases it is less honest.

Everyone knows your job application is written to puff you up, and they price it in. If you don’t have the correct amount of puffery, you’re misleading people into thinking you’re worse than you are.

It’s a bad way to communicate and a bad race-to-the-bottom equilibrium but not actually dishonest.

You can write “Dear X” on a letter to a person you don’t know. People used to sign off letters “Your obedient servant”. It evolves for weird signaling reasons but is not taken literally.

Anonymous 10 Oct 2022 13:32 UTC
1 point
0
on: The Teacup Test
“Systems that would adapt their policy if their actions would influence the world in a different way”
Does the teacup pass this test? It doesn’t necessarily seem like it.
We might want to model the system as “Heat bath of Air → teacup → Socrates’ tea”. The teacup “listens to” the temperature of the air on its outside, and according to some equation transmits some heat to the inside. In turn the tea listens to this transmitted heat and determines its temperature.
You can consider the counterfactual world where the air is cold instead of hot. Or the counterfactual world where you replace “Socrates’ tea” with “Meletus’ tea”, or with a frog that will jump out of the cup, or whatever. But in all cases the teacup does not actually change its “policy”, which is just to transmit heat to the inside of the cup according to the laws of physics.
To put it in the terminology of “Discovering Agents”, one can add mechanism variables $M_{a}, M_{c}, M_{t}$ going into the object level variables. But there are no arrows between these, so there’s no agent.
Of course, my model here is bad and wrong physically speaking, even if it does capture crude cause-effect intuition about the effect of air temperature on beverages. However I’d be somewhat surprised if a more physically correct model would introduce an agent to the system where there is none.

Anonymous 25 Sep 2022 10:16 UTC
1 point
0
in reply to: Algon’s comment on: Funding is All You Need: Getting into Grad School by Hacking the NSF GRFP Fellowship
There are industry places that will, at least as stated, take you seriously with no PhD as long as you have some publications (many job postings don’t require a PhD or say “or equivalent research experience”), and it’s unusual but not unheard of for people do this.
The thing is, a PhD program is a reliable way to build a research track record. And you don’t see too many PhD dropouts who want to be scientists because if you’ve got a research track record, the extra cost of just finishing your dissertation and graduating is pretty low.

Anonymous 24 Jul 2022 3:54 UTC
6 points
1
on: A Bias Against Altruism
People sometimes seem to act like unsolved problems are exasperating, aesthetically offensive, or somehow unappealing, so they have no choice but to roll up their sleeves and try to help fix them, because it’s just so irritating to see the problem go unsolved. So one can do purely altruistic stuff, but with this selfish posture (which also shifts focus away from motivation and psychology) it won’t trip the hypocrisy alarms. It may also genuinely be a better attitude to cultivate, if it helps deflate one’s ego a little bit—I’m not quite sure.

Anonymous 7 Jun 2022 23:34 UTC
12 points
0
on: AGI Safety FAQ / all-dumb-questions-allowed thread
A lot of the AI risk arguments seem to come mixed together with assumptions about a particular type of utilitarianism, and with a very particular transhumanist aesthetic about the future (nanotech, von Neumann probes, Dyson spheres, tiling the universe with matter in fixed configurations, simulated minds, etc.).
I find these things (especially the transhumanist stuff) to not be very convincing relative to the confidence people seem to express about them, but they also don’t seem to be essential to the problem of AI risk. Is there a minimal version of the AI risk arguments that are disentangled from these things?

Anonymous 2 Jun 2022 2:24 UTC
8 points
on: Reshaping the AI Industry
most academic research work is done by grad students, and grad students need incremental, legible wins to put on their CV so they can prove they are capable of doing research. this has to happen pretty fast. an ML grad student who hasn’t contributed to any top conference papers by their second or third year in grad school might get pulled aside for a talk about their future.
ideally you want a topic where you can go from zero to paper in less than a year, with multiple opportunities for followup work. get a few such projects going and you have a very strong chance of getting at least one through in time to not get managed out of your program—and of course, usually more will succeed and you’ll be doing great.
I don’t think there’s anything like this in AI safety research. Section 3.4 seems to acknowledge this a little bit. If you want AI safety to become more popular, you’d hope that an incoming PhD student could say “I want to work on AI Safety” and be confident that in a year or two, they’ll have a finished research project that they can claim as a success and submit to a top venue. Otherwise, they are taking a pretty huge career risk, and most people won’t take it.

Anonymous 20 May 2022 1:43 UTC
3 points
in reply to: DirectedEvolution’s comment on: What’s up with the recent monkeypox cases?
“Does the disease heavily affect career-age people (age 25-65), or frequently leave survivors with lasting disability?”

This is rightly ticked off as “No”, but I think it morally counts as “Yes” if there is more danger to young children. That’s scarier in itself, and from COVID it seems people are also more likely to accept very extreme NPIs to protect children, meaning there might well be a large economic impact.

Anonymous 7 May 2022 18:08 UTC
11 points
on: Is there a convenient way to make “sealed” predictions?
Historically, scientists would use anagrams to do this. Galileo famously said “Smaismrmilmepoetaleumibunenugttauiras”. Later he revealed that it could be unscrambled into “Altissimum planetam tergeminum observavi” which per Wikipedia is Latin for “I have observed the most distant planet to have a triple form”, establishing his priority in discovering the rings of Saturn.
Obviously hashing and salting is better, nowadays.

Anonymous 13 Mar 2022 14:26 UTC
3 points
in reply to: Charlie Steiner’s comment on: Book review of “Mengzi”
From my limited knowledge, that’s definitely one of the purposes Ruism/Confucianism was put to—especially once the civil service exams were instituted.
In one way, “philosophy of the establishment” seems mostly correct to me, as the Mengzi seemingly makes a core assumption that the current social order is legitimate. But it mostly isn’t making excuses for that social order (as philosophy and social science often does), it’s challenging rulers to live up to an ideal and serve the people. At one point, Mengzi says that any king who “mutilates benevolence” is a “mere fellow” who can be rightfully executed by the people.
And I don’t know enough about history, but it seems like nearly every Chinese philosopher of any school—even maybe Zhuangzi (??) -- was involved in some kind of government position. Maybe that’s what every literate person did. So it’s hard to draw a clear “establishment/anti-establishment” line.

Book review of “Mengzi”

Anonymous12 Mar 2022 4:54 UTC

31 points

2 comments12 min readLW link

Anonymous 4 Mar 2022 1:41 UTC
9 points
on: Introducing myself: Henry Lieberman, MIT CSAIL, whycantwe.org
Schizophrenia is the wrong metaphor here—it’s not the same disease as split personalities (i.e. dissociative identity disorder). I think it would be clearer and more accurate to rewrite that paragraph without it. I don’t intend this as an attack or harsh criticism, it’s just that I have decided to be a pedant about this point whenever I encounter it, as I think it would be good for the general public to develop a more accurate and realistic understanding of schizophrenia.

Anonymous 21 Feb 2022 2:00 UTC
12 points
in reply to: Alexander Gietelink Oldenziel’s comment on: 12 interesting things I learned studying the discovery of nature’s laws
Rubin’s framework says basically, suppose all our observations are in a big data table. Now consider the counterfactual observations that didn’t happen (i.e. people in the control group getting the treatment) -- these are called “potential outcomes”—treat those like missing cells in the data table. Then causal inference is just to fill in potential outcomes using missing data imputation techniques, although to be valid these require some assumptions about conditional independence.
Pearl’s framework and Rubin’s are isomorphic in the sense that any set of causal assumptions in Pearl’s framework (a structural causal model, which has a DAG structure), can be translated into a set of causal assumptions in Rubin’s framework (a bunch of conditional independence assumptions about potential outcomes), and vice versa. This is touched on somewhat in Ch. 7 of “Causality”.
Pearl argues that despite this equivalence, his framework is superior because it’s a better tool for thinking. In other words, writing down your assumptions as DAG/SCM is intuitive and can be explained and argued about, while he claims the Rubin model independence assumptions are opaque and hard to understand.

Anonymous 20 Feb 2022 21:17 UTC
48 points
on: 12 interesting things I learned studying the discovery of nature’s laws
I will give a potted history of Pearl’s discovery as I understand it.
In the late 70s/early 80s, people wanted to deal with uncertainty in logic-based AI. The obvious thing to use is probability, but doing a Bayesian update to compute a posterior is exponentially expensive.
Pearl wanted to come up with a good data structure for doing computations over probability distributions in less-than-exponential time.
He introduced the idea of Bayesian networks in his paper Reverend Bayes On Inference Engines where he represents factorized probability distributions using DAGs. Here, the direction of the arrows is arbitrary and there are many DAGs corresponding to one probability distribution.
He was not thinking about causality at all, it was just a problem in data structures. The idea was this would be used for the same sort of thing as an “expert system” or other logic based AI systems, but taking into account uncertainty expressed probabilistically.
Later, people including Pearl noticed that you can and often should interpret the arrows as causal, this amounts to choosing one DAG from many. The fact that there are many possible DAGs is related to the fact that there are seemingly always multiple incompatible causal stories, to explain observations absent making additional assumptions about the world. But if you pick one, you can start using it to see whether your causal question can be answered from observational data alone.
Finally, he realized that the assumptions encoded in a DAG aren’t sufficient for fully general counterfactuals, and realized that in full generality you have to specify exactly what functional relationship goes along each edge of the graph.
As someone originally concerned with AI, not with problems in the natural sciences, Pearl is probably unusual. Pearl himself looks back on Sewall Wright as his progenitor for coming up with path diagrams—he was working in genetics. If you are interested in this, you should also look at Don Rubin’s experience—his causal framework is isomorphic to Pearl’s. He was a 100 percent classic statistician, motivated by looking at medical studies.

Anonymous

Book re­view of “Mengzi”

Book review of “Mengzi”