Soapspud

Karma: 106

Soapspud 11 Oct 2025 3:48 UTC
1 point
0
in reply to: VojtaKovarik’s comment on: Irresponsible Companies Can Be Made of Responsible Employees
it might not work as explicitly as people knowingly paying attention to it. It could be implicitly present in something else (“culture fit”)

Yes, that sounds plausible to me as well. I did not mention those because I found it much harder to think of ways to tell when those dynamics are actually in play.

If I understand this correctly:

FWIW, I think this concern is important, but we need to be very cautious about it. Since one could always go “yeah, your results say nothing like this is going on, but the real mechanism is even more indirect”.

I think you are gesturing at the same issue?

I imagine that the fields like fairness & bias have to encounter this a lot, so they might be some insights.

It makes sense to me that the implicit pathways for these dynamics would be an area of interest to the fields of fairness and bias. But I would not expect them to have any better tools for identifying causes and mechanisms than anyone else^[1]. What kinds of insights would you expect those fields to offer?
1. ↩︎
  To be clear, I have only a superficial awareness of the fields of fairness and bias.

Soapspud 8 Oct 2025 17:23 UTC
2 points
0
on: Irresponsible Companies Can Be Made of Responsible Employees
But I would be curious to know what are the dynamics that actually take place, which of them matter how much, and what is the overall effect.
I would be interested to know more about those too! However, I don’t have any direct experience with the insides of AI companies, and I don’t have any friends who do either, so I’m hoping that other readers of this post might have insights that they are willing to share.
For those who have worked for AI companies or have reliable info from others who have worked for an AI company, these are a few things I am especially curious about, categorised by the mechanisms mentioned in the post:
Selective hiring
- What kinds of personality traits are application-reviewers and interviewers paying attention for? To what extent are the following traits advantages or disadvantages to candidates getting hired?
  - bias to action?
  - agreeableness?
  - outspokenness?
  - conscientiousness?
- Ignoring any explicit criteria that are supposed to factor into the decision to hire someone, what was the typical distribution of personality traits in people who were actually hired?
- What about the distribution of traits in people who continued to work there for longer than two years?
- To what extent are potential hires expected to have thought about the wider impacts of the work they will be doing?
- Does an applicant’s level of concern on catastrophic risk typically come up during the hiring process? Is that a thing that factors into the decision to hire them? (Ignoring the question of whether it should be a factor for now.)
Firing dissenters as a coordination problem
- How often have employees had consequential disagreements with the team or company’s direction and voiced it?
- How often have employees had consequential disagreements with the team or company’s direction and not voiced it? Why not?
- How often do employees in the company see their colleagues express consequential disagreements with the work they are doing?
- How often have you seen employees express consequential disagreements and then have them addressed to their satisfaction?
- Among the employees who have expressed consequential disagreements at some point, how many of them work on the same team two years later?
  - How many of them still work at the company at all two years later?
Compartmentalising undesirable information (and other types of information control)
- How long has the average Risk Team member worked for that team?
- How often do people on the teams addressing risks talk to people outside their teams about the implications of their work?
- How often do people working on the teams addressing risks find themselves softening or otherwise watering down their risk assessments/recommendations when communicating with people outside their teams?

Soapspud 27 Aug 2025 0:55 UTC
10 points
0
in reply to: aaaaaaaaaaa’s comment on: A Comprehensive Guide to Running
I’m a new-ish runner, and I think I had a similar problem when I started. I couldn’t jog slowly enough to not be immediately winded. I say immediately, but I could go for a few tens of seconds, I think? Less than a minute, and I would be surprised if it was even 30 seconds.

So, I started jogging for just long enough that I was touching the edge of being out of breath, but nothing really hurt, and then I would walk until I caught my breath and felt like I could do another short jog.

I still can’t jog for that long, but it’s way longer than before; and now I can go harder once or twice a week and it feels pleasantly intense rather than intensely painful.

Also, I don’t want to make it sound like a heart-rate monitor is necessary, because I don’t think it is, but I found that getting a (reliable) heart-rate monitor was really illuminating for me. In my case, there was initially just no jogging pace that would not put me well above zone 2. I experimented a bit and found that speed-walking was the only way I could stay in zone 2, and even then, I would slowly climb up into 3 by the end of my route. (Even more interesting was finding out that, for me, that “Gah! I’m dying!” feeling was me smacking into zone 5, which I can now use to calibrate the intensity of my harder runs.)

One of the more valuable insights I got from my heart-rate monitor was that I was trying to train at a way higher intensity than my fitness level allowed, which helped me chill out and stop injuring myself so often.

Soapspud 29 May 2024 20:19 UTC
73 points
61
on: MIRI 2024 Communications Strategy
We understand that we may be discounted or uninvited in the short term, but meanwhile our reputation as straight shooters with a clear and uncomplicated agenda remains intact.

I don’t have any substantive comments, but I do want to express a great deal of joy about this approach.

I am really happy to see people choosing to engage with the policy, communications, and technical governance space with this attitude.

Soapspud 11 Apr 2024 19:24 UTC
5 points
4
on: RTFB: On the New Proposed CAIP AI Bill
We only have people who cry wolf all the time. I love that for them, and thank them for their service, which is very helpful. Someone needs to be in that role, if no one is going to be the calibrated version. Much better than nothing. Often their critiques point to very real issues, as people are indeed constantly proposing terrible laws.
The lack of something better calibrated is still super frustrating.

This mental (or emotional) move here, where you manage to be grateful for people doing a highly imperfect job while also being super frustrated that no one is doing a genuinely good job: how are you doing that?

I see this often in rationalist spaces, and I’m confused about how people learn to do this. I would probably end up complaining about the failings of the best (highly inadequate) strategies we’ve got without the additional perspective of “how would things be if we didn’t even have this?”

For people who remember learning how to do this, how did you practice?

Soapspud 16 Oct 2023 3:12 UTC
5 points
0
in reply to: Jacob G-W’s comment on: Noticing confusion in physics
It’s definitely intuition gained from a few years of doing those kinds of problems.

Also, there’s an important point that makes my intuition a bit less impressive, and it’s the fact that the problem-statement sounded like an intro-physics problem, so I assumed away many ambiguities that would make the solution particularly complicated to think through.

For example, thought it’s not specified, it matters whether your gas is in a fixed volume or not, but if you assume the gas can expand, you’re getting into solutions where you might need to know the boundary conditions at the edge of your gas, and you might need to figure out relative pressures and/or temperature gradients. Since the question doesn’t specify any of that, I guessed that it’s probably not that kind of problem.

Soapspud 13 Oct 2023 15:43 UTC
2 points
0
on: Noticing confusion in physics
My thought after reading the first sentence of your post, and before reading any of the comments, was that gases become less compressible at higher temperatures, which should make it more responsive to a pressure-wave, raising the speed of sound in that medium.

Soapspud 1 Aug 2023 21:54 UTC
6 points
1
on: AI Awareness through Interaction with Blatantly Alien Models
The less-misleading user interface seems good to me, but I have strong reservations about the other four interventions.

To use the shoggoth-with-smiley-face-mask analogy, the way the other strategies are phrased sounds like a request to create new, creepier masks for the shoggoth so people will stop being reassured by the smiley-face.

From the conversation with 1a3orn, I understand that the creepier masks are meant to depict how LLMs / future AIs might sometimes behave.

But I would prefer that the interventions removed the mask altogether, that seems more truth-tracking to me.

(Relatedly, I’d be especially interested to see discussions (from anyone) on what creates the smiley-face-mask, and how entangled the mask is with the rest of the shoggoth’s behaviour.)

Note: I believe my reservations are similar to some of 1a3orn’s, but expressed differently.

Soapspud 28 Feb 2023 4:18 UTC
2 points
0
on: Buddhist Psychotechnology for Withstanding Apocalypse Stress
drawn black balls that I do not have adequate defenses for the informational content of directly.
I don’t know what drawing black balls means in this context. Would someone be able to clarify?

Soapspud 12 Dec 2022 23:04 UTC
1 point
0
in reply to: Lone Pine’s comment on: Take 6: CAIS is actually Orwellian.
Do you believe the MIRI view is Orwellian? If so, could you elaborate?

Soapspud 19 May 2022 20:20 UTC
10 points
0
in reply to: Daniel Kokotajlo’s comment on: Risks from AI persuasion

I would love to see research that collects data on metrics like cost of censorship and effectiveness of propaganda, and plots them over time to see if there is any general trend and how much it has varied over the course of human history, and whether the latest AI techniques are changing these metrics significantly.

If you have not already seen it, this report from CSET discusses the extent to which something as capable as GPT-3 changes the cost and effectiveness of disinformation and propaganda.

There was also a recorded discussion/seminar of the same topics with the authors of the report.

I don’t think it’s exactly what you’re looking for, but it seemed adjacent enough to be worth mentioning.