Great stuff—super impressed on how much you achieved on this budget! Those numbers seem incredibly impressive, and definitely seems to indicate a lot of value can be had by this type of program. Running it in Sydney/Melbourne also means you have less diminishing returns—normally I’d expect this to be the best program you could run since you’re picking from the most pent-up demand for this course, but I can’t imagine people likely to come from abroad for a part-time 14-week course.
Jay Bailey
Karma: 789
I can see why you might find that frustrating. I think a lot of us, myself included, do think that the science is the most important part of the argument—but we don’t understand the science well enough to distinguish true arguments from false ones, in this domain.
I don’t have the proper expertise to evaluate your claims about pain receptors properly, but I do have the proper expertise to conclude that SWP calling their opponents “irrational, evil, or both” is bad, and that this correlates with shoddy reasoning everywhere else, including the parts I don’t understand. Thus, there’s a limit to how much I can update even from an incredibly strong neurological takedown, if that takedown requires knowledge of neurology that I don’t have in order to fully appreciate its correctness.
So, in terms of what we care about, think of it less as “How many bits of information should this point be worth” and more “How many bits of information can your audience actually draw from this piece of information?”
If the above is true, I think this is really good information that would have been very nice to have cited within the article. That would make me a lot more skeptical of SWP and of their conclusions, and it’d be great to see links for these examples if you could provide them.
Especially this paragraph:
“It just intuitively seems like they are.” This is proposed as a rebuttal for critiques of the shrimp welfare project, not very convincing to me, yet they claim that those who don’t support them are “irrational, evil or both”. I find that making that claim with sparse, scattered and unclear evidence is not great, and paints anyone who opposes their views as as flawed person.
I agree with the value claims in this paragraph completely, so if you have sources for those quotes I think that would be very persuasive to a lot of us here on this site, and it might even be worth a labelled edit to the main post.
I think you make some interesting points here, but there are two points I would disagree with:
First is “The Shrimp Welfare Project wants shrimp to suffer so they can have a new problem to solve.” This claim is made with no supporting evidence whatsoever. You don’t even argue for why it might be the case, and show no curiosity about other explanations for this. They claim to disagree with you, so clearly they have ulterior, malicious motives. (I would say knowingly creating a charity that doesn’t solve a real problem, just to be able to say you’re solving a new problem, is quite unethical!) Why is it so hard to believe the people who founded SWP did so with the intent of reducing as much suffering as possible, and just happened to be incorrect? What makes you completely dismiss this hypothesis so much that it isn’t even worth mentioning the alternative in your article?
Second is “At best, a shrimp sentium would encode only the most surface-level sensory experience: raw sensation without context, meaning, or emotional depth. Think of the mildest irritation you can imagine, like the persistent squeak of a shopping cart wheel at Walmart.”
I don’t see how the second sentence follows from the first. When I imagine a migraine, the worst pain I personally have ever experienced (being a rather fortunate individual) it doesn’t seem to me like the reason I am suffering is because of the context, meaning, or emotional depth of my pain. I’m suffering because it hurts. A lot. It doesn’t seem that complicated. It seems like it would be much more principled, using your analysis, to treat 860,000 shrimps freezing to death as suffering equivalent to one human experiencing the sensation of freezing to death, not experiencing mild irritation. I say “experiencing the sensation of” because things like being aware of one’s own mortality does seem like a thing that’s out of reach of a shrimp. So it’s not equivalent to a human actually dying, in my view, but freezing to death is likely still quite unpleasant, and not something I’d do for fun, and I’d much rather experience a squeaky wheel at Walmart even if I was fine as soon as I lost consciousness and had no chance of mental trauma from the incident, which I think still matches the shrimp equivalence.
What I still think makes this article interesting is that 550 humans experiencing the sensation of freezing to death across twenty minutes is bad, but not as bad as even one human death, which could be prevented by orders of magnitude less cost than a shrimp stunner. So even despite this article’s flaws I still think it’s a good article on net and worth engaging with for a proponent of shrimp welfare.
While a reply isn’t required, if you are going to engage with only one of these points, I would prefer it be the first one even though I wrote a lot less about it. The second point doesn’t actually change the overall conclusion very much imo, but the first point is generally quite confusing to me, and makes me less confident about rest of the article given the quality of reasoning in that claim.
This is, as I understand it, the correct pronunciation.
I do remember when I was learning poker strategy, and I learned about the idea of wanting to get your entire stack in the middle preflop if you can get paid off with AA—was a very fundamental lesson to young me! That said, there’s a key insight that goes along with the “pocket aces principle” that is missing here, and that’s bankroll management.
In poker, there is standard advice for how much money to have in your bankroll before you sit down at a table at all. E.g, for cash games, it’s at least 2000 big blinds (20x the largest stack you can buy). This is what allows you to bet all-in on pocket aces—if your entire bankroll is on the table, you should bet more conservatively. The point of bankroll management is to allow you to make the +EV play of putting your entire stack into the middle without caring about the variance when you lose 20% of the time.
To apply this metaphor to real life, you might say something like “Consider how much you’re willing to lose in the event things turn out badly (e.g, a year or two on a startup, six months on a relationship) and then, within that amount, bet the house.”
There’s a counterargument to the AGI hype that basically says—of course the labs would want to hype this technology, they make money that way, just because they say they believe in short timelines doesn’t mean it’s true. Specifically, the claim here is not that the AI lab CEO’s are mistaken, but rather that they are actively lying, and they know AGI isn’t around the corner.
What actions have frontier AI labs taken in the last year or two that wouldn’t make sense, given the above explanation? Stuff like GDM’s merger or OpenAI (reportedly) operating at a massive loss. Ideally these actions would be reported on by entities other than those companies themselves, in order to help convince skeptics. I’ve definitely seen stuff like this around but I can’t remember where and the search terms are too vague.
I’ve also tried using Deep Research for this, but it doesn’t seem to understand the idea of only looking at actions that are far more likely in the non-hype world than the hype-world, talking about things like investor decks projecting high returns that are very compatible with them hyping themselves up.
Significant Digits is (or was, a few years ago) considered the best one, to my recollection.
I bought a month of Deep Research and am open to running queries if people have a few but don’t want to spend 200 bucks for them. Will spend up to 25 queries in total.
A paragraph or two of detail is good—you can send me supporting documents via wnlonvyrlpf@tznvy.pbz (ROT13) if you want. Offer is open publicly or via PM.
Having reflected on this decision more, I have decided I no longer endorse those feelings in point B of my second-to-last paragraph. In fact, I’ve decided that “I donated roughly 1k to a website that provided way more expected value than that to me over my lifetime, and also if it shut down I think that would be a major blow to one of the most important causes in the world” is something to be proud of, not embarrassed by, and something worthy of being occasionally reminded of.
So if you’re still sending them out I’d gladly take one after all :)
I’ve been procrastinating on this, but I heard it was the last day to do this, so here I am. I’ve utilised LessWrong for years, but am also a notoriously cheap bastard. I’m working on this. That said, I feel I should pay something back, for what I’ve gotten out of it.
When I was 20 or so, I was rather directionless, and didn’t know what I wanted to do in life, bouncing between ideas, never finishing them. I was reading LessWrong at the time. At some point, a LessWrong-ism popped into my head—“Jay—this thing you’re doing isn’t working. Your interests change faster than you can commit to a career. Therefore, you need a career strategy that does not rely on your interests.” This last sentence definitely would not have occurred to me without LessWrong. It felt like a quantitative shift in thinking, that I had finally truly learned a new pattern. Nowadays it seems obvious, and it would be obvious to many of my friends...but back then, I remember that flash of insight, and I’ve never forgotten it.
I came up with a series of desiderata—something I’d be good at, not hate, and get to work indoors for a reasonable salary. I decided to be an accountant, which is evidence for this whole “One-shot the problem” thing being hard, but wisely pivoted into pursuing a software engineering degree a year later.
While EA was what got me into AI safety, even ignoring the effect LessWrong has had on EA, the skills I decided to learn thanks to LessWrong principles are potentially the only reason I have much of a say in the future at all. Not to mention I’ve made a pretty solid amount of money out of it.
Considering the amount of value I’ve gotten out of LessWrong, I’m far too cheap to donate an amount that would be truly “fair”, but I wanted to donate a solid amount anyway—an amount that at least justifies the years of use I’ve gotten out of the site. I talked myself into donating $1,000, but then I realised that A) I didn’t want a shirt to affect my donation decisions, and B) I’d be a bit embarassed to have a shirt that symbolises how I donated four figures to a website that has helped me think good. I feel like I’ll forget the money easily once I donate it, and it won’t affect my day to day life at all. Unless, of course, I have a physical reminder of it.
Thus, I have donated $999 USD to the cause.
Hi Giorgi,
Not an expert on this, but I believe the idea is that over time the agent will learn to assign negligible probabilities to actions that don’t do anything. For instance, imagine a game where the agent can move in four directions, but if there’s a wall in front of it, moving forward does nothing. The agent will eventually learn to stop moving forward in this circumstance. So you could probably just make it work, even if it’s a bit less efficient, if you just had the environment do nothing if an invalid action was selected.
Thanks for this! I’ve changed the sentence to:
The target network gets to see one more step than the Q-network does, and thus is a better predictor.
Hopefully this prevents others from the same confusion :)
pandas is a good library for this—it takes CSV files and turns them into Python objects you can manipulate. plotly / matplotlib lets you visualise data, which is also useful. GPT-4 / Claude could help you with this. I would recommend starting by getting a language model to help you create plots of the data according to relevant subsets. Like if you think that the season matters for how much gold is collected, give the model a couple of examples of the data format and simply ask it to write a script to plot gold per season.
To provide the obvious advice first:
Attempt a puzzle.
If you didn’t get the answer, check the comments of those who did.
Ask yourself how you could have thought of that, or what common principle that answer has. (e.g, I should check for X and Y)
Repeat.
I assume you have some programming experience here—if not, that seems like a prerequisite to learn. Or maybe you can get away with using LLM’s to write the Python for you.
I don’t know about the first one—I think you’ll have to analyse each job and decide about that. I suspect the answer to your second question is “Basically nil”. I think that unless you are working on state-of-the-art advances in:
A) Frontier models B) Agent scaffolds, maybe.
You are not speeding up the knowledge required to automate people.
I guess my way of thinking of it is—you can automate tasks, jobs, or people.
Automating tasks seems probably good. You’re able to remove busywork from people, but their job is comprised of many more things than that task, so people aren’t at risk of losing their jobs. (Unless you only need 10 units of productivity, and each person is now producing 1.25 units so you end up with 8 people instead of 10 - but a lot of teams could also quite use 12.5 units of productivity well)
Automating jobs is...contentious. It’s basically the tradeoff I talked about above.
Automating people is bad right now. Not only are you eliminating someone’s job, you’re eliminating most other things this person could do at all. This person has had society pass them by, and I think we should either not do that or make sure this person still has sufficient resources and social value to thrive in society despite being automated out of an economic position. (If I was confident society would do this, I might change my tune about automating people)
So, I would ask myself—what type of automation am I doing? Am I removing busywork, replacing jobs entirely, or replacing entire skillsets? (Note: You are probably not doing the last one. Very few, if any, are. The tech does not seem there atm. But maybe the company is setting themselves up to do so as soon as it is, or something)
And when you figure out what type you’re doing, you can ask how you feel about that.
Makes sense. Would be curious about estimated costs if this wasn’t the case!