It is hard to do as a prefix in German, I think. It sounds a bit antiquated to me, but you could try “Jung war X”. But yes, in general, I think you are going to run into problems here because German inflects a lot of words based on the gender.
Morpheus
Karma: 990
Your German also gives away the gender. Probably use some language model to double check your sentences.
I queried my brain (I am German) and noticed my claim doesn’t predict the result. Then checked online and I had male and female backwards from what I read in a dictionary once
After checking random words I noticed the bias is the other way around and female is more likely. Google gave me the same. Now I am confused.
I don’t find it surprising. For example, IIRC in German 1⁄2 of nouns are male, 1⁄3 is female, 1⁄6 is neuter. I’d expect correlations/frequencies in English and other European languages, but harder to spot if you don’t have gendered nouns.
In the spirit of “All stable processes we shall predict, all unstable processes we shall control.” I was thinking about how you would control the weather and earthquakes. One big problem for both of these is convection. For example, earthquakes are powered by hot material from inside the earth being transported out. I noticed my day-to-day intuition had been really confused by convection in solids. Intuitively, moving mass around feels much more inefficient than conduction. I still don’t have great intuition for this, but one thing that helped was learning about the Rayleigh number (not to be confused with Reynolds number) which quantifies in which regimes convection rather than conduction dominates. Intuitively, the problem with conduction is that it works best if there are large energy distances locally, but the longer the distance the heat has to travel, the flatter the local heat gradient is going to be. Convection moves things in bulk, which works better with more volume. Heat transfer is so slow inside the earth, a large part of earths internal heat is still left over from the potential energy released by its formation. So if heat wasn’t moved by all of the mantle collectively moving centimetres a year, even less heat would escape.
So...why, then? My working theory is that I was cursed by an ancient totem I touched as a child, but I’m open to suggestions.
Your active search and your writing are both selecting for smart non-conformists. Sanity and integrity might even be correlated with those, but not strongly enough.
I have a blog, sure, but is it really that weird to like learning about some science stuff and sometimes tell people about what you learned?
It is, in fact, deeply weird. If you know a bunch of bloggers who write good posts like yourself, I would love to know your recommendations, though. I already knew 3 out of 4 of the YouTube channels you mentioned, which made me more pessimistic about how much high-quality, easy-to-digest material I might be missing. Your posts on chemistry and engineering revived some curiosity that had been withering. Since then, I’ve learned a lot more basic chemistry, biochemistry and geology, sometimes just because I was curious what things are made of and why, so thank you!
I am in the waiting room at the doctor and the ~1 year old child next to me is scrolling through YouTube shorts on his mother’s phone with the mom watching along. Incredibly incoherent AI slop. Not an expert at early development, but this seems very suboptimal. I know that scrolling is not good for my reward system, but a 1-year old? At least put some random video on instead of letting the child scroll? If I was in charge at Google and was living by “don’t be evil”, I would maybe make a classifier to identify children scrolling like this and giving a reminder once in a while to suggest some alternative activities to the parent?
Thanks for writing this post! I do think some people are pursuing interp for this “wrong reason” of trying to prevent scheming, and the road where you get interp to improve that well to make that work seems unlikely (understanding general circuits doesn’t seem impossible to me but extremely hard and nonzero people are working on this).
I think that perhaps the mistake comes from mistaking the simplicity of the optimizer for a property of the mesa-optimizer. SGD by backprop is one algorithm so people put a single label, “deep learning,” on all models it produces. But there is no reason that all of these models must use similar circuits. They may all use an array of unique fantastically complex circuits. Understanding every circuit that can be produced by SGD at once is not a cohesive research program, and it is not a plan that will succeed.
There would be reasons to believe that models are going to use similar algorithms if they use similar training data. Understanding every circuit that is possible to be produced by SGD given “infinite training data” seems intractable, but “in practice” I’d expect different algorithms produced by SGD to produce modular structures with common “motifs” just like evolution does. Evolutionary developmental biology is indeed a field (that just like interp seems more bottlenecked on better theory rather than measurement capabilities). It’s why I am still excited about developmental interpretability, even though I don’t have a coherent plan for how it will help us with safety beyond “more theory and foundations” seems nice (with the general caveat that I am very confused about capability externalities, but this seems kind of unavoidable for actually broad insights).
The fact that this type of thing tends to get such large emotional responses out of people makes me wonder to what extent rendering counterfactuals would be useful to combat the lack of imagination for big decisions that actually are ahead of someone?
I would get a vaccine again if I thought I was at the risk of getting it?
Yes! For the individual, it does not make sense to adjust the sex ratio with changes in climate, but for the species overall it’s not that bad or even positive. Sex chromosomes do a much better job for the individual here (And would be selected for if temperature changes were happening too often). I do think I had confused thinking at the time, because I also had just read about a theory in humans that high status females supposedly produce more males (so payoff could be different between low and high status, but that doesn’t apply in this case, and it’s not born out well empirically anyway).
What I do think is true: It might not be that much of a coincidence that climate change would lead to more females. If it was the other way around, there is a higher likelihood the species would have gone extinct from a meteor, and we wouldn’t observe it today. Having 1% females and 99% males is going to lead to a really bad population bottleneck, but having 1% males and 99% females just means the remaining males have little competition (the effective population is still reduced to some extent). The fact that turtles don’t fit this pattern and look like they should have gone extinct with the last meteor tells you how weak and slow selection between species is.
I was thinking of this when I saw these articles complaining how sex ratios are supposedly endangering sea turtles. It doesn’t seem obvious to me that the sex ratios alone should be a problem (although 1⁄99 is probably bad).
I thought about this a bunch of years ago, and I am not quite sure how much this matters. Often ants get sperm from multiple fathers, and then they’d be less related than siblings (although maybe they only start doing that once they are out of the unstable intermediate period between being not eusocial and being eusocial). Also, this leads to colonies producing fewer males than would be game theoretically optimal for the queen and to conflict between queen and workers (50/50 would be optimal), although that might be helpful on the species level? So I think most of the cohesion benefits from 75% kinship between daughters is offset by the 25% kinship between brothers and sisters and if the queen mates with 2 males instead of 1, then the kinship is actually less than with monogamous termites (which is most termites, while I think most ant queens mate with multiple males).
I do think the difference in kinship could make it easier to develop eusociality, but I think my above argument that you just need a queen, so adapting behaviour is a simpler program might explain better how ants got through the unstable period between being individualistic and being sort of eusocial better (also if you start out with two parents there is less marginal returns to another worker for the nest, so that might also explain why it happened so often in Hymenoptera rather than other nest building insects). There are some wasp species that sometimes decide to build nests together and do division of labour (protect nest, search food) even though they are unrelated, but I forgot their name (Most of my opinion above is informed by the Superorganism book from EO Wilson. He has some idiosyncratic beliefs when it comes to group selection, but I think when it comes to his analysis that the kinship seems marginally important, he seems right). The kinship could explain why ants developed it multiple times, but it doesn’t explain why termites even after developing eusociality didn’t expand further.
I tend to have some luck with concrete analogies sometimes. For example I asked for the equivalent of Tonedeff (His polymer album is my favorite album) in other genres and it recommended me Venetian Snares. I then listened to some of his songs and it seemed like the kind of experimental stuff where I might find something I find interesting. Venetian Snares has 80k monthly listeners while Tonedeff has 14K, so there might be some weighting towards popularity, but that seems mild.
I think when you wrote postmortem you meant to write premortem?
Sounds interesting and like something I might miss if true. I would be interested in examples.
I was engaging with this, because I thought maybe you are advocating for some kind of doubles think, and I might spell you out of that, but this doesn’t seem to be the case. I am not interested to get deep into that religion argument (too many different people and different religions). Yes, there are some topics like ethics where most people don’t benefit from reasoning about them explicitly, and even smart people tend to get very confused by them. I remember I was confused for days by my first introduction to ethics.
Reframing your thoughts such that you don’t step on your own toes is great. I am not quite sure what you are trying to argue for with the religion point. Do you actually endorse this on a practical basis? Have you convinced yourself that you need to yourself belief in untrue things or that it is good if other people are believing untrue things in just the right way they cancel each other out? Believing in God seems a particularly bad example. Having high alieve you can solve a problem that lots of people haven’t solved before you might be fine for minutes or a few days. But I can’t see how you would get psychological benefits from believing in a god or fate and whatever and that not messing up your epistemics.
Thank you for this post! My impression is that this post makes real progress at identifying some of the upstream cruxes between people’s models of how AGI is going to go.
As examples, if you look at @So8res’s AGI ruin scenarios are likely (and disjunctive), I claim that a bunch of his AGI ruin scenarios rely on his belief that alignment is hard. I think that belief is correct! But still, it makes his argument less disjunctive than it might seem. Likewise, I now recognize that my own What does it take to defend the world against out-of-control AGIs? sneaks in a background assumption that alignment is hard (or alignment tax is high) in various places.
This seems correct to me.
But I figured out that I can occupy that viewpoint better if I say to myself: “Claude seems nice, by and large, leaving aside some weirdness like jailbreaks. Now imagine that Claude keeps getting smarter, and that the weirdness gets solved, and bam, that’s AGI. Imagine that we can easily make a super-Claude that cares about your long-term best interest above all else, by simply putting ‘act in my long-term best interest’ in the system prompt or whatever.” Now, I don’t believe that, for all the reasons above, but when I put on those glasses I feel like a whole bunch of the LLM-focused AGI discourse—e.g. writing by Paul Christiano, OpenPhil people, Redwood people, etc.—starts making more sense to me.
That seems to represent well the viewpoint that I now have even less faith in, but that didn’t seem strictly ruled out to me 4 years ago when GPT-3 had been around for a while, and it seemed likely we were headed for more scaling:
39 Nothing we can do with a safe-by-default AI like GPT-3 would be powerful enough to save the world (to ‘commit a pivotal act’), although it might be fun. In order to use an AI to save the world it needs to be powerful enough that you need to trust its alignment, which doesn’t solve your problem.
What exactly makes people sure that something like GPT would be safe/unsafe?
If what is needed is some form of insight/break through: Some smarter version of GPT-3 seems really useful? The idea that GPT-3 produces better poetry than me while GPT-5 could help to come up with better alignment ideas, doesn’t strongly conflict with my current view of the world?
For all Arbital content, there is the Arbital scrape index. Most (all?) of that material has been incorporated into Lesswrong’s concept pages.