Malmesbury
I had the impression while reading that, at some point during the FAQ you stopped caring and had fun writing, and I thought this illustrated the point of the post pretty nicely.
It no longer works!
I just found out about this and want to join (probably just for quarter-haven, like Rauno), but the discord link is expired. Can I still join?
Making sense of parameter-space decomposition
Doesn’t Claude’s training data include all the tutorials and step by step walkthroughs of this game ever published on the internet? How is it not using this information?
In the sequel: the mirror mold evolves into intelligent life, develops a brain on its own, and threatens to take over the world. U3 begrudgingly sends an ambassador to the human city to collect samples of mirror-mirror-pathogens.
That’s impressive work! Out of curiosity, how long did it take to figure all of this out?
Putting error-correction codes in the genetic code is an interesting idea. In the context of the Pikachu thought experiment, though, here what I think would happen in the long run: because of the drift barrier, evolution can’t distinguish between a ~1/N error rate and a zero error rate. So, there’s nothing to prevent the rest of the machinery to become less accurate, until the error rate reaches 1/N after error correction. Now that I think about, you could probably keep things in control by systematically sequencing the genes for the replication machinery and breeding based on that. There is a spark of hope.
Oh yeah, I mean to compare things that have the same functionality (e.g. human-made butterfly robot vs natural butterfly). Obviously shovels are more simple than butterflies. But, seeing stuff like the wax motor and other examples people have posted, humans are definitely capable of coming up with great simple mechanisms, and I underestimated that. Thanks for bringing it up.
Kudos for taking the challenge! If I understand correctly, your first point is actually pretty similar to how E. coli follows gradients of nutrients, even when the scale of the gradient is much larger than the size of a cell.
Mechanisms too simple for humans to design
You might enjoy this story I wrote a few months ago, also about AI doom and also set in the future: https://www.lesswrong.com/posts/BgTsxMq5bgzKTLsLA/this-is-already-your-second-chance
Self-review: It’s been long enough that I’ve forgotten most of the details of the post, so it’s a good time to re-read it and get a sense of what it reads like for someone who’s discovering the content for the first time. I still believe most of the ideas here are correct. My goal here was to write a bottom-up overview of how the basic molecular structure of DNA might “inevitably” lead to stuff like sexually-dimorphic ornaments, after a long chain of events. The point I wanted to get at was that the male/female binary, which human cultures often depict as a metaphysical fundamental principle of the universe, results from a series of pretty prosaic evolutionary constraints. Love itself isn’t quite as magical and central to the Universe as we make it out to be. This post hints at the idea, but I could have expanded more in that direction. In hindsight, I think most people saw this as a kind of textbook explanation of the evolution of sex, and I guess this post does a OK job at this. However, it’s completely skipping over some important theories, namely the Red Queen Race and the handicap principle. Maybe I should add some links at the end. (To my defense, I did include a disclaimer about this not being a complete overview of the field.)
Which one? I hope it’s not the one where you have to put chocolate, because this is the most crucial instruction.
This is already your second chance
it’s no biology lab
I’m afraid you’re overestimating how well biologists follow the safety procedures. I wouldn’t be surprised if we all had fluorescent bacteria in our guts.
Oh, that’s a really good point. Actually, it might be common for chemists to work with panels of related molecules, while in clinical trials they only work with one purified drug candidate. This makes it less likely for them to discover things by accident. Surely a piece of the puzzle!
Sure, all these stories totally sound like urban legends, but the sweeteners are out there and I don’t see how they could have been discovered otherwise (unless they were covertly screening drugs on a large number of people).
That’s a great question, this is totally mysterious to me. There are a lot of examples of people putting thaumatin in transgenic fruits or vegetables (and somehow in the milk of transgenic mice because there’s always one creepy study), but I don’t know why it hasn’t been commercialized. It sounds like superfruits would make a nice healthy alternative to palm-oil-and-chocolate-based comfort foods. Maybe it’s a regulatory problem?
The fact that the tech changes all the time sounds like a major obstacle. I think a big reason why so few people refine their LLM usage the way Janus does is that any skill you’ll acquire through practice might become irrelevant in the next few months, when the company releases the next update. Likewise, you could probably design some nice thick interesting AI-based musical instruments right now, but the threat of immediate obsolescence would discourage people from actually practicing. Instead, we get Suno, because that’s the only thing that is viable on these timescales. So, unless AI progress slows down dramatically, making thick AI tools will require figuring out a way to make things plausibly future-proof. That sounds hard, but maybe there’s a way.