Mainly commenting on your footnote, I generally agree that it’s fine to put low amounts of effort into one-off simple events. The caveat here is that this is an event that is 1) treated pretty seriously in past years and 2) is a symbol of a certain mindset that I think typically includes double-checking things and avoiding careless mistakes.
NaiveTortoise
I don’t know all the details of what testing was done, but I would not describe code review and then deploying as state-of-the-art as this ignores things like staged deploys, end-to-end testing, monitoring, etc. Again, I’m not familiar with the LW codebase and deploy process so it’s possible all these things are in place, in which case I’d be happy to retract my comment!
I know this is going to come off as overly critical no matter how I frame it but I genuinely don’t mean it to be.
Another takeaway from this would seem to be an update towards recognizing the difference between knowing something and enacting it or, analogously, being able to identify inadequacy vs. avoid it. People on LW often discuss, criticize, and sometimes dismiss folks who work at companies that fail to implement all security best practices or do things like push to production without going through proper checklists. Yet, this is a case where exactly that happened, even though there was not strong financial or (presumably) top down pressure to act quickly.
EDIT: I now see you research these questions and so want to add a disclaimer that I have not thought about these things nearly as deeply as you probably have...
Epistemic status: very speculative.
Cool post, I’ve long been fond of the, likely less difficult, thought experiment of whether we can grow a house using synthetic biology.
At first, I was thinking growing vs. building was just about the amount of labor involved to go from raw materials to final product. Then I realized this doesn’t work because under this definition a fully automated robot factory would qualify as growing a car.
My next best guess is that “growing” is related to:
the system containing its own description,
the system maintaining itself given “raw” materials, and
an aesthetic component that connects growing to things that look and feel biological.
In terms of KPIs, the first things that come to mind are metrics like:
How small a seed can the system bootstrap itself from given raw materials and otherwise little to no outside intervention?
Can the system repair itself when damaged?
This and the linked post have been really helpful for my attempts to better internalize the Kelly Criterion. Thanks!
EDIT: I see you’ve corrected the mistake with the 12% and 2.7x return that I originally discussed below in a subsequent post so the details below aren’t necessary. Maybe consider linking that post in the Addendum?
Mostly unrelated to the above, this is sort of a nitpick but between the body and the addendum, you (implicitly) switch from the odds-as-ratio-of-probabilities representation of to one in which is net fractional odds and is assumed to be . I know this makes sense because you’re explicitly talking about bets in the final section but I’m bringing it up because it might throw off someone who hasn’t read as many discussions of Kelly.
Another great example of this is Striped Smith-Waterman, which takes advantage of SIMD instructions to achieve a 2-8 speed-up (potentially much more on modern CPUs though) for constructing sequence local alignments.
(I’m the author of the post.) This is a totally reasonable critique, which I tried to make of myself:
Wrapping things up, I find all this analysis still a bit dissatisfying. While I’ve tried to use the commonalities amongst and differences between energetic aliens to understand them better, I feel like all the factors I identified describe rather than explain what’s going on with energetic aliens. A more satisfying understanding would instead at least suggest candidate causal factors which are predictive of energetic alienness warranting further investigation. Of course, this is also why this essay is called the neglected mystery rather than the resolved mystery.
That said, part of the challenge is that coming up with good theories is really hard here and risks touching on controversial issues, so warrants being careful! If you, or anyone else has such theories, I’d love to hear them.
As someone who has also struggled with similar issues, although in a different context than writing papers, I found some of the answers here helpful and could imagine some of them as good “tactical advice” to go along with cultural norms. I also ended up looking through Google’s SRE book as recommended in Gwern’s answer and benefited from it even though it’s focused on software infrastructure. In particular, the idea of treating knowledge production as a complex system helped knock me out of my “just be careful” mindset, which I think is often one of the harder things to scale. Of course, YMMV.
That makes sense.
For what it’s worth, I took notes on the event and did not share them publicly because it was pretty clear to me doing so would have been a defection, even though it was implicit. Obviously, just because it was clear to me doesn’t mean it was clear to everyone but I thought it still made sense to share this as a data point in favor of “very well known person doing a not recorded meetup” implying “don’t post and promote your notes publicly.”
I am also disappointed to see that this post is so highly upvoted and positively commented upon despite:
Presumably others were aware of the fact that the meetup was not supposed to be recorded and LW is supposedly characteristically aware of coordination problems/defection/impact on incentives. At least to me, it seems likely that in expectation this post spreading widely would make Sam and people like him less likely to speak at future events and less trustworthy of the community that hosted the event. This seems not worth the benefit of having the notes posted given that people who were interested could have attended the event or asked someone about it privately.
As Sean McCarthy and others pointed out, there were some at best misleading portrayals of what Altman said during his Q&A.
Returning to my number of muscle cells an adult human body example (from the initial stable equilibrium post), for the purposes of calculating lean vs. fat mass (or just weight), we don’t care about the fact that the distribution shifts as the person ages and experiences sarcopenia.
For predator-prey population size ratios, the ratio fluctuates slightly on a daily basis assuming the predators hunt at certain times of the day and potentially seasonally. Assuming both species live more than a year, neither matters for estimating the carrying capacity of the ecosystem for the predator species.
For calculating the average body temperature of a species, we can mostly ignore real but small fluctuations that occur throughout the day due to circadian rhythms, digestion, etc.
Number of cells in an adult human body. Also, cell type composition in an adult human body (over the timescale of months but not years because aging).
Relative size of predator/prey species population in a mature, mostly otherwise static ecosystem.
Warm-blooded mammal body temperature.
Most complex eukaryotic organisms are either dead or alive. Yes, they can be sick, which is sort of in between, but sick is still “alive”. In general, going from dead to alive is hard… Going from alive to dead requires disrupting any of several important core sub-equilibria of the living system.
It’s snowing out vs. not. Note: didn’t use raining because “misting” felt like more of an in between edge case than lightly snowing.
A door is either open or closed. Depending on the door, switching from closed to open or open to closed requires applying force and maybe adding some sort of friction device to keep the system in its new state.
(Cheating because I’ve seen this before.) Some natural and designed proteins function as switches with multiple stable states of comparable free energies.
Main exercise:
Amount of muscle a person who doesn’t exercise regularly has.
Level of clutter on a person’s desk/counter/etc.
Quantity of light that reaches a forest floor at a given time.
(Extra:) Number of organisms in an all-male group.
I recognize 1 and 3 are borderline dynamic equilibria but I think they changes on a slow enough timescale that they count.
Bonus exercise:
Watch their diet over the timescale of weeks to months and their physical activity. Can ignore incidental activity, like running to catch a bus or lifting lots of stuff for a move. More generally, can ignore any activity that’s acute.
Persistent changes to their cleaning behaviors (for reduction) or accumulation patterns (do they switch to computer notetaking?). Can ignore temporary changes or routine cleaning behaviors that have been going on for a while.
Pay attention to introduction of organisms/factors that change the forest to a different type of environment with less (or more) coverage. Can ignore both temporary disturbances like a human walking through and crushing plants and introduction of organisms which only consume one of several types of trees that form the canopy, since others will presumably fill the void.
I enjoyed this post a lot but in the weeks since reading it, one unaddressed aspect has been bugging me and I’ve finally put my finger on it: the recommendation to “Specialize in Things Which Generalize” neglects the all-important question of “how much?” Put a different way, at least in my experience, one can always go deeper into one of these subjects—probability theory, information theory, etc. -- but doing so takes time away from expanding one’s breadth. Therefore, as someone looking to build general knowledge, you’re constantly presented with the trade-off of continuing to learn one area deeply vs. switching to the next area you’d like to learn.
If I try to inhabit the mindset of the OP, I can generate two potential answers to this quandary, but none of them are super satisfying:
Learn enough to be able to leverage what you’ve learned for novel problems.
Learn enough to be able to build gears-level models using what you’ve learned.
Created some PredictionBook predictions based off of this:
Nice point. I wanted to note that the converse is also true and seems like an example of Berkson’s Paradox. If you only see individuals who passed the test, it will look like teachability is anti-correlated with the other two factors even though this may purely be a result of the selection process.
This may seem pedantic but the point I’m making is that it’s equally important not to update in the other direction and assume less alignment between past experience and current skillset is better, since it may not be once you correct for this effect.
Sounds like Metacademy.
This is basically a souped up version of TagTime (by the Beeminder folks) so you might be able to start with their implementation.
What do you think about synthetic biology as a manufacturing technology?