It’s the whole ‘and that’s bad’ that the pattern is made of as an energetic pattern of dismissal. I have issues with dismissal that I’m working on.
romeostevensit
Karma: 10,869
Yeah, it seems reasonable that part of what ethical injunctions are for is ‘this is where the tribal map of the world ends.’
There’s quite a lot of people trying to deconfuse the concept, but it’s so cursed that I think clean sheet is often appropriate. Just split out whatever other things one means and ignore agency as a monolithic concept block.
If a problem seems hard the representation is probably wrong.
Beginning to really dislike the word agency.
Another generalization I’ve found useful is that moralizing is what happens specifically when you reach the end of people’s causal model of a situation but they still have leftover emotional charge to expend. I’ve noticed this internally as well.
Update, I have purchased a microwave grill pan and consumed a grilled cheese. It works well. I’m looking forward to running experiments.
On reflection, I am somewhat sad that people find Smith’s book worthy of mockery. It feels a bit like a microcosm of the mockery of rationality in general. Sincere person runs lots of experiments with new tech to see what’s possible. Tries to render this available to others. Lol what a low status dumbass. :(
As one pushes into the tails of risks bc ones lifespan grows arbitrarily long, the logical conclusion is that one must control the entire universe in order to control all possible threats. Uncontrolled processes must eventually be curtailed. And a smart wizard sees this far enough in advance that they try to move for decisive strategic advantage now. The general trope is that they then fail to see the consequences of self modification corrupting the things they cared about anyway, which is also weirdly prescient, as it turns out that this really is a wicked problem, philosophically, computationally etc.
My fuzzy unjustified research sense is that people seem to be doing far too much in the direction of assuming that future AIs will maintain properties of current AIs as they scale, whereas I’m expecting more surprising qualitative shifts. Like if evolution built a hind brain and dusted off its hands with how aligned it is and then oops prefrontal cortex.
Edit: to add more explicitly, I think it’s something like ontology shifts introduce degrees of freedom for reward goodharting.
Thanks for doing some sensitivity analysis. It’s often number one on my list of frustrating things not included in a model.
This feels like a throwback to the LW of yore in an excellent way. Bravo.
two useful things:
putting things in the new larger context windows, like the books of authors you respect and having the viewpoints discuss things back and forth between several authors. Helps avoid the powerpoint slop attractor.
learning to prompt better via the dual use of practicing good business writing techniques. Easy to do via the above by putting a couple business writing books in context and then prompting the model to give you exercises that it then grades you on.
I recently found out that a lifetime of dairy consumption is likely to only kill about 1 cow on expectation. This was about 6-8x off what I naively guessed. This made me curious and I did some napkin math with Gemini:
https://imgur.com/a/dyAWdUs
and fish are horrendous: https://imgur.com/a/EpwcWAm
I recently found out that one of the reasons it took Darwinism so long to emerge given animal husbandry was that many people had a similar idea, but the universe was considered to be too young for it to lead to anything but minor variation. It was shortly after cosmological times were pushed way way back that the idea took on more steam.
I think it’s pretty reasonable that people discount the 3-5 years exercise adds at the end of life. But the quality of life boost is not reasonable to discount.
Aumann’s agreement is pragmatically wrong. For bounded levels of compute you can’t necessarily converge on the meta level of evidence convergence procedures.
I’ve thought a lot of the past dating roundups had good advice, but one based primarily on twitter discourse, as this one is, is very bad. Many of these takes reek of mental illness and should not be updated on IMO.
faster is often the wrong dimension for me to focus on. Though good optimization does lead to speed, it often feels like a result rather than a direct optimization target. I think something closer to how could I have learned that more easily, more completely. What was extraneous in retrospect? Were there any signs that I could have noticed earlier? What could I have included earlier that was obviously helpful?
One dynamic these questions reveal is often subtle bike-shedding: focusing on those areas that gave quick hits of sense of progress rather than what moves the needle on the original goal the most. A shorthand I have for this is ‘decision leverage’, as in how will what I am doing currently connect to something I do differently in the future?
More generally, the four pedagogical interventions with the largest effect size AFAIK are
deliberate practice
elaboration of context (connection to other areas of knowledge)
frequent low stakes quizzing
teaching the material to others
we identify seven epistemic fault lines, divergences in grounding, parsing, experience, motivation, causal reasoning, metacognition, and value. We call the resulting condition Epistemia: a structural situation in which linguistic plausibility substitutes for epistemic evaluation, producing the feeling of knowing without the labor of judgment.
https://arxiv.org/abs/2512.19466
This may crisp up some intuitions about specific ways slop feels bad.
probabilistic sanctioning as the simplest way of distributing the duty (enforcement costs).
prior art for Eliezer’s cooperation-punishment theorem from Planecrash?
https://iopscience.iop.org/article/10.1088/1367-2630/16/8/083016/meta
Tbh I find the topic exhausting right now and have trouble reading any posts that talk about it