Decaeneus

Karma: 37

Decaeneus 7 May 2024 14:44 UTC
20 points
5
on: Decaeneus’s Shortform
Pretending not to see when a rule you’ve set is being violated can be optimal policy in parenting sometimes (and I bet it generalizes).
Example: suppose you have a toddler and a “rule” that food only stays in the kitchen. The motivation is that each time food is brough into the living room there is a small chance of an accident resulting in a permanent stain. There’s cost to enforcing the rule as the toddler will put up a fight. Suppose that one night you feel really tired and the cost feels particularly high. If you enforce the rule, it will be much more painful than it’s worth in that moment (meaning, fully discounting future consequences). If you fail to enforce the rule, you undermine your authority which results in your toddler fighting future enforcement (of this and possibly all other rules!) much harder, as he realizes that the rule is in fact negotiable / flexible.
However, you have a third choice, which is to credibly pretend to not see that he’s doing it. It’s true that this will undermine your perceived competence, as an authority, somewhat. However, it does not undermine the perception that the rule is to be fully enforced if only you noticed the violation. You get to “skip” a particularly costly enforcement, without taking steps back that compromise future enforcement much.
I bet this happens sometimes in classrooms (re: disruptive students) and prisons (re: troublesome prisoners) and regulation (re: companies that operate in legally aggressive ways).
Of course, this stops working and becomes a farce once the pretense is clearly visible. Once your toddler knows that sometimes you pretend not to see things to avoid a fight, the benefit totally goes away. So it must be used judiciously and artfully.

Decaeneus 26 Jan 2024 14:56 UTC
6 points
0
on: Decaeneus’s Shortform
Infertility rates are rising and nobody seems to quite know why. Below is what feels like a possible (trivial) explanation that I haven’t seen mentioned anywhere.
I’m not in this field personally so it’s possible this theory is out there, but asking GPT about it doesn’t yield the proposed explanation: https://chat.openai.com/share/ab4138f6-978c-445a-9228-674ffa5584ea
Toy model:
- a family is either fertile or infertile, and fertility is hereditary
- the modal fertile family can have up to 10 kids, the modal infertile family can only have 2 kids
- in the olden days families aimed to have as many kids as they could
- now families aim to have 2 kids each
Under this model, in the olden days we would find a high proportion of fertile people in the gene pool, but in the modern world we wouldn’t. Put differently, the old convention lead to a strong positive correlation between fertility and participation in the gene pool, and the new convention leads to 0 correlation. This removes the selective pressure on fertility, hence we should expect fertility to drop / infertility to rise.
Empirical evidence for this would be something like an analysis of the time series of family size variance and infertility—is lower variance followed by increased infertility?

Decaeneus 10 May 2024 19:00 UTC
3 points
0
on: Decaeneus’s Shortform
Causality is rare! The usual statement that “correlation does not imply causation” puts them, I think, on deceptively equal footing. It’s really more like correlation is almost always not causation absent something strong like an RCT or a robust study set-up.
Over the past few years I’d gradually become increasingly skeptical of claims of causality just by updating on empirical observations, but it just struck me that there’s a good first principles reason for this.
For each true cause of some outcome we care to influence, there are many other “measurables” that correlate to the true cause but, by default, have no impact on our outcome of interest. Many of these measures will (weakly) correlate to the outcome though, via their correlation to the true cause. So there’s a one-to-many relationship between the true cause and the non-causal correlates. Therefore, if all you know is that something correlates with a particular outcome, you should have a strong prior against that correlation being causal.
My thinking previously was along the lines of p-hacking: if there are many things you can test, some of them will cross a given significance threshold by chance alone. But I’m claiming something more specific than that: any true cause is bound to be correlated to a bunch of stuff, which will therefore probably correlate with our outcome of interest (though more weakly, and not guaranteed since correlation is not necessarily transitive).
The obvious idea of requiring a plausible hypothesis for the causation helps somewhat here, since it rules out some of the non-causal correlates. But it may still leave many of them untouched, especially the more creative our hypothesis formation process is! Another (sensible and obvious, that maybe doesn’t even require agreement with the above) heuristic is to distrust small (magnitude) effects, since the true cause is likely to be more strongly correlated with the outcome of interest than any particular correlate of the true cause.

Decaeneus 8 May 2024 16:42 UTC
3 points
2
in reply to: RobertM’s comment on: Decaeneus’s Shortform
Perhaps that can work depending on the circumstances. In the specific case of a toddler, at the risk of not giving him enough credit, I think that type of distinction is too nuanced. I suspect that in practice this will simply make him litigate every particular application of any given rule (since it gives him hope that it might work) which raises the cost of enforcement dramatically. Potentially it might also make him more stressed, as I think there’s something very mentally soothing / non-taxing about bright line rules.
I think with older kids though, it’s obviously a really important learning to understand that the letter of the law and the spirit of the law do not always coincide. There’s a bit of a blackpill that comes with that though, once you understand that people can get away with violating the spirit as long as they comply with the letter, or that complying with the spirit (which you can grok more easily) does not always guarantee compliance with the letter, which puts you at risk of getting in trouble.

Decaeneus 9 Feb 2024 18:14 UTC
3 points
0
in reply to: porby’s comment on: porby’s Shortform
For what it’s worth (perhaps nothing) in private experiments I’ve seen that in certain toy (transformer) models, task B performance gets wiped out almost immediately when you stop training on it, in situations where the two tasks are related in some way.
I haven’t looked at how deep the erasure is, and whether it is far easier to revive than it was to train it in the first place.

Decaeneus 9 Feb 2024 18:08 UTC
3 points
0
on: Decaeneus’s Shortform
Reflecting on the particular ways that perfectionism differs from the optimal policy (as someone who suffers from perfectionism) and looking to come up with simple definitions, I thought of this:
- perfectionism looks to minimize the distance between an action and the ex-post optimal action but heavily dampening this penalty for the particular action “do nothing”
- optimal policy says to pick the best ex-ante action out of the set of all possible actions, which set includes “do nothing”
So, perfectionism will be maximally costly in an environment where you have lots of valuable options of new things you could do (breaking from status quo) but you’re unsure whether you can come close to the best one, like you might end up choosing something that’s half as good as the best you could have done. Optimal policy would say to just give it your best, and that you should be happy since this is an amazingly good problem to have, whereas perfectionism will whisper in your ear how painful it might be to only get half of this very large chunk of potential utility, and wouldn’t it be easier if you just waited.

Decaeneus 12 Jan 2024 20:23 UTC
LW: 3 AF: 3
0
AF
in reply to: Steven Byrnes’s comment on: My AGI safety research—2022 review, ’23 plans
Thanks for the thoughtful reply. I read the fuller discussion you linked to and came away with one big question which I didn’t find addressed anywhere (though it’s possible I just missed it!)
Looking at the human social instinct, we see that it indeed steers us towards not wanting to harm other humans, but it weakens when extended to other creatures, somewhat in proportion to their difference from humans. We (generally) have lots of empathy for other humans, less so for apes, less so for other mammals (who we factory farm by the billions without most people particularly minding it) probably less so for octopi (who are bright but quite different) and almost none to the zillion microorganisms, some of which we allegedly evolved from. I would guess that even canonical Good Person Paul Christiano probably doesn’t lose much sleep over his impact on microorganisms.
This raises the question of whether the social instinct we have, even if fully reverse engineered, can be deployed separately from the identity of the entity to which it is attached. In other words, if the social instinct circuitry humans have is “be nice to others in proportion to how similar to yourself they are”, which seems to match the data, then we would need more than just the ability to place that circuitry in AGIs (which would presumably make the AGIs want to be nice to other similar AGIs). We would in fact need to be able to tease apart the object of empathy, and replace it with something that is very different than how humans operate, since no human is nice to microorganisms, so I see no evidence that the existing social instincts ever make any person be nice to something very different, and much weaker, than them, so I would expect it to work similarly in an AGI.
This is speculative, but it seems reasonably likely to me to turn out to be an actual problem. Curious if you have thoughts on it.

Decaeneus 11 Jan 2024 20:53 UTC
LW: 3 AF: 2
0
AF
on: My AGI safety research—2022 review, ’23 plans
This is drifting a bit far afield from the neurobio aspect of this research, but do you have an opinion about the likelihood that a randomly sampled human, if endowed with truly superhuman powers, would utilize those powers in a way that we’d be pleased to see from an AGI?
It seems to me like we have many salient examples of power corrupting, and absolute power corrupting to a great degree. Understanding that there’s a distribution of outcomes, do you have an opinion about the likelihood of benevolent use of great power, among humans?
This is not to say that this understanding can’t still be usefully employed, but somehow it seems like a relevant question. E.g. if it turns out that most of what keeps humans acting pro-socially is the fear that anti-social behavior will trigger their punishment by others, that’s likely not as juicy a mechanism since it may be hard to convince a comparatively omniscient and omnipotent being that it will somehow suffer if it does anti-social things.

Decaeneus 18 Mar 2024 14:42 UTC
1 point
0
in reply to: Razied’s comment on: Decaeneus’s Shortform
Agreed with your example, and I think that just means that L2 norm is not a pure implementation of what we mean by “simple”, in that it also induces some other preferences. In other words, it does other work too. Nevertheless, it would point us in the right direction frequently e.g. it will dislike networks whose parameters perform large offsetting operations, akin to mental frameworks or beliefs that require unecessarily and reducible artifice or intermediate steps.
Worth keeping in mind that “simple” is not clearly defined in the general case (forget about machine learning). I’m sure lots has been written about this idea, including here.

Decaeneus 17 Mar 2024 14:26 UTC
1 point
0
on: Decaeneus’s Shortform
Regularization implements Occam’s Razor for machine learning systems.

When we have multiple hypotheses consistent with the same data (an overdetermined problem) Occam’s Razor says that the “simplest” one is more likely true.

When an overparameterized LLM is traversing the subspace of parameters that solve the training set seeking the smallest l2-norm say, it’s also effectively choosing the “simplest” solution from the solution set, where “simple” is defined as lower parameter norm i.e. more “concisely” expressed.

Decaeneus 15 Mar 2024 15:28 UTC
1 point
0
on: Rule Thinkers In, Not Out
In early 2024 I think it’s worth noting that deep-learning based generative models (presently, LLMs) have the property of generating many plausible hypotheses, not all of which are true. In a sense, they are creative and inaccurate.
An increasingly popular automated problem-solving paradigm seems to be bolting a slow & precise-but-uncreative verifier onto a fast & creative-but-imprecise (deep learning based) idea fountain, a la AlphaGeometry and FunSearch.
Today, in a paper published in Nature, we introduce FunSearch, a method to search for new solutions in mathematics and computer science. FunSearch works by pairing a pre-trained LLM, whose goal is to provide creative solutions in the form of computer code, with an automated “evaluator”, which guards against hallucinations and incorrect ideas. By iterating back-and-forth between these two components, initial solutions “evolve” into new knowledge. The system searches for “functions” written in computer code; hence the name FunSearch.
Perhaps we’re getting close to making the valuable box you hypothesize.

Decaeneus 21 Feb 2024 21:46 UTC
1 point
0
in reply to: Zac Hatfield-Dodds’s comment on: Decaeneus’s Shortform
To be sure, let’s say we’re talking about something like “the entirety of published material” rather than the subset of it that comes from academia. This is meant to very much include the open source community.
Very curious, in what way are most CS experiments not replicable? From what I’ve seen in deep learning, for instance, it’s standard practice to include a working github repo along with the paper (I’m sure you know lots more about this than I do). This is not the case in economics, for instance, just to pick a field I’m familiar with.

Decaeneus 21 Feb 2024 19:27 UTC
1 point
0
on: Decaeneus’s Shortform
I wonder how much of the tremendously rapid progress of computer science in the last decade owes itself to structurally more rapid truth-finding, enabled by:
- the virtual nature of the majority of the experiments, making them easily replicable
- the proliferation of services like github, making it very easy to replicate others’ experiments
- (a combination of the points above) the expectation that one would make one’s experiments easily available for replication by others
There are other reasons to expect rapid progress in CS (compared to, say, electrical engineering) but I wonder how much is explained by this replication dynamic.

Decaeneus 14 Feb 2024 16:59 UTC
1 point
0
on: Decaeneus’s Shortform
It feels like (at least in the West) the majority of our ideation about the future is negative, e.g.
- popular video games like Fallout
- zombie apocalypse themed tv
- shows like Black Mirror (there’s no equivalent White Mirror)
Are we at a historically negative point in the balance of “good vs bad ideation about the future” or is this type of collective pessimistic ideation normal?
If the balance towards pessimism is typical, is the promise of salvation in the afterlife in e.g. Christianity a rare example of a powerful and salient positive ideation about our futures (conditioned on some behavior)?

Decaeneus 12 Feb 2024 18:16 UTC
1 point
0
on: Decaeneus’s Shortform
From personal observation, kids learn text (say, from a children’s book, and from songs) back-to-front. That is, the adult will say all but the last word in the sentence, and the kid will (eventually) learn to chime in to complete the sentence.
This feels correlated to LLMs learning well when tasked with next-token prediction, and those predictions being stronger (less uniform over the vocabulary) when the preceding sequences get longer.
I wonder if there’s a connection to having rhyme “live” in the last sound of each line, as opposed to the first.

Decaeneus 9 Feb 2024 20:04 UTC
1 point
0
in reply to: Decaeneus’s comment on: Decaeneus’s Shortform
Kind of related Quanta article from a few days ago: https://www.quantamagazine.org/what-your-brain-is-doing-when-youre-not-doing-anything-20240205/

Decaeneus 28 Jan 2024 1:52 UTC
1 point
0
in reply to: Perhaps’s comment on: Decaeneus’s Shortform
The parallel to athlete pre game rituals is an interesting one, but I guess I’d be interested in seeing the comparison between the following two groups:
group A: is told to meditate the usual way for 30 minutes / day, and does
group B: is told to just sit there for 30 minutes / day, and does
So both of the groups considered are sitting quietly for 30 minutes, but one group is meditating while the other is just sitting there. In this comparison, we’d be explicitly ignoring the benefit from meditation which acts via the channel of just making it more likely you actually sit there quietly for 30 minutes.

Decaeneus 27 Jan 2024 19:58 UTC
1 point
0
on: Decaeneus’s Shortform
Is meditation provably more effective than “forcing yourself to do nothing”?
Much like sleep is super important for good cognitive (and, of course, physical) functioning, it’s plausible that waking periods of not being stimulated (i.e. of boredom) are very useful for unlocking increased cognitive performance. Personally I’ve found that if I go a long time without allowing myself to be bored, e.g. by listening to podcasts or audiobooks whenever I’m in transition between activities, I’m less energetic, creative, sharp, etc.
The problem is that as a prescription “do nothing for 30 minutes” would be rejected as unappealing by most. So instead of “do nothing” it’s couched as “do this other thing” with a focus on breathing and so on. Does any of that stuff actually matter or does the benefit just come from doing nothing?

Decaeneus 26 Jan 2024 21:34 UTC
1 point
0
in reply to: tailcalled’s comment on: Decaeneus’s Shortform
To be sure, I’m not an expert on the topic.
Declines in male fertility I think are regarded as real, though I haven’t examined the primary sources.
Regarding female fertility, this report from Norway outlines the trend that I vaguely thought was representative of most of the developed world over the last 100 years.
Female fertility is trickier to measure, since female fertility and age are strongly correlated, and women have been having kids later, so it’s important (and likely tricky) to disentangle this confounder from the data.

Decaeneus 22 May 2023 16:25 UTC
1 point
0
in reply to: Andy_McKenzie’s comment on: The basic reasons I expect AGI ruin
Understood, and agreed, but I’m still left wondering about my question as it pertains to the first sigmoidal curve that shows STEM-capable AGI. Not trying to be nitpicky, just wondering how we should reason about the likelihood that the plateau of that first curve is not already far above the current limit of human capability.
A reason to think so may be something to do with irreducible complexity making things very hard for us at around the same level that it would make them hard for a (first-gen) AGI. But a reason to think the opposite would be that we have line of sight to a bunch of amazing tech already, it’s just a question of allocating the resources to support sufficiently many smart people working out the details.
Another reason to think the opposite is that having a system that’s (in some sense) directly optimized to be intelligent might just have a plateau drawn from a higher-meaned distribution than one that’s optimized for fitness, and develops intelligence as a useful tool in that direction, since the pressure-on-intelligence for that sort of caps out at whatever it takes to dominate your immediate environment.