Old man 1: Life is one trouble after another. I’d be better off dead, better yet, I wish I was never born
Old man 2: True, true, but who has such luck ?.. maybe one in a thousand.
My blog: https://blog.cerebralab.com
I’m also building an open source generic ML library: https://github.com/mindsdb/mindsdb & https://github.com/mindsdb/lightwood …. which I guess might be of interest to some people here
I’ve been thinking a lot about replacing statistics with machine learning and how one could go about that. I previously tried arguing that the “roots” of a lot of classical statistical approaches are flawed, i.e. they make too many assumptions about the world and thus lead to faulty conclusions and overly complex models with no real insight.
I kind of abandoned that avenue once I realized people back in the late 60s and early 70s were making that point and proposing what are now considered machine learning techniques as a replacement.
So instead I’ve decided to just focus any further anger at bad research and people using nonsensical constructs like p-value on trying to popularize better approaches based on predictive modeling.
There is no relationship between neurons and the “neurons” of an ANN. It’s just a naming mishap at this point.
I consider myself a skeptic empiricist, to the extent I can, for it’s a difficult view to hold.
I don’t think this community or Eliezer’s ideas are so, they are fundamentally rational:
Timeless decision theory
Assumptions about experimental perfection that lead to EZs incoherent rambling on physics
Everything that’s part of the AI doomsday cult views
These are highly rational things, I suspect steming from a pre school “intelligence is useful” prior that most people failed to introspect, and that is pretty correct unless taken to an extreme. But it’s reasoning from that uncommon a prior (after al empiricists also start from something, it’s just that their starting point is one that’s commonly shared by all or most humans, e.g. obvious seen features), and other like it, that lead to the sequences and to most discussion on LW.
Which is not to say that it’s bad, I’ve personally come to believe it’s as ok as any religion, but it shouldn’t be confused with empiricism and empiricists methods.
Sometimes, those tokens represent words and sometimes they represent single characters.
Hmh, ok, quick update to my knowledge that I should have done before: https://huggingface.co/transformers/tokenizer_summary.html
Seems to indicate that GPT-2 uses a byte-level BPE, though maybe the impl here is wrong, where I’d have expected it to use something closer to a word-by-wrod tokenizer with exceptions for rare words (i.e. a sub-word tokenizer that’s basically acting as a word tokenizer 90% of the time). And maybe GPT-3 uses the same?
Also it seems that sub-word tokenizer split much more aggressively than I’d have assumed before.
Complaint retracted.
<retracted>
interesting.
Wasn’t Feynman basically known for:
His contribution to computing, formalizing problems into code, parallelizing, etc
His mathematical contributions (Feynman diagrams, Feynman integrals)
His contributions to teaching/reasoning methods in general.
I agree that I’d want to learn physics from him, I’m just not sure he was an exceptional physicist. Good, but not Von Neuman. He says as much in his biographies (e.g. pointing out one of his big contributions came from randomly point to a valve on a schematic and getting people to think about the schematic).
He seems to be good at “getting people to think reasonably and having an unabashedly open, friendly, mischievous and perseverant personality”, which seems to be what he’s famous for and the only thing he thinks of himself as being somewhat good at. Though you could always argue it’s due to modesty.
To give a specific example, this is him “explaining magnets”, except that I’m left knowing nothing extra about magnets, but I do gain a new understanding of concepts like “level of abstraction” and various “human guide to word”-ish insights about language use and some phenomenology around what it means to “understand”.
But the use-case for learning from the best is completely different: you study the best when there are no other options. You study the best when the best is doing something completely different, so they’re the only one to learn it from.
I feel like I do mention this when I say one ought to learn from similar people.
If you spent 10 years learning how to <sport> and you are nr 10 in <sport> and someone else is nr 1 in <sport>, the heuristic of learning from someone similar to you applies.
For instance, back in college I spent a semester on a project with the strongest programmer in my class, and I picked up various small things which turned out to be really important (like “choose a good IDE”).
What you are describing here though is simply a category error, “the best in class” is not “the best programmer”, there were probably hundreds of thousands better than him on all possible metrics.
So I’m not sure how it’s relevant.
It might pay to hang out with him, again, based on the similarity criteria I point out: He’s someone very much like you, that is somewhat better at the thing you want to learn (programming).
Maybe weird writing on my end, the working out example that I’m referring is the section on professional athletes (aka them never necessarily having learnt how to do casual health-focused workouts). While physics teacher might have forgotten how it is not to know physics 101, but she still did learn physics 101 at some point.
Hopefully that makes it more clear?
The aliens seem to have also included with their boon:
Cheap and fast eec GPU RAM with minute electricity consumptions
A space time disruptor that allows you to have CMOS transistors smaller than electrons to serve as the L1&L2
A way of getting rid of electron tunneling at a very small scale
12 OOMs better SSDs and fiber optic connections and cures for the host of physical limitations plaguing mere possibility of those 2.
Ah, ok, maybe I was discussing the wrong thing then.
I think sleeping 4-6 hours on some days ought to be perfectly fine, even 0, I’d just argue that keeping the mean around 7-9 is probably ideal for most (but again, low confidence, think it boils down to personalized medicine).
The theory I heard postulated (by the guy that used to record the ssc podcast) is that once people start thinking “better” in reductionist frameworks they fail to account non quantifiable metrics (e.g. death is quantifiable in qaly, being more isolated isn’t)
The rest of your arguments (bpm, cortisol..) apply fully to sports as well I believe.
I don’t think so. BPM is slower when one practices sports (see athletes heart), in that it will be higher during the activity itself, but mean BPM during the day and especially at night is lower.
Personally I’ve observed this correlation as well and it seems to be causal~ish, i.e I can do 3 days on / 3 days off physical activity and notice decreased resting & sleeping heart rate on the 2nd day of activity up until the 2nd day of inactivity after which it picks back up.
With cortisol, the mechanism I’m aware of is the same, i.e. exercising increases cortisol afterwards but decreases the baseline. Though here I’m not 100% sure.
This might not hold for the very extreme cases though (strongmen, ultra-marathon runners, etc). Since then you’re basically under physical stress for most of the day instead of a few minutes or hours.
re: not encountering info re dangers of oversleep: do you want to comment on the bit about sleep deprivation therapy? Isn’t this rather compelling evidence of sleep directly causing bad mood?
Sleep deprivation, I’d assume, would work through cortisol and adrenaline, which do give a “better than awfully depressed mood” but can’t build up to great moods and aren’t sustainable (at least if I am to trust models ala the one championed by Sapolsky about the effects of cortisol).
Granted, I think it depends, and afaik most people don’t feel the need to sleep more than 8-9 hours. The ones I know that “sleep” a lot tend to just hang around in a half-comatosed state after overeating or while procrastinating. I think it becomes an issue of “actual sleep” vs “waking up every 30 minutes, checking phone, remembering life is hard and trying to sleep again | rinse and repeat 2 to 8 times”.
I’d actually find it interesting to study “heavy sleepers” in a sleep lab or with a semi-capable portable EEG (even just 2-4 electrodes should be enough, I guess?) and see if what they do is actually “sleep” past the 9 hour mark. But I’m unaware of any such studies.
But I have low confidence in all of these claims and I personally dislike epidemiological evidence, I think that there’s a horrible practice of people trying to “”“control””” shitty experiments with made-up statistics models that come with impossible assumptions built-in. My main decisions about sleep come from pulse-oximeter based monitoring and correlating that with how I feel plus other biomarkers (planning to upgrade to an openbci based eeg soon, been holding out for a freeEEG32 for a while, but I see only radio silence around that). So ultimately the side I fall on is that I dislike the evidence on way or another and think that, much like anything that uses epidemiology as it’s almost sole source of evidence, you could just scrap the whole thing in favour of a personalized goal-oriented approach.
Oh gosh, you’re right… both here and on my blog. Sometimes things go wrong during translation to markdown and I don’t even notice. Thanks for pointing it out, corrected.
The “think” here is more prosaic, as in, it’s just not my intuition that this is the case and I think that applies for most other people based on the memeplexes I see circulating out there.
As for why my intuition is that I can boil it down to, say I said in the post: Everyone told me so and everyone warned me about the effects of not sleeping but not vice versa.
Is this correct if analyzed on a rational basis?
I don’t know, it’s not relevant as far as the post is concerned.
From personal experience I know that for myself I associate shorter sleep with:
increase cortisol (urine, so arguable quality)
Increased time getting into ketosis (and overall lower ketone+glucose balance, i.e. given an a*G + b*BhB = y where y is the number at which I feel ok and seems in-line with what epidemiology would recommend as optimal glucose levels for lifespan | I will tend to fall constantly bellow y given lack of sleep, which manifests as being tired and sometimes feeling lightheaded and being able to walk, lift and swim less … hopefully that makes sense? I’m not sure how mainstream of a nutrition framework this is )
increased heart rate (significant, ~9bpm controlling~ish for effort and, ~7bpm o.o at night), *
feeling a bit off and feeling time passes faster.
But that’s correlated, not causal, e.g. if I smoke or vape during a day I’m likely to sleep less that night, so is smoking/vaping fucking my body or is lack of sleep or is the distinction even possible given the inferential capabilities of biology in the next 500+years? I don’t know
More broadly I know that looking back to months with plenty of sleep I feel much better than when i sleep less. But maybe that’s because I sleep less overall when I’m feeling down and also feel less happy (by definition) and am less productive, and maybe low sleep is actually a mechanism against feeling even worst.
Overall I assume most people notice these correlations, though probably less in-depth, based on how commonly people seem to complain about bad sleep, needing to get more sleep… etc and how rarely the opposite is true.
That is the best example I had of how one could, e.g, disagree with a scientific field by just erring on scepticism rather than taking the opposite view.
***
To answer your critique of that point, though again, I think it bares no or little relation to the article itself:
The “predictions” by which the theory is judges here are just as fuzzy and inferentially distant.
I am not a cosmologist, what I’ve read regrading cosmology have been mainly papers around unsupervised and semi-supervised clustering on noisy data, incidental evidence from those has made me doubt the complex ontologies proposed by cosmologists, given the seemingly huge degree of error acceptable in the process of “cleaning” data.
There are many examples of people fooling themselves into making experiments to confirm a theory and “correcting” or discarding results that don’t confirm it (see e.g. phlogiston, mass of the electron, the pushback against proton gradients as a fundamental mechanism of cell energy production, vitalism-confirming experiments, roman takes on gravity)
One way science can be guarded against modelling an idealized reality that no longer is related to the real world is by making something “obviously” real (e.g. electric lightbulb, nuclear bomb, vacuum engines).
Focusing on real-world problem also allows for different types of skin-in-the-game, i.e. going against the consensus for profit, even if you think the consensus is corrupt.
Cosmology is a field that requires dozens of years to “get into”, it has no practical applications that validate it’s theories, it’s only validation comes from observational evidence using data that is supposed to describe objects that are, again, a huge inferential distance away in both time/space and SNR… data which is heavily cleaned based on models created and validated by cosmology.
So I tend to err on the side of “bullshit” provided lack of relevant predictions that can be validate by literally anyone other than a cosmologist or a theoretical physicist, could be someone that’s proveably good in high energy physics validating an anomaly (e.g. gravitational anomaly causing a laser to behave weird around the time it was predicted that two black holes would be validated).
Hopefully this completes the picture to exhibit my point better.,
And what I’m saying is that I agree. As in, I’m not arguing that there’s no reason for the slope to be the way that is, I’d think most slopes are asymmetric exactly because of very real asymmetric risks/rewards they map to.
yes, that’s what I had intended, thanks for correcting that.
“Progress” can be a terminal goal, and many people might be much happier if they treated it as such. I love the fact that there are fields I can work in that are both practical and unregulated, but if I had to choose between e.g. medical researcher and video-game pro, I’m close to certain I’d be happier as the latter. I know many people which basically ruined their lives by choosing the wrong answer and going into dead-end fields that superficially seem open to progress (or to non-political work).
Furthermore, fields bleed into each other. Machine learning might well not be the optimal paradigm in which to treat <gestures towards everything interesting going on in the world>, but it’s the one that works for cultural reasons, and it will likely converge to some of the same good ideas that would have come about had other professions been less political.
Also, to some extent, the problem is one of “culture” not regulation. EoD someone could have always sold a covid vaccine as a supplement, but who’d have bought it? Anyone is free to make their own research into anything, but who’ll take them seriously?… etc