Daniel_Eth

Karma: 614

Daniel_Eth Dec 28, 2021, 12:29 AM
−3 points
AF
on: larger language models may disappoint you [or, an eternally unfinished draft]
Play with GPT-3 for long, and you’ll see it fall hard too.
...
This sample is a failure. No one would have written this, not even as satire or surrealism or experimental literature. Taken as a joke, it’s a nonsensical one. Taken as a plot for a film, it can’t even keep track of who’s alive and who’s dead. It contains three recognizable genres of writing that would never appear together in this particular way, with no delineations whatsoever.
This sample seems pretty similar to the sort of thing that a human might dream, or that a human might say during/immediately after a stroke, a seizure, or certain types of migraines. It’s clear that the AI is failing here, but I’m not sure that humans don’t also sometimes fail in somewhat similar ways, or that there’s a fundamental limitation here that needs to be overcome in order to reach AGI.
The first time you see it, it surprises you, a crack in the floor… Eventually, you no longer picture of a floor with cracks in it. You picture a roiling chaos which randomly, but regularly, coalesces into ephemeral structures possessing randomly selected subsets of the properties of floors.
^I guess the corollary here would be that human minds may also be roiling chaos which randomly coalesce into ephemeral structures possessing properties of floors, but just are statistically much more likely to do so than current language models.

Daniel_Eth Dec 3, 2021, 8:46 AM
LW: 1 AF: 1
AF
on: Fractional progress estimates for AI timelines and implied resource requirements
FWIW, Hanson has elsewhere promoted the idea that algorithmic progress is primarily due to hardware progress. Relevant passage:
Maybe there are always lots of decent ideas for better algorithms, but most are hard to explore because of limited computer hardware. As hardware gets better, more new ideas can be explored, and some of them turn out to improve on the prior best algorithms. This story seems to at least roughly fit what I’ve heard about the process of algorithm design.
So he presumably would endorse the claim that HLMI will likely requires several tens of OOM more compute than we currently have, but that a plateauing in other inputs (such as AI researchers) won’t be as relevant. (Here’s also another post of Hanson where he endorses a somewhat related claim that we should expect exponential increases in hardware to translate to ~linear social impact and rate of automation.)

Daniel_Eth Nov 10, 2021, 10:51 AM
1 point
in reply to: Annapurna’s comment on: Worth checking your stock trading skills
“uranium, copper, lithium, oil”
These are commodities, not equities (unless OP meant invested in companies in those industries?)

Daniel_Eth Nov 10, 2021, 10:12 AM
3 points
0
in reply to: Thomas Sepulchre’s comment on: Great-Filter Hard-Step Math, Explained Intuitively
So again, I wasn’t referring to the expected value of the number of steps, but instead how we should update after learning about the time – that is, I wasn’t talking about $E [k | t],$ but instead $P (k | t) / P (k)$ for various $k$ .
Let’s dig into this. From Bayes, we have: $P (k | t) / P (k) = P (t | k) / P (t)$ . As you say, $P (t | k)$ ~ kt^(k-1). We have the pesky $P (t)$ term, but we can note that for any value of $t$ , this will yield a constant, so we can discard it and recognize that now we don’t get a value for the update, but instead just a relative value (we can’t say how large the update is at any individual $k$ , but we can compare the updates for different $k$ ). We are now left with $P (k | t) / P (k)$ ~ kt^(k-1), holding $t$ constant. Using the empirical value on Earth of $t = (4.5 / 5.5) = 0.82$ , we get $P (k | t = 0.82) / P (k)$ ~ k*0.82^(k-1).
If we graph this, we get:
which apparently has its maximum at 5. That is, whatever the expected value for the number of steps is after considering the time, if we do update on the time, the largest update is in favor of there having been 5 steps. Compared to other plausible numbers for $k$ , the update is weak, though – this partiuclar piece of evidence is a <2x update on there having been 5 steps compared to there having been 2 steps or 10 steps; the relative update for 5 steps is only even ~5x the size of the update for 20 steps.
Considering the general case (where we don’t know $t$ ), we can find the maximum of the update by setting the derivative of kt^(k-1) equal to zero. This derivative is (k ln(t) + 1)t^(k-1), and so we need $k * l n (t) = - 1$ , or $k = - 1 / l n (t)$ . If we replace $t$ with $x / (x + 1)$ , such that $x$ corresponds to the naive number of steps as I was calculating before, then that’s $k = - 1 / l n (x / (x + 1))$ . Here’s what we get if we graph that:

This is almost exactly my original guess (though weirdly, ~all values for $k$ are ~0.5 higher than the corresponding values of $x$ ).

Daniel_Eth Nov 8, 2021, 10:37 PM
1 point
in reply to: Thomas Sepulchre’s comment on: Great-Filter Hard-Step Math, Explained Intuitively
The intuition, I assume, is that this is the inverse function of the previous estimator.
So the estimate for the number of hard steps doesn’t make sense in the absence of some prior. Starting with a prior distribution for the likelihood of the number of hard steps, and applying bayes rule based on the time passed and remaining, we will update towards more mass on k = t/(T–t) (basically, we go from P( t | k) to P( k | t)).

By “gives us reason to expect” I didn’t mean “this will be the expected value”, but instead “we should update in this direction”.

Daniel_Eth Nov 5, 2021, 9:39 PM
1 point
in reply to: Thomas Sepulchre’s comment on: Great-Filter Hard-Step Math, Explained Intuitively
Having a model for the dynamics at play is valuable for making progress on further questions. For instance, knowing that the expected hard-step time is ~identical to the expected remaining time gives us reason to expect that the number of hard steps passed on Earth already is perhaps ~4.5 (given that the remaining time in Earth’s habitability window appears to be ~1 billion years). Admittedly, this is a weak update, and there are caveats here, but it’s not nothing.
Additionally, the fact that the expected time for hard steps is ~independent of the difficulty of those steps tells us that, among other things, the fact that abiogenesis was early in Earth’s history (perhaps ~400 MY after Earth formed) is not good evidence that abiogenesis is easy, as this is arguably around what we’d expect from the hard-step model (astrobiologists have previously argued that early abiogenesis on Earth is strong evidence for life being easy and thus common in the universe, and that if it was hard/rare that we’d expect abiogenesis to have occurred around halfway through Earth’s lifetime).
Of course, this model doesn’t help solve the latter two questions you raised, as it doesn’t touch on future steps. A separate line of research that made progress on those questions (or a different angle of attack on the questions that this model can address) would also be a valuable contribution to the field.

Daniel_Eth Nov 5, 2021, 9:15 PM
4 points
AF
in reply to: Steven Byrnes’s comment on: Modeling the impact of safety agendas
I like this comment, though I don’t have a clear-eyed view of what sort of research makes (A) or (B) more likely. Is there a concrete agenda here (either that you could link to, or in your head), or is the work more in the exploratory phase?

Daniel_Eth Oct 28, 2021, 10:23 PM
4 points
in reply to: JohnofCharleston’s comment on: Covid 10/28: An Unexpected Victory
Yeah, that also triggered my “probably false or very misleading” alarm. People are making all sorts of wild claims about covid online for political points, and I don’t even know who the random person on twitter making that claim was.

Daniel_Eth Oct 21, 2021, 7:10 PM
LW: 1 AF: 1
AF
in reply to: Richard_Ngo’s comment on: Emergent modularity and safety
Yeah, I’m not trying to say that the point is invalid, just that phrasing may give the point more appeal than is warranted from being somewhat in the direction of a deepity. Hmm, I’m not sure what better phrasing would be.

Daniel_Eth Oct 21, 2021, 5:28 PM
LW: 3 AF: 3
AF
in reply to: johnswentworth’s comment on: Emergent modularity and safety
The statement seems almost tautological – couldn’t we somewhat similarly claim that we’ll understand NNs in roughly the same ways that we understand houses, except where we have reasons to think otherwise? The “except where we have reasons to think otherwise” bit seems to be doing a lot of work.

Daniel_Eth Oct 18, 2021, 8:56 PM
1 point
in reply to: Elizabeth’s comment on: Long Covid Informal Study Results
Thanks. I feel like for me the amount of attention for a marginal daily pill is negligibly small (I’m already taking a couple supplements, and I leave the bottles all on the kitchen table, so this would just mean taking one more pill with the others), but I suppose this depends on the person, and also the calculus is a bit different for people who aren’t taking any supplements now.

Daniel_Eth Oct 18, 2021, 3:25 AM
2 points
on: Long Covid Informal Study Results
“the protocol I analyze later requires a specific form of niacin”
What’s the form? Also, do you know what sort of dosage is used here?
If niacin is helpful for long covid, I wonder if taking it decreases the chances of getting long covid to begin with. Given how well tolerated it is, it might be worth taking just in case.

Daniel_Eth Oct 17, 2021, 10:04 PM
2 points
in reply to: Snorkelfarsan’s comment on: What if we should use more energy, not less?
“at least nanotech and nano-scale manufacturing at a societal scale would require much more energy than we have been willing to provide it”

Maybe, but:
1) If we could build APM on a small scale now we would
2) We can’t
3) This has nothing to do with energy limits
(My sense is also that advanced APM would be incredibly energy efficient and also would give us very cheap energy – Drexler provides arguments for why in Radical Abundance.)
I don’t think regulatory issues have hurt APM either (agree they have in biotech, though). Academic power struggles have hurt nanotech (and also biotech), though this seems to be the case in every academic field and not particularly related to creeping institutional sclerosis (over the past several hundred years, new scientific ideas have often had trouble breaking in through established paradigms, and we seem less bad on this front than we used to be). Regardless, neither of these issues would be solved with more energy, and academic power struggles would still exist even in the libertarian state Hall wants.

Daniel_Eth Oct 17, 2021, 6:55 AM
9 points
on: What if we should use more energy, not less?
I can see how oodles more energy would mean more housing, construction, spaceflight, and so on, leading to higher GDP and higher quality of life. I don’t see how it would lead to revolutions in biotech and nanotech – surely the reason we haven’t cured aging or developed atomically precise manufacturing aren’t the energy requirements to do those things.

Daniel_Eth Oct 16, 2021, 2:46 AM
4 points
in reply to: lsusr’s comment on: Cheap food causes cooperative ethics
Worth noting that Northern states abolished slavery long before industrialization. Perhaps even more striking, the British Empire (mostly) abolished slavery during the peak of its profitability. In both cases (and many others across the world), moral arguments seem to have played a very large role.

Daniel_Eth Oct 14, 2021, 11:42 PM
16 points
on: Covid 10/14: Less Long Cvoid
“Mandates continue to make people angry”
True for some people, but also worth noting that they’re popular overall. Looks like around 60% of Americans support Biden’s mandate, for instance (this is pretty high for a cultural war issue).
“Republicans are turning against vaccinations and vaccine mandates in general… would be rather disastrous if red states stopped requiring childhood immunizations”
Support has waned, and it would be terrible if they stopped them, but note that:
- Now republicans are split ~50:50; so it’s not like they have a consensus either way
- Republicans being split and others (including independents) being in favor means that majority is clearly in favor, even in red states
- Republican support has recovered somewhat already, and I’d expect support will continue to revert closer to pre-COVID levels as we progress further (especially years out); we might not reach pre-covid levels, but I’d be surprised if the general view of republicans was against several years from now (though OTOH, perhaps those against are more strongly against, so you could wind up in a single-issue voter type problem)
“3%”
This seems to be at the 12 week mark, which is somewhat arbitrary. Even according to the same study, looks like long covid rates are closer to 1% after 19 weeks.
“To be blunt, they cheated (intentionally or otherwise)”
Flagging that I don’t like this language, for a couple reasons:
- I think it’s inaccurate/misrepresentative. “Cheating”, in my mind, implies some dishonesty. Yes, words can obviously be defined in any way, but I’m generally not a fan of redefining words with common definitions unless there’s a good reason. If, on the other hand, your claim is that they were indeed being dishonest, then I think you should come out and say that (otherwise what you’re doing is a little motte-and-bailey-ish).
- I think it’s unnecessarily hostile. People make mistakes, including scientists making dumb mistakes. It’s good that they corrected their mistake (which is not something lots of people—including scientists—do). The fact that none of us caught it shows just how easy it is to make these sort of mistakes. (Again, this point doesn’t stand if you are trying to imply that it was intentional, but then I think you should state that.) I similarly don’t think it’s apt to call it “fessing up” when they correct their mistake.
Looks like this dropped after your post here so you wouldn’t have been able to incorporate it – advisors to the FDA are recommending moderna boosters for the same group of people that are getting pfizer boosters (65+, risk for health reasons, or risk for job), and also this will be at half dose. They should make a recommendation on J&J tomorrow.

Daniel_Eth 6 Oct 2021 21:50 UTC
LW: 1 AF: 1
AF
in reply to: Said Achmiz’s comment on: Comments on “The Singularity is Nowhere Near”
Also, these physical limits – insofar as they are hard limits – are limits on various aspects of the impressiveness of the technology, but not on the cost of producing the technology. Learning-by-doing, economies of scale, process-engineering R&D, and spillover effects should still allow for costs to come down, even if the technology itself can hardly be improved.

Daniel_Eth 6 Oct 2021 20:42 UTC
LW: 1 AF: 1
AF
in reply to: Steven Byrnes’s comment on: Draft report on AI timelines
Potentially worth noting that if you add the lifetime anchor to the genome anchor, you most likely get ~the genome anchor.

Daniel_Eth 1 Oct 2021 1:37 UTC
5 points
on: Covid 9/30: People Respond to Incentives
“Resources are always limited (as they should be) and prioritization is necessary. Why should they focus on who is and isn’t wearing a mask over enforcing laws against, I don’t know, robbery, rape and murder?”

I’m all for the police prioritizing serious crimes over more minor crimes (potentially to the extent of not enforcing the minor crime at all), but I have a problem, as a general rule, with the police telling people that they won’t enforce a law and will instead just be asking for voluntary compliance. That sort of statement is completely unnecessary, and seems to indicate that the city doesn’t have as strong control of their police as they should.

Daniel_Eth 24 Sep 2021 10:37 UTC
3 points
in reply to: TurnTrout’s comment on: Covid 9/23: There Is a War
Also, the train of thought seems somewhat binary. If doctors are somewhat competent, but the doctors who worked at the FDA were unusually competent, then having an FDA would still make sense.