gwern
Karma: 41,517
NewTop
Internet Search Tips: how I use Google/Google Scholar/Libgen
Where did the rest of this article go? There’s just a paragraph at the start, on both LW2/GW.
The gossip I hear is that the Gods of Straight Lines continued somewhat but prices took a breather because of the Illumina quasi-monopoly (think Intel vs AMD). Several of the competitors stumbled badly for what appear to be reasons unrelated to the task itself: BGI gambled on an acquisition to develop its own sequencers which famously blew up in its face for organizational reasons, and rumor has it that 23andMe spent a ton of money on an internal effort which it eventually discarded for unknown reasons. You’ll notice that after WGS prices paused for years on end, suddenly, now that other companies have begun to catch up, Illumina has begun talking about $100 WGS next year and we’re seeing DTC WGS drop like a stone.
That’s not very imaginative. Here’s how a chess tree search algorithm—let’s take AlphaZero for concreteness—could learn to kill other processes, even if it has no explicit action which corresponds to interaction with other processes and is apparently sandboxed (aside from the usual sidechannels like resource use). It’s a variant of the evolutionary algorithm which learned to create a board so large that its competing GAs crashed/were killed while trying to deal with it (the Tic-tac-toe memory bomb). In this case, position evaluations can indirectly reveal that an exploration strategy caused enough memory use to trigger the OOM, killing rival processes, and freeing up resources for the tree search to get a higher win rate by more exploration:
-
one of the main limits to tree evaluation is memory consumption, due to the exponential growth of breadth-first memory requirements (this is true regardless of whether an explicit tree or implicit hash-based representation is used); to avoid this, memory consumption is often limited to a fixed amount of memory or a mix of depth/breadth-first strategies are used to tame memory growth, even though this may not be optimal, as it may force premature stopping to expansion of the game tree (resorting to light/heavy playouts) or force too much exploitation depthwise along a few promising lines of play and too little exploration etc. (One of the criticisms of AlphaZero, incidentally, was that too little RAM was given to the standard chess engines to permit them to reach their best performance.)
-
when a computer OS detects running out of memory, it’ll usually invoke an ‘OOM killer’, which may or may not kill the program which makes the request which uses up the last of free memory
-
so, it is possible that if a tree search algorithm exhausts memory (because the programmer didn’t remember to include a hard limit, the hard limit turns out to be incorrect for the machine being trained on, the limit is defined wrong like in terms of max depth instead of total nodes, etc), it may not crash or be killed but other programs, using unknown & potentially large percentages of memory, may be killed instead to free up memory. (I’ve observed this on Linux, to my frustration, where the programs I don’t want killed get killed by the OOM reaper instead of the haywire program.)
-
once other programs are killed to free up memory, all that memory is now available for the tree search algorithm to use; using this memory will increase performance by allowing more of the game tree to be explicitly evaluated, either wider or deeper.
-
in AlphaZero, the choice of widening or deepening is inherently controlled by the NN, which is trained to predict the result of the final values of each position and increase win probabilities.
-
reaching a position (which can be recognized by its additional complexity, indicating it lies at a certain additional depth in the tree and thus indirectly reveals how much memory is being used by the NN’s cumulative exploration) which triggers an OOM killing other programs will result in more accurate position evaluations, leading to higher values/higher win probability; so it will reinforce a strategy where it learns to aggressively widen early in the game to exhaust memory, waits for an OOM to happen, and then in the rest of the game proceeds to explore more aggressively (rather than depth-first exploit) given the new memory.
(Depending on the exact details of how the tree expansion & backups are done, it’s possible that the AlphaZero NN couldn’t observe the benefits of wide-then-deep—it might just look like noise in value estimates—but there are expert iteration variants where the NN directly controls the tree expansion rather than merely providing value estimates for the MCTS algorithm to explore using, and those should be able to observe indirect benefits of exploration strategies over a game.)
At no point does it interact directly with other processes, or even know that they exist; it just implicitly learns that expanding a decision tree in a particular wide-then-deep fashion leads to better evaluations more consistent with the true value and/or end-game result (because of side-effects leading to increased resource consumption leading to better performance). And that’s how a tree-search algorithm can hit upon killing other processes.
-
That’s a good question, and the answer may be that they don’t have color vision in any normal sense; what they have is the ability to use chromatic aberration to focus their eyes for various colors, and this serial focusing scan lets them decide how to adjust their skin to match surroundings: “Spectral discrimination in color blind animals via chromatic aberration and pupil shape”, Stubbs & Stubbs 2016.
Should this be considered color vision? It seems safe to say that whatever the qualia of scanning chromatic-aberration vision would be, it would be very different from our simultaneous realtime trichromatic color vision. And it’s worth noting that under the Stubbs model, to explain why behavioral assays (not just the finding that they only have one kind of photoreceptor) found them to be color blind, there’s a lot of things they can’t do with the chromatic aberration trick:
Second, some behavioral experiments (7⇓⇓⇓–11) designed to test for color vision in cephalopods produced negative results by using standard tests of color vision to evaluate the animal’s ability to distinguish between two or more adjacent colors of equal brightness. This adjacent color comparison is an inappropriate test for our model (Fig. 4R). Tests using rapidly vibrating (8, 9) color cues are also inappropriate. Although these dynamical experiments are effective tests for conventional color vision, they would fail to detect spectral discrimination under our model, because it is difficult to measure differential contrast on vibrating objects. These results corroborate the morphological and genetic evidence: any ability in these organisms for spectral discrimination is not enabled by spectrally diverse photoreceptor types
...In our proposed mechanism, cephalopods cannot gain spectral information from a flat-field background or an edge between two abutting colors of comparable intensity (Fig. 3). This phenomenology would explain why optomotor assays and camouflage experiments using abutting colored substrates (7, 9, 11) fail to elicit a response different from a flat-field background. Similarly, experiments (10) with monochromatic light projected onto a large uniform reflector or training experiments (8, 9) with rapidly vibrating colored cues would defeat a determination of chromatic defocus
...We predict that the animals will fail to match flat-field backgrounds with no spatial structure as previously shown in figure 3B in the work by Mäthger et al. (7) just as a photographer could not determine best focus when imaging a screen with no fine-scale spatial structure. If, for instance, their ability to spectrally match backgrounds was conferred by the skin or another potential unknown mechanism, they would successfully match on flat-field backgrounds. However, under our model, they should succeed when there is a spatial structure allowing for the calculation of chromatically induced defocus, such as in our test patterns (Fig. 4) or the more naturally textured backgrounds by Kühn (21). If, however, cephalopods truly cannot accurately match their background color but solely use luminance and achromatic contrast to determine camouflage, we would expect the response on colored substrates to be identical to that on a gray substrate of similar apparent brightness with identical spatial structure.
November 2018 gwern.net newsletter
Watching the Q&A was mindbending, and reminded me why I dislike bioethicists so much. Asking He Jiankui whether the participants were literate enough to read the forms! (You got one question, with so many important things to ask, and that was the point you decided to make?)
Well, do you have anything in mind specifically to do with it? If you do, it may not be worthwhile to wait. But if you don’t have something which needs to be done with a WGS right now, you probably aren’t going to be struck with inspiration once you get your download either.
No, I was mentioning the sales because they offer a measurement of what WGS costs end to end now—presumably Veritas/Dante or Nebula are offering at close to their marginal cost (as they aren’t big or wealthy enough to afford to give it away and WGSes aren’t exactly a repeat-customer business). As far as Dante goes, I have seen some complaints about very slow or inconsistent service; on IRC, one of us did a previous sale and their original spit didn’t work, so they sent him another tube and forgot the postage. Not sure if he’s gotten his WGS yet either.
Yes. In fact, I am already a PGP participant.
I am not sure you necessarily want to use Veritas/Dante Labs (Veritas might be sold out already based on their Twitter), as WGS reports are usually pretty raw and you won’t get all of the interpretive services somebody like 23andMe would provide. I don’t believe 23andMe or the other major services let you just upload sequencing data either, only download. Offhand, I’m not sure how easy it would be to even use Promethease (not that Promethease is very worthwhile, as most of their report is candidate-gene junk). Personally, I am holding off on getting a WGSes done. I don’t know what I would do with mine, and the price should keep getting lower.
I wonder if Jill Bolte Taylor was temporarily enlightened? (Incidentally, The God Formula is now on Libgen.)
Just got a Veritas-related email:
Veritas Genetics will be offering their MyGenome product (30x whole genome sequencing) normally $999, for $199 to the first 1000 customers, starting tomorrow, Monday, 9 AM ET.
Even allowing for promotional discounts, I’m still impressed. EDIT: Dante Labs too!
Galaxy brain take: one man’s modus ponens is another man’s modus tollens—he wouldn’t wear a car helmet, so by the same logic, it’s not worthwhile to wear a helmet while biking: https://www.forbes.com/sites/carltonreid/2018/10/21/i-do-not-wear-a-bicycle-helmet/
I felt some deja vu reading this and looked it up. Did anything in particular prompt crossposting it to LW over a year later, or is this just when you got around to it?
It’s worth remembering that the chicken experiment was specifically designed to elicit that effect, and chickens are unusual in being confined to extremely small cages with other chickens. That doesn’t happen with cows or apples or wheat or… As far as I know, animal/plant breeders typically totally ignore such indirect genetic effects/group-level effects (or even model them away, absorbing them into fixed/random effects), along with ignoring apparently vital stuff like epistasis/dominance, and yet the dumb simple selection methods based on additivity work fine and still realize all the improvements they are supposed to. Yields go up reliably every year.
Consumer WGSes hit ~$1000 with Veritas in 2016. In 2018, Dante Labs began offering WGS at ~$600, with a sale of $350. And we now have a rumor that Illumina will announce a $100 genome in a few months (presumably in early 2019): https://twitter.com/coregenomics/status/1058790189752049664
$100 might be a little questionable here (apparently Illumina has a history of making the most favorable possible assumptions about volume/amortization) but revisiting my original prediction from 7 years ago:
As part of my work for Luke, I looked into price projections for whole genome sequencing, as in not SNP genotyping, which I expect to pass the $100 mark by 2014. The summary is that I am confident whole-genome sequencing will be <$1000 by 2020, and slightly skeptical <$100 by 2020.
I was too pessimistic about SNP genotyping (it was actually more like $50 in 2014, I was completely unaware of UK Biobank at the time or its scale or savings), definitely right about ‘<$1000 by 2020’, and I think I will turn out to be somewhat wrong about WGS being <$100 by 2020: even if Illumina is fudging some numbers for early 2019 at $100, it’ll have almost a whole year to drop the cost a little more, and honestly, even if it’s actually $110 does it make a difference considering how many things you can use whole genomes for & general medical overhead? You can hardly get some prescription aspirin these days for $100...
Overall self-assessment: I was more right than I had any right to be in that set of predictions given I was using some simple extrapolating and adding some pessimism/mean-reversion. Not bad, past-self!
I was thinking more of Harry Potter: it’s similar in that we have a pervasively magical/fantasy universe in which the main characters often are engaged in adventures and are special or elevated in various ways, but there’s enough worldbuilding that you can easily imagine writing a fanfic simply about ordinary everyday life (eg about a Ministry of Magic clerk struggling with goblin negotiations or Ollivander experimenting with wand R&D tweaks—“I have not failed 100 times, I have discovered 100 ways to not process unicorn hair for wands”). The Potterverse is entirely flexible enough to let you write almost any story you want—just set it among muggles, or in America, if you need to. Even J.K. Rowling does it… That doesn’t seem like a major driver of HP fanfics, though. And the MLP fics I’ve personally read do tend to draw heavily on the main characters or at least fantasy or genre tropes (eg that one based on Death Note, or the isekai Myou’ve got to be kidding). One counterexample that comes to mind is Twilight fics which discarded all of the vampire stuff entirely, and that niche spawned 50 Shades of Grey, but I never would’ve heard of that niche if not for 50 Shades, of course, so maybe it’s not the best example. If we sampled fimfiction.net randomly (whether weighting by work, chapter, words, or pageviews), would we really come up with a lot of MLP fanfics which weren’t either very based on MLP adventure/fantasy plot or based ‘on wizards’ or ‘about vampires’ or ‘about ninjas’? I’m doubtful.
(I agree that from what I’ve used of it, fimfiction.net does seem a lot better technically than fanfiction.net, which certainly can’t hurt. That thing must be running on 30 year old code by now… and I’ve always been irritated by their dumb little tricks like CSS for disabling copy-paste.)
At https://www.gwern.net/MLP specifically.
Theranos, as I understand it, was promising blood testing of all sorts of biomarkers like blood glucose, and nothing to do with DNA. DNA sequencing is different from measuring concentration—at least in theory, you only need a single strand of DNA and you can then amplify that up arbitrary amounts (eg in PGD/embryo selection, you just suck off a cell or two from the young embryo and that’s enough to work with). If you were trying to measure the nanograms of DNA per microliter, that’s a bit different.
I don’t know anything about RNA sequencing, since it’s not relevant to anything I follow like GWASes.