gwern
Would it be possible to use something like Maia, instead of Stockfish? Then you could more plausibly get ‘higher human-strength moves’, where you are not left trying to followup impossibly sharp tactical play you can’t pull off without a chess engine, or other issues with doing direct naive behavior cloning from superhuman chess engines.
(Review moved.)
Or, Zhihu has a very low weight in the training set of LLMs.
This would be my default expectation: “Zhihu is hard to crawl in some way and so just doesn’t get into training datasets”. For example, Twitter is notably absent from most LLMs… Because Twitter invests a lot of effort into blocking crawlers in order to preserve tweets for Grok and do price-discrimination on the API. This means that people who primarily tweet will be under-represented in LLMs. (Even Grok doesn’t actually seem to train much on tweets.)
You also have to write about yourself. I expect for a lot of these ‘project only’ names, there just isn’t anything about them online—as opposed to the project. What else is the LLM going to say...? If someone translates a bunch of essays, but doesn’t say a word about themselves, what else is the LLM going to say other than talk about the people who wrote the essays?
Maybe source from https://arxiv.org/abs/1202.3936 https://gwern.net/doc/math/2013-hisano.pdf as pre-AI sets of conjectures to monitor or target? (I have many errors listed in my math error essay but not sure how useful the ad hoc set is compared to the Hisano & Sornette work.)
No, Guardian Angels. But to fix pretraining/dynamic evaluation, you need to enrich the principal’s data a lot, I think, and dumping in fulltext of references is a good way to ensure the LLM personas have access to the principal’s context and avoid encouraging confabulation. (Gwern.net essays/annotations/Wikipedia serve as a kind of implicit ‘reference wiki’, as do the analyses/writeups we are having the LLMs generate to reverse-engineer writing.)
You did say it was cheap. Most RSS feeds are just not that big, even with fulltext. (For a project, we’re taking my entire history of web writing and trying to put in fulltext of all links and context and elaborate metadata with verbose XML formatting… and it still winds up being only like 10b tokens.)
And I guess if you do it in the background, you can probably find an even cheaper LLM somewhere. (Does OA still do that ’50% off’ batch background thing?)
Also, note that if you summarize all the items, not just the ones the user is about to look at, you now have a useful resource for searching, embeddings, recommendations, etc.
1s is still annoying UI jank and lag. If it’s cheap, why not just run it all in advance and cache the result? RSS is the perfect case for this because the items don’t change.
sometimes just serve that version. I assume that’s what Gwern is doing
Correct. I serve Markdown in two ways: first, every page includes the HTML-standardized metadata, which points to the Markdown file version of pages (using a simple template line:
<link rel="alternate" type="text/markdown" href="https://gwern.net$url$.md” title=”Markdown source of ‘$title-plain$’ page”>); second, I support standard HTTP content-negotiation, so any HTTP request which header specifies anywhere in it that it would accept atext/markdownortext/plainresult will then be given the.mdcontents instead. (Done in nginx where this is unpleasantly complicated.)So while the features may be a trifle obscure to any given programmer (although not web developer), they are well-known standardized features going back decades being used as they are supposed to be used, and as such should be entirely obvious and easy to use for coding agents (eg. adding
text/markdownto an agent’s HTTP Accept header downloads is basically free and it should be able to blindly put that in every web request without breaking anything).
Most obvious problem to me is that if you just did a finetune on your own normal writings, then “Rewrite this passage in your own words:” is very out of distribution, because you’re trying to create a persona or base model. But no one ever talks like that in those settings; only assistant chatbots converse in these kinds of abusive, peremptory prompts. You never talk to yourself or write that in your writings (do you?), so whatever follows is going to be strange.
If you are trying to get a ‘rewrite this to sound like me’, you might want to try something like creating a small dataset of of rewrite examples with that exact formatting, where you create the pairs by grabbing random paragraphs from your own writings and have a chatbot rewrite them to be ‘better’, and reversing them. Now when you do the atomic example, it’s a well-understood task with many examples of what to do, and it should behave more normally, like ending after 1 translated paragraph with quotation marks and EOT.
For GA, we’re experimenting with heavy use of role tags, which let us synthesize transcripts of things like the ‘LLM agent’ researching/writing stuff and then ‘Gwern’ rewriting it or writing a final essay. Seems to be working.
(I have also heard that there may be some things wrong with the TMI infrastructure where it trains fine and will run locally fine, but their runtime deployment is screwed up subtly. I doubt this is the case; but you could try downloading the Kimi checkpoint to run locally slowly, just as a sanity check.)
I’ll outsource my opinion on that to Nesov. I like density / fine-grained sparsity in principle, and since Western hyperscalers are so hardware-advantaged, I tend to expect less coarse-grained sparsity than Chinese labs are forced into.
Sure, why not. If you are bored and killing time, you may be quite curious—far more curious about the teacher’s (hopefully entertaining) past of drug use and rock and roll and hedonism than about the assigned curricular material… Nevertheless.
I don’t buy the intro example as being analogous to your two posts, or indeed, much of a conundrum about Bayesian epistemology.
The question was obviously bad to ask because it was either asked in bad faith or to kill time/boredom. You don’t need ‘advanced epistemology’ to note that the teacher’s personal anecdotal testimony has only epsilon bearing on the truth of DARE, because the badness of drug addiction is based on the experiences of billions of people over millennia and a vast amount of research, and this is as obvious as, say, the Big Bang. ‘Teacher, you say that the universe started in a Big Bang. But have you ever seen a Big Bang yourself?’ Such a question should not be dignified with an answer. Or Christopher Columbus, or, or.… in fact, ~100% of the things taught in middle school have no relationship to the teacher’s personal testimony. (Even in things like music or gym class.) Every middle schooler knows this and would regard as insane as a classmate who asks if the teacher had gone to the moon, and when the teacher responded they hadn’t, then took seriously the possibility that maybe the moon is made of papier-mâché because the science teacher hadn’t personally gone there and is ‘just’ relaying the reports of people like Neil Armstrong or astronomer consensus about it being made of rocks. The claims of DARE may be wrong (and in fact I think they broadly are from what I remember of it), but the teacher’s personal drug use tells one little—and that is before you get into anything that could reasonably be considered ‘advanced epistemology’ for middle schoolers (such as selection effects—eg. if drugs really did destroy peoples’ lives with high probability, a drug-user teacher probably wouldn’t be standing there teaching them).
Meanwhile, in both your posts, your personal experiences and judgments are a major part of them. You are not a neutral far-downstream reporter of major topics; even in the sense that you are writing about research papers in the LLM post, those research areas are extremely new and controversial and complex and have results rapidly changing at an almost daily rate, so you are applying a lot of your critical judgment and beliefs in which research to highlight, how to interpret them, and how to combine them along with your personal experiences and commentary. (Consider if the teacher had titled that class day lecture “How I Stopped Being Sure Drugs Are So Bad” and filled it with anecdotes about LSD use and the latest MAPS papers on clinical trials.)
My point is that in your scenario, it is entirely possible that S’ had managed to obtain a job she liked because she was genetically predisposed (eg. having a nice personality), but that S had not for exogenous random bad luck, and that if they were measured at age 39, they would seem unusually discordant twins—except it was just bad luck, and S would regress to her mean at age 40 and now they would be concordant. You can’t just handwave it and say, ‘S just got a new better job! Heritability must be going down!’ The new job could well be heritability going up. Your ‘random example’ is just poorly chosen and irrelevant to the discussion since it’s not at all obvious that it is an example of E going up and ACD going done just because “she obviously has the same genes now as she did last year.” There could be a time reversal: the bad job could have been the E, not the good one!
Arbitrarily high sparsity seems useful with unlimited data, see Figure 11 from this paper (the horizontal axis is log-sparsity; it looks like infinite sparsity gives an infinite compute multiplier).
Not sure that’s of interest, because that seems like it might be a trivial phenomenon of asymptotic reasoning. In the limit of unlimited or infinite data, you are simply constructing a giant look-up table from question to answer, which of course requires zero compute to associate an input with a pre-specified output. (Even the overhead of a nearest-neighbors lookup will effectively disappear, as that’s just log or something.) Which could be said of every possible algorithm, you could always just hybridize it with a lookup table to get an infinite data limit like that, so it can’t tell us anything interesting about real NNs.
I agree it’s too soon. It’s easy to forget, in our bubbles, that lots of insurers and providers are still balking at even semaglutide and trying to ration it, or, as we go to parties where people have been offering free retatrutide shots for years that it’s still not officially approved by the FDA! A lot of people are still talking about how their ‘new pickle juice diet’ finally worked for them...
But she obviously has the same genes now as she did last year.
Leaving the job is not an example because it could have increased the heritability of her personality survey. Pretty much all traits change in heritability over a lifetime (eg. Wilson effect for IQ or think about, say, psychiatric disorders) and other variance components as well, so you can’t point to a single change and say it is an example of E because ‘A didn’t change’ - well, neither did C (if an adult) or D, they could all be increasing due to that single change (you can view it as regression to a mean as she finally leaves an outlier job, as most people do not ‘hate’ their job), so in fact, it may be any of ACD!
See also: “‘Winning’ AI Arms Races: Then What?” (mirror).
Does this really work? It doesn’t work for non-code things; ask Sol or Fable to ‘write like gwern’, and assuming they don’t refuse outright because I am a living author, it is certainly not a gwern-like output ‘to the extent you can’t tell whether gwern wrote it’! Or if it does work, what would make code different?