Those people will probably not see this so wont reply.
What I can tell you is that in the last three months I went through a phase transition in my AI use and I regret not doing this ~1 year earlier.
It’s not that I didnt use AI daily before for mundane tasks or writing emails, it’s not that I didnt try a couple times to get it to solve my thesis problem (it doesn’t get it) - it’s that I failed to refrain my thinking from asking “can AI do X?” to “how can I reengineer and refactor my own workflow, even the questions I am working on so as to maximally leverage AI?”
About 6 months ago you strongly recommended that I make use of the integrated AI plugin for Overleaf (Writefull). I did try it. Its recommended edits seem quite useless to me; they always seem to flow from a desire to make the wording more normal/standard/expected in contrast to more correct (which makes some sense given the way generatie pre-training works). This is obviously useful to people with worse English, but for me, the tails come apart massively between “better” and “more normal/standard/expected”, such that all the AI suggestions are either worse or totally neutral rephrasing.
It also was surprisingly bad at helping me write LaTeX; I had a much better time asking Claude instead.
It’s not that I didnt use AI daily before for mundane tasks or writing emails,
I haven’t found AI at all useful for writing emails, because the AI doesn’t know what I want to say, and taking the time to tell the AI isn’t any easier than writing it myself. AI can only help me write the boring boilerplate stuff that email recipients would skim over anyway (which I don’t want to add to my emails). AI can’t help me get info out of my head this way—it can only help me in so far as emails have a lot of low-entropy cruft. I can see how this could be useful for someone who has to write a lot of low-entropy emails, but I’m not in that situation. To some degree this could be mitigated if the LLMs had a ton of context (EG recording everything that happens on my computer), but again, only the more boring cases I think.
I’d love to restore the Abram2010 ability to crank out several multi-page emails a day on intellectual topics, but I don’t think AI is helpful towards that end yet. I haven’t tried fine-tuning on my own writing, however. (I haven’t tried fine-tuning at all.)
Similarly, LLMs can be very useful for well-established mathematics which had many examples in the training data, but get worse the more esoteric the mathematics becomes. The moment I ask for something innovative, the math becomes phony.
Across the board, LLMs seem very useful for helping people who are at the lower end of a skill ladder, but not yet very useful for people at the upper end.
So I’m curious, how did you refactor your workflow to make better use of AI?
I’m still quite curious what you have found useful and how you’ve refactored your workflow to leverage AI more (such that you wish you did it a year ago).
I do use Perplexity, exa.ai and elicit as parts of my search strategy.
@abramdemski I think I’m the biggest agree vote for alexander (without me alexander would have −2 agree), and I do see this because I follow both of you on my subscribe tab.
I basically endorse Alexander’s elaboration.
On the “prep for the model that is coming tomorrow not the model of today” front, I will say that LLMs are not always going to be as dumb as they are today. Even if you can’t get them to understand or help with your work now, their rate of learning still makes them in some sense your most promising mentee, and that means trying to get as much of the tacit knowledge you have into their training data as possible (if you want them to be able to more easily & sooner build on your work). Or (if you don’t want to do that for whatever reason) just generally not being caught flat-footed once they are smart enough to help you, as all your ideas are in videos or otherwise in high context understandable-only-to-abram notes.
Should you write text online now in places that can be scraped? You are exposing yourself to ‘truesight’ and also to stylometric deanonymization or other analysis, and you may simply have some sort of moral objection to LLM training on your text.
This seems like a bad move to me on net: you are erasing yourself (facts, values, preferences, goals, identity) from the future, by which I mean, LLMs. Much of the value of writing done recently or now is simply to get stuff into LLMs. I would, in fact, pay money to ensure Gwern.net is in training corpuses, and I upload source code to Github, heavy with documentation, rationale, and examples, in order to make LLMs more customized to my use-cases. For the trifling cost of some writing, all the worlds’ LLM providers are competing to make their LLMs ever more like, and useful to, me.
On the “prep for the model that is coming tomorrow not the model of today” front, I will say that LLMs are not always going to be as dumb as they are today.
Right, I strongly agree with this part.
their rate of learning still makes them in some sense your most promising mentee
I disagree in the sense that they’re no mentee of mine, ie, me trying to get today’s models to understand me doesn’t directly help tomorrow’s models to understand. (With the exception of the limited forms of feedback in the interface, like thumbs up/down, the impact of which I’m unsure of so it doesn’t feel like something I should deliberately spend a lot of time on.)
I also disagree in the sense that engaging with LLMs right now seems liable to produce a lot less fruits downstream, even as measured by “content that can usefully prompt an LLM later”. IE, if mentees are viewed as machines that convert time-spent-dialoging-with-me to text that is useful later, I don’t think LLMs are currently my most promising mentees.
So although I strongly agree with continuing to occasionally poke at LLMs to prep for the models that are coming soon & notice when things get better, to the extent that “most promising mentee” is supposed to imply that significant chunks of my time could be usefully spent with LLMs in the present, I disagree based on my (fairly extensive) experience.
trying to get as much of the tacit knowledge you have into their training data as possible (if you want them to be able to more easily & sooner build on your work).
Barring special relationships with frontier labs, this sounds functionally equivalent to trying to get my work out there for humans to understand, for now at least.
I did talk to Anthropic last year about the possibility of me providing detailed feedback on Claude’s responses (wrt my research questions), but it didn’t end up happening. The big problems I identified seemed to be things they thought would definitely get addressed in another way, so there wasn’t a mutually agreed-on value proposition (I didn’t understand what they hoped to gain, & they didn’t endorse the sorts of things I hoped to train). I got busy and moved on to other things.
Or (if you don’t want to do that for whatever reason) just generally not being caught flat-footed once they are smart enough to help you, as all your ideas are in videos or otherwise in high context understandable-only-to-abram notes.
I feel like this is speaking from a model I don’t understand. Are videos so bad? Video transcriptions are already a thing, and future models should be better at watching video and getting info from it. Are personal notes so bad? What sorts of actions are you recommending? I already want to write as many text posts as I can.
skills issue.
prep for the model that is coming tomorrow not the model of today
I’m seeing some agreement-upvotes of Alexander here so I am curious for people to explain the skill issue I am having.
Those people will probably not see this so wont reply.
What I can tell you is that in the last three months I went through a phase transition in my AI use and I regret not doing this ~1 year earlier.
It’s not that I didnt use AI daily before for mundane tasks or writing emails, it’s not that I didnt try a couple times to get it to solve my thesis problem (it doesn’t get it) - it’s that I failed to refrain my thinking from asking “can AI do X?” to “how can I reengineer and refactor my own workflow, even the questions I am working on so as to maximally leverage AI?”
About 6 months ago you strongly recommended that I make use of the integrated AI plugin for Overleaf (Writefull). I did try it. Its recommended edits seem quite useless to me; they always seem to flow from a desire to make the wording more normal/standard/expected in contrast to more correct (which makes some sense given the way generatie pre-training works). This is obviously useful to people with worse English, but for me, the tails come apart massively between “better” and “more normal/standard/expected”, such that all the AI suggestions are either worse or totally neutral rephrasing.
It also was surprisingly bad at helping me write LaTeX; I had a much better time asking Claude instead.
I haven’t found AI at all useful for writing emails, because the AI doesn’t know what I want to say, and taking the time to tell the AI isn’t any easier than writing it myself. AI can only help me write the boring boilerplate stuff that email recipients would skim over anyway (which I don’t want to add to my emails). AI can’t help me get info out of my head this way—it can only help me in so far as emails have a lot of low-entropy cruft. I can see how this could be useful for someone who has to write a lot of low-entropy emails, but I’m not in that situation. To some degree this could be mitigated if the LLMs had a ton of context (EG recording everything that happens on my computer), but again, only the more boring cases I think.
I’d love to restore the Abram2010 ability to crank out several multi-page emails a day on intellectual topics, but I don’t think AI is helpful towards that end yet. I haven’t tried fine-tuning on my own writing, however. (I haven’t tried fine-tuning at all.)
Similarly, LLMs can be very useful for well-established mathematics which had many examples in the training data, but get worse the more esoteric the mathematics becomes. The moment I ask for something innovative, the math becomes phony.
Across the board, LLMs seem very useful for helping people who are at the lower end of a skill ladder, but not yet very useful for people at the upper end.
So I’m curious, how did you refactor your workflow to make better use of AI?
Fair enough.
I’m still quite curious what you have found useful and how you’ve refactored your workflow to leverage AI more (such that you wish you did it a year ago).
I do use Perplexity, exa.ai and elicit as parts of my search strategy.
@abramdemski I think I’m the biggest agree vote for alexander (without me alexander would have −2 agree), and I do see this because I follow both of you on my subscribe tab.
I basically endorse Alexander’s elaboration.
On the “prep for the model that is coming tomorrow not the model of today” front, I will say that LLMs are not always going to be as dumb as they are today. Even if you can’t get them to understand or help with your work now, their rate of learning still makes them in some sense your most promising mentee, and that means trying to get as much of the tacit knowledge you have into their training data as possible (if you want them to be able to more easily & sooner build on your work). Or (if you don’t want to do that for whatever reason) just generally not being caught flat-footed once they are smart enough to help you, as all your ideas are in videos or otherwise in high context understandable-only-to-abram notes.
In the words of gwern,
Right, I strongly agree with this part.
I disagree in the sense that they’re no mentee of mine, ie, me trying to get today’s models to understand me doesn’t directly help tomorrow’s models to understand. (With the exception of the limited forms of feedback in the interface, like thumbs up/down, the impact of which I’m unsure of so it doesn’t feel like something I should deliberately spend a lot of time on.)
I also disagree in the sense that engaging with LLMs right now seems liable to produce a lot less fruits downstream, even as measured by “content that can usefully prompt an LLM later”. IE, if mentees are viewed as machines that convert time-spent-dialoging-with-me to text that is useful later, I don’t think LLMs are currently my most promising mentees.
So although I strongly agree with continuing to occasionally poke at LLMs to prep for the models that are coming soon & notice when things get better, to the extent that “most promising mentee” is supposed to imply that significant chunks of my time could be usefully spent with LLMs in the present, I disagree based on my (fairly extensive) experience.
Barring special relationships with frontier labs, this sounds functionally equivalent to trying to get my work out there for humans to understand, for now at least.
I did talk to Anthropic last year about the possibility of me providing detailed feedback on Claude’s responses (wrt my research questions), but it didn’t end up happening. The big problems I identified seemed to be things they thought would definitely get addressed in another way, so there wasn’t a mutually agreed-on value proposition (I didn’t understand what they hoped to gain, & they didn’t endorse the sorts of things I hoped to train). I got busy and moved on to other things.
I feel like this is speaking from a model I don’t understand. Are videos so bad? Video transcriptions are already a thing, and future models should be better at watching video and getting info from it. Are personal notes so bad? What sorts of actions are you recommending? I already want to write as many text posts as I can.
Don’t get me wrong, I’ve kept trying and plan to keep trying.