Some principles of using Thinking Assistants for more than “just keep you vaguely accountable”*
* (for me, it’s not at all surprising if others need radically different things)
Often, I’m in the middle of figuring out a confusing thing, which is taking up all my working memory. The exact right question can help direct my attention towards an important place. The wrong question is annoying and overloads my stack.
Sometimes assistants just magically intuit the exact right thing to ask, but it’s hard to rely on this.
Thus:
Principle #1: Construct an if-then tree of prompts, which both I and the assistant can iterate on. The job of the assistant isn’t (usually) to magically intuit the right thing. We’re working together on writing phrases that are helpful for them to say periodically when situations seem to call for it.
Principle #2: Prompts should be very easy to respond to. Usually simple yes/no questions, or questions where I don’t really have to respond, they’re just might nudge my semi-conscious brain in the right direction.
Worse question: “What are you confused by?” (I dunno, that’s the problem. It is sometimes helpful to fully articulate the confusion to see if anything reveals itself, but usually not while I’m in the middle of trying to figure it out)
Less-worse question: “Are you confused right now?” (This is a’ight but trying to figure out “does my current qualia count as ‘confusion’?” is also a bit overloading)
Better questions:
Do you know what to do next?
Have you tried more than 2 things to figure this out?
I don’t have a comprehensive satisfying one – one of the issues is realistically I can only handle a couple prompt suggestions before they’re getting too distracting, so there’s some art to reading the situation and trying to pick a prompt likely to be useful.
But, a working draft
Top level prioritization:
Are you tired?
Will napping work? Will walking around? Will caffeine?
Are you oriented?
Is it better to get momentum or get more clarity?
(with something like a default towards ~10 minutes getting more clarity at least once a day unless I’m feeling like it’s a particularly momentum-centric day)
Are you having trouble focusing?
Are there any other projects you could move forward, with a brief action?
For code debugging in particular:
Do you feel a little confused / Does this feel hard?
Have you tried solving the current problem multiple times?
What questions do you not know the answer to?
Do you feel like you’re getting anywhere with what you’re doing right now? (reword to be more concise)
Where would a smart programmer look to find the most useful information?
Are you getting an error message?
If so, can you trace that anywhere useful?
Have you searched for that message online?
Is this something an LLM could help with?
Is the issue something you introduced recently, or did it already exist?
Is the issue occurring consistently or intermittently?
If it’s intermittent:
Can you cause it to happen?
What observations can you make about it?
Do the logs look different when something is wrong?
Some principles of using Thinking Assistants for more than “just keep you vaguely accountable”*
* (for me, it’s not at all surprising if others need radically different things)
Often, I’m in the middle of figuring out a confusing thing, which is taking up all my working memory. The exact right question can help direct my attention towards an important place. The wrong question is annoying and overloads my stack.
Sometimes assistants just magically intuit the exact right thing to ask, but it’s hard to rely on this.
Thus:
Principle #1: Construct an if-then tree of prompts, which both I and the assistant can iterate on. The job of the assistant isn’t (usually) to magically intuit the right thing. We’re working together on writing phrases that are helpful for them to say periodically when situations seem to call for it.
Principle #2: Prompts should be very easy to respond to. Usually simple yes/no questions, or questions where I don’t really have to respond, they’re just might nudge my semi-conscious brain in the right direction.
Worse question: “What are you confused by?” (I dunno, that’s the problem. It is sometimes helpful to fully articulate the confusion to see if anything reveals itself, but usually not while I’m in the middle of trying to figure it out)
Less-worse question: “Are you confused right now?” (This is a’ight but trying to figure out “does my current qualia count as ‘confusion’?” is also a bit overloading)
Better questions:
Do you know what to do next?
Have you tried more than 2 things to figure this out?
Do you have a clear understanding of the problem?
What is your current working tree of prompts?
I don’t have a comprehensive satisfying one – one of the issues is realistically I can only handle a couple prompt suggestions before they’re getting too distracting, so there’s some art to reading the situation and trying to pick a prompt likely to be useful.
But, a working draft
Top level prioritization:
Are you tired?
Will napping work? Will walking around? Will caffeine?
Are you oriented?
Is it better to get momentum or get more clarity?
(with something like a default towards ~10 minutes getting more clarity at least once a day unless I’m feeling like it’s a particularly momentum-centric day)
Are you having trouble focusing?
Are there any other projects you could move forward, with a brief action?
For code debugging in particular:
Do you feel a little confused / Does this feel hard?
Have you tried solving the current problem multiple times?
What questions do you not know the answer to?
Do you feel like you’re getting anywhere with what you’re doing right now? (reword to be more concise)
Where would a smart programmer look to find the most useful information?
Are you getting an error message?
If so, can you trace that anywhere useful?
Have you searched for that message online?
Is this something an LLM could help with?
Is the issue something you introduced recently, or did it already exist?
Is the issue occurring consistently or intermittently?
If it’s intermittent:
Can you cause it to happen?
What observations can you make about it?
Do the logs look different when something is wrong?
Is it a race condition?
Is it a caching issue?