This is great to see laid out! I’ve done this kind of work for around a decade as well, but the logistics of assembling it into a full analysis has not passed a cost-benefit calculation.
How much of your process would you say is scalable/generalizable vs your own ad hoc refinement with LLMs? I think a general method or chain of prompts would be useful for others.
Unrelated, but I hadn’t seen your work on diagramming before. I do something similar, but as a causal loop diagram. I think the main difference is that yours look like the have a set source/sink, while I’ve found most of my problems have more sustaining feedback structure. I’ll also put +/- for the sign of the effect. Maybe just a difference in the problems themselves.
Thanks! I think this process is fairly generalizable and could work for many people. Most people probably have a smaller pile of notes that takes less time to preprocess. I think the year range rather than the number of notes makes the most difference for the quality of AI output. I added the prompts I used as an appendix to the post.
For the diagrams, the way I drew them on paper was to put the problems on a circle and then draw edges between things that contribute to each other. The resulting diagram often (but not always) ended up having a source and/or a sink node. I agree that the feedback loops are the most interesting part of this, as they suggest multiple points of intervention that can have outsized effects (because they weaken the whole loop), while the sources (like “should” mindset) were often things that were hard to budge directly.
This is great to see laid out! I’ve done this kind of work for around a decade as well, but the logistics of assembling it into a full analysis has not passed a cost-benefit calculation.
How much of your process would you say is scalable/generalizable vs your own ad hoc refinement with LLMs? I think a general method or chain of prompts would be useful for others.
Unrelated, but I hadn’t seen your work on diagramming before. I do something similar, but as a causal loop diagram. I think the main difference is that yours look like the have a set source/sink, while I’ve found most of my problems have more sustaining feedback structure. I’ll also put +/- for the sign of the effect. Maybe just a difference in the problems themselves.
Thanks! I think this process is fairly generalizable and could work for many people. Most people probably have a smaller pile of notes that takes less time to preprocess. I think the year range rather than the number of notes makes the most difference for the quality of AI output. I added the prompts I used as an appendix to the post.
For the diagrams, the way I drew them on paper was to put the problems on a circle and then draw edges between things that contribute to each other. The resulting diagram often (but not always) ended up having a source and/or a sink node. I agree that the feedback loops are the most interesting part of this, as they suggest multiple points of intervention that can have outsized effects (because they weaken the whole loop), while the sources (like “should” mindset) were often things that were hard to budge directly.