I’m doing a physics PhD, and you’re making me feel better about my coding practices. I appreciate your explicit example as well, as I’m interested in trying my hand at ML research and curious about what it looks like in terms of toolsets and typical sort-of-thing-one-works-on. I want to chime in down here in the comments to assure people that at least one horrible coder in a field which has nothing to do with machine learning (most of the time) thinks that the sentiment of this post is true. I admit that I’m biased by having very little formal CS training, so proper functional programming is more difficult for me than writing whatever has worked for me in the past writing ad-hoc Bash scripts. My sister is a professional software developer, and she winces horribly at my code. However, you point out that it is often the case that any particular piece of research code you are running has a particular linear set of tasks to achieve, and so:
You don’t need to worry much about resilient code which handles weird edge cases.
It is often better to have everything in one place where you can see it than to have a bunch of broken up functions scattered across a folder full of files.
Nobody else will need to use the code later, including yourself, so legibility is less important
As an example of the Good/Good-enough divide, here’s a project I’m working on. I’m doing something which requires speed, so I’m using c++ code built on top of old code someone else wrote. I’m extremely happy that the previous researcher did not follow your advice, at least when they cleaned up the code for publishing, because it makes life easier for me to have most of the mechanics of my code hidden away out of view. Their code defines a bunch of custom types which rather intuitively match certain physical objects. They wrote a function which parses arg files so that you don’t need to recompile the code to rerun a calculation with different physical parameters. Then there’s my code which uses all of that machinery: My main function that I have written is sort of obviously a nest of loops over discrete tasks which could easily be separate functions, but I just throw them all together into one file, and I rewrite the whole file for different research questions so I have a pile of “main” files which reuse a ton of structure. As an example of a really ugly thing I did, I hard-code indices corresponding to momenta I want to study into the front of my program instead of making a function which parses momenta and providing an argument file listing the sets I want. I might have done that for the sake of prettiness, but I needed to provide a structure which lets me easily find momenta of opposite parity. Hard-coding the momenta let me keep the structure I was using at front of mind when I created the four other subtasks in the code which exploited that structure to let me construct subtasks which needed to easily find objects of opposite parity.
still trying to figure out the “optimal” config setup. The “clean code” method is roughly to have dedicated config files for different components that can be composed and overridden etc (see for example, https://github.com/oliveradk/measurement-pred). But I don’t like how far away these configs are from the main code. On the other hand, as the experimental setup gets more mature I often want to toggle across config groups. Maybe the solution is making a “mode” an optional config itself with overrides within the main script
At the start of my Ph.D. 6 months ago, I was generally wedded to writing “good code”. The kind of “good code” you learn in school and standard software engineering these days: object oriented, DRY, extensible, well-commented, and unit tested.
I think you’d like Casey Muratori’s advice. He’s a software dev who argues that “clean code” as taught is actually bad, and that the way to write good code efficiently is more like the way you did it intuitively before you were taught OOP and stuff. He advises “Semantic Compression” instead- essentially you just straightforwardly write code that works, then pull out and reuse the parts that get repeated.
just read both posts and they’re great (as is The Witness). It’s funny though, part of me wants to defend OOP—I do think there’s something to finding really good abstractions (even preemptively), but that its typically not worth it for self-contained projects with small teams and fixed time horizons (e.g. ML research projects, but also maybe indie games).
Can’t agree more with this post! I used to be afraid of long notebooks but they are powerful in allowing me to just think.
Although while creating a script I tend to use “#%%” of vscode to run cells inside the script to test stuff. My notebooks usually contain a bunch of analysis code that don’t need to be run, but should stay.
I’m doing a physics PhD, and you’re making me feel better about my coding practices. I appreciate your explicit example as well, as I’m interested in trying my hand at ML research and curious about what it looks like in terms of toolsets and typical sort-of-thing-one-works-on. I want to chime in down here in the comments to assure people that at least one horrible coder in a field which has nothing to do with machine learning (most of the time) thinks that the sentiment of this post is true. I admit that I’m biased by having very little formal CS training, so proper functional programming is more difficult for me than writing whatever has worked for me in the past writing ad-hoc Bash scripts. My sister is a professional software developer, and she winces horribly at my code. However, you point out that it is often the case that any particular piece of research code you are running has a particular linear set of tasks to achieve, and so:
You don’t need to worry much about resilient code which handles weird edge cases.
It is often better to have everything in one place where you can see it than to have a bunch of broken up functions scattered across a folder full of files.
Nobody else will need to use the code later, including yourself, so legibility is less important
As an example of the Good/Good-enough divide, here’s a project I’m working on. I’m doing something which requires speed, so I’m using c++ code built on top of old code someone else wrote. I’m extremely happy that the previous researcher did not follow your advice, at least when they cleaned up the code for publishing, because it makes life easier for me to have most of the mechanics of my code hidden away out of view. Their code defines a bunch of custom types which rather intuitively match certain physical objects. They wrote a function which parses arg files so that you don’t need to recompile the code to rerun a calculation with different physical parameters. Then there’s my code which uses all of that machinery: My main function that I have written is sort of obviously a nest of loops over discrete tasks which could easily be separate functions, but I just throw them all together into one file, and I rewrite the whole file for different research questions so I have a pile of “main” files which reuse a ton of structure. As an example of a really ugly thing I did, I hard-code indices corresponding to momenta I want to study into the front of my program instead of making a function which parses momenta and providing an argument file listing the sets I want. I might have done that for the sake of prettiness, but I needed to provide a structure which lets me easily find momenta of opposite parity. Hard-coding the momenta let me keep the structure I was using at front of mind when I created the four other subtasks in the code which exploited that structure to let me construct subtasks which needed to easily find objects of opposite parity.
thanks for the detailed (non-ML) example! exactly the kind of thing I’m trying to get at
also see Cognitive Load Is What Matters
still trying to figure out the “optimal” config setup. The “clean code” method is roughly to have dedicated config files for different components that can be composed and overridden etc (see for example, https://github.com/oliveradk/measurement-pred). But I don’t like how far away these configs are from the main code. On the other hand, as the experimental setup gets more mature I often want to toggle across config groups. Maybe the solution is making a “mode” an optional config itself with overrides within the main script
I think you’d like Casey Muratori’s advice. He’s a software dev who argues that “clean code” as taught is actually bad, and that the way to write good code efficiently is more like the way you did it intuitively before you were taught OOP and stuff. He advises “Semantic Compression” instead- essentially you just straightforwardly write code that works, then pull out and reuse the parts that get repeated.
just read both posts and they’re great (as is The Witness). It’s funny though, part of me wants to defend OOP—I do think there’s something to finding really good abstractions (even preemptively), but that its typically not worth it for self-contained projects with small teams and fixed time horizons (e.g. ML research projects, but also maybe indie games).
Can’t agree more with this post! I used to be afraid of long notebooks but they are powerful in allowing me to just think.
Although while creating a script I tend to use “#%%” of vscode to run cells inside the script to test stuff. My notebooks usually contain a bunch of analysis code that don’t need to be run, but should stay.
Thanks! huh yeah the python interactive windows seems like a much cleaner approach, I’ll give it a try
This is a great post, and I like the research process. Do you know if the LLM code completion in cursor is compatible with ipynb notebooks?
thanks! yup curser is notebook compatible